Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
TypeScript
fix(store): merge schema lists per entry, personas as records, shared namespace mirror (#307)
Serg Eroshe0789dc1aff6c
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 21c1ae27..9b9cbf19 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Added+- **Panel personas on a store-only machine.** `o2b brain panel personas` lists the personas in force, `o2b brain panel persona-set <slug> [--lens L] [--file F]` adds or replaces one (the prompt from the file or stdin) and `o2b brain panel persona-remove <slug>` removes one. Before the store switch a persona stays a note under `Brain/personas/`; after it, it is a store record, so a persona you curate on one machine is the panel's lens on every machine of the store, with or without a vault. Removing the last custom persona brings back the built-in set. The verbs refuse while the machine is frozen, on both kinds of machine (on a switched machine every lane page write, store-mode settings write and schema apply now also refuses while it is frozen), and, on a machine that has not switched, while a migration runs on its vault (on a switched machine a migration gate left naming the vault refuses nothing), over a note of another kind at the persona's path, over a persona that lives in `private` (remove it first, then set it again as shared), and for a private persona (text with a `<private>` region or naming a private page, which machines without `private` would never see); a prompt holds at most 16 KB and a lens at most 200 characters on one line. On a machine that has not switched and runs the dual-write trial, removing a persona also removes the copy the trial wrote to the store, so it does not come back after the switch; the same holds for every Brain page a lane removes or moves (a pledge, a staged dream folder, an archived dead end). The store copy of a moved page is the page as it is after the move, a page moved onto its own path keeps its store copy, and a private page moved over a shared one, on a switched machine too, leaves no shared copy behind. A moved page that is a symbolic link, or sits in a linked folder, is never read into the store (the move removes the old copy, and the store copy of a page it overwrote, as the migration never imports such a page), and a folder moved into its own subfolder or into a folder that holds it is refused on both kinds of machine, as the rename refuses it, where a switched machine used to lose an entry. A page moved over a private one stays private, as a page written over it does; a page the trial never stored, moved over one it did, replaces that store copy instead of leaving the overwritten text to come back after the switch; a page moved onto itself on a switched machine writes nothing; and a lane page or folder moved out of the Brain lanes is refused on both kinds of machine, where a switched machine used to lose it. While a record of the store has not reached this machine whole (a shard read only in part, or a record in `private` whose body blob has not arrived, quarantined or not), a moved or written lane page is kept out of `shared`, since the missing record may be the one that makes it private. A record in `shared` whose blob has not arrived never makes a page private, so it holds nothing back: a machine that holds only `shared` goes on writing lane pages after a lost blob there is quarantined.- **Topics: the `brain_topic` tool, `o2b topic`, and planning in dream.** `brain_topic` and the `o2b topic` verbs list, show and review topics and change them (create, rename, status, goal, assign, unassign, merge, split, link, archive, undo), and let an agent record its own decisions, open questions, log entries and milestones on the topic it works on. A command is stored as your record; a tool call carries the agent's authority unless the agent names yours on your explicit request, which a remote caller cannot, so an agent's goal never replaces the one you wrote and shows as an open question instead. A record names only an item of its own topic. A caller of a remote MCP transport never sees a private topic: it is absent from every list, `show` and `osb://topic/{id}` answer it as an unknown id, and a write that names it is refused the same way. The dream pass plans topics on the view writer only, under a lock, so two machines or two passes never create the same topic: it routes a new record of a project to the topic it clearly belongs to, creates a topic when enough related records of one project (`topics.new_min_records`, default 5) come from at least two sessions (`topics.new_min_sessions`), lists every create and route in its report with the `o2b topic undo` command that takes it back, and never routes or recreates what you undid. Merges and splits are only proposed, and a topic with no news for `topics.quiet_days` (default 30) is offered for a status decision in the digest; nothing changes until you answer. Unticking a done milestone on its page opens it again. Undoing an undo of a private topic's record stays in `private`, private titles, descriptions and goals never route a shared record, a `record` whose `owner` holds no such item (or is neither the topic nor one merged into it) is refused, and a remote caller gets the scope of `list` and `show` fenced like the other topic text. A drafted title of characters outside the Basic Multilingual Plane is cut to fit the record limit instead of never becoming a topic. A view writer move that arrives while a pass plans stops it before it writes, a dream pass whose deadline passed while it waited for the planning lock writes nothing, and a pass that fails midway still lists the topics and routes it wrote, with their undo commands. A split moves an item of a topic merged into the one it names, and refuses an item the topic does not show; an item id a split moved away is not taken again for a new item, and a decision reworded on its page keeps the milestones it serves. When two topics exchange folder names, each keeps its own companion files. A companion move or a conflict copy never deletes a newer version a sync tool delivers under the same name meanwhile; it is kept, and a copy is ingested on the next pass. A folder or symbolic link named like a conflict copy is refused where it is, never moved or read. Editing a private item of a shared topic keeps the new text in `private`, a private record is routed by the private text of a topic, and `o2b topic split --owner` names whose item moves when two merged topics share its id. `brain_topic` refuses a `private` or `removed` value that is not a boolean instead of ignoring it, a dream pass plans topics on its own date and its undo commands name the config it ran with, and a topic counts as active when a topic merged into it or a split into it brings news. A private completion or resolution makes its item private, so a later one written without the flag stays private, and a route you took back from a topic stays blocked after that topic is merged into another. The state digests now carry a `topics` fold hash, so doctor reports machines whose topic folds diverge; a digest from an earlier release has none and is not compared on it. The topics guide is `docs/topics.md`.- **`osb://topic/{slug}` is now `osb://rule-topic/{slug}`.** The resource that summarises the signals and the rule of a preference slug moved to its new name, and `osb://topic/{id}` serves topics; the old name still answers a slug for one minor version. Code that names the preference slug says `ruleTopic` now; the `topic` field of signals keeps its name.- **Topics in the brain store and the human view (first part).** A topic is one area of work or one personal matter with an id that never changes: its title, description, status, goal, milestones, decisions, open questions, log, links, merges, splits and record assignments are records of the `topics` stream (`topic-create`, `topic-rename`, `topic-status`, `topic-goal`, `decision`, `open-question`, `question-resolved`, `log-entry`, `milestone`, `milestone-done`, `assign`, `unassign`, `merge`, `split`, `link`, `undo`, documented in `FORMAT.md`), folded the same way on every machine, with your records beating an agent's and an agent's beating a decision model's. A rename keeps the id and every reference, an undo voids the record it names and keeps a route you took back from being taken again, a milestone is done only when you mark it or an agent cites evidence, and a personal topic goes to the `private` partition by default. The view writer renders each shared topic to `topics/<date>-<slug>/topic.md` in a fixed section order (title, description, status, goal, milestones, decisions, open questions, the log newest first, related), moves the oldest entries over the page budget to a companion `archive.md`, shows only earned log entries, and moves the folder, your companion files included, when the topic is renamed. Your edits of a topic page become topic records with your authority, so a goal you rewrite stays even when an agent later proposes another, which then shows as an open question. Research reports and source summaries render to `reports/` and `sources/`. A note tool refuses a topic, report or source page of the view and names the tool to use instead; with the view at the vault root, your own notes in folders of those names stay writable. The view writer only ever touches a page of the exact shapes it writes, inside its view folder and never through a symbolic link; a folder at a topic page's name is reported as refused while the rest of the view renders, and a companion file the writer moves is never left in both folders, also when a sync tool replaces it mid-move or Windows refuses to remove it. Undoing an automatic route or taking a record out of a topic blocks that route whether you or an agent wrote the undo.@@ -16,6 +17,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Fixed+- **Schema field lists no longer lose an edit made on another machine.** A schema apply on a store-only machine writes one record per entry it adds or removes (`setting-entry`, documented in `FORMAT.md`) instead of the whole list, and a list setting folds per entry, so two machines that change the same schema field before they sync keep both changes and read one list, whatever order the records arrive in. An add and a remove of the same entry resolve to your record over an agent's, then to the later one; a keyed entry such as a prefix holds one value per name, and two spellings of one token (case, spaces, quotes) are one entry. Only `brain.schema.<field>` keys take entry records, so no entry record can remove a setting that holds one value. Lists written whole by the migration or `o2b brain settings set` read as before. An agent cannot change an entry you set or removed, through an apply or through a whole-list `brain_settings` write, and an accepted write reports the list in force. A type removed on one machine and a reference to it (an alias, a prefix, a link constraint pair) added on another no longer block every later apply: the pack drops references to undeclared tokens when it is read, the same on every machine, and the next apply removes them from the records. Removing a type now also removes the link constraint pairs that name it (and a link type's constraints once none is left), and an apply refuses a pair whose source or target is not declared, so a removed type no longer keeps allowing or blocking typed links of pages whose `type:` still names it. A pack merged from two applies stays attested: each apply's audit row names the records it wrote and the records rows it had seen attest, the migration's import attests what it imported, and a record no apply wrote reports the pack as modified and makes the next store-only apply refuse until you adopt it with `o2b brain schema apply --adopt-unattested`, alone or with the apply's mutations. A row attests each record in the version it saw (`<id>:<sha256>`), so another body written under an attested record's id is a finding too, and a record under `brain.schema.` for a name the pack does not read is neither part of the pack nor something to adopt. A whole-list record of a schema field whose value is no list of one-line entries (a single string or number, an object, an entry with a line break) is not applied, so it can no longer hide the list from the pack and from the adoption check. An apply reads the pack, the records that decide it and the stored entries once, so an entry another machine adds while the apply runs is neither removed by it as a repair nor attested by it, and the integrity check reads the pack and its records once the same way. On a machine that has not switched, `o2b brain schema apply --adopt-unattested` with an empty batch (`--mutation '[]'` too) is refused and leaves `_brain.yaml` as it is, and `--mutation '[]'` alone counts as no mutation. On a store-only machine, adoption alone writes no entry, the removal of a dangling reference included; an apply's removal of a dangling reference is skipped when another machine replaced that entry meanwhile, and is written as a conditional removal (`if_decided_by`) that every machine ignores once a replacement it never saw decides the entry, so a replacement that syncs in only after the removal is kept too; an attribute or frontmatter tier whose `type.field` name has spaces around its dot (`paper . year`) is one entry with `paper.year`, as the pack reads it, and is no longer removed by an unrelated apply as a dangling reference; a schema field holding only an empty list or an unset beside such a conditional removal no longer makes every settings read fail; and `o2b brain schema report` reports the pack as modified, naming the records, whenever the next apply would refuse them, also when an unaudited record left the pack as it was.+- **The shared namespace on the brain store.** `shared_namespace` gets no store of its own: where the shared vault is store-only on a machine, it is that machine's brain store, which every store-only agent reads. The mirror now writes there when the origin record went only to the origin vault's legacy tree, and still refuses, with the reason, on a switched machine, for a store-only origin vault, with dual writes on (the record is in that store already), and when the machine has no store; these checks read the machine config the write was made with. A namespace path that is not a directory (unmounted, missing or a file) is reported as failed instead of being mirrored into this machine's store.- **A lane entry id maps to exactly one legacy path.** A page and a JSON file of one name (`x.json.md` and `x.json`) no longer share an entry id, and a name that would not map back to itself (a file with another extension or none, a folder named like another folder's lane, such as `exact-state/`) is refused by name on a store-only machine and listed by the migration as an item it could not convert, instead of being filed under an id that names another path. A folder of your own under `Brain/` spelled otherwise than a lane name (`My Notes/`, `project_x/`) is still imported, under its lane (`my-notes`), and a rollback puts it back at its own path; two folders that would take one lane (`My Notes/` beside `my-notes/`) are listed and archived instead, and `brain_doctor` names them before a migration (`lane-folder-unimportable`) so you can rename one first. A page an earlier release's migration filed under the id of the file beside it is moved to its own id by the next dream pass on a store-only machine that holds `private`, once per machine, and again once new records arrive (or after a pass that could not read every shard), and the file gets its id back; a page removed after its repair stays removed. `brain_doctor` names any page still filed that way (`lane-id-misfiled`) with the remedy, also on a store where no machine holds `private`.- **A procedure use no longer appends store records.** The procedural memory usage counters, graph and recall hints are machine state, kept in the device-local state once a machine is switched, so `mark_used` and `mark_outcome` rewrite one local file instead of appending the index, the usage file, the graph and the hints to the store on every use. The migration moves the counters into the device-local state, and counters an index still carries are read and moved on the next reconcile. Counters are per machine from now on. A machine switched by an earlier release reads the counters, graph and hints from their store records until it has its own copy, adopting them on the first read, and the removal of the legacy tree retires those records.- **A schema apply is attested on a machine that holds only `shared`.** A store-only apply writes its audit row to `shared`, beside the schema settings records it attests, so such a machine reports another device's apply as `ok` instead of `modified` or unverified. The shared row carries the attestation and the mutation list; the reason stays on the whole row in `private`, where audit rows always went on a machine that holds it, and a batch or reason that names a private page or carries a `<private>` region leaves only the attestation on the shared row.diff --git a/docs/brain-store.md b/docs/brain-store.mdindex 4b08824b..83f4956e 100644--- a/docs/brain-store.md+++ b/docs/brain-store.md@@ -638,7 +638,14 @@ machine writes is refused, because its writer may still be appending. Aquarantined record whose blob arrives later reads with its body, with nostep of yours. The rollback, the late import of the transition, themigration preflight and the legacy removal still refuse while any record-waits for its blob, quarantined or not. One exception:+waits for its blob, quarantined or not. While a record in `private` waits+for its blob, quarantined or not, a Brain lane page written or moved on+this machine is kept out of `shared` (refused on a machine without+`private`), since the record may be the one that makes the page private. A+record in `shared` that waits for its blob does not hold lane pages back:+a record there never makes a page private, so a machine that holds only+`shared` goes on writing lane pages after you quarantine its lost blob.+One exception:`o2b brain migrate-store --remove-legacy --force` goes past a shard readonly in part, a quarantined or pending blob included, as it goes past acohort device still on the legacy layout. That risks data loss: a record@@ -667,6 +674,93 @@ arrived) leaves the shard whole. When another process holds the ledger'slock past the wait budget, `quarantine` and `release` refuse with`ledger-busy` and change nothing; run the command again.+## Settings, schema and personas across machines++Shared settings are records in `shared/settings/`, so a change on one+machine applies on every other once the record syncs. A setting holds one+value, and when two machines change it before they sync, your change beats+an agent's and then the later one wins.++The schema fields (`brain.schema.<field>`, what `o2b brain schema apply` and+`schema_apply_mutations` change) are lists, and they merge per entry: an+apply writes one record per entry it adds or removes, never the whole list.+Two machines that change the same field before they sync keep both changes.+An add on one machine and a remove of the same entry on the other resolve+like a setting: your record beats an agent's, then the later one wins. For+fields of `name=value` entries that hold one value per name (`prefixes`,+`expert_routing`, `attributes`, `frontmatter_tiers`) a new value replaces+the old entry; in `attributes` and `frontmatter_tiers` the name is+`type.field`, and spaces around its dot do not make a second name. Two+spellings of one token (case, spaces, quotes) are one entry. An agent cannot change an entry you set or removed, through an apply+or through a whole-list `o2b brain settings set` or `brain_settings` write,+and an accepted write reports the list in force.++When a type is removed on one machine and a reference to it (an alias, a+prefix, an expert route, an attribute) is added on another, the merged+fields hold a reference to a type that is gone. The pack drops such+references when it is read, the same way on every machine, and the next+apply removes them from the records, so no apply is blocked by them. An+apply skips that removal when another machine has replaced the entry in+the meantime, and the removal it writes names the record it judged, so+every machine ignores it once a replacement it never saw, even one that+syncs in later, decides the entry.++The schema integrity check accepts a pack merged from applies on several+machines: each apply's audit row names the records it wrote and the+records rows it had seen already attest, and the migration's import+attests the records it imported. A record no apply wrote (a settings write,+a hand-made line) still reports the pack as modified, even when it leaves+the pack as it was, and the next+store-only apply refuses with `unattested_schema_records`, naming the+records. A row attests each record in the version it saw, so another body+written under the id of an attested record counts as unattested too. Look+at them with `o2b brain schema report`; if they are right, adopt them with+`o2b brain schema apply --adopt-unattested` (alone, or added to the apply+that refused), which records the records and your name in the audit row.+Adoption alone works only on a store-only machine; an empty batch there+(no `--mutation`, or `--mutation '[]'`) changes no entry.+An agent cannot adopt. A record under `brain.schema.` for a name the pack+does not read is no part of the pack and needs no adoption.++Panel personas are store records once a machine is switched: curate them+with `o2b brain panel persona-set`, `persona-remove` and `personas`, on any+machine of the store, with or without a vault. Before the switch they are+notes under `Brain/personas/`. A persona is read on every machine, so a+private one (a `<private>` region, or a private page named in its lens or+prompt) is refused; the prompt holds at most 16 KB and the lens at most 200+characters on one line. The verbs refuse in a frozen vault, on a switched+machine too, and over a note of another kind at the persona's path. A+persona that lives in `private` (written before these rules, or marked+private) is not replaced, since the replacement would stay private: remove+it with `persona-remove`, then set it again. On a machine that has not+switched yet and runs the dual-write trial, `persona-remove` also removes+the copy the trial wrote to the store, so the persona stays removed after+the switch. The same holds for every Brain page a lane removes or moves+there: a moved page's store copy is the page as it is after the move, and+a private page moved over a shared one leaves no shared copy behind. A+page moved over a private one stays private, as a page written over it+does. A page that never reached the store, moved over one that did,+replaces the store copy of the page it overwrote. A moved page that is a+symbolic link, or sits in a linked folder, is never read into the store:+the move removes the old copy, and the store copy of a page it+overwrote. During the trial the legacy file is+the truth: when a switched machine has already stored a newer version of+a page and this machine moves the page, the store copy at the new path is+this machine's file, and the newer version stays only in the store's+history. A lane page or folder is never moved out of `Brain/` lanes, and a+lane folder is never moved into its own subfolder or into a folder that+holds it, on either kind of machine.++The `shared_namespace` mirror gets no store of its own. Where the shared+vault is store-only on a machine (it has no legacy tree), it is that+machine's brain store, which every store-only agent reads: the mirror writes+there when the origin record went only to the origin vault's legacy tree,+and refuses, with the reason, when the record is in the store already (a+switched machine, a store-only origin vault, dual writes on) or when the+machine has no store. These checks read the machine config the write+itself was made with. A `shared_namespace` path that is not a directory+(an unmounted share, a missing path, a file) is reported as failed.+## What the partitions protect, and what they do notThe partitions decide **which machines receive which records**. They are adiff --git a/docs/cli-reference.md b/docs/cli-reference.mdindex 3180b4a3..1bfd13a0 100644--- a/docs/cli-reference.md+++ b/docs/cli-reference.md@@ -485,7 +485,7 @@ o2b brain sources Read-only dashboard of signals by (agent, source_to2b brain schema Runtime schema report/admin (since v0.26.0): report|stats|lint|graph|explain|orphans|apply|sync; mutation writes are locked and auditedreport names the pack's integrity: `ok` (digests to what the newest audited apply recorded), `modified` (with both digests), or `unverified` with the reason - `no-apply-recorded`, `audit-unreadable`, or `config-absent`. The digest is over the rendered `schema:` block, so comment and whitespace churn elsewhere in `_brain.yaml` never moves itsync is NOT implemented and exits non-zero. It previously returned `{updated: 0, skipped: 0}` with a "no backfill was required" note without reading the vault at all, so a caller could not tell "nothing needed doing" from "nothing was done"-o2b brain schema apply --mutation '<json>' (repeatable) applies audited, locked mutations to Brain/_brain.yaml; --actor NAME --reason TEXT annotate the audit record+o2b brain schema apply --mutation '<json>' (repeatable) applies audited, locked mutations to Brain/_brain.yaml (the shared schema records on a store-only machine); --actor NAME --reason TEXT annotate the audit record; --adopt-unattested (store-only, the operator only) adopts records deciding the pack that no audited apply wrote instead of refusing, and the audit row names them and the actor; given alone, without --mutation (or with --mutation '[]'), it adopts and changes no entry, and only on a store-only machine--dry-run previews instead (since v1.43.0): it returns the pack that would result plus its leaf-level diff and writes nothing at all - no config write, no audit record, no lock file. Before v1.43.0 `apply` parsed the flag and mutated anyway. The preview runs the same pack validator the apply runs, but NOT the two checks that exist only because the apply writes (the vault-identity write guard, and the atomic writer's re-parse of the rendered YAML), so a batch that renders to unparseable YAML previews clean and fails on applyo2b brain watchdog Probe Brain config/dirs/search index and plan safe recovery (since v0.26.0): --remediate [--dry-run], --restore <run_id> [--force-restore], --jsono2b brain graph-export Serialise the vault knowledge graph (pages, wikilinks, typed relations) to a stable graph.json (since v0.22.0): stdout or --out <file>@@ -660,7 +660,7 @@ o2b brain session open --target <Brain/...md> [--schema-type S] [--iapprove <id> - operator-side commit of a needs-review sessionabandon <id> - terminal abandonstatus <id> | list | sweep - inspect or clean the session store (Brain/.sessions/write/, lazy TTL default 24h)-o2b brain panel open <topic...> [--personas a,b,c] [--target T] [--require-review] - convene a decision panel; personas from Brain/personas/ (built-in defaults: technical, strategic, risk, user-experience)+o2b brain panel open <topic...> [--personas a,b,c] [--target T] [--require-review] - convene a decision panel with the operator's custom personas (built-in defaults: technical, strategic, risk, user-experience); personas [--json] lists them, persona-set <slug> [--lens L] [--file F] adds or replaces one (prompt from F or stdin), persona-remove <slug> removes one. A custom persona is a note under Brain/personas/ before the store switch and a store record after it. The verbs refuse in a frozen vault, over a note of another kind at that path, and for a private persona; the prompt holds at most 16 KB, the lens at most 200 characters on one linesubmit <id> [--file F|-] - answer the current persona step or the synthesis; the committed note lands under Brain/decisions/panels/status <id> - live envelope of a panel session```diff --git a/docs/how-it-works.md b/docs/how-it-works.mdindex 74959fdd..006304a9 100644--- a/docs/how-it-works.md+++ b/docs/how-it-works.md@@ -1602,9 +1602,9 @@ A calling agent opens a session and receives a JSON envelope: what to generate (Three rules are enforced at the commit boundary, not by convention: `create` intent never overwrites an existing file, `merge` appends a session-stamped delimited section without touching existing bytes, and a target outside the agent page lanes (`sources/`, `reports/`, `distillations/`, `notes/`, `decisions/panels/` under `Brain/`) is refused before a session even opens, then again at commit against where the bytes actually land. Every terminal transition - committed, failed, abandoned, approved - lands exactly one `write-session` audit event in the Brain log.-The decision panel is the first consumer of the kernel: persona steps (operator-curated notes under `Brain/personas/`, or the built-in technical / strategic / risk / user-experience set) walk in declared order, the synthesis step sees every accepted answer, and the committed decision note under `Brain/decisions/panels/` records who argued what through which lens. The Brain contributes sequencing, validation, and the commit; every word of analysis comes from the calling agent.+The decision panel is the first consumer of the kernel: persona steps (operator-curated personas, or the built-in technical / strategic / risk / user-experience set; a custom persona is a note under `Brain/personas/` before the store switch and a store record after it, curated with `o2b brain panel persona-set` and `persona-remove`) walk in declared order, the synthesis step sees every accepted answer, and the committed decision note under `Brain/decisions/panels/` records who argued what through which lens. The Brain contributes sequencing, validation, and the commit; every word of analysis comes from the calling agent.-Two adjacent seams complete the contract. The memory backend boundary (`memory_backend` config key) routes memory import through a format-adapter registry - the Claude Code adapter is the default and byte-identical to the pre-seam behavior, and a second runtime becomes one self-contained module. The shared namespace (`shared_namespace` config key) mirrors explicit remember-writes into a second vault after the primary write succeeds, attributed by agent and origin vault; mirroring is fail-soft by contract, so a broken shared vault degrades to a reported `mirror: "failed"` and the primary write never notices.+Two adjacent seams complete the contract. The memory backend boundary (`memory_backend` config key) routes memory import through a format-adapter registry - the Claude Code adapter is the default and byte-identical to the pre-seam behavior, and a second runtime becomes one self-contained module. The shared namespace (`shared_namespace` config key) mirrors explicit remember-writes into a second vault after the primary write succeeds, attributed by agent and origin vault; mirroring is fail-soft by contract, so a broken shared vault degrades to a reported `mirror: "failed"` and the primary write never notices. The namespace has no store of its own: where the shared vault is store-only on a machine, it is that machine's brain store, so the mirror writes there only when the origin record went to the legacy tree alone, and otherwise reports `mirror: "misconfigured"` with the reason.## Time-aware recall and activation (since v0.42.0)diff --git a/openclaw/index.js b/openclaw/index.jsindex d82f3407..16ee2a6d 100644--- a/openclaw/index.js+++ b/openclaw/index.js@@ -2417,7 +2417,7 @@ var init_format_wikilink = __esm(() => {});// src/core/store/constants.ts-var STORE_FORMAT_VERSION = 1, STORE_FORMAT_FILE = "FORMAT.md", STORE_IDENTITY_FILE = "store.json", STORE_PARTITIONS, STORE_STREAMS, LEDGER_STREAM = "ledgers", STORE_BLOBS_DIR = "blobs", STORE_SHARD_EXT = "jsonl", ENVELOPE_FIELDS, BLOB_REF_KEY = "$blob", BLOB_REF_BYTES_KEY = "bytes", COMPACTED_KIND = "compacted", STORE_RECORD_KINDS, STORE_LINE_CAP_BYTES, STORE_BODY_MAX_DEPTH = 64, STORE_BLOB_MAX_BYTES, STORE_ROOT_CONFIG_KEY = "store_root", STORE_ROOT_ENV = "OPEN_SECOND_BRAIN_STORE_ROOT", STORE_DEFAULT_DIR = "store", STORE_STATE_DIR = "stores", STORE_LOCKS_DIR = "locks", STORE_CLOCK_DIR = "clock", SEEN_SHARDS_FILE = "seen-shards.json", AUTHORITY_FIELD = "authority", VAULT_ID_FIELD = "vault_id", AUTHORITIES, PREFERENCE_SET_KIND = "preference-set", PREFERENCE_REMOVE_KIND = "preference-remove", PREFERENCE_EVIDENCE_KIND = "preference-evidence", PINNED_SET_KIND = "pinned-set", PINNED_CLEAR_KIND = "pinned-clear", SETTING_SET_KIND = "setting-set", SETTING_UNSET_KIND = "setting-unset", DEVICE_RETIRE_KIND = "device-retire", STATE_DIGEST_KIND = "state-digest", FOLD_RECORD_KINDS, EVIDENCE_RESULTS, PREFERENCE_COUNTER_FIELDS, SETTING_VIEW_LANGUAGE = "view.language", SETTING_TOPIC_MIN_RECORDS = "topics.new_min_records", SETTING_TOPIC_MIN_SESSIONS = "topics.new_min_sessions", SETTING_TOPIC_QUIET_DAYS = "topics.quiet_days", SETTING_KEYS, STATE_DIGEST_FOLDS, STATE_DIGEST_REQUIRED_FOLDS, DAY_MS, STORE_DUAL_WRITE_CONFIG_KEY = "store_dual_write", STORE_DUAL_WRITE_ENV = "OPEN_SECOND_BRAIN_STORE_DUAL_WRITE", SIGNAL_KIND = "signal", LOG_EVENT_KIND = "log-event", LEDGER_ROW_KIND = "ledger-row", DUAL_WRITE_RECORD_KINDS, DUAL_WRITE_STATUS_FILE = "dual-write.json", STORE_TMP_DIR = "tmp", BLOB_TEMP_PREFIX = ".tmp-";+var STORE_FORMAT_VERSION = 1, STORE_FORMAT_FILE = "FORMAT.md", STORE_IDENTITY_FILE = "store.json", STORE_PARTITIONS, STORE_STREAMS, LEDGER_STREAM = "ledgers", STORE_BLOBS_DIR = "blobs", STORE_SHARD_EXT = "jsonl", ENVELOPE_FIELDS, BLOB_REF_KEY = "$blob", BLOB_REF_BYTES_KEY = "bytes", COMPACTED_KIND = "compacted", STORE_RECORD_KINDS, STORE_LINE_CAP_BYTES, STORE_BODY_MAX_DEPTH = 64, STORE_BLOB_MAX_BYTES, STORE_ROOT_CONFIG_KEY = "store_root", STORE_ROOT_ENV = "OPEN_SECOND_BRAIN_STORE_ROOT", STORE_DEFAULT_DIR = "store", STORE_STATE_DIR = "stores", STORE_LOCKS_DIR = "locks", STORE_CLOCK_DIR = "clock", SEEN_SHARDS_FILE = "seen-shards.json", AUTHORITY_FIELD = "authority", VAULT_ID_FIELD = "vault_id", AUTHORITIES, PREFERENCE_SET_KIND = "preference-set", PREFERENCE_REMOVE_KIND = "preference-remove", PREFERENCE_EVIDENCE_KIND = "preference-evidence", PINNED_SET_KIND = "pinned-set", PINNED_CLEAR_KIND = "pinned-clear", SETTING_SET_KIND = "setting-set", SETTING_UNSET_KIND = "setting-unset", SETTING_ENTRY_KIND = "setting-entry", DEVICE_RETIRE_KIND = "device-retire", STATE_DIGEST_KIND = "state-digest", FOLD_RECORD_KINDS, EVIDENCE_RESULTS, PREFERENCE_COUNTER_FIELDS, SETTING_VIEW_LANGUAGE = "view.language", SETTING_TOPIC_MIN_RECORDS = "topics.new_min_records", SETTING_TOPIC_MIN_SESSIONS = "topics.new_min_sessions", SETTING_TOPIC_QUIET_DAYS = "topics.quiet_days", SETTING_KEYS, STATE_DIGEST_FOLDS, STATE_DIGEST_REQUIRED_FOLDS, DAY_MS, STORE_DUAL_WRITE_CONFIG_KEY = "store_dual_write", STORE_DUAL_WRITE_ENV = "OPEN_SECOND_BRAIN_STORE_DUAL_WRITE", SIGNAL_KIND = "signal", LOG_EVENT_KIND = "log-event", LEDGER_ROW_KIND = "ledger-row", DUAL_WRITE_RECORD_KINDS, DUAL_WRITE_STATUS_FILE = "dual-write.json", STORE_TMP_DIR = "tmp", BLOB_TEMP_PREFIX = ".tmp-";var init_constants = __esm(() => {STORE_PARTITIONS = Object.freeze(["shared", "private"]);STORE_STREAMS = Object.freeze([@@ -2452,6 +2452,7 @@ var init_constants = __esm(() => {PINNED_CLEAR_KIND,SETTING_SET_KIND,SETTING_UNSET_KIND,+ SETTING_ENTRY_KIND,DEVICE_RETIRE_KIND,STATE_DIGEST_KIND]);@@ -5169,10 +5170,152 @@ var init_fold_preferences = __esm(() => {]);});+// src/core/brain/types.ts+var BRAIN_SIGNAL_SOURCE_TYPE, BRAIN_SIGNAL_SOURCE_TYPES, BRAIN_MEMORY_LAYER, BRAIN_MEMORY_LAYER_VALUES, BRAIN_LOG_EVENT_KIND, BRAIN_COMMITMENT_TIER, BRAIN_COMMITMENT_TIER_VALUES, BRAIN_LOG_EVENT_KIND_SET, BRAIN_SNAPSHOT_REASON, BRAIN_SNAPSHOT_REASONS;+var init_types = __esm(() => {+ BRAIN_SIGNAL_SOURCE_TYPE = {+ live: "live",+ inline: "inline",+ session: "session",+ extracted: "extracted",+ autoExtract: "auto_extract"+ };+ BRAIN_SIGNAL_SOURCE_TYPES = Object.freeze(Object.values(BRAIN_SIGNAL_SOURCE_TYPE));+ BRAIN_MEMORY_LAYER = {+ L0: "L0",+ L1: "L1",+ L2: "L2",+ L3: "L3"+ };+ BRAIN_MEMORY_LAYER_VALUES = Object.values(BRAIN_MEMORY_LAYER);+ BRAIN_LOG_EVENT_KIND = {+ dream: "dream",+ feedback: "feedback",+ applyEvidence: "apply-evidence",+ forceConfirmed: "force-confirmed",+ reject: "reject",+ promote: "promote",+ retire: "retire",+ notedRedundant: "noted-redundant",+ skipCorruptedFrontmatter: "skip-corrupted-frontmatter",+ pin: "pin",+ unpin: "unpin",+ rollback: "rollback",+ snapshot: "snapshot",+ signalSuppressed: "signal-suppressed",+ scanInline: "scan-inline",+ importSession: "import-session",+ merge: "merge",+ upgrade: "upgrade",+ importClaudeMemory: "import-claude-memory",+ promoteHostMemory: "promote-host-memory",+ note: "note",+ sessionLifecycle: "session-lifecycle",+ reconcile: "reconcile",+ writeSession: "write-session",+ entityAnchorSkip: "entity-anchor-skip",+ attributeWrite: "attribute-write",+ durabilitySkip: "durability-skip",+ ambientWithheld: "ambient-withheld",+ writeConflictAdvisory: "write-conflict-advisory",+ captureRoutingHint: "capture-routing-hint",+ signalRetire: "signal-retire",+ tombstone: "tombstone",+ temporalReplace: "temporal-replace",+ chainDecay: "chain-decay",+ decisionRecord: "decision-record",+ decisionOutcome: "decision-outcome",+ decisionRating: "decision-rating",+ decisionChangeReceipt: "decision-change-receipt",+ decisionOpen: "decision-open",+ decisionResolved: "decision-resolved",+ decisionDiscarded: "decision-discarded",+ authoredAtBackfill: "authored-at-backfill",+ tension: "tension",+ sourceCitation: "source-citation",+ doctorRepair: "doctor-repair",+ vectorBackfill: "vector-backfill",+ eventAnchorBackfill: "event-anchor-backfill",+ expirationSet: "expiration-set",+ freeze: "freeze",+ unfreeze: "unfreeze",+ writeRefused: "write-refused",+ noteWrite: "note-write",+ lessonsApproved: "lessons-approved",+ lessonsDenied: "lessons-denied",+ lessonsPromoted: "lessons-promoted"+ };+ BRAIN_COMMITMENT_TIER = {+ exploring: "exploring",+ leaning: "leaning",+ decided: "decided",+ locked: "locked"+ };+ BRAIN_COMMITMENT_TIER_VALUES = Object.values(BRAIN_COMMITMENT_TIER);+ BRAIN_LOG_EVENT_KIND_SET = new Set(Object.values(BRAIN_LOG_EVENT_KIND));+ BRAIN_SNAPSHOT_REASON = Object.freeze({+ dream: "dream",+ upgrade: "upgrade",+ importClaudeMemory: "import-claude-memory",+ promoteHostMemory: "promote-host-memory",+ deleteBySource: "delete-by-source",+ entityPrune: "entity-prune",+ entityMerge: "entity-merge",+ noteDelete: "note-delete",+ noteRevert: "note-revert",+ knowledgePackUninstall: "knowledge-pack-uninstall",+ payloadGc: "payload-gc",+ receiptCompaction: "receipt-compaction",+ storeMigration: "store-migration",+ sessionBoundary: "session-boundary",+ planBoundary: "plan-boundary",+ decisionBoundary: "decision-boundary",+ manual: "manual"+ });+ BRAIN_SNAPSHOT_REASONS = Object.freeze(Object.values(BRAIN_SNAPSHOT_REASON));+});++// src/core/brain/schema-vocab.ts+var SCHEMA_VOCAB_CATEGORIES, DEFAULT_SCHEMA_VOCAB;+var init_schema_vocab = __esm(() => {+ init_types();+ SCHEMA_VOCAB_CATEGORIES = Object.freeze([+ "preference_types",+ "signal_types",+ "page_types",+ "log_event_kinds"+ ]);+ DEFAULT_SCHEMA_VOCAB = Object.freeze({+ preference_types: Object.freeze(["preference"]),+ signal_types: Object.freeze(["feedback"]),+ page_types: Object.freeze(["note"]),+ log_event_kinds: Object.freeze(Object.values(BRAIN_LOG_EVENT_KIND))+ });+});++// src/core/store/fold-setting-lists.ts+var KEYED_LIST_SETTINGS, COMPOUND_KEYED_LIST_SETTINGS;+var init_fold_setting_lists = __esm(() => {+ init_schema_vocab();+ init_fold();+ init_constants();+ KEYED_LIST_SETTINGS = new Set([+ "brain.schema.prefixes",+ "brain.schema.expert_routing",+ "brain.schema.attributes",+ "brain.schema.frontmatter_tiers"+ ]);+ COMPOUND_KEYED_LIST_SETTINGS = new Set([+ "brain.schema.attributes",+ "brain.schema.frontmatter_tiers"+ ]);+});+// src/core/store/fold-settings.tsvar positiveInt = (value) => typeof value === "number" && Number.isSafeInteger(value) && value >= 1, LANGUAGE_RE, SETTING_RULES;var init_fold_settings = __esm(() => {init_constants();+ init_fold_setting_lists();init_fold();LANGUAGE_RE = /^[A-Za-z]{2,3}(-[A-Za-z0-9]{2,8}){0,3}$/;SETTING_RULES = Object.freeze({@@ -8329,111 +8472,6 @@ var init_artifact_ref_resolve = __esm(() => {// src/core/brain/note-path.tsvar init_note_path = () => {};-// src/core/brain/types.ts-var BRAIN_SIGNAL_SOURCE_TYPE, BRAIN_SIGNAL_SOURCE_TYPES, BRAIN_MEMORY_LAYER, BRAIN_MEMORY_LAYER_VALUES, BRAIN_LOG_EVENT_KIND, BRAIN_COMMITMENT_TIER, BRAIN_COMMITMENT_TIER_VALUES, BRAIN_LOG_EVENT_KIND_SET, BRAIN_SNAPSHOT_REASON, BRAIN_SNAPSHOT_REASONS;-var init_types = __esm(() => {- BRAIN_SIGNAL_SOURCE_TYPE = {- live: "live",- inline: "inline",- session: "session",- extracted: "extracted",- autoExtract: "auto_extract"- };- BRAIN_SIGNAL_SOURCE_TYPES = Object.freeze(Object.values(BRAIN_SIGNAL_SOURCE_TYPE));- BRAIN_MEMORY_LAYER = {- L0: "L0",- L1: "L1",- L2: "L2",- L3: "L3"- };- BRAIN_MEMORY_LAYER_VALUES = Object.values(BRAIN_MEMORY_LAYER);- BRAIN_LOG_EVENT_KIND = {- dream: "dream",- feedback: "feedback",- applyEvidence: "apply-evidence",- forceConfirmed: "force-confirmed",- reject: "reject",- promote: "promote",- retire: "retire",- notedRedundant: "noted-redundant",- skipCorruptedFrontmatter: "skip-corrupted-frontmatter",- pin: "pin",- unpin: "unpin",- rollback: "rollback",- snapshot: "snapshot",- signalSuppressed: "signal-suppressed",- scanInline: "scan-inline",- importSession: "import-session",- merge: "merge",- upgrade: "upgrade",- importClaudeMemory: "import-claude-memory",- promoteHostMemory: "promote-host-memory",- note: "note",- sessionLifecycle: "session-lifecycle",- reconcile: "reconcile",- writeSession: "write-session",- entityAnchorSkip: "entity-anchor-skip",- attributeWrite: "attribute-write",- durabilitySkip: "durability-skip",- ambientWithheld: "ambient-withheld",- writeConflictAdvisory: "write-conflict-advisory",- captureRoutingHint: "capture-routing-hint",- signalRetire: "signal-retire",- tombstone: "tombstone",- temporalReplace: "temporal-replace",- chainDecay: "chain-decay",- decisionRecord: "decision-record",- decisionOutcome: "decision-outcome",- decisionRating: "decision-rating",- decisionChangeReceipt: "decision-change-receipt",- decisionOpen: "decision-open",- decisionResolved: "decision-resolved",- decisionDiscarded: "decision-discarded",- authoredAtBackfill: "authored-at-backfill",- tension: "tension",- sourceCitation: "source-citation",- doctorRepair: "doctor-repair",- vectorBackfill: "vector-backfill",- eventAnchorBackfill: "event-anchor-backfill",- expirationSet: "expiration-set",- freeze: "freeze",- unfreeze: "unfreeze",- writeRefused: "write-refused",- noteWrite: "note-write",- lessonsApproved: "lessons-approved",- lessonsDenied: "lessons-denied",- lessonsPromoted: "lessons-promoted"- };- BRAIN_COMMITMENT_TIER = {- exploring: "exploring",- leaning: "leaning",- decided: "decided",- locked: "locked"- };- BRAIN_COMMITMENT_TIER_VALUES = Object.values(BRAIN_COMMITMENT_TIER);- BRAIN_LOG_EVENT_KIND_SET = new Set(Object.values(BRAIN_LOG_EVENT_KIND));- BRAIN_SNAPSHOT_REASON = Object.freeze({- dream: "dream",- upgrade: "upgrade",- importClaudeMemory: "import-claude-memory",- promoteHostMemory: "promote-host-memory",- deleteBySource: "delete-by-source",- entityPrune: "entity-prune",- entityMerge: "entity-merge",- noteDelete: "note-delete",- noteRevert: "note-revert",- knowledgePackUninstall: "knowledge-pack-uninstall",- payloadGc: "payload-gc",- receiptCompaction: "receipt-compaction",- storeMigration: "store-migration",- sessionBoundary: "session-boundary",- planBoundary: "plan-boundary",- decisionBoundary: "decision-boundary",- manual: "manual"- });- BRAIN_SNAPSHOT_REASONS = Object.freeze(Object.values(BRAIN_SNAPSHOT_REASON));-});-// src/core/brain/temporal/period-common.tsfunction extractId(linkOrId) {const stripped = linkOrId.replace(LINK_REASON_RE, "").trim();diff --git a/openspec/changes/brain-store-and-topics/design.md b/openspec/changes/brain-store-and-topics/design.mdindex 520fccfc..34a5ed3b 100644--- a/openspec/changes/brain-store-and-topics/design.md+++ b/openspec/changes/brain-store-and-topics/design.md@@ -286,7 +286,7 @@ The inventory for task 2.10 (316 modules reach the legacy tree) raised questions10. **Integrity.** Legacy day-file hash chains stay verifiable by the legacy reader. Store integrity is the record sums plus partition digests over cuts; there is no per-record chain.11. **Derived ledgers** (rollup ladder, cross-topic) are device-local and recomputable; machines need not agree on them.12. **Exports** (`export`, knowledge pack, OKF, bundle) read the folds and may keep the legacy layout as an interchange format. They are exporter modules for the 2.10 grep.-13. **Write guard.** In store-only mode the vault identity guard is replaced by the store identity check (store id and device id bound in the device-local state); the freeze marker is device-local. Note tools keep the vault identity guard through the adapter.+13. **Write guard.** In store-only mode the vault identity guard is replaced by the store identity check (store id and device id bound in the device-local state); the freeze marker is device-local and still holds every content write (lane pages, settings, schema applies), with or without a vault; a migration gate left naming the vault is a leftover of the migration that switched the machine and refuses nothing. Note tools keep the vault identity guard through the adapter.14. **Vault filtering.** Store reads do not filter by `vault_id` (one brain across vaults, D1); `vault_id` stays as provenance. The search index lives in the store's device-local state, one per vault (keyed by a hash of the vault's real path), and is keyed by the store id when no vault is configured.15. **MCP tokens** move from per-vault files to device config, so a server with no vault still enforces them; migration copies them. As built in 2.10 only the read side has moved: the transport reads `mcp-tokens.json` beside the machine config when it exists (with or without a vault) and otherwise the per-vault file (`src/mcp/token-source.ts`). The migration copies the per-vault file there (never over a device copy already present), and a rollback removes that copy again (or sets it aside when it changed since), so the per-vault file is the token map once more. On a switched machine minting, rotation and revocation write the device-config copy too (starting from the per-vault map when no copy exists yet), with the custody audit in the store, so a token minted after the switch authenticates with or without a vault.16. **The 2.10 checks.** The test preload configures no vault; tests that need one create it. The grep is an architecture test with an allowlist and a reason per entry, covering the `Brain/` literal and `"Brain"` path segments in code, not comments; comments are updated by the package that owns the file.@@ -349,6 +349,18 @@ Details the specs leave open, settled for the first part of phase 3:10. **Second re-review follow-ups.** Planning checks the view writer again from the catch-up read it plans from, so a move that arrives after the gate under the lock read the settings still stops the pass (`not-writer`). The dream pass asks its deadline again under the planning lock, after the read and before the first write, and a pass past it writes nothing (`deferred`). A write that fails after others landed reports `error` with the topics created and records routed before it, each with its undo command, and the dream summary counts them as a change. A `split` request takes only items the topic shows and writes the split of an item of a merged topic under that topic, the item's owner (the fold looks up a split's items by the topic it names); an id two merged owners share needs `owner`. An item id is refused for a new item of a topic only when a split moved that topic's item of the id to another topic: the fold still resolves the id to the moved item. A removed item stays its topic's, so its id can be recorded again. A `decision` record that leaves `serves` out keeps the links the item has, so a decision reworded on its page keeps the milestones it serves, and `"serves": []` clears them. When topics exchange folders, each retiring page moves only the companion names its folder held before any move of the pass. A companion move never unlinks a live name: the source and, when a move is taken back, the new name are each renamed to a temp name first and removed only when they are the version this move checked or placed, so a version a sync tool delivers under either name meanwhile is put back. A conflict-copy name that is not a regular file (a folder, a symbolic link) is refused where it is, and a capture that meets one puts it back under its name without replacing a name that exists. A conflict copy is taken out of its name before it is read, in a render and while retiring, so a copy written again after that read stays for the next pass. A private record routes by the private head text of a topic too, since its assign lands in `private`, and so does any record routed to a private topic; a shared route still uses shared text only. An edit, completion or resolution of an item that has a record in `private` is written to `private`, whatever the caller asks. `o2b topic split` takes `--owner` as `brain_topic` does; a split whose named topic moves no item of its own carries only its records. `brain_topic` refuses a `private` or `removed` that is not a boolean instead of dropping it. A dream pass plans on its own clock when it has one, and its undo commands name the config it ran with. A topic's activity time includes the records of topics merged into it and a split that hands it items or records. An item's privacy counts its completions and resolutions, whose evidence and answer are its text, so a later one written without the flag stays private. A route taken back from a topic is also blocked for the topic it merged into. Companion moves resolve each path once and use path-based calls: the runtime has no descriptor-relative rename or link, so a folder swapped for a symbolic link between the check and the call is not caught. That needs a process with write access to the view folder at that instant, which can already change the user's files directly.+### D19. Schema field lists, personas and the shared namespace (task 3.12)++Three questions the store left open after phase 2, settled by the option that fits the store already built (append-only records, order-independent folds, human over agent over model, shared and private partitions) with the least new machinery.++1. **Schema field lists merge per entry.** A schema field (`brain.schema.<field>`) was one settings register over the whole list, so two devices that each added an entry before they synced kept only the later list (a whole-list last writer wins). A schema apply now writes one `setting-entry` record (`{ authority, key, entry, present }`) per entry it adds or removes, never the list. A settings key that holds such a record folds per entry (`store/fold-setting-lists.ts`): each entry is one register under D4, keyed by its slot, the entry in the normalised form the pack parser reads (NFC, quotes stripped, trimmed, lower case, each side of a `name=value` line on its own and each side of a `->` pair too), or for the keyed lists (`prefixes`, `expert_routing`, `attributes`, `frontmatter_tiers`, whose `name=value` lines hold one value per name) the normalised name, a compound `type.field` name of `attributes` and `frontmatter_tiers` split at its first `.` with each half normalised on its own (the split function is the parser's own, `splitCompoundSchemaKey`, so `paper . year` is never a second slot beside `paper.year` and never repaired away as a dangling entry), so a changed value replaces the old entry and a variant spelling of one token is the same slot, never a second one. Only the keys `brain.schema.<field>` take entry records; the fold rejects an entry record for any other key, so a key that holds one value (a human `permissions.document`, a boolean, an object) stays on the plain register and no entry record can turn it into a removed list. The other way round too: a `setting-set` of a `brain.schema.<field>` key whose value is no list of one-line scalars (a string, a number, an object, an element with a line break) is rejected by the fold, so it can neither win the plain register and hide the stored list from the pack reader and the attestation, nor remove every entry of a key folded per entry, nor read as several pack lines; the settings writer already refuses such a value. An add on one device and a remove of the same entry on another resolve as every register does: the stronger authority, then the later clock. A whole-list `setting-set` (the migration's import, `o2b brain settings set`) offers every entry the key ever named, its own as present and the rest as removed, and a `setting-unset` offers all as removed, so whole-list histories resolve exactly as the register did; a key with no entry record folds as the plain register. The list is ordered by the fold order of each entry's deciding record. Rejected: keeping the whole-list register and asking the user to re-apply after a conflict (silent loss is the defect the task names); a three-way list merge against a base the record would have to carry (more machinery, and it needs the base both devices saw, which delayed sync does not guarantee); add-wins or remove-wins as a special rule for lists (a second precedence rule beside D4 for no gain).+ 1.1 **Authority per entry.** The writers refuse an agent change the fold would ignore. An apply's entry change is refused when its slot is decided by a stronger authority; a slot no record named yet is decided by the whole-list records alone (the strongest `setting-set` or `setting-unset` of the key), never by the newest entry record of another slot. For a key folded per entry, an agent's whole-list `o2b brain settings set` or `unset` (MCP `brain_settings`) is refused when any slot it would add, remove or respell is decided by a stronger authority, and an accepted write reports the folded value, not the requested one. A key not yet folded per entry keeps the key-level refusal.+ 1.2 **References across fields are repaired on read.** The fold merges per field, so a `remove_type` on one device and an `add_alias` or `add_prefix` for that type on another meet in a pack no apply could produce. The fold cannot refuse a record that arrives later, so the rule is enforced on the read side: the pack built from the store drops every reference to an undeclared token (aliases, prefixes, extractable entries, expert routes and attributes of a token the vocabulary does not hold, a link constraint pair whose source or target the vocabulary does not hold, and the constraints of an undeclared link type or of one left with no pair), the cascade `remove_type` and `remove_link_type` apply. A pair is dropped rather than kept as inert: the indexer reads a page's type from its frontmatter `type:`, which removing the type from the vocabulary does not rewrite, so a kept pair would still allow or refuse typed edges of such pages, and an apply refuses a pair with an undeclared endpoint, so a machine that reads `_brain.yaml` never holds one either. The drop is deterministic, so every machine reads one consistent pack and a later apply never fails on a dangling reference. The next apply writes the drop as entry removals, so the records converge too; a removal the applying authority may not write (an agent, an operator's entry) is skipped, since every reader drops it anyway.+ 1.3 **What an audit row attests.** A pack merged from two applies is no single apply's output, so its digest matches no audit row. Each apply's row also names records (`decided`): the records this apply wrote, plus the deciding records that rows this machine has seen already attest. A row names each record as `<id>:<sha256>` of the record's canonical JSON, never by id alone: a record's checksum covers its body, not its id, so another version under an attested id would otherwise pass as attested. An apply reads the store's schema records once, under its lock (`readStoreSchemaPack` in `brain/schema-pack.ts`): the pack in force, the records deciding it and the stored field lists a repair compares against all come from that one fold of the shared settings stream, so the race is impossible rather than unlikely. A record that syncs in while the apply runs is neither part of the pack it builds on without being among the records it checks, nor attested by its row, nor removed as a repair: a repair removes only an entry the pack in force dropped on read from the same field lists (item 1.2), never one the read did not hold. A repair removal also names the record that decided the entry's slot in that read, and the settings writer skips it when another record decides the slot by the time it is written (a replacement another device wrote meanwhile). Sync delivers a peer's record at any time and no lock orders it against this machine's append, so a write-time check alone still loses a replacement written before the repair and delivered after it, whose clock is lower. The repair is therefore written as a conditional removal (`if_decided_by`, the record it judged): the fold applies it only while that record decides the slot among the key's unconditional records, and ignores it otherwise, whatever its clock and the delivery order. So a repair never removes what it did not judge; skipping or ignoring it is always safe, since every reader drops the dangling entry anyway. Rejected: re-checking the slot under the shard lock right before the append (it narrows the window and closes it only against writers on this machine, not against sync). `if_decided_by` belongs to `setting-entry` alone: on a `setting-set` or `setting-unset` it is no field of the kind and is ignored, as the format ignores every field a kind does not define, while on an addition, or empty, it is a malformed `setting-entry` and the record is rejected. A key whose records name no entry at all once the inapplicable conditional removals are dropped (an empty whole list, an unset) is a removed key, won by its strongest, then latest, record as the whole-list register would win it; the list fold never throws on any record set, which a seeded property test over small random record sets pins together with order independence. The integrity read takes the pack's digest and the records it checks from one read the same way. Never every record that happens to decide the pack: one apply would then attest, on every machine, a record no apply wrote. A store-only pack that differs from the newest digest is still `ok` (`merged`) when every record deciding it is named by some row. A deciding record no row attests under an apply is a finding: the apply refuses with `unattested_schema_records`, naming the records, and the integrity read reports the pack as `modified`, naming the same records (field `decided`), even when the pack digests as the newest row recorded (a record that sets an entry the pack already held changes no digest but still decides the entry). Only a store whose rows all predate `decided` keeps the digest comparison alone. The operator may adopt them with `o2b brain schema apply --adopt-unattested` (human authority only; MCP has no such option), with or without mutations (on its own it is an empty batch, and only on a store-only machine), and the row records them as `adopted` with the operator as `adopted_by`. Adoption alone writes no entry record, not even the repair of a dangling entry: it attests, it does not edit. The migration's import writes a baseline row (`schema_import_baseline`, no digest) attesting the schema records it wrote. A store whose rows all predate `decided` keeps the digest comparison: there an apply counts the records deciding the pack as attested only when the pack in force is exactly the newest apply's output. The attestation reads the same pack fields the pack reader does, the field names the pack parser reads and nothing else, so a record under `brain.schema.` that is no pack field (a name the parser does not read, a dotted key, a value that is no list) changes nothing attested.+ 1.4 **Versioning.** No released version of Open Second Brain contains the brain store; the store ships for the first time in the release that also carries `setting-entry`. So no earlier release can meet the kind, and format version 1 includes it from the start. From that first release on, the versioning rule of the format (`store/FORMAT.md`, Versioning) applies unchanged: a later change that a reader of the released format would misread raises the format version, so that reader skips and reports the newer records instead of folding a different value.+2. **Personas are records.** D14 item 2 already folds personas as `lane-entry` records of the lane `personas`, and the panel read them from the fold, but nothing could write one on a store-only machine, where there is no `Brain/personas/` folder to edit. The lane page layer now writes and removes them through `o2b brain panel persona-set <slug> [--lens L] [--file F]` and `persona-remove <slug>` (`personas` lists them), with human authority from the CLI: a note on an unswitched machine, a record on a store-only one, with or without a vault. A persona curated on one machine is the lens on every machine of the store, and two devices editing one persona resolve by D4. The verbs run the lane write guard first, as every lane write: a frozen machine refuses them in both modes, and a running migration refuses them on a machine that has not switched; on a store-only machine the guard checks the store identity in place of the vault marker (D14 item 13) and keeps the freeze, as the store-mode settings writer and the store-only schema apply do, while a migration gate left naming the vault refuses nothing there. The verbs then refuse a page of another kind at the persona's path, refuse to replace a persona that lives in `private` (the lane writer keeps a replaced entry in its partition and never lowers its privacy, so the operator removes it and sets it again), and refuse a private persona by name: a lens or prompt with a `<private>` region or naming a private page would be stored in `private`, which machines that do not hold it never read. A prompt holds at most 16 KB (every panel step carries it, and the record syncs to every machine), a lens at most 200 characters on one line, Unicode line separators included. On an unswitched machine with the phase 1 dual write on, `persona-set` also writes the entry to the store, so `persona-remove` there dual-writes the stored entry's tombstone, and a persona removed before the switch is not back after it; this is the lane page layer's removal and move (`store/lane-dual-moves.ts`), so every lane's removed or moved page (a pledge, a staged folder, an archived dead end) leaves its stored entry the same way. A page that never reached the store is removed as a file alone. The records of one removal or move, a folder's included, are one store operation built from one fold of the lane entries, so an entry's value and its partition come from the same read. A move is the write of the destination page plus the removal of the source, judged as those two writes would be judged, in both modes and for a page and a folder alike; one table-driven test checks it over every state a move can start from (`tests/core/lanes/lane-move-table.test.ts`). A moved entry is built from the page as it is on disk after the rename (a legacy-only edit the store never saw included; when nothing the store would import is there, the move removes the source entry and the destination entry the rename overwrote, and writes nothing else), lives where the old entry lived, or in `private` when the page is private by its own content, the fold may miss a record, or the destination it replaces is private (its stored entry lives in `private` or its fields reserve it, or, on an unswitched machine, the legacy page the rename replaced was private by its frontmatter): the rule of a write over that page, since nothing lowers an entry's partition, and a move is a write at its destination. A move that replaces a shared destination entry with a private one writes that entry's shared tombstone first (D14 item 18), the store-only move too. A page the store never saw, renamed over a path whose entry the store holds (a single page, or a file of a folder moved onto stored entries), is dual-written as the write of that page alone (there is no source entry to remove): the page as it now is on disk, judged as a fresh write would be (the replaced entry or legacy page private keeps it private, its own content, a fold that may miss a record), the replaced shared entry's tombstone first when the new value leaves `shared`; when no page the store would import is there, the stored entry's tombstone alone. A folder's stored entries are paired from the entry ids below the source and the destination folders, never from the disk, so a link where an entry's file was does not hide that entry. Each moved entry is placed by itself (D14 item 18), so a private page of a folder does not raise the others, and the lane's private-when-held rule does not apply to a move (no caller moves a page into such a lane). So the destination's stored entry never stays live with content the tree no longer has. A move whose source is a lane entry and whose destination is none (outside `Brain/`, or not in a lane folder) is refused before anything moves, in both modes: an unswitched machine used to rename it and write the tombstones, while a store-only machine has no file to leave there and its tombstones alone would lose the page. A store-only move of a single page onto itself (the same path, or one that normalises to it, a trailing slash included) writes nothing, like every same-identity move. A dual-written move builds the new entry from the legacy file even when the store holds a newer value from a switched peer: the legacy write wins during the trial (D11), so a peer's newer value of a page this machine moves is replaced by the file's; no staleness guard is added, since the trial's rule is that the legacy tree is the truth until the switch. The page at the new path is never read through a symbolic link, at its own name or at a folder below `Brain/`, nor when it is a special file (`openRegularFileForRead`, the rule of the migration's legacy scan, which never imports such a page): it writes the tombstones of the old entry and of the destination entry the rename overwrote, so a link moved as a lane page cannot carry a file from outside the vault into `shared`. A folder is never moved into its own subfolder nor into a folder that holds it, refused in both modes before anything moves: the rename refuses both (a folder cannot move into itself, and the folder that holds the source is never empty), and on a store-only machine one entry's destination can be another entry's source in the same batch (`s1` moved to its parent, holding `s1/z.md` and `s1/s1/z.md`), whose tombstone, written after it, would remove the entry the batch just wrote there. Refusal was chosen over writing every tombstone of the batch before every new entry: that order keeps every entry, but it accepts moves the rename refuses, so the two modes would end in different states, and a move into an ancestor would merge the folder's entries into the ancestor's own. With overlapping folders refused, the only destination that is also a source is the folder moved onto itself, which writes nothing. A fold that may miss a record is one with an `events` shard it could not see or read only in part, or a record in `private` whose body blob has not arrived, quarantined on this machine or not (a quarantine lets the shard read as whole, yet the record is still left out, and it may be the newer private value of the entry that moves): one predicate (`laneFoldIncomplete`) answers it for every lane write, move and privacy read, so a quarantined blob in `private` no longer lets a lane page into `shared` while the record waits. A record in `shared` whose blob has not arrived does not count: every writer judges a value's fields before it places the record, so a record in `shared` never carries a value that makes an entry private, and leaving one record out changes an entry's winner only when it is that winner (the register keeps the last record of the strongest authority), so the fold without it is at least as private as the fold with it. Counting it would block every lane write on a machine that holds only `shared` until the lost blob arrives, which is what a quarantine is there to end. The table test crosses both with the move states. A move onto the entry it moves (the same path, or one that normalises to it) writes nothing; a case-only rename names two entries in the store, whose entry ids keep their case, so it moves the entry like any rename. Rejected: personas as files in the view folder, ingested by the view writer (a second edit path, and the view is rendered from `shared` only and written by one device, so a machine that is not the writer could not curate); personas as device-local files (machines would disagree on the panel's lenses).+3. **The shared namespace gets no store of its own.** The `shared_namespace` mirror wrote remember-writes into a second vault. A write to a vault that is store-only on this machine lands in this machine's brain store, which every store-only agent of every machine of the store reads: a store-only namespace is that store. So the mirror writes there when the origin record is not in it already (the origin vault still writes the legacy layout and dual writes are off), attributed by `origin_vault` as before, and refuses before any I/O when it is (a switched machine, a store-only origin vault, dual writes on), naming the reason, and when the machine has no store at all. Every check reads the machine config the caller names, the one its origin write used. A namespace path that is no directory (unmounted, missing, a file) is reported `failed` before any write, never taken for a store-only namespace. A second store would need its own store identity, clock floor, held partitions, seen-shards ledger, digest roster and index per machine for a fact every agent of the brain already shares; a namespace shared with a different brain stays a legacy-vault feature. Rejected: a `shared_namespace_store` root opened beside the machine's own store (all of that machinery, and one brain across vaults, D14 item 14, already covers one user's agents); keeping the blanket refusal (it also refused the case where the mirror is the only way the record reaches the store).+## Risks / Trade-offs- [A fold bug makes machines diverge silently] → state digests with doctor alerts (D6), property tests that shuffle delivery order and duplicate lines, and a rebuild command.diff --git a/openspec/changes/brain-store-and-topics/specs/brain-store/spec.md b/openspec/changes/brain-store-and-topics/specs/brain-store/spec.mdindex 173c147c..23871504 100644--- a/openspec/changes/brain-store-and-topics/specs/brain-store/spec.md+++ b/openspec/changes/brain-store-and-topics/specs/brain-store/spec.md@@ -168,3 +168,11 @@ Settings that every machine must agree on (the view writer, topic thresholds, th#### Scenario: Setting changed on one machine- **WHEN** the user changes the topic creation threshold on machine A- **THEN** machine B applies the new threshold after the record syncs++#### Scenario: Schema field edited on two machines before they sync+- **WHEN** machine A adds a page type and machine B removes another page type of the same schema field before either has the other's records+- **THEN** once both sync, both machines read one list that holds A's addition and lacks B's removal, whatever order the records arrive in; an add and a remove of the same entry resolve to the stronger authority, then the later clock++#### Scenario: A type removed on one machine while another adds a reference to it+- **WHEN** machine A removes a page type and machine B adds an alias or a prefix for that type before either has the other's records+- **THEN** once both sync, both machines read one pack without the type and without the references to it, and the next apply on either machine succeeds and removes the references from the recordsdiff --git a/openspec/changes/brain-store-and-topics/tasks.md b/openspec/changes/brain-store-and-topics/tasks.mdindex bc5e010f..3756945d 100644--- a/openspec/changes/brain-store-and-topics/tasks.md+++ b/openspec/changes/brain-store-and-topics/tasks.md@@ -88,7 +88,7 @@ Phases follow design.md D13. Each phase ends in its own release PR, and the vers- [x] 3.9 Make lane entry ids a bijection with legacy paths (every extension, `x.json` vs `x.json.md`, folders that are not lane names), or refuse the names that do not round-trip (OCR F295). Verify: a round-trip test over the lane path table. Done: refused names, design D15 item 1.- [x] 3.10 Keep the procedural graph and hint counters as device-local state instead of re-appending whole `lane-entry` records on every usage tick (OCR F360). Verify: a usage tick appends no store record, and a migration test moves the counters into the device-local state. Done: design D15 item 2.- [x] 3.11 Write the schema-apply audit row in the partition of the pack records it attests (`shared`), with one privacy verdict over the batch and its reason, or have the schema integrity check accept the newest settings records as attested (OCR F394). Verify: a machine that holds only `shared` reports a schema apply from another device as attested, not `modified`. Done: the row moves to `shared`, design D15 item 3.-- [ ] 3.12 Settle the cross-device merge of the schema field lists (a set merge rather than last writer wins on the whole list), personas as files or records, and whether the shared namespace gets a store of its own; the mirror into this machine's own store refuses until then (OCR F397, F464, F398). Verify: a concurrent edit test on two devices for the field lists, and a test per decision taken.+- [x] 3.12 Settle the cross-device merge of the schema field lists (a set merge rather than last writer wins on the whole list), personas as files or records, and whether the shared namespace gets a store of its own; the mirror into this machine's own store refuses until then (OCR F397, F464, F398). Verify: a concurrent edit test on two devices for the field lists, and a test per decision taken. Done: design D19.- [x] 3.13 Keep private-partition log events out of the OKF `log.md`, or carry a privacy marker for them: a log entry has no frontmatter for the `visibility: [private]` stamp pages from the private partition now get (OCR F333, log side). Verify: an export test on a switched machine with one private and one shared log event. Done in change `lane-pages-split`.- [x] 3.14 Split the store side of `src/core/store/lane-pages.ts` (fold accessors, partition keep-rule, lane-entry writers) into its own module: the file is over the 700-line limit after the replace-keeps-private fix. Verify: no behaviour change, the lanes and store suites pass unchanged. Done in change `lane-pages-split`.- [x] 3.15 Name a shard that reads only in part (a corrupt middle line, a record of a newer format, a blob that never arrives) in a doctor finding with its path, and add a repair that quarantines the bad line, so a protection read that fails closed on it can be released without a hand edit. Also decide whether rollback and the late import wait for a pending blob rather than leave its record out. Verify: a doctor test per gap kind, a quarantine test that reopens capture. Done in change `store-hardening`.diff --git a/src/cli/brain/help-text.ts b/src/cli/brain/help-text.tsindex 2c270863..606eaf1e 100644--- a/src/cli/brain/help-text.ts+++ b/src/cli/brain/help-text.ts@@ -840,6 +840,8 @@ export const VERB_HELP: Record<string, string> = {"usage: o2b brain schema [report|stats|lint|graph|explain|orphans|apply|sync] [--vault <path>] [--json]\n" +"Inspect and mutate the active runtime schema vocabulary through locked, audited writes.\n" +"report names the pack's integrity: ok, modified, or unverified with the reason.\n" ++ "apply --adopt-unattested (store-only machines, the operator only; alone or with\n" ++ "--mutation): adopt the records deciding the pack that no audited apply wrote.\n" +"sync is not implemented and refuses; it never reported real work.\n","session-hook":"usage: o2b brain session-hook [--vault <path>] [--agent <name>] [--dry-run] [--json]\n" +@@ -1584,12 +1586,18 @@ export const VERB_HELP: Record<string, string> = {"status <id> | list | sweep inspect or clean the session store.\n" +"Envelopes are JSON with --json: status, step, prompt, errors, attempts_left.",panel:- "usage: o2b brain panel <open <topic...>|submit <id>|status <id>> [flags]\n" ++ "usage: o2b brain panel <open <topic...>|submit <id>|status <id>|personas|\n" ++ " persona-set <slug>|persona-remove <slug>> [flags]\n" +"open <topic> [--personas a,b,c] [--target T] [--require-review] convene a\n" +- "decision panel; personas come from Brain/personas/ (built-in default set:\n" +- "technical, strategic, risk, user-experience). Each submit answers the\n" ++ "decision panel; personas are the operator's custom ones (built-in default\n" ++ "set: technical, strategic, risk, user-experience). Each submit answers the\n" +"current persona step; after synthesis the decision note commits under\n" +- "Brain/decisions/panels/. The calling agent generates every word.",+ "Brain/decisions/panels/. The calling agent generates every word.\n" ++ "personas [--json] list the personas in force. persona-set <slug> [--lens L]\n" ++ "[--file F] add or replace a custom persona, its prompt read from F or\n" ++ "stdin. persona-remove <slug> remove one. A custom persona is a note under\n" ++ "Brain/personas/ before the store switch and a store record after it, so\n" ++ "it reaches every machine of the store and needs no vault.",architect:"usage: o2b brain architect <project-path> [--vault V] [--progress] [--json]\n" +"Scan a project tree deterministically (built-in runtime only, no dependency,\n" +diff --git a/src/cli/brain/verbs/feedback.ts b/src/cli/brain/verbs/feedback.tsindex efb8a858..8546bda3 100644--- a/src/cli/brain/verbs/feedback.ts+++ b/src/cli/brain/verbs/feedback.ts@@ -140,7 +140,7 @@ export async function cmdBrainFeedback(argv: string[]): Promise<number> {const mirror =sharedNamespace === null || vault === null? undefined- : mirrorSignal(sharedNamespace, vault, signalInput, writeOpts);+ : mirrorSignal(sharedNamespace, vault, signalInput, writeOpts, config);try {appendLogEvent(diff --git a/src/cli/brain/verbs/panel.ts b/src/cli/brain/verbs/panel.tsindex 8bc20d4b..b486cf81 100644--- a/src/cli/brain/verbs/panel.ts+++ b/src/cli/brain/verbs/panel.ts@@ -2,7 +2,10 @@* `o2b brain panel <op>` (Agent Write Contract Suite, t_0cc6fdff):* convenience face of the decision-panel session kind. `open` takes* the topic as positional text; `submit`/`status` are sugar over the- * shared session surface for panel-shaped sessions.+ * shared session surface for panel-shaped sessions. `personas`,+ * `persona-set` and `persona-remove` curate the panel's personas (task+ * 3.12): notes on an unswitched machine, records on a store-only one,+ * where no vault is needed.*/import { readFileSync } from "node:fs";@@ -12,13 +15,27 @@ import {sessionEnvelope,} from "../../../core/brain/write-session/engine.ts";import { openPanelSession, submitToPanelSession } from "../../../core/brain/write-session/panel.ts";+import {+ listPersonas,+ removePersona,+ setPersona,+} from "../../../core/brain/write-session/personas.ts";import { readWriteSession } from "../../../core/brain/write-session/store.ts";import type { WriteSessionEnvelope } from "../../../core/brain/write-session/types.ts";-import { brainVerbContext, fail, ok, okJson, parse, resolveBrainAgent } from "../helpers.ts";+import {+ brainVerbContext,+ fail,+ ok,+ okJson,+ parse,+ resolveBrainAgent,+ type BrainVerbFlags,+} from "../helpers.ts";const USAGE =- "usage: o2b brain panel <open <topic...>|submit <session-id>|status <session-id>> " +- "[--personas a,b,c] [--target T] [--file F] [--require-review] [--agent A] [--vault V] [--json]";+ "usage: o2b brain panel <open <topic...>|submit <session-id>|status <session-id>|personas|" ++ "persona-set <slug>|persona-remove <slug>> [--personas a,b,c] [--target T] [--file F] " ++ "[--lens L] [--require-review] [--agent A] [--vault V] [--json]";function renderEnvelope(env: WriteSessionEnvelope, asJson: boolean): void {if (asJson) {@@ -59,9 +76,17 @@ export async function cmdBrainPanel(argv: string[]): Promise<number> {file: { type: "string" },agent: { type: "string" },"require-review": { type: "boolean" },+ lens: { type: "string" },});const op = positional[0];const asJson = flags["json"] === true;+ if (op === "personas" || op === "persona-set" || op === "persona-remove") {+ try {+ return personaVerb(op, positional[1], flags, asJson);+ } catch (err) {+ return emitCliError(err instanceof Error ? err.message : String(err), asJson);+ }+ }const { config, vault } = brainVerbContext(flags);try {@@ -135,3 +160,43 @@ export async function cmdBrainPanel(argv: string[]): Promise<number> {return emitCliError(err instanceof Error ? err.message : String(err), asJson);}}++/** The persona verbs: list, add or replace, remove (no vault needed on a store-only machine). */+function personaVerb(+ op: "personas" | "persona-set" | "persona-remove",+ slug: string | undefined,+ flags: BrainVerbFlags,+ asJson: boolean,+): number {+ const { vault } = brainVerbContext(flags, { vault: "optional" });+ if (op === "personas") {+ const listed = listPersonas(vault);+ if (asJson) {+ okJson({ source: listed.source, personas: listed.personas.map((p) => ({ ...p })) });+ return 0;+ }+ ok(`personas (${listed.source}):`);+ for (const p of listed.personas) ok(` ${p.slug}: ${p.lens}`);+ return 0;+ }+ if (!slug) {+ process.stderr.write(`${USAGE}\n`);+ return 2;+ }+ if (op === "persona-remove") {+ const removed = removePersona(vault, slug);+ if (asJson) okJson({ slug, removed });+ else ok(removed ? `removed persona ${slug}` : `no persona ${slug}`);+ return 0;+ }+ const fileFlag = flags["file"] as string | undefined;+ const prompt =+ fileFlag !== undefined && fileFlag !== "-"+ ? readFileSync(fileFlag, "utf8")+ : readFileSync(0, "utf8");+ const lens = flags["lens"] as string | undefined;+ const result = setPersona(vault, { slug, prompt, ...(lens !== undefined ? { lens } : {}) });+ if (asJson) okJson({ ...result });+ else ok(result.changed ? `persona ${slug} saved` : `persona ${slug} unchanged`);+ return 0;+}diff --git a/src/cli/brain/verbs/schema.ts b/src/cli/brain/verbs/schema.tsindex 7abc5af1..fcb72097 100644--- a/src/cli/brain/verbs/schema.ts+++ b/src/cli/brain/verbs/schema.ts@@ -15,7 +15,10 @@ import {SCHEMA_PACK_INTEGRITY,type SchemaPackIntegrity,} from "../../../core/brain/schema-integrity.ts";-import { previewSchemaMutations } from "../../../core/brain/schema-mutate.ts";+import {+ previewSchemaMutations,+ SCHEMA_ADOPT_ONLY_UNSWITCHED,+} from "../../../core/brain/schema-mutate.ts";import { formatStampMismatch } from "../../../core/integrity/stamp.ts";import {buildSchemaReport,@@ -23,6 +26,7 @@ import {type SchemaReportFinding,} from "../../../core/brain/schema-report.ts";import { SCHEMA_VOCAB_CATEGORIES } from "../../../core/brain/schema-vocab.ts";+import { isStoreOnly } from "../../../core/store/store-only.ts";import { fail, parse, resolveBrainVault } from "../helpers.ts";export async function cmdBrainSchema(argv: string[]): Promise<number> {@@ -51,6 +55,7 @@ export async function cmdBrainSchema(argv: string[]): Promise<number> {actor: { type: "string", default: "cli" },reason: { type: "string" },"dry-run": { type: "boolean" },+ "adopt-unattested": { type: "boolean" },"batch-size": { type: "string", default: "100" },});const config = defaultConfigPath();@@ -99,8 +104,20 @@ export async function cmdBrainSchema(argv: string[]): Promise<number> {return writeResult(reviewSchemaOrphans(vault), Boolean(flags["json"]), renderGenericText);case "apply": {const mutationPayloads = (flags["mutation"] as string[] | undefined) ?? [];- if (mutationPayloads.length === 0) throw new Error("schema apply requires --mutation JSON");+ // Judged on the parsed batch, so `--mutation '[]'` is no mutation+ // either. Adoption runs on its own too, on a store-only machine: an+ // empty batch that changes no entry and records the adopted records+ // in its audit row. The lane refuses it on an unswitched machine,+ // where an empty apply would rewrite and re-attest `_brain.yaml`.const mutations = parseSchemaMutationPayloads(mutationPayloads);+ if (mutations.length === 0) {+ if (!flags["adopt-unattested"]) {+ throw new Error("schema apply requires --mutation JSON (or --adopt-unattested alone)");+ }+ if (!isStoreOnly({ vault, configPath: config })) {+ throw new Error(SCHEMA_ADOPT_ONLY_UNSWITCHED);+ }+ }// `--dry-run` is parsed for every schema subcommand. Until the// preview existed, `apply` accepted the flag and wrote anyway - an// operator asking to look first got a mutated vault schema.@@ -118,6 +135,7 @@ export async function cmdBrainSchema(argv: string[]): Promise<number> {// The operator's own change (store-only machines write it as a// human-authority settings record).authority: "human",+ ...(flags["adopt-unattested"] ? { adoptUnattested: true } : {}),});return writeResult(result, Boolean(flags["json"]), renderGenericText);}diff --git a/src/core/brain/note.ts b/src/core/brain/note.tsindex f7905bc1..2cb1ccd2 100644--- a/src/core/brain/note.ts+++ b/src/core/brain/note.ts@@ -172,11 +172,12 @@ function appendJudgedNote(const mirror =shared === null || vault === null? undefined- : mirrorNote(shared, vault, {- text: sanitised,- agent,- ...(input.now ? { now: input.now } : {}),- });+ : mirrorNote(+ shared,+ vault,+ { text: sanitised, agent, ...(input.now ? { now: input.now } : {}) },+ input.configPath,+ );return {logged_at: timestamp,log_path: reportedLogPath(res, vault),diff --git a/src/core/brain/policy/settings-entry-authority.ts b/src/core/brain/policy/settings-entry-authority.tsnew file mode 100644index 00000000..b4bac15e--- /dev/null+++ b/src/core/brain/policy/settings-entry-authority.ts@@ -0,0 +1,105 @@+/**+ * Who decides one entry of a list setting (task 3.12, design D19 item 1),+ * for the shared settings writer (`./settings-write.ts`): an entry folds+ * as its own register (`../../store/fold-setting-lists.ts`), so an agent's+ * change of an entry a stronger authority decided would be ignored by the+ * fold, and is refused instead of written. A whole-list write or unset of+ * a key folded per entry is judged the same way, for every entry it would+ * change.+ */++import { SETTING_SET_KIND, SETTING_UNSET_KIND, type Authority } from "../../store/constants.ts";+import { authorityRank, type RegisterState } from "../../store/fold.ts";+import { settingEntrySlot } from "../../store/fold-setting-lists.ts";+import type { SettingsFold } from "../../store/fold-settings.ts";+import { BrainSettingError, type SettingAuthority, type SettingTarget } from "./settings-names.ts";++/** The authority that decides the entry slot `slot` of a list setting now, if any record does. */+function slotAuthority(fold: SettingsFold, key: string, slot: string): Authority | undefined {+ const named = fold.lists.get(key)?.get(slot);+ if (named !== undefined) return named.winner.authority;+ const state = fold.settings.get(key);+ if (state === undefined) return undefined;+ // A slot no record named yet is decided by the whole-list records alone+ // (a `setting-set` or `setting-unset` offers it as absent once it is+ // named), so the strongest of those decides it; entry records of other+ // slots do not.+ let strongest: Authority | undefined;+ for (const offer of state.history) {+ if (offer.record.kind !== SETTING_SET_KIND && offer.record.kind !== SETTING_UNSET_KIND)+ continue;+ if (strongest === undefined || authorityRank(offer.authority) > authorityRank(strongest)) {+ strongest = offer.authority;+ }+ }+ return strongest;+}++/**+ * The id of the record that decides the slot of `entry` in the list+ * setting `key` now: the slot's own winning offer once an entry record+ * named it, else the key's winning whole-list record; `null` when no record+ * decides the key.+ */+export function slotDecider(fold: SettingsFold, key: string, entry: string): string | null {+ const named = fold.lists.get(key)?.get(settingEntrySlot(key, entry));+ if (named !== undefined) return named.winner.record.id;+ return fold.settings.get(key)?.winner.record.id ?? null;+}++function refuseSlot(+ target: SettingTarget,+ fold: SettingsFold,+ slot: string,+ shown: string,+ authority: SettingAuthority,+): void {+ const decided = slotAuthority(fold, target.key, slot);+ if (decided === undefined || authorityRank(decided) <= authorityRank(authority)) return;+ throw new BrainSettingError(+ "human_value_in_force",+ `${target.name}: the entry "${shown}" was decided by the operator (human authority); an ` ++ "agent cannot change it. Ask the operator to make the change with " ++ "`o2b brain schema apply`",+ );+}++/** Refuse an agent's change of one entry a stronger authority decides. */+export function refuseOverStrongerEntry(+ target: SettingTarget,+ fold: SettingsFold,+ entry: string,+ authority: SettingAuthority,+): void {+ refuseSlot(target, fold, settingEntrySlot(target.key, entry), entry, authority);+}++/**+ * Refuse an agent's whole-list write (`value`, or `null` for an unset) of+ * a key folded per entry when any entry slot it would change, added,+ * removed or respelled, is decided by a stronger authority (finding 1 of+ * the task 3.12 review). A slot is the normalised form the pack parser+ * reads, so a variant spelling is the same slot, not a second one.+ */+export function refuseWholeListOverStrongerEntries(+ target: SettingTarget,+ fold: SettingsFold,+ value: unknown,+ authority: SettingAuthority,+): void {+ const slots: ReadonlyMap<string, RegisterState<string>> = fold.lists.get(target.key) ?? new Map();+ const requested = new Map<string, string>();+ if (Array.isArray(value)) {+ for (const item of value) {+ const text = String(item);+ requested.set(settingEntrySlot(target.key, text), text);+ }+ }+ for (const slot of new Set([...slots.keys(), ...requested.keys()])) {+ const state = slots.get(slot);+ const before = state === undefined || state.removed ? undefined : state.value;+ const after = requested.get(slot);+ if (before === after) continue;+ refuseSlot(target, fold, slot, after ?? before ?? slot, authority);+ }+}diff --git a/src/core/brain/policy/settings-store-commit.ts b/src/core/brain/policy/settings-store-commit.tsnew file mode 100644index 00000000..fc73042e--- /dev/null+++ b/src/core/brain/policy/settings-store-commit.ts@@ -0,0 +1,108 @@+/**+ * The store-mode commit of the shared settings writer+ * (`./settings-write.ts`): the records of one settings write appended as+ * one operation, `setting-set`, `setting-unset` or, for one entry of a+ * list setting, `setting-entry` (task 3.12). Split from the writer by+ * cohesion: this is the only part that builds and appends records.+ */++import {+ AUTHORITY_FIELD,+ SETTING_ENTRY_KIND,+ SETTING_SET_KIND,+ SETTING_UNSET_KIND,+} from "../../store/constants.ts";+import { assertStoreContentWrite } from "../../store/content-write-gates.ts";+import { IF_DECIDED_BY_FIELD } from "../../store/fold-setting-lists.ts";+import { callerInput, opaqueFields, operationPrivacy } from "../../store/operation-privacy.ts";+import { withOperationPrivacy } from "../../store/operation-scope.ts";+import { storeWrite } from "../../store/store-write.ts";+import { BrainSettingError, type SettingTarget } from "./settings-names.ts";+import { REMOVE_KEY } from "./settings-yaml-edit.ts";+import type { SettingAuthority } from "./settings-names.ts";++/** What a commit needs of a settings write: `BrainSettingsWriteContext` of the writer. */+export interface SettingsCommitContext {+ readonly vault: string | null;+ readonly configPath?: string;+ readonly authority: SettingAuthority;+ readonly agent: string;+}++/** One change of a multi-key store write: a value, {@link REMOVE_KEY}, or one list entry. */+export interface StoreChange {+ readonly target: SettingTarget;+ readonly value: unknown;+ /**+ * One list entry. `ifDecidedBy` makes a removal conditional+ * (`if_decided_by`): it applies only while that record decides the+ * entry's slot, so a replacement it never saw is kept.+ */+ readonly entry?: {+ readonly text: string;+ readonly present: boolean;+ readonly ifDecidedBy?: string;+ };+}++/**+ * Append the records of `changes` as ONE operation: one privacy verdict+ * over the whole caller input, every record in `shared`. Returns the ids+ * written, in order. Refused while this machine is frozen, as the legacy+ * settings write is by the vault guard.+ */+export function storeCommit(+ ctx: SettingsCommitContext,+ changes: ReadonlyArray<StoreChange>,+): string[] {+ assertStoreContentWrite(ctx.vault);+ const bodies = changes.map(({ target, value, entry }) => {+ const body: Record<string, unknown> = { [AUTHORITY_FIELD]: ctx.authority, key: target.key };+ if (entry !== undefined) {+ body["entry"] = entry.text;+ body["present"] = entry.present;+ if (entry.ifDecidedBy !== undefined) body[IF_DECIDED_BY_FIELD] = entry.ifDecidedBy;+ } else if (value !== REMOVE_KEY) body["value"] = value;+ const kind =+ entry !== undefined+ ? SETTING_ENTRY_KIND+ : value === REMOVE_KEY+ ? SETTING_UNSET_KIND+ : SETTING_SET_KIND;+ return { target, kind, body };+ });+ // A shared setting is read from `shared` only, so a value that would+ // route the record to `private` is refused rather than written where no+ // reader looks (one verdict over the whole caller input, fail closed).+ const verdict = operationPrivacy(ctx.vault, {+ content: [+ callerInput(+ bodies.map(({ body }) => ({ key: body["key"], value: body["value"] ?? body["entry"] })),+ opaqueFields("key"),+ ),+ ],+ });+ if (verdict()) {+ throw new BrainSettingError(+ "private_setting_value",+ `${changes.map((c) => c.target.name).join(", ")}: the value names a private page or ` ++ `carries a <private> region; shared settings are read by every machine`,+ );+ }+ const written = withOperationPrivacy(verdict, () =>+ storeWrite(+ () =>+ bodies.map(({ kind, body }) => ({+ stream: "settings" as const,+ kind,+ agent: ctx.agent,+ body,+ })),+ {+ vault: ctx.vault,+ ...(ctx.configPath !== undefined ? { configPath: ctx.configPath } : {}),+ },+ ),+ );+ return written.map((w) => w.record.id);+}diff --git a/src/core/brain/policy/settings-write.ts b/src/core/brain/policy/settings-write.tsindex f54910a9..71eeb811 100644--- a/src/core/brain/policy/settings-write.ts+++ b/src/core/brain/policy/settings-write.ts@@ -36,15 +36,11 @@ import lockfile from "proper-lockfile";import { canonicalJson } from "../../integrity/digest.ts";import { atomicWriteFileSync } from "../../fs-atomic.ts";-import { AUTHORITY_FIELD, SETTING_SET_KIND, SETTING_UNSET_KIND } from "../../store/constants.ts";import { foldSettings, type SettingsFold } from "../../store/fold-settings.ts";-import { callerInput, opaqueFields, operationPrivacy } from "../../store/operation-privacy.ts";-import { withOperationPrivacy } from "../../store/operation-scope.ts";import { protectionStore, StoreUnavailableError } from "../../store/read/protection-store.ts";import { readStreamRecords } from "../../store/read/stream-read.ts";import { storeOnlyMode } from "../../store/store-only.ts";import type { StoreInspection } from "../../store/store-open.ts";-import { storeWrite } from "../../store/store-write.ts";import { brainConfigPath } from "../paths.ts";import { assertVaultIdentityForWrite } from "../vault-identity.ts";import { parseBrainYaml } from "../yaml-parse.ts";@@ -69,6 +65,12 @@ import {type YamlSettingValue,} from "./settings-yaml-edit.ts";import { validateBrainConfigDetailed } from "./validate.ts";+import {+ refuseOverStrongerEntry,+ slotDecider,+ refuseWholeListOverStrongerEntries,+} from "./settings-entry-authority.ts";+import { storeCommit, type StoreChange } from "./settings-store-commit.ts";export {BrainSettingError,@@ -305,102 +307,68 @@ function refuseOverOperatorYaml();}-/** One change of a multi-key store write: a value, or {@link REMOVE_KEY}. */-interface StoreChange {- readonly target: SettingTarget;- readonly value: unknown;-}--/**- * Append the records of `changes` as ONE operation: one privacy verdict- * over the whole caller input, every record in `shared`. Returns the ids- * written, in order.- */-function storeCommit(- ctx: BrainSettingsWriteContext,- changes: ReadonlyArray<StoreChange>,-): string[] {- const bodies = changes.map(({ target, value }) => {- const body: Record<string, unknown> = { [AUTHORITY_FIELD]: ctx.authority, key: target.key };- if (value !== REMOVE_KEY) body["value"] = value;- return { target, value, body };- });- // A shared setting is read from `shared` only, so a value that would- // route the record to `private` is refused rather than written where no- // reader looks (one verdict over the whole caller input, fail closed).- const verdict = operationPrivacy(ctx.vault, {- content: [- callerInput(- bodies.map(({ body }) => ({ key: body["key"], value: body["value"] })),- opaqueFields("key"),- ),- ],- });- if (verdict()) {- throw new BrainSettingError(- "private_setting_value",- `${changes.map((c) => c.target.name).join(", ")}: the value names a private page or ` +- `carries a <private> region; shared settings are read by every machine`,- );- }- const written = withOperationPrivacy(verdict, () =>- storeWrite(- () =>- bodies.map(({ value, body }) => ({- stream: "settings" as const,- kind: value === REMOVE_KEY ? SETTING_UNSET_KIND : SETTING_SET_KIND,- agent: ctx.agent,- body,- })),- {- vault: ctx.vault,- ...(ctx.configPath !== undefined ? { configPath: ctx.configPath } : {}),- },- ),- );- return written.map((w) => w.record.id);-}--/** A change of {@link writeSharedSettingRecords}: a value, or `null` to unset. */-export interface SharedSettingRecordChange {+/** A change of {@link writeSharedSettingEntries}: one entry of a list setting. */+export interface SharedSettingEntryChange {readonly name: string;- /** The new value; `null` writes a `setting-unset`. */- readonly value: unknown;+ readonly entry: string;+ /** `true` adds the entry, `false` removes it. */+ readonly present: boolean;+ /**+ * A removal every reader applies already (a reference the pack drops on+ * read): skipped, not refused, where this authority may not write it.+ * `decidedBy` is the record that decided the entry's slot in the read+ * the repair was judged on: when another record decides the slot now (a+ * replacement another device wrote meanwhile), the repair is skipped,+ * and the removal it writes is conditional on that record+ * (`if_decided_by`), so the fold keeps a replacement delivered only+ * after the repair was written too: it never removes what it did not+ * judge.+ */+ readonly repair?: { readonly decidedBy: string };}/**- * Write several shared settings as one store operation, for a writer- * that validated the values itself against a richer validator than the- * settings one (the schema pack). Store mode only: the names are- * classified, and an agent change to a key the operator set is refused- * for the whole batch before anything is written. Unchanged keys are- * skipped. Returns the ids of the records written.+ * Write entry changes of list settings as one store operation, one+ * `setting-entry` record each (task 3.12): a list setting folds per entry,+ * so an edit another device makes meanwhile is kept. Store mode only. An+ * entry whose register a stronger authority decides (an agent changing an+ * entry the operator set or left out) refuses the whole batch, since the+ * fold would ignore it. Returns the ids of the records written.*/-export function writeSharedSettingRecords(+export function writeSharedSettingEntries(ctx: BrainSettingsWriteContext,- changes: ReadonlyArray<SharedSettingRecordChange>,+ changes: ReadonlyArray<SharedSettingEntryChange>,): string[] {- // An unswitched reader ignores settings records: written there, they- // would change nothing anyone reads.if (modeOf(ctx) !== "store") {- throw new Error("writeSharedSettingRecords: store mode only (this machine reads _brain.yaml)");+ throw new Error("writeSharedSettingEntries: store mode only (this machine reads _brain.yaml)");}const read = sharedRead(ctx);- const fold = read.fold;const pending: StoreChange[] = [];for (const change of changes) {const target = resolveSettingName(change.name);+ if (+ change.repair !== undefined &&+ slotDecider(read.fold, target.key, change.entry) !== change.repair.decidedBy+ ) {+ continue;+ }refuseOperatorOnly(target, ctx.authority);refuseUnseen(target, read, ctx.authority);- const before = storeView(target, fold);- const value = change.value === null ? REMOVE_KEY : change.value;- const same =- value === REMOVE_KEY- ? !before.set- : before.set && canonicalJson(before.value) === canonicalJson(value);- if (same) continue;- refuseOverHuman(target, fold, ctx.authority);- pending.push({ target, value });+ try {+ refuseOverStrongerEntry(target, read.fold, change.entry, ctx.authority);+ } catch (err) {+ if (change.repair !== undefined && err instanceof BrainSettingError) continue;+ throw err;+ }+ pending.push({+ target,+ value: null,+ entry: {+ text: change.entry,+ present: change.present,+ ...(change.repair !== undefined ? { ifDecidedBy: change.repair.decidedBy } : {}),+ },+ });}return pending.length === 0 ? [] : storeCommit(ctx, pending);}@@ -663,7 +631,12 @@ function write(mode: "store" as const,};if (same) return { ...result, value: before.value, changed: false };- refuseOverHuman(target, fold, ctx.authority);+ // A key folded per entry (task 3.12) is judged entry by entry: the key's+ // newest record says nothing about who decides each entry.+ const perEntry = fold.lists.has(target.key);+ if (perEntry && ctx.authority !== "human") {+ refuseWholeListOverStrongerEntries(target, fold, removing ? null : value, ctx.authority);+ } else refuseOverHuman(target, fold, ctx.authority);if (!native) {validateCandidate(readRawBrainSettingsFromStore(@@ -675,7 +648,16 @@ function write();}const [record] = storeCommit(ctx, [{ target, value }]);- return { ...result, changed: true, ...(record !== undefined ? { record } : {}) };+ // What is in force now: for a key folded per entry the fold can keep+ // what a stronger authority decided, so the requested value is not+ // reported as if it were.+ const inForce = perEntry ? storeView(target, sharedFold(ctx)).value : result.value;+ return {+ ...result,+ value: inForce,+ changed: true,+ ...(record !== undefined ? { record } : {}),+ };}const vault = ctx.vault!;// The read the comparison and the refusal judge, and the edit, are onediff --git a/src/core/brain/schema-integrity.ts b/src/core/brain/schema-integrity.tsindex b98b67bf..6047e158 100644--- a/src/core/brain/schema-integrity.ts+++ b/src/core/brain/schema-integrity.ts@@ -61,7 +61,13 @@ import { readLedgerRows } from "../store/read/ledger-rows.ts";import { isStoreOnly } from "../store/store-only.ts";import { listShardedFiles } from "./ledger-shards.ts";import { brainDirs } from "./paths.ts";-import { readSchemaPackSource, renderSchemaBlock, type SchemaPack } from "./schema-pack.ts";+import {+ readSchemaPackSource,+ readStoreSchemaPack,+ renderSchemaBlock,+ type SchemaPack,+} from "./schema-pack.ts";+import type { SchemaDecidingRecord } from "./schema-pack-store.ts";/** The three answers a schema-pack integrity read can honestly give. */export const SCHEMA_PACK_INTEGRITY = Object.freeze({@@ -198,6 +204,12 @@ export interface SchemaPackIntegrity {readonly audit_path?: string;/** Present exactly when `status` is `unverified`. */readonly unverified_reason?: SchemaPackUnverifiedReason;+ /**+ * Set when the pack is no single apply's output but a merge of audited+ * applies from several devices (task 3.12): every record that decides it+ * is one an audit row attests, so it is `ok` though `expected` differs.+ */+ readonly merged?: true;/*** Non-empty exactly when `status` is `modified`, carrying the shared* mismatch shape so it renders through `formatStampMismatch`.@@ -322,7 +334,10 @@ function newestRecordedDigest(* recorded, and say which of the four conditions this vault is in.*/export function assessSchemaPackIntegrity(vault: string): SchemaPackIntegrity {- const source = readSchemaPackSource(vault);+ // A store-only pack and the records deciding it are one read, so the+ // merge check below judges the records the digest was taken of.+ const store = isStoreOnly({ vault }) ? readStoreSchemaPack() : null;+ const source = store?.source ?? readSchemaPackSource(vault);if (!source.present) {return Object.freeze({status: SCHEMA_PACK_INTEGRITY.unverified,@@ -346,10 +361,42 @@ export function assessSchemaPackIntegrity(vault: string): SchemaPackIntegrity {// The ungated comparison: this surface reports what it finds regardless// of any operator gate, and it reports it in the shape every other// integrity surface in this project emits.- const mismatches = compareStamps(+ const digestMismatches = compareStamps({ [SCHEMA_PACK_DIGEST_FIELD]: recorded.digest },{ [SCHEMA_PACK_DIGEST_FIELD]: digest },);+ // A store whose rows name records: a deciding record no row attests is+ // a finding even when the pack digests as recorded (a record that sets+ // an entry the pack already held), since the next apply refuses it.+ const deciding = store?.deciding ?? [];+ const unattested = store === null ? null : unattestedDeciding(vault, deciding);+ const mismatches =+ unattested === null || unattested.length === 0+ ? digestMismatches+ : Object.freeze([+ ...digestMismatches,+ Object.freeze({+ field: SCHEMA_DECIDED_RECORDS_FIELD,+ expected: "every deciding record attested by an audit row",+ actual: `unattested: ${unattested.join(", ")}`,+ }),+ ]);+ if (+ digestMismatches.length > 0 &&+ unattested !== null &&+ unattested.length === 0 &&+ deciding.length > 0+ ) {+ return Object.freeze({+ status: SCHEMA_PACK_INTEGRITY.ok,+ digest,+ expected: recorded.digest,+ recorded_at: recorded.recorded_at,+ audit_path: recorded.audit_path,+ merged: true as const,+ mismatches: NO_MISMATCHES,+ });+ }return Object.freeze({status: mismatches.length === 0 ? SCHEMA_PACK_INTEGRITY.ok : SCHEMA_PACK_INTEGRITY.modified,digest,@@ -360,6 +407,81 @@ export function assessSchemaPackIntegrity(vault: string): SchemaPackIntegrity {});}+/**+ * Audit-row field naming the records that decided the pack after an+ * apply, each as `<id>:<digest>` of the version the row attests.+ */+export const SCHEMA_DECIDED_RECORDS_FIELD = "decided";++/**+ * The `action` of the row the migration's import writes (task 3.12): it+ * attests the schema records the import wrote, carries no digest, and is+ * never the newest apply.+ */+export const SCHEMA_IMPORT_BASELINE_ACTION = "schema_import_baseline";++/** What the schema audit rows of a store attest. */+export interface SchemaAttestation {+ /**+ * Every record attestation a row names (`decided`), across all rows+ * seen: `<id>:<digest>` of the one version of the record the row saw+ * (`schemaRecordAttestation` in `./schema-pack-store.ts`).+ */+ readonly attested: ReadonlySet<string>;+ /**+ * No row names records: every row predates the field, so only the+ * digest comparison can attest the pack.+ */+ readonly digestOnly: boolean;+}++/**+ * The records the schema audit rows of this store attest (task 3.12,+ * design D19 item 1), each in the version a row saw. An apply's row names+ * the records that apply wrote plus records rows it had seen already+ * attest (or the operator adopted by name); the migration's baseline row+ * names the records the import wrote. The attested set is the union over+ * every row. An entry that is a bare id binds no content and attests+ * nothing.+ */+export function schemaAttestation(vault: string): SchemaAttestation {+ const attested = new Set<string>();+ let named = false;+ for (const { record } of storeAuditRecords(vault)) {+ if (!isRecord(record)) continue;+ const action = record["action"];+ if (action !== SCHEMA_MUTATION_AUDIT_ACTION && action !== SCHEMA_IMPORT_BASELINE_ACTION) {+ continue;+ }+ const details = record["details"];+ const decided = isRecord(details) ? details[SCHEMA_DECIDED_RECORDS_FIELD] : undefined;+ if (!Array.isArray(decided)) continue;+ named = true;+ for (const entry of decided) if (typeof entry === "string") attested.add(entry);+ }+ return { attested, digestOnly: !named };+}++/**+ * The ids of the records deciding a store-only pack that no audit row+ * attests in the version they hold (task 3.12, design D19 item 1), or+ * `null` when no row names records (rows written before the field+ * existed), where the digest comparison alone decides. A pack that matches+ * no single apply's digest is the merge of audited applies when this is+ * empty; a pack changed by any other write (a settings write, a forged+ * record, another body under an attested id) has an id here, even when+ * the record left the pack as it was. `live` is the deciding records of+ * the same read the pack's digest was taken of.+ */+function unattestedDeciding(+ vault: string,+ live: ReadonlyArray<SchemaDecidingRecord>,+): string[] | null {+ const { attested, digestOnly } = schemaAttestation(vault);+ if (digestOnly) return null;+ return live.filter((record) => !attested.has(record.attestation)).map((record) => record.id);+}+function notFound(reason: SchemaPackUnverifiedReason): RecordedSchemaPackDigest {return Object.freeze({ found: false as const, reason });}diff --git a/src/core/brain/schema-mutate-store.ts b/src/core/brain/schema-mutate-store.tsindex a8e335ba..a9c06800 100644--- a/src/core/brain/schema-mutate-store.ts+++ b/src/core/brain/schema-mutate-store.ts@@ -14,6 +14,21 @@* store-only machine). Nothing under the vault's `Brain/` is written; the* lock is device-local.*+ * The pack in force, the records deciding it and the stored entries a+ * repair may remove come from one read of the shared settings stream+ * (`readStoreSchemaPack` in `./schema-pack.ts`), taken under the lock.+ *+ * The audit row names the records it attests (`decided`, task 3.12), each+ * bound to the version it saw (`<id>:<digest>`): the records this apply+ * wrote, plus the deciding records rows this machine has seen attest+ * already. A deciding record no row attests (a settings+ * write, a hand-made record) is a finding: the apply refuses with+ * {@link UnattestedSchemaRecordsError}, unless the operator adopts it+ * (`adoptUnattested`, human authority only), which the row records with+ * the operator's name. A store whose rows predate `decided` attests by+ * digest only: when the pack in force is the newest apply's output, the+ * records deciding it count as attested.+ ** The audit row attests the shared pack records, so it is written to* `shared` too (task 3.11): a machine that holds only `shared` reads the* digest it must hold the pack against, and reports another device's@@ -34,6 +49,7 @@ import {type AuditRecord,} from "../reliability/audit.ts";import { withFileLock } from "../reliability/lock.ts";+import { assertStoreContentWrite } from "../store/content-write-gates.ts";import { authorityFromOrigin } from "../store/dual-write.ts";import { configuredStore } from "../store/read/store-entities.ts";import { valueCarriesPrivateRegion } from "../store/private-region.ts";@@ -41,12 +57,20 @@ import { storesStateDir } from "../store/store-paths.ts";import { brainDirs } from "./paths.ts";import {computeSchemaPackDigest,+ readRecordedSchemaPackDigest,+ schemaAttestation,+ SCHEMA_DECIDED_RECORDS_FIELD,SCHEMA_MUTATION_AUDIT_ACTION,SCHEMA_MUTATION_AUDIT_DIR,SCHEMA_PACK_DIGEST_FIELD,} from "./schema-integrity.ts";-import { loadSchemaPack, renderSchemaBlock, type SchemaPack } from "./schema-pack.ts";-import { STORE_SCHEMA_PACK_SOURCE, writeSchemaPackToStore } from "./schema-pack-store.ts";+import { readStoreSchemaPack, renderSchemaBlock, type SchemaPack } from "./schema-pack.ts";+import {+ schemaDecidingRecords,+ STORE_SCHEMA_PACK_SOURCE,+ type SchemaDecidingRecord,+ writeSchemaPackToStore,+} from "./schema-pack-store.ts";/** Device-local lock of a store-only schema apply (under the stores state dir). */const SCHEMA_PACK_STORE_LOCK = "schema-pack";@@ -69,6 +93,8 @@ export interface StoreSchemaApplyOptions {readonly lockStaleMs?: number;/** Authority of the records (default: the process's origin). */readonly authority?: "human" | "agent";+ /** Adopt deciding records no row attests (the operator only). */+ readonly adoptUnattested?: boolean;/** The next pack from the one in force (the lane's own validator runs here). */readonly next: (current: SchemaPack) => SchemaPack;}@@ -79,19 +105,50 @@ export async function applySchemaMutationsToStore(mutations: ReadonlyArray<unknown>,opts: StoreSchemaApplyOptions,): Promise<StoreSchemaApplyResult> {+ // The vault identity guard is the legacy tree's; the freeze holds here+ // too, before the lock is taken.+ assertStoreContentWrite(vault);const lockDir = storesStateDir(processDirsEnv());mkdirSync(lockDir, { recursive: true });const lockPath = join(lockDir, SCHEMA_PACK_STORE_LOCK);return await withFileLock(lockPath, { staleMs: opts.lockStaleMs ?? 30_000, retries: 3 }, () => {- const nextPack = opts.next(loadSchemaPack(vault));- writeSchemaPackToStore(- {- vault,- authority: opts.authority ?? (authorityFromOrigin() === "human" ? "human" : "agent"),- agent: resolveAgentName(),- },- renderSchemaBlock(nextPack),+ const authority = opts.authority ?? (authorityFromOrigin() === "human" ? "human" : "agent");+ // One read of the store's schema records (`readStoreSchemaPack`): the+ // pack in force, the records deciding it and the stored entries a+ // repair may remove all come from it, so a record that syncs in while+ // this apply runs is neither built on unattested nor removed unseen.+ const snapshot = readStoreSchemaPack();+ const currentPack = snapshot.source.pack;+ const deciding = snapshot.deciding;+ const attested = attestedBefore(vault, currentPack, deciding);+ const unattested = deciding.filter((record) => !attested.has(record.attestation));+ const adopted = unattested.map((record) => record.id);+ if (adopted.length > 0 && (opts.adoptUnattested !== true || authority !== "human")) {+ throw new UnattestedSchemaRecordsError(adopted, authority);+ }+ const nextPack = opts.next(currentPack);+ // Adoption alone (an empty batch with `adoptUnattested`) records the+ // adopted records in its row and writes no entry, not even a repair.+ const adoptionOnly = mutations.length === 0 && opts.adoptUnattested === true;+ const written = new Set(+ adoptionOnly+ ? []+ : writeSchemaPackToStore(+ { vault, authority, agent: resolveAgentName() },+ renderSchemaBlock(nextPack),+ renderSchemaBlock(currentPack),+ snapshot,+ ),);+ for (const record of unattested) attested.add(record.attestation);+ // The records deciding the pack now that this row may attest (see the+ // module docblock): attested before, adopted, or written just now (a+ // fresh id, so the version read back is the one this apply wrote). A+ // pack merged with another device's apply is still attested, and a+ // record nobody audited is not.+ const decided = schemaDecidingRecords()+ .filter((record) => attested.has(record.attestation) || written.has(record.id))+ .map((record) => record.attestation);const packDigest = computeSchemaPackDigest(nextPack);const record: AuditRecord = {timestamp: opts.now.toISOString(),@@ -103,6 +160,8 @@ export async function applySchemaMutationsToStore(applied: mutations.length,mutations,[SCHEMA_PACK_DIGEST_FIELD]: packDigest,+ [SCHEMA_DECIDED_RECORDS_FIELD]: decided,+ ...(adopted.length > 0 ? { adopted, adopted_by: opts.actor } : {}),...(opts.reason ? { reason: opts.reason } : {}),},};@@ -123,6 +182,50 @@ export async function applySchemaMutationsToStore(});}+/**+ * The record attestations an apply finds attested before it writes: those+ * the rows it has seen name, or, on a store whose rows predate `decided`,+ * the records `deciding` the pack when that pack is exactly the newest+ * apply's output.+ */+function attestedBefore(+ vault: string,+ current: SchemaPack,+ deciding: ReadonlyArray<SchemaDecidingRecord>,+): Set<string> {+ const { attested, digestOnly } = schemaAttestation(vault);+ const out = new Set(attested);+ if (digestOnly) {+ const recorded = readRecordedSchemaPackDigest(vault);+ if (recorded.found && recorded.digest === computeSchemaPackDigest(current)) {+ for (const record of deciding) out.add(record.attestation);+ }+ }+ return out;+}++/**+ * An apply over a pack decided by records no audit row attests (task+ * 3.12, design D19 item 1): something other than an audited apply changed+ * the pack, and an apply must not attest that change implicitly.+ */+export class UnattestedSchemaRecordsError extends Error {+ readonly code = "unattested_schema_records";+ constructor(+ readonly records: ReadonlyArray<string>,+ authority: "human" | "agent",+ ) {+ super(+ `unattested_schema_records: the schema pack in force is decided by ${records.length} ` ++ `record(s) no audited apply wrote (${records.join(", ")}); see \`o2b brain schema report\`. ` ++ (authority === "human"+ ? "Rerun with `--adopt-unattested` to adopt them as yours"+ : "Only the operator can adopt them, with `o2b brain schema apply --adopt-unattested`"),+ );+ this.name = "UnattestedSchemaRecordsError";+ }+}+/** What the shared row says in place of a batch that names a private page. */export const SCHEMA_AUDIT_WITHHELD = "the mutations name a private page";diff --git a/src/core/brain/schema-mutate.ts b/src/core/brain/schema-mutate.tsindex 6cbd1021..557e9275 100644--- a/src/core/brain/schema-mutate.ts+++ b/src/core/brain/schema-mutate.ts@@ -33,6 +33,7 @@ import {parseSchemaPack,renderSchemaBlock,replaceSchemaBlock,+ pairEndpoints,schemaPackTokens,validateEndpointPair,FRONTMATTER_TIERS,@@ -140,6 +141,8 @@ export interface ApplySchemaMutationsOptions {* the CLI, `agent` from MCP. Absent: decided by the process's origin.*/readonly authority?: "human" | "agent";+ /** Store-only: adopt unattested deciding records (operator only; `./schema-mutate-store.ts`). */+ readonly adoptUnattested?: boolean;}export interface ApplySchemaMutationsResult {@@ -188,6 +191,11 @@ export interface PreviewSchemaMutationsResult {readonly diff: ReadonlyArray<SchemaPackDiffEntry>;}+/** The refusal of an adoption with no mutation on an unswitched machine. */+export const SCHEMA_ADOPT_ONLY_UNSWITCHED =+ "schema apply --adopt-unattested without a mutation is for a store-only machine; " ++ "this one reads its schema from the vault file, which has no records to adopt";+/*** Separates a leaf's path from its value when keying a list MEMBER, so two* members of the same list do not collide in the flat map the diff walks. A@@ -226,6 +234,12 @@ export async function applySchemaMutations(// A store-only machine checks the store's identity on the write itself;// the vault identity guard is the legacy tree's.if (!storeOnly) assertVaultIdentityForWrite(vault);+ // Adoption alone (an empty batch) is for a store-only machine: the file+ // path has no records to adopt, and an empty apply there would rewrite+ // and re-attest `_brain.yaml` as it stands.+ if (!storeOnly && mutations.length === 0 && opts.adoptUnattested === true) {+ throw new Error(SCHEMA_ADOPT_ONLY_UNSWITCHED);+ }// Deny-consulted evidence (write-side trust, PR #247 review M3 residual,// schema sibling): a schema batch rewrites the operator's vocabulary pack// - the same mutation-lane shape as update, append and delete_by_source -@@ -249,6 +263,7 @@ export async function applySchemaMutations(...(opts.reason !== undefined ? { reason: opts.reason } : {}),...(opts.lockStaleMs !== undefined ? { lockStaleMs: opts.lockStaleMs } : {}),...(opts.authority !== undefined ? { authority: opts.authority } : {}),+ ...(opts.adoptUnattested === true ? { adoptUnattested: true } : {}),next: (current) => applyMutationsToPack(current, mutations),});}@@ -455,6 +470,11 @@ function applyOne(pack: MutableSchemaPack, mutation: SchemaMutation): void {removeValue(pack.extractable, token);delete pack.expert_routing[token];delete pack.attributes[token];+ for (const [linkType, pairs] of Object.entries(pack.link_constraints)) {+ const kept = pairs.filter((pair) => !pairEndpoints(pair).includes(token));+ if (kept.length > 0) pack.link_constraints[linkType] = kept;+ else delete pack.link_constraints[linkType];+ }return;}case "update_type": {@@ -680,10 +700,15 @@ function validateSchemaPackReferences(pack: SchemaPack): void {throw new Error(`schema.expert_routing.${token}: token is not declared`);}}- for (const linkType of Object.keys(pack.link_constraints)) {+ for (const [linkType, pairs] of Object.entries(pack.link_constraints)) {if (!pack.link_types.includes(linkType)) {throw new Error(`schema.link_constraints.${linkType}: link type is not declared`);}+ for (const pair of pairs) {+ if (!pairEndpoints(pair).every((token) => tokens.has(token))) {+ throw new Error(`schema.link_constraints.${linkType}: ${pair}: token is not declared`);+ }+ }}for (const type of Object.keys(pack.attributes)) {if (!tokens.has(type)) {diff --git a/src/core/brain/schema-pack-store.ts b/src/core/brain/schema-pack-store.tsindex 3646dbec..7c743145 100644--- a/src/core/brain/schema-pack-store.ts+++ b/src/core/brain/schema-pack-store.ts@@ -10,16 +10,33 @@* reader of the pack works unchanged on a store-only machine.* - {@link writeSchemaPackToStore} writes a pack the mutation lane* already validated as one store operation through the shared settings- * writer (`./policy/settings-write.ts`): changed fields as- * `setting-set`, fields the pack no longer has as `setting-unset`.+ * writer (`./policy/settings-write.ts`): one `setting-entry` record per+ * entry the pack adds or drops (task 3.12, design D19 item 1), never the+ * whole list, so an entry another device adds or removes meanwhile is+ * kept by the per-entry fold (`../store/fold-setting-lists.ts`).* Nothing under the vault's `Brain/` is written.+ * - {@link schemaDecidingRecords} names the records that decide the pack+ * in force, which an apply's audit row attests, each bound to its+ * content ({@link schemaRecordAttestation}).+ * - {@link readStoreSchemaRecords} returns the field lists, their text and+ * their deciding records from one read, for a caller that must hold all+ * three to one snapshot (an apply, the integrity read).*/-import { foldSettings } from "../store/fold-settings.ts";+import { canonicalJson, sha256Hex } from "../integrity/digest.ts";+import type { StoreRecord } from "../store/envelope.ts";+import { settingEntrySlot } from "../store/fold-setting-lists.ts";+import type { RegisterState } from "../store/fold.ts";+import { foldSettings, type SettingsFold } from "../store/fold-settings.ts";import { configuredStore } from "../store/read/store-entities.ts";import { readStreamRecords } from "../store/read/stream-read.ts";import { brainSettingKey } from "./policy/settings-keys.ts";-import { writeSharedSettingRecords, type SettingAuthority } from "./policy/settings-write.ts";+import {+ writeSharedSettingEntries,+ type SettingAuthority,+ type SharedSettingEntryChange,+} from "./policy/settings-write.ts";+import { SCHEMA_VOCAB_CATEGORIES } from "./schema-vocab.ts";/** The settings block the pack lives in. */const SCHEMA_BLOCK = "schema";@@ -27,30 +44,168 @@ const SCHEMA_BLOCK = "schema";/** What a store-only machine names as the place its schema pack lives. */export const STORE_SCHEMA_PACK_SOURCE = `store:shared/settings/${brainSettingKey(SCHEMA_BLOCK)}`;+/** The shared settings fold the pack is read from (empty without `shared`). */+function schemaFold(configPath?: string): SettingsFold {+ const store = configuredStore(configPath);+ if (store === null || !store.partitions.has("shared")) return foldSettings([]);+ return foldSettings(readStreamRecords(store.root, "shared", { stream: "settings" }).records);+}++const SCHEMA_PREFIX = `${brainSettingKey(SCHEMA_BLOCK)}.`;++/**+ * The fields of the `schema:` block the pack parser reads+ * (`parseSchemaPack` in `./schema-pack.ts`). A record under+ * `brain.schema.` for any other name changes no pack, so it is neither+ * rendered into the pack's text nor attested.+ */+export const SCHEMA_PACK_FIELDS: ReadonlySet<string> = new Set([+ ...SCHEMA_VOCAB_CATEGORIES,+ "aliases",+ "prefixes",+ "link_types",+ "extractable",+ "expert_routing",+ "labels",+ "link_constraints",+ "attributes",+ "frontmatter_tiers",+]);++/** One record that decides the pack in force. */+export interface SchemaDecidingRecord {+ readonly id: string;+ /** What an audit row names to attest this record: {@link schemaRecordAttestation}. */+ readonly attestation: string;+}++/**+ * What an audit row names to attest one record: `<id>:<digest>`, the+ * digest the sha256 of the record's canonical JSON (envelope and body).+ * The id alone is not enough: a record's checksum covers its body, never+ * its id, so another version under an attested id (a different body with+ * a matching checksum) would pass as attested. A row attests the one+ * version it saw.+ */+export function schemaRecordAttestation(record: StoreRecord): string {+ return `${record.id}:${sha256Hex(canonicalJson(record))}`;+}++/**+ * The schema records of the store as one read of the shared settings+ * stream (task 3.12, design D19 item 1): the field lists in force, the+ * config text they render to, and the records deciding them. An apply and+ * the integrity read take all three from this one fold, so a record that+ * syncs in meanwhile can neither change the pack without being among its+ * deciding records nor reach the entries a repair removes.+ */+export interface StoreSchemaRecords {+ /** The `brain.schema.<field>` values in force, by field. */+ readonly fields: Readonly<Record<string, ReadonlyArray<string>>>;+ /** {@link storeSchemaConfigText} of those fields. */+ readonly text: string | null;+ /** {@link schemaDecidingRecords} of the same fold. */+ readonly deciding: ReadonlyArray<SchemaDecidingRecord>;+ /**+ * The id of the record deciding each entry of {@link fields}, by field+ * and entry: what a repair of that entry checks the slot against when+ * it writes (`writeSharedSettingEntries`).+ */+ readonly decidedBy: Readonly<Record<string, Readonly<Record<string, string>>>>;+}++/** Read the store's schema records once: see {@link StoreSchemaRecords}. */+export function readStoreSchemaRecords(configPath?: string): StoreSchemaRecords {+ const fold = schemaFold(configPath);+ const fields = fieldsOf(fold);+ return {+ fields,+ text: configTextOf(fields),+ deciding: decidingOf(fold),+ decidedBy: decidedByOf(fold, fields),+ };+}++function decidedByOf(+ fold: SettingsFold,+ fields: Readonly<Record<string, ReadonlyArray<string>>>,+): Record<string, Record<string, string>> {+ const out: Record<string, Record<string, string>> = {};+ for (const [field, entries] of Object.entries(fields)) {+ const key = `${SCHEMA_PREFIX}${field}`;+ const slots = fold.lists.get(key);+ const whole = fold.settings.get(key)!.winner.record.id;+ out[field] = Object.fromEntries(+ entries.map((entry) => [+ entry,+ slots?.get(settingEntrySlot(key, entry))?.winner.record.id ?? whole,+ ]),+ );+ }+ return out;+}++/**+ * The records that decide the pack in force, sorted by id: for a field+ * folded per entry each entry's deciding record (removals included, so a+ * removal nobody audited is seen), else the field's winning record. An+ * apply's audit row names them (`decided`), so a pack merged from applies+ * on two devices is still attested (`./schema-integrity.ts`).+ */+export function schemaDecidingRecords(configPath?: string): SchemaDecidingRecord[] {+ return decidingOf(schemaFold(configPath));+}++function decidingOf(fold: SettingsFold): SchemaDecidingRecord[] {+ const records = new Map<string, StoreRecord>();+ const add = (record: StoreRecord) => records.set(record.id, record);+ for (const [key, state] of fold.settings) {+ if (!isPackField(key, state)) continue;+ const slots = fold.lists.get(key);+ if (slots === undefined) add(state.winner.record);+ else for (const slot of slots.values()) add(slot.winner.record);+ }+ return [...records.keys()]+ .toSorted()+ .map((id) => ({ id, attestation: schemaRecordAttestation(records.get(id)!) }));+}+/** The `brain.schema.<field>` values in force, by field. */export function storeSchemaFields(configPath?: string): Record<string, ReadonlyArray<string>> {- const store = configuredStore(configPath);- if (store === null || !store.partitions.has("shared")) return {};- const fold = foldSettings(- readStreamRecords(store.root, "shared", { stream: "settings" }).records,- );- const prefix = `${brainSettingKey(SCHEMA_BLOCK)}.`;+ return fieldsOf(schemaFold(configPath));+}++function fieldsOf(fold: SettingsFold): Record<string, ReadonlyArray<string>> {const out: Record<string, ReadonlyArray<string>> = {};for (const [key, state] of fold.settings) {- if (!key.startsWith(prefix) || state.removed) continue;- const field = key.slice(prefix.length);- if (field.includes(".") || !Array.isArray(state.value)) continue;- out[field] = state.value.map(String);+ if (!isPackField(key, state) || state.removed) continue;+ out[key.slice(SCHEMA_PREFIX.length)] = (state.value as unknown[]).map(String);}return out;}+/**+ * Whether the settings key `key` is a field of the pack: a name under+ * `brain.schema.` the pack parser reads ({@link SCHEMA_PACK_FIELDS}),+ * holding a list or removed (a removal decides the field too). The pack+ * reader and the attestation use this one filter, so a record the pack+ * never reads cannot change what is attested.+ */+function isPackField(key: string, state: RegisterState<unknown>): boolean {+ if (!key.startsWith(SCHEMA_PREFIX)) return false;+ if (!SCHEMA_PACK_FIELDS.has(key.slice(SCHEMA_PREFIX.length))) return false;+ return state.removed || Array.isArray(state.value);+}+/*** The config text of the store's schema pack, or `null` when no schema* field is set (the pack is then the built-in vocabulary alone).*/export function storeSchemaConfigText(configPath?: string): string | null {- const fields = storeSchemaFields(configPath);+ return configTextOf(storeSchemaFields(configPath));+}++function configTextOf(fields: Readonly<Record<string, ReadonlyArray<string>>>): string | null {const names = Object.keys(fields).toSorted();if (names.length === 0) return null;const lines = ["schema_version: 1", `${SCHEMA_BLOCK}:`];@@ -94,23 +249,65 @@ export interface SchemaPackStoreWrite {/*** Write the pack whose rendered `schema:` block is `schemaBlock` over the- * pack in force: one `setting-set` per changed field, one `setting-unset`- * per field the new pack drops. Returns the ids of the records written.+ * pack in force, rendered as `currentBlock`: a `setting-entry` record+ * adding each entry the new pack has and the one in force lacks, and one+ * removing each entry it drops (a keyed entry whose value changed is+ * replaced by its new entry alone). Entries are compared by slot, the+ * normalised form the pack parser reads, so a stored entry in another+ * spelling of the same token is no change. An entry the records still+ * hold but the pack in force dropped on read (a reference to an undeclared+ * token, `dropDanglingReferences` in `./schema-pack.ts`) is removed too,+ * as a repair: skipped where this authority may not remove it, since every+ * reader drops it anyway. `stored` holds the field lists `currentBlock` was+ * read from ({@link readStoreSchemaRecords}): the same read, so a repair+ * removes only an entry the pack in force dropped, never one that synced+ * in after the pack was read; and a repair is skipped when another record+ * decides the entry's slot by the time it is written (`decidedBy`), so a+ * replacement another device wrote meanwhile is kept. Returns the ids of+ * the records written.*/-export function writeSchemaPackToStore(ctx: SchemaPackStoreWrite, schemaBlock: string): string[] {+export function writeSchemaPackToStore(+ ctx: SchemaPackStoreWrite,+ schemaBlock: string,+ currentBlock: string,+ stored: Pick<StoreSchemaRecords, "fields" | "decidedBy">,+): string[] {const next = fieldsOfSchemaBlock(schemaBlock);- const current = storeSchemaFields(ctx.configPath);- const names = [...new Set([...Object.keys(current), ...Object.keys(next)])].toSorted();- return writeSharedSettingRecords(+ const current = fieldsOfSchemaBlock(currentBlock);+ const names = [+ ...new Set([...Object.keys(stored.fields), ...Object.keys(current), ...Object.keys(next)]),+ ].toSorted();+ const changes: SharedSettingEntryChange[] = [];+ for (const field of names) {+ const name = `${SCHEMA_BLOCK}.${field}`;+ const key = `${SCHEMA_PREFIX}${field}`;+ const slot = (entry: string) => settingEntrySlot(key, entry);+ const before = current[field] ?? [];+ const after = next[field] ?? [];+ const kept = new Set(after.map(slot));+ const shown = new Set(before.map(slot));+ for (const entry of after) {+ if (!before.includes(entry)) changes.push({ name, entry, present: true });+ }+ for (const entry of before) {+ if (!kept.has(slot(entry))) changes.push({ name, entry, present: false });+ }+ for (const entry of stored.fields[field] ?? []) {+ const s = slot(entry);+ if (!shown.has(s) && !kept.has(s)) {+ // An entry with no deciding record named is never repaired.+ const decidedBy = stored.decidedBy[field]?.[entry] ?? "";+ changes.push({ name, entry, present: false, repair: { decidedBy } });+ }+ }+ }+ return writeSharedSettingEntries({vault: ctx.vault,authority: ctx.authority,agent: ctx.agent,...(ctx.configPath !== undefined ? { configPath: ctx.configPath } : {}),},- names.map((field) => ({- name: `${SCHEMA_BLOCK}.${field}`,- value: next[field] ?? null,- })),+ changes,);}diff --git a/src/core/brain/schema-pack.ts b/src/core/brain/schema-pack.tsindex 9380fdef..d9e21916 100644--- a/src/core/brain/schema-pack.ts+++ b/src/core/brain/schema-pack.ts@@ -3,10 +3,15 @@ import { existsSync, readFileSync } from "node:fs";import { isStoreOnly } from "../store/store-only.ts";import { escapeRegex } from "../strings.ts";import { brainConfigPath } from "./paths.ts";-import { STORE_SCHEMA_PACK_SOURCE, storeSchemaConfigText } from "./schema-pack-store.ts";+import {+ readStoreSchemaRecords,+ STORE_SCHEMA_PACK_SOURCE,+ type StoreSchemaRecords,+} from "./schema-pack-store.ts";import {SCHEMA_VOCAB_CATEGORIES,resolveSchemaVocabulary,+ splitCompoundSchemaKey,validateSchemaDeclarations,validateSchemaToken,type BrainSchemaDeclarations,@@ -85,22 +90,39 @@ export function readSchemaPackSource(vault: string | null): SchemaPackSource {// A store-only machine keeps the pack in the shared settings records// `brain.schema.<field>` (design D14 item 3), never in `_brain.yaml`.// No vault means store-only too.- if (vault === null || isStoreOnly({ vault })) {- const stored = storeSchemaConfigText();- const text = stored ?? ABSENT_SCHEMA_CONFIG_TEXT;- return Object.freeze({- path: STORE_SCHEMA_PACK_SOURCE,- present: stored !== null,- text,- pack: parseSchemaPack(text),- });- }+ if (vault === null || isStoreOnly({ vault })) return readStoreSchemaPack().source;const path = brainConfigPath(vault);const present = existsSync(path);const text = present ? readFileSync(path, "utf8") : ABSENT_SCHEMA_CONFIG_TEXT;return Object.freeze({ path, present, text, pack: parseSchemaPack(text) });}+/** The store's schema pack with the records it was read from, one read of them. */+export interface StoreSchemaPack extends StoreSchemaRecords {+ readonly source: SchemaPackSource;+}++/**+ * The pack of a store-only machine together with the field lists and the+ * deciding records it was parsed from, all from one read of the shared+ * settings stream (task 3.12, design D19 item 1). An apply and the+ * integrity read take the pack, its deciding records and the entries a+ * repair may remove from here, so no record can sync in between them.+ */+export function readStoreSchemaPack(configPath?: string): StoreSchemaPack {+ const records = readStoreSchemaRecords(configPath);+ const text = records.text ?? ABSENT_SCHEMA_CONFIG_TEXT;+ const source: SchemaPackSource = Object.freeze({+ path: STORE_SCHEMA_PACK_SOURCE,+ present: records.text !== null,+ text,+ // Fields merge per entry (task 3.12), so a removal from one device+ // can meet a reference another device added: repaired on read.+ pack: dropDanglingReferences(parseSchemaPack(text)),+ });+ return { ...records, source };+}+export function loadSchemaPack(vault: string | null): SchemaPack {return readSchemaPackSource(vault).pack;}@@ -183,6 +205,57 @@ export function parseSchemaPack(configText: string): SchemaPack {});}+/**+ * The pack without its references to undeclared tokens (design D19 item+ * 1): the aliases, prefixes, extractable entries, expert routes and+ * attributes of a token the vocabulary does not hold, the constraint pairs+ * whose source or target it does not hold, and the constraints of an+ * undeclared link type or of one left with no pair. That is the cascade+ * `remove_type` and `remove_link_type` apply. A store's fields merge per entry, so a type+ * removed on one device and a reference to it added on another meet in+ * the merged records; every machine drops the reference the same way when+ * it reads the pack, so the pack in force always passes the reference+ * check an apply runs, and the next apply writes the drop as removals.+ */+export function dropDanglingReferences(pack: SchemaPack): SchemaPack {+ const tokens = schemaPackTokens(pack);+ return freezeSchemaPack({+ declarations: pack.declarations,+ vocabulary: pack.vocabulary,+ aliases: keep(pack.aliases, (token) => tokens.has(token)),+ prefixes: keep(pack.prefixes, (_, token) => tokens.has(token)),+ link_types: pack.link_types,+ extractable: pack.extractable.filter((token) => tokens.has(token)),+ expert_routing: keep(pack.expert_routing, (token) => tokens.has(token)),+ labels: pack.labels,+ link_constraints: keep(+ Object.fromEntries(+ Object.entries(pack.link_constraints).map(([type, pairs]) => [+ type,+ Object.freeze(pairs.filter((pair) => pairEndpoints(pair).every((t) => tokens.has(t)))),+ ]),+ ),+ (type, pairs) => pack.link_types.includes(type) && pairs.length > 0,+ ),+ attributes: keep(pack.attributes, (type) => tokens.has(type)),+ frontmatter_tiers: pack.frontmatter_tiers,+ });+}++/** The entries of `record` that `ok` keeps, frozen. */+function keep<T>(+ record: Readonly<Record<string, T>>,+ ok: (key: string, value: T) => boolean,+): Readonly<Record<string, T>> {+ return Object.freeze(Object.fromEntries(Object.entries(record).filter(([k, v]) => ok(k, v))));+}++/** The source and target tokens of a normalized `source->target` pair. */+export function pairEndpoints(pair: string): [string, string] {+ const arrow = pair.indexOf("->");+ return [pair.slice(0, arrow), pair.slice(arrow + 2)];+}+/** Validate and normalize a `source->target` endpoint pair value. */export function validateEndpointPair(value: string, field: string): string {const arrow = value.indexOf("->");@@ -341,12 +414,12 @@ function parseCompoundMapField<T extends string>(const flat = /^ -\s+(.+?)\s*$/.exec(line);if (!flat) continue;const [rawKey, rawValue] = splitPair(flat[1]!);- const dot = rawKey.indexOf(".");- if (dot <= 0 || dot === rawKey.length - 1) {+ const halves = splitCompoundSchemaKey(rawKey);+ if (halves === null) {throw new Error(`${field}: "${rawKey}" must use a compound type.field key`);}- const outer = validateSchemaToken(rawKey.slice(0, dot), `${field}.${rawKey}`);- const inner = validateSchemaToken(rawKey.slice(dot + 1), `${field}.${outer}`);+ const outer = validateSchemaToken(halves[0], `${field}.${rawKey}`);+ const inner = validateSchemaToken(halves[1], `${field}.${outer}`);const value = validateValue(rawValue, `${field}.${outer}.${inner}`);out[outer] = out[outer] ?? {};out[outer]![inner] = value;diff --git a/src/core/brain/schema-vocab.ts b/src/core/brain/schema-vocab.tsindex 532e4ac3..86564d0a 100644--- a/src/core/brain/schema-vocab.ts+++ b/src/core/brain/schema-vocab.ts@@ -48,6 +48,20 @@ export function normalizeSchemaToken(raw: string): string {return raw.normalize("NFC").trim().toLowerCase();}+/**+ * The two halves of a compound `type.field` key (the `attributes` and+ * `frontmatter_tiers` entries), split at its first `.`, or `null` when+ * either half would be empty. `rawKey` is the trimmed text before the+ * entry's `=`; the halves are not normalised. The pack parser and the+ * store's per-entry fold of those lists both split here, so they agree on+ * which entries name one key.+ */+export function splitCompoundSchemaKey(rawKey: string): readonly [string, string] | null {+ const dot = rawKey.indexOf(".");+ if (dot <= 0 || dot === rawKey.length - 1) return null;+ return [rawKey.slice(0, dot), rawKey.slice(dot + 1)];+}+export function validateSchemaDeclarations(declarations: BrainSchemaDeclarations | undefined | null,): BrainSchemaDeclarations {diff --git a/src/core/brain/shared-namespace.ts b/src/core/brain/shared-namespace.tsindex b3855cc0..07a6cef9 100644--- a/src/core/brain/shared-namespace.ts+++ b/src/core/brain/shared-namespace.ts@@ -27,11 +27,23 @@* records as ordinary first-class signals. Reading it back is a separate* mechanism - it is enumerated as a search origin by* `portability/origins.ts`.+ *+ * The brain store (task 3.12, design D19 item 3): the namespace gets no+ * store of its own. A write to a vault that is store-only on this machine+ * lands in this machine's brain store, which every store-only agent of+ * every machine of the store reads, so a store-only namespace IS that+ * store. The mirror writes there only when the origin record is not in it+ * already: the origin vault still writes the legacy layout here and dual+ * writes are off. It refuses, before any I/O, on a switched machine, for a+ * store-only origin vault, with dual writes on (each would put the record+ * into the store a second time), and when there is no store at all.*/import { realpathSync, statSync } from "node:fs";import { basename, resolve } from "node:path";+import { isDualWriteEnabled } from "../store/dual-write.ts";+import { configuredStore } from "../store/read/store-entities.ts";import { storeOnlyMode } from "../store/store-only.ts";import { resolveSharedNamespace } from "../config.ts";import { appendLogEvent } from "./log.ts";@@ -99,26 +111,61 @@ const SELF_MIRROR: MirrorReport = Object.freeze({});/**- * The refusal when the shared vault is store-only on this machine (this- * machine switched, or the shared vault has no legacy tree): a store write- * goes to this machine's own store, never to the shared namespace, so the- * mirror would only duplicate the origin record in the same store.+ * The refusal when the shared vault is store-only on this machine and the+ * origin record is already in this machine's store (this machine switched,+ * the origin vault is store-only too, or dual writes copied it there): a+ * mirror would only write the same record into the same store again.*/const STORE_ONLY_MIRROR: MirrorReport = Object.freeze({outcome: MIRROR_OUTCOME.misconfigured,reason:- "the shared namespace is store-only on this machine, so a mirror would write this " +- "machine's own brain store again and never reach the shared vault; nothing was mirrored",+ "the shared namespace is store-only on this machine, and the origin record is already in " ++ "this machine's brain store, which is that namespace; nothing was mirrored",+});++/** The refusal when the shared vault is store-only but this machine has no store. */+const NO_STORE_MIRROR: MirrorReport = Object.freeze({+ outcome: MIRROR_OUTCOME.misconfigured,+ reason:+ "the shared namespace is store-only on this machine (it has no legacy tree), but this " ++ "machine has no brain store to write it into; run `o2b store init`, or `o2b brain init` " ++ "in the shared vault; nothing was mirrored",+});++/** The report when the shared namespace path is no directory. */+const NOT_A_DIRECTORY: MirrorReport = Object.freeze({+ outcome: MIRROR_OUTCOME.failed,+ reason:+ "shared_namespace is not a directory (unmounted, missing, or a file); nothing was mirrored",});-/** Why a mirror cannot run before any I/O, or `null` when it can. */-function mirrorRefusal(sharedVault: string, originVault: string): MirrorReport | null {+/**+ * Why a mirror cannot run before any I/O, or `null` when it can. Every+ * check reads the caller's machine config (`configPath`, default: the+ * resolved one), the same config the origin write and the namespace came+ * from.+ */+function mirrorRefusal(+ sharedVault: string,+ originVault: string,+ configPath: string | undefined,+): MirrorReport | null {if (isSelfMirror(sharedVault, originVault)) return SELF_MIRROR;- const mode = storeOnlyMode({ vault: sharedVault });- // A shared path that is no directory at all is a broken mount: the- // attempted write reports it as `failed`, as before.+ const withConfig = configPath !== undefined ? { configPath } : {};+ const mode = storeOnlyMode({ vault: sharedVault, ...withConfig });if (mode.reason === "device-switched") return STORE_ONLY_MIRROR;- if (mode.reason === "no-legacy-tree" && isDirectory(sharedVault)) return STORE_ONLY_MIRROR;+ if (mode.reason !== "no-legacy-tree") return null;+ // A shared path that is no directory at all (an unmounted share, a+ // file) is a broken namespace, not a store-only one: reported `failed`+ // before any write, never mirrored into this machine's store.+ if (!isDirectory(sharedVault)) return NOT_A_DIRECTORY;+ if (configuredStore(configPath) === null) return NO_STORE_MIRROR;+ if (+ storeOnlyMode({ vault: originVault, ...withConfig }).storeOnly ||+ isDualWriteEnabled(configPath)+ ) {+ return STORE_ONLY_MIRROR;+ }return null;}@@ -141,15 +188,18 @@ function failureReason(exc: unknown): MirrorReport {/*** Mirror a feedback signal into the shared vault. The record is the* same signal with `origin_vault` attribution; slug collisions resolve- * through the normal allocator.+ * through the normal allocator. `configPath` is the caller's machine+ * config: the refusal checks read it. The signal writer itself resolves+ * the process's config, as the origin write did.*/export function mirrorSignal(sharedVault: string,originVault: string,input: WriteSignalInput,options: WriteSignalOptions = {},+ configPath?: string | null,): MirrorReport {- const refused = mirrorRefusal(sharedVault, originVault);+ const refused = mirrorRefusal(sharedVault, originVault, configPath ?? undefined);if (refused !== null) return refused;try {writeSignal(sharedVault, { ...input, origin_vault: basename(originVault) }, options);@@ -165,13 +215,17 @@ export interface MirrorNoteInput {readonly now?: Date;}-/** Mirror a narrative note event into the shared vault's log. */+/**+ * Mirror a narrative note event into the shared vault's log. The refusal+ * checks and the write read the caller's machine config (`configPath`).+ */export function mirrorNote(sharedVault: string,originVault: string,input: MirrorNoteInput,+ configPath?: string | null,): MirrorReport {- const refused = mirrorRefusal(sharedVault, originVault);+ const refused = mirrorRefusal(sharedVault, originVault, configPath ?? undefined);if (refused !== null) return refused;try {appendLogEvent(@@ -186,6 +240,7 @@ export function mirrorNote(},},// Not fail-open: the mirror reports its own failure to the caller.+ configPath !== undefined && configPath !== null ? { configPath } : {},);return OK;} catch (exc) {diff --git a/src/core/brain/write-session/personas.ts b/src/core/brain/write-session/personas.tsindex 6c8077cb..1e4c315d 100644--- a/src/core/brain/write-session/personas.ts+++ b/src/core/brain/write-session/personas.ts@@ -2,17 +2,35 @@* Panel personas (Agent Write Contract Suite, t_0cc6fdff).** A persona is one analytical lens the calling agent must answer- * through. Operators own custom personas as ordinary vault notes under- * `Brain/personas/<slug>.md` (frontmatter `kind: persona`, `lens`; the- * body is the generation instruction). When the directory is absent or- * holds no persona notes, the built-in default set applies - the panel- * works out of the box and stays operator-curatable.+ * through: a slug, a lens and the generation instruction. When no custom+ * persona exists, the built-in default set applies - the panel works out+ * of the box and stays operator-curatable.+ *+ * Where a persona lives (task 3.12, design D19 item 2): on an unswitched+ * machine it is a vault note under `Brain/personas/<slug>.md` (frontmatter+ * `kind: persona`, `lens`; the body is the instruction). On a store-only+ * machine, with or without a vault, it is a record: a `lane-entry` of the+ * lane `personas` in the brain store, folded by entry id like every lane+ * entry, so a persona curated on one machine is the lens on every machine+ * of the store. {@link setPersona} and {@link removePersona} write either+ * form through the lane page layer (`o2b brain panel persona-set` and+ * `persona-remove`; the CLI writes with human authority).*/import { join } from "node:path";import type { WriteSessionPersona } from "./types.ts";-import { lanePageExists, listLaneFiles, parseLanePage } from "../../store/lane-pages.ts";+import { legacyPagePrivate, storedEntryPrivate } from "../../store/lane-entry-store.ts";+import { callerInput, opaqueFields, operationPrivacy } from "../../store/operation-privacy.ts";+import {+ assertLaneWrite,+ lanePageExists,+ laneVaultRoot,+ listLaneFiles,+ parseLanePage,+ removeLanePage,+ writeLanePage,+} from "../../store/lane-pages.ts";/** Default lens set, in deliberation order. */export const DEFAULT_PERSONAS: ReadonlyArray<WriteSessionPersona> = Object.freeze([@@ -53,9 +71,15 @@ export function personasDir(vault: string): string {* non-persona notes and malformed files are skipped silently - the* panel must keep working when a stray note lands in the directory.*/-export function loadPersonas(vault: string): ReadonlyArray<WriteSessionPersona> {+export function loadPersonas(vault: string | null): ReadonlyArray<WriteSessionPersona> {+ return customPersonas(vault) ?? DEFAULT_PERSONAS;+}++/** The custom personas, or `null` when there are none and the defaults apply. */+function customPersonas(vaultOrNull: string | null): ReadonlyArray<WriteSessionPersona> | null {+ const vault = laneVaultRoot(vaultOrNull);const dir = personasDir(vault);- if (!lanePageExists(vault, dir)) return DEFAULT_PERSONAS;+ if (!lanePageExists(vault, dir)) return null;const personas: WriteSessionPersona[] = [];for (const name of listLaneFiles(vault, dir, "").toSorted()) {if (!name.endsWith(".md")) continue;@@ -74,5 +98,143 @@ export function loadPersonas(vault: string): ReadonlyArray<WriteSessionPersona>if (prompt === "") continue;personas.push(Object.freeze({ slug, lens, prompt }));}- return personas.length > 0 ? Object.freeze(personas) : DEFAULT_PERSONAS;+ return personas.length > 0 ? Object.freeze(personas) : null;+}++/** The personas in force and whether they are the operator's or the defaults. */+export function listPersonas(vault: string | null): {+ readonly source: "custom" | "default";+ readonly personas: ReadonlyArray<WriteSessionPersona>;+} {+ const custom = customPersonas(vault);+ return custom === null+ ? { source: "default", personas: DEFAULT_PERSONAS }+ : { source: "custom", personas: custom };+}++/** A persona request the verbs refuse (a bad slug, lens or prompt). */+export class PersonaError extends Error {+ constructor(message: string) {+ super(message);+ this.name = "PersonaError";+ }+}++/** A persona slug: a plain lowercase name, also its note's file name. */+const PERSONA_SLUG_RE = /^[a-z0-9][a-z0-9-]{0,63}$/;++function checkedSlug(slug: string): string {+ if (!PERSONA_SLUG_RE.test(slug)) {+ throw new PersonaError(+ `persona slug "${slug}" must be 1 to 64 lowercase letters, digits and hyphens, ` ++ "starting with a letter or digit",+ );+ }+ return slug;+}++function personaPath(vault: string, slug: string): string {+ return join(personasDir(vault), `${checkedSlug(slug)}.md`);+}++/** The largest prompt a persona may hold, in UTF-8 bytes: it rides in every panel step. */+export const PERSONA_PROMPT_MAX_BYTES = 16 * 1024;++/** The longest lens, in characters: one short label. */+export const PERSONA_LENS_MAX_CHARS = 200;++/** Line breaks a single-line lens may not hold, Unicode separators included. */+const LINE_BREAK_RE = /[\r\n\u0085\u2028\u2029]/;++/**+ * The persona note at `path`, refused when a page of another kind lives+ * there: the verbs curate personas, never another note at that path.+ */+function refuseOtherPage(vault: string, path: string, slug: string): void {+ if (!lanePageExists(vault, path)) return;+ let kind: unknown;+ try {+ kind = parseLanePage(vault, path)[0]["kind"];+ } catch {+ kind = undefined;+ }+ if (kind === "persona") return;+ throw new PersonaError(+ `Brain/personas/${slug}.md is no persona (kind: ${String(kind)}); the persona verbs ` ++ "leave it alone. Pick another slug, or move that note first",+ );+}++/**+ * The persona at `path`, refused when it lives in `private` (or its note+ * is marked private): the lane writer keeps a replaced entry where it+ * lives and never lowers its privacy, so a shared replacement would stay+ * private and machines without `private` would never see it.+ */+function refusePrivateTarget(vault: string, path: string, slug: string): void {+ if (!legacyPagePrivate(vault, path) && !storedEntryPrivate(vault, path)) return;+ throw new PersonaError(+ `persona ${slug} lives in private, and a replacement would stay there, out of reach of ` ++ `machines without private; remove it first (\`o2b brain panel persona-remove ${slug}\`), ` ++ "then set it again as a shared persona",+ );+}++/**+ * Add or replace the custom persona `slug`: a note on an unswitched+ * machine, a `lane-entry` record on a store-only one (`vault` may then be+ * `null`). Writing what the persona already holds changes nothing.+ *+ * Refused (task 3.12 review): in a frozen vault or while a migration runs+ * on it (the lane write guard; on a store-only machine the freeze), over a page+ * of another kind at that path, over a persona that lives in `private`,+ * a prompt over {@link PERSONA_PROMPT_MAX_BYTES} or a lens over+ * {@link PERSONA_LENS_MAX_CHARS} or with a line break, and a private+ * persona: text with a `<private>` region or naming a private page would+ * be stored in `private`, which machines that do not hold it never read,+ * while a persona is the lens on every machine of the store.+ */+export function setPersona(+ vaultOrNull: string | null,+ input: { readonly slug: string; readonly lens?: string; readonly prompt: string },+): { readonly slug: string; readonly changed: boolean } {+ const vault = laneVaultRoot(vaultOrNull);+ const path = personaPath(vault, input.slug);+ const prompt = input.prompt.trim();+ if (prompt === "") throw new PersonaError("a persona needs a non-empty prompt");+ if (Buffer.byteLength(prompt, "utf8") > PERSONA_PROMPT_MAX_BYTES) {+ throw new PersonaError(+ `a persona prompt holds at most ${PERSONA_PROMPT_MAX_BYTES} bytes; every panel step carries it`,+ );+ }+ const lens = input.lens?.trim() || input.slug;+ if (LINE_BREAK_RE.test(lens)) throw new PersonaError("a persona lens must be a single line");+ if (lens.length > PERSONA_LENS_MAX_CHARS) {+ throw new PersonaError(`a persona lens holds at most ${PERSONA_LENS_MAX_CHARS} characters`);+ }+ const isPrivate = operationPrivacy(vaultOrNull, {+ content: [callerInput({ lens, prompt }, opaqueFields())],+ });+ if (isPrivate()) {+ throw new PersonaError(+ `persona ${input.slug} is private: its lens or prompt carries a <private> region or names ` ++ "a private page, and a persona is read by every machine of the store",+ );+ }+ assertLaneWrite(vault);+ refuseOtherPage(vault, path, input.slug);+ refusePrivateTarget(vault, path, input.slug);+ const changed = writeLanePage(vault, path, { kind: "persona", lens }, `${prompt}\n`, {+ overwrite: true,+ });+ return { slug: input.slug, changed };+}++/** Remove the custom persona `slug`; `false` when there was none. */+export function removePersona(vaultOrNull: string | null, slug: string): boolean {+ const vault = laneVaultRoot(vaultOrNull);+ const path = personaPath(vault, slug);+ assertLaneWrite(vault);+ refuseOtherPage(vault, path, slug);+ return removeLanePage(vault, path);}diff --git a/src/core/brain/write-session/types.ts b/src/core/brain/write-session/types.tsindex 68d28bd3..0487f032 100644--- a/src/core/brain/write-session/types.ts+++ b/src/core/brain/write-session/types.ts@@ -51,7 +51,7 @@ export interface WriteSessionError {readonly message: string;}-/** One panel persona (loaded from Brain/personas/ or built-in). */+/** One panel persona (custom: a note or a store record; or built-in). */export interface WriteSessionPersona {readonly slug: string;readonly lens: string;diff --git a/src/core/store-migration/brain-settings.ts b/src/core/store-migration/brain-settings.tsindex 09be9b30..9f4b816f 100644--- a/src/core/store-migration/brain-settings.ts+++ b/src/core/store-migration/brain-settings.ts@@ -17,6 +17,11 @@* default is neither written nor reported; the store's settings reader* falls back to the same default.*+ * The schema fields it writes (`brain.schema.<field>`) are attested by a+ * baseline row in the store's audit ledger (task 3.12, design D19 item+ * 1): no audited apply wrote them, and an apply refuses over a deciding+ * record no row attests.+ ** `_permissions.yaml` rides along (review finding F461): its text becomes* the operator-only `permissions.document` setting at `human` authority,* never over a value the store already holds, so the deny rules stay in@@ -34,10 +39,17 @@ import {import { brainConfigPath } from "../brain/paths.ts";import { DEFAULT_BRAIN_CONFIG } from "../brain/policy/defaults.ts";import { brainConfigKey, brainKeyHome, brainSettingKey } from "../brain/policy/settings-keys.ts";+import {+ SCHEMA_DECIDED_RECORDS_FIELD,+ SCHEMA_IMPORT_BASELINE_ACTION,+ SCHEMA_MUTATION_AUDIT_DIR,+} from "../brain/schema-integrity.ts";+import { schemaRecordAttestation, STORE_SCHEMA_PACK_SOURCE } from "../brain/schema-pack-store.ts";import { parseBrainYaml } from "../brain/yaml-parse.ts";import { canonicalJson } from "../integrity/digest.ts";import { appendRecord } from "../store/appender.ts";-import { AUTHORITY_FIELD, SETTING_SET_KIND } from "../store/constants.ts";+import { AUTHORITY_FIELD, LEDGER_ROW_KIND, SETTING_SET_KIND } from "../store/constants.ts";+import { DUAL_WRITE_LEDGERS } from "../store/dual-write-records.ts";import { foldSettings } from "../store/fold-settings.ts";import type { StoreHandle } from "../store/store-open.ts";import { heldRecords } from "./journal.ts";@@ -124,6 +136,7 @@ export function importBrainSettings(unknown: [] as string[],};const fold = foldSettings(heldRecords(store));+ const schemaAttestations: string[] = [];// `_permissions.yaml` becomes the operator-only `permissions.document`// setting (review finding F461): its text exactly as the file holds it,// at human authority, so a deny rule stays in force once the legacy tree@@ -174,7 +187,7 @@ export function importBrainSettings(result.unchanged.push(key);continue;}- appendRecord(store, {+ const written = appendRecord(store, {partition: "shared",stream: "settings",kind: SETTING_SET_KIND,@@ -182,6 +195,32 @@ export function importBrainSettings(body: { [AUTHORITY_FIELD]: "human", key, value },});result.written.push(key);+ if (block === "schema") schemaAttestations.push(schemaRecordAttestation(written.record));}+ if (schemaAttestations.length > 0) appendSchemaBaseline(store, agent, schemaAttestations);return result;}++/**+ * The baseline row attesting the schema records the import wrote, each+ * bound to its content (`<id>:<digest>`), in `shared` like every apply's+ * attesting row (`../brain/schema-mutate-store.ts`).+ */+function appendSchemaBaseline(store: StoreHandle, agent: string, attestations: string[]): void {+ const record = {+ timestamp: new Date().toISOString(),+ actor: agent,+ action: SCHEMA_IMPORT_BASELINE_ACTION,+ target: STORE_SCHEMA_PACK_SOURCE,+ ok: true,+ details: { [SCHEMA_DECIDED_RECORDS_FIELD]: attestations },+ };+ appendRecord(store, {+ partition: "shared",+ stream: "ledgers",+ ledger: DUAL_WRITE_LEDGERS.reliabilityAudit,+ kind: LEDGER_ROW_KIND,+ agent,+ body: { row: { audit_dir: SCHEMA_MUTATION_AUDIT_DIR, record } },+ });+}diff --git a/src/core/store/FORMAT.md b/src/core/store/FORMAT.mdindex df29e5ef..9c84769f 100644--- a/src/core/store/FORMAT.md+++ b/src/core/store/FORMAT.md@@ -181,6 +181,10 @@ Settings every machine must agree on are records in the `settings` stream of theKeys `brain.<block>.<field>` (and `brain.<key>` for a scalar) hold the settings of a vault's legacy `_brain.yaml` that every machine must agree on. The migration from the legacy layout writes only the values the user changed from the shipped default, with `authority` `human`; a value equal to its default is not written, and the key falls back to that default. Their values are the JSON the file held.+A list setting can also be edited one entry at a time: a `setting-entry` record adds or removes one entry of the list. Only the schema field lists `brain.schema.<field>` (one field name after `brain.schema.`) are list settings; a `setting-entry` record for any other key is not applied, so a key that holds one value stays one register. A key that holds at least one `setting-entry` record folds per entry, so two machines that edit the list before they sync keep both edits. Each entry is one register, resolved by authority, then clock, like every register. The register of an entry is its slot: the entry in the form the schema pack reads it, NFC-normalised, with surrounding quotes stripped, trimmed and in lower case, each side of a `name=value` line normalised on its own (split at the first `=`), and each side of a `source->target` pair too; quotes around the whole line are stripped first unless the same quote character also stands inside it. So two spellings of one token are one register. The keyed lists `brain.schema.prefixes`, `brain.schema.expert_routing`, `brain.schema.attributes` and `brain.schema.frontmatter_tiers` hold `name=value` lines with one value per name: their slot is the normalised text before the first `=`, so a new value replaces the old entry. In `brain.schema.attributes` and `brain.schema.frontmatter_tiers` that text is a compound `type.field` name: as the schema pack reads it, a compound `type.field` name is split at its first `.` and each half normalised on its own, so `paper . year` and `paper.year` are one slot. A `setting-set` of the key with a list value offers every slot the key has ever named, its own entries as present and every other slot as removed; a `setting-unset` offers every slot as removed. A `setting-set` of a list setting whose value is not a list of one-line strings, numbers or booleans is not applied, whether or not the key holds a `setting-entry` record, so such a value never hides the list. So a key written only by whole lists resolves as one register over the list. The value is the list of entries present, each as the record that decided its slot wrote it, ordered by the fold order of that record, then by the entry's place in that record; when no entry is present the key is removed and falls back to its default. A key with no `setting-entry` record folds as one register. The schema fields are written this way: a schema apply writes one `setting-entry` per entry it adds or removes, never the whole list. A removal may carry `if_decided_by`, the id of the record that decided the entry's slot when the writer judged the removal: such a conditional removal applies only while the record it names decides the entry's slot among the key's records that carry no `if_decided_by`, and is otherwise ignored, wherever its clock falls and in whatever order the records arrive. `if_decided_by` is a field of `setting-entry` only: on a `setting-set` or `setting-unset` it is ignored, like any field a kind does not define. When the records of a key folded per entry name no entry at all once the conditional removals that do not apply are left out (an empty list, a `setting-unset`), the key is removed, and its strongest, then latest, record stands for it. Format version 1 includes `setting-entry` from the first release that contains the store.++A schema pack is read from these records with every reference to an undeclared token dropped (aliases, prefixes, extractable entries, expert routes and attributes of a token the vocabulary does not hold, a link constraint pair whose source or target the vocabulary does not hold, and the constraints of an undeclared link type or of one left with no pair), so two applies merged per entry never yield a pack with a dangling reference. Each schema apply is attested by a `reliability-audit` row in `shared` (`audit_dir` `schema-mutations`, `action` `schema_apply_mutations`) carrying the digest of the pack it produced and `decided`, the records it attests: the records it wrote and records rows it had seen already attest, plus records the operator adopted, whose ids the row also lists under `adopted` with `adopted_by`. Each entry of `decided` is `<id>:<sha256>`, the record's id and the sha256 of the record's canonical JSON (the whole envelope, body included), so a row attests the one version of the record it saw: another body under the same id, with a checksum that matches it, is not attested. An entry that is a bare id attests nothing. The migration's import writes a row with `action` `schema_import_baseline` whose `decided` names the schema records it imported. A pack that matches no single row's digest is attested when every record deciding it is named by some row. Once rows name records, a deciding record no row names is unattested even when the pack matches the newest row's digest (a record that sets an entry the pack already held changes no digest but still decides the entry). A schema apply and the integrity check read the pack, the records deciding it and the entries an apply compares against from one read of the `settings` stream, so a record that arrives meanwhile is never attested, and never removed, by that apply: an apply writes the removal of an entry the pack dropped on read only while the record that decided it in that read still decides the entry's slot when the removal is written, and writes it as a conditional removal (`if_decided_by`) naming that record, so another device's replacement of that entry is kept even when it arrives after the removal. An apply that only adopts records writes no `setting-entry` record. Only the fields the pack reads decide it: `preference_types`, `signal_types`, `page_types`, `log_event_kinds`, `aliases`, `prefixes`, `link_types`, `extractable`, `expert_routing`, `labels`, `link_constraints`, `attributes` and `frontmatter_tiers`; a record for any other name under `brain.schema.` is neither part of the pack nor attested.+A record that sets a known key to an invalid value is not applied. A key this version does not know is kept as the JSON value written, so an older reader keeps a newer release's setting instead of losing it.## Reading@@ -250,6 +254,7 @@ Kinds of the mutable-entity folds, with their bodies (fields marked optional may- `pinned-clear`: `{ "authority" }`, written to `events`. The tombstone of the pinned context.- `setting-set`: `{ "authority", "key": "<key>", "value": <json, not null> }`, written to `settings` in `shared`.- `setting-unset`: `{ "authority", "key": "<key>" }`, written to `settings` in `shared`. The tombstone of a setting.+- `setting-entry`: `{ "authority", "key": "<key>", "entry": "<one non-empty line>", "present": true | false, "if_decided_by": "<record id>" (optional, removals only) }`, written to `settings` in `shared`. Adds (`present` true) or removes (`present` false) one entry of a list setting; a removal with `if_decided_by` is conditional; see "Shared settings". A record with `if_decided_by` on an addition, or with an empty one, is not applied. A record for a key other than `brain.schema.<field>` is not applied.Kinds of the state digests:@@ -260,7 +265,7 @@ Kinds written by the dual writes of format version 1 (a trial: while the dual-wrAll records one write produces land in one partition. The write is judged once: it is private when the page it writes or is about is private (or cannot be read), or when any of its content fields (a signal's principle, sources and raw text, a preference's principle, evidence and relationship fields, an event's text or artifact) carries a `<private>` region, links a private page, or names a preference whose page is not on disk; its own id, generated paths, hashes, timestamps, keys and agent names are not judged as links. That one verdict places every record of the write: a signal and its `idempotency` receipt, the records of a preference write, an apply-evidence event with its `preference-evidence` record and receipt, a pin's `preference-set` and its event, and every event and receipt of a write batch.-One exception: a `lane-entry-remove` carries no content, so it lands in the partition of the entry it removes, whatever the rest of its write is. A folder move or removal places each entry and its tombstone in that entry's own partition (a moved entry can still be raised to `private`, never lowered), and a write that raises a shared lane entry to `private` writes the entry's shared tombstone first, in the same write. A device that holds only `shared` then sees a shared entry go, and a device that holds both reads the private value, whose clock is later.+One exception: a `lane-entry-remove` carries no content, so it lands in the partition of the entry it removes, whatever the rest of its write is. A folder move or removal places each entry and its tombstone in that entry's own partition (a moved entry can still be raised to `private`, never lowered), and a write that raises a shared lane entry to `private` writes the entry's shared tombstone first, in the same write, as does a move that replaces a shared entry with a private one at its destination. A move lands in `private` when the entry it replaces at its destination is private, as a write over that entry does: nothing lowers an entry's partition. A move onto the entry it moves writes nothing. A move of a lane entry to a path that is no lane entry (outside `Brain/`, or not in a lane folder) is refused and writes nothing. A folder is never moved into its own subfolder, nor into a folder that holds it: the move is refused and writes nothing, as the rename on a machine that has not switched refuses it, so no tombstone of a folder move can remove an entry the same move wrote. A folder move or removal builds every record from one read of the lane entries, so an entry's value and its partition come from the same read, and while that read may miss a record (a shard it could not see or read only in part, or a record in `private` whose body blob has not arrived, quarantined on this machine or not) the moved entries go to `private`, as every lane write does then. A record in `shared` whose body blob has not arrived does not count: a record there never carries a value that makes an entry private. A device that holds only `shared` then sees a shared entry go, and a device that holds both reads the private value, whose clock is later.Every dual-written body also carries `"vault_id": "<id>" | null`: the id in the `Brain/vault-id.json` marker of the vault the legacy write went to, or `null` when that vault has no marker. One store serves every vault on a machine. In format version 1 the field only records where a record came from: the search index of each vault reads every record of the store, and nothing filters by it yet. A later version decides whether a vault reads only its own records.@@ -323,7 +328,7 @@ Kinds of attestations (a signed statement of the state one machine measured, madKinds of record lanes (the Brain record lanes that have no kind of their own: decisions, obligations, pledges, theses, tensions, gap tasks, attention flows, entities, procedures, procedural memory, skill proposals and benchmarks, captures, write sessions, exact-state pages, standing rules and the rest):- `lane-entry`: `{ "authority", "lane": "<lane>", "entry": "<entry id>", "fields": { ... }, "text": "<Markdown>", "raw": "<file text>" (optional) }`, written to `events`. The whole current entry: `fields` are the structured fields the entry carried (a page's frontmatter, or the object a JSON file held), `text` its Markdown body. `raw`, when present, is the file's text byte for byte, written only when `fields` and `text` do not reproduce it (a page without frontmatter, a quoted value, CRLF line ends, the formatting of a JSON file). A reader serves `raw` only when it parses back to the record's own `fields` and `text`, and renders `fields` and `text` otherwise; a reader that does not know `raw` ignores it. The rollback export serves `raw` under the same rule: it writes `raw` into the restored file when it parses back, and the rendering of `fields` and `text` otherwise. `raw` counts for the entry's privacy like `fields` and `text`, and spills into a blob with the rest of the body. A lane name is 1 to 64 lowercase letters, digits and hyphens, starting with a letter or digit; an entry id is a non-empty single line of at most 512 characters, unique within its lane. Standing rules are entries of the lane `standing-rule` and carry `authority` `human`. An entry goes to `private` when its page is private by its visibility markers, or when its fields or text name a private page or one that cannot be judged; the text of a large entry spills into a blob like any other body.-- `lane-entry-remove`: `{ "authority", "lane": "<lane>", "entry": "<entry id>" }`, written to `events`. The tombstone of an entry.+- `lane-entry-remove`: `{ "authority", "lane": "<lane>", "entry": "<entry id>" }`, written to `events`. The tombstone of an entry. While the dual-write setting is on, a machine that has not switched writes it beside a removed or moved legacy page whose entry the store holds (a move also writes the entry at its new path, built from the page as it is on disk after the move, or only the tombstones when no page is there, the old entry's and that of a destination entry the rename overwrote; the page is never read through a symbolic link, at its own name or at a folder below `Brain/`, nor when it is a special file, so such a page also leaves only the tombstones, as the migration never imports it), so the store holds what the legacy tree holds. A page whose entry the store does not hold, renamed over a path whose entry it does hold, is written as that page's `lane-entry` (or, when no page the store would import is there, that entry's tombstone), placed as a write of the page on that machine would be, so the store never keeps the content the rename replaced.Entries fold by lane and entry id: each entry is one register, resolved by authority, then clock.diff --git a/src/core/store/constants.ts b/src/core/store/constants.tsindex ea1e0b75..17c15f68 100644--- a/src/core/store/constants.ts+++ b/src/core/store/constants.ts@@ -166,6 +166,11 @@ export const PINNED_CLEAR_KIND = "pinned-clear";/** Shared setting records (`settings` stream): a value and its removal. */export const SETTING_SET_KIND = "setting-set";export const SETTING_UNSET_KIND = "setting-unset";+/**+ * One entry of a list setting added or removed (`settings` stream): a list+ * setting folds per entry, so concurrent edits on two devices both apply.+ */+export const SETTING_ENTRY_KIND = "setting-entry";/** A human record that takes a device off the digest roster (`settings` stream). */export const DEVICE_RETIRE_KIND = "device-retire";@@ -182,6 +187,7 @@ export const FOLD_RECORD_KINDS = Object.freeze([PINNED_CLEAR_KIND,SETTING_SET_KIND,SETTING_UNSET_KIND,+ SETTING_ENTRY_KIND,DEVICE_RETIRE_KIND,STATE_DIGEST_KIND,] as const);diff --git a/src/core/store/content-write-gates.ts b/src/core/store/content-write-gates.tsnew file mode 100644index 00000000..2cb59075--- /dev/null+++ b/src/core/store/content-write-gates.ts@@ -0,0 +1,23 @@+/**+ * The content-write gate of a store-only write (design D14 item 13).+ *+ * On an unswitched machine every content writer passes the vault identity+ * guard (`../brain/vault-identity.ts`), which holds the freeze marker and+ * the migration gate before it checks the vault's identity marker. On a+ * store-only machine the store write checks the store identity in place of+ * the vault marker, and a migration gate left naming the vault is a+ * leftover of the migration that switched it, which refuses nothing. The+ * freeze still holds: its marker lives in the device-local state there+ * and freezes this machine. A writer that skips the vault guard on a+ * store-only machine calls {@link assertStoreContentWrite} instead. With+ * no vault, the freeze is read from the device-local state through the+ * no-vault root, so no vault is needed.+ */++import { assertVaultNotFrozen } from "../brain/freeze-marker.ts";+import { noVaultRoot } from "../platform-dirs.ts";++/** Refuse a store-only content write while this machine is frozen (`VaultFrozenError`). */+export function assertStoreContentWrite(vault: string | null): void {+ assertVaultNotFrozen(vault ?? noVaultRoot());+}diff --git a/src/core/store/fold-setting-lists.ts b/src/core/store/fold-setting-lists.tsnew file mode 100644index 00000000..def0c99c--- /dev/null+++ b/src/core/store/fold-setting-lists.ts@@ -0,0 +1,307 @@+/**+ * The set merge of list settings (task 3.12, design D19 item 1).+ *+ * A list setting (the schema fields `brain.schema.<field>`, the only keys+ * that take entry records) used to be one register over the whole list: two+ * devices that each added an entry at once kept only the later list, and+ * the other addition was lost. A `setting-entry` record instead adds or+ * removes one entry, and a key that holds at least one such record folds+ * per entry:+ *+ * - Each entry is one register, keyed by its *slot*: the entry in the+ * normalised form the schema pack parser reads (NFC, quotes stripped,+ * trimmed, lower case; each side of a `name=value` line on its own),+ * or for a keyed list (`name=value` lines, where one name holds one+ * value) the normalised name before the first `=` (a compound+ * `type.field` name split at its first `.`, each half on its own).+ * So two spellings of one token are one register. Registers resolve as+ * every register does (design D4): authority first, then the later clock.+ * - A whole-list `setting-set` (the migration's import, `o2b brain+ * settings set`; a value that is no list of one-line scalars is rejected,+ * {@link listValueRejection}) offers every slot the key+ * has ever named: its own entries as present, every other slot as+ * removed. A `setting-unset` offers every slot as removed. So a+ * history of whole lists alone resolves as the whole-list register did.+ * - The value is the present entries, ordered by the fold order of the+ * record that decided each, then by its place in that record. When no+ * entry is present the key is removed.+ *+ * A key with no `setting-entry` record folds as the plain register.+ * The fold is a pure function of the record set: every delivery order+ * gives the same list.+ */++import { normalizeSchemaToken, splitCompoundSchemaKey } from "../brain/schema-vocab.ts";+import type { StoreRecord } from "./envelope.ts";+import { recordAuthority, RegisterSet, TOMBSTONE, type RegisterState } from "./fold.ts";+import { SETTING_ENTRY_KIND, SETTING_UNSET_KIND } from "./constants.ts";++/**+ * List settings whose entries are `name=value` lines holding one value per+ * name: a change of the value replaces the entry instead of adding one.+ */+export const KEYED_LIST_SETTINGS: ReadonlySet<string> = new Set([+ "brain.schema.prefixes",+ "brain.schema.expert_routing",+ "brain.schema.attributes",+ "brain.schema.frontmatter_tiers",+]);++/**+ * The keys that take `setting-entry` records: the schema field lists+ * `brain.schema.<field>` (design D19 item 1). Every other key holds one+ * value and stays on the plain register; an entry record for it is+ * rejected, so it can never turn a human value into a removed list.+ */+const LIST_SETTING_KEY_RE = /^brain\.schema\.[a-z][a-z0-9_]*$/;++/** Whether `key` is a list setting that folds per entry. */+export function isListSettingKey(key: string): boolean {+ return LIST_SETTING_KEY_RE.test(key);+}++/**+ * Keyed lists whose name is a compound `type.field` key: the pack parser+ * splits it at its first `.` and reads each half as a token, so+ * `paper . year` and `paper.year` name one key.+ */+const COMPOUND_KEYED_LIST_SETTINGS: ReadonlySet<string> = new Set([+ "brain.schema.attributes",+ "brain.schema.frontmatter_tiers",+]);++/**+ * One part of an entry in the form the schema pack parser reads it+ * (`../brain/schema-pack.ts`): surrounding quotes stripped, then the+ * parser's own `normalizeSchemaToken` (NFC, trimmed, lower case).+ */+function canonicalPart(raw: string): string {+ let text = raw.normalize("NFC").trim();+ if (+ text.length >= 2 &&+ ((text.startsWith('"') && text.endsWith('"')) || (text.startsWith("'") && text.endsWith("'")))+ ) {+ text = text.slice(1, -1);+ }+ return normalizeSchemaToken(text);+}++/** The name of a keyed entry: for a compound key each half normalised on its own, as the parser splits it. */+function keyedName(key: string, raw: string): string {+ const name = canonicalPart(raw);+ if (!COMPOUND_KEYED_LIST_SETTINGS.has(key)) return name;+ const halves = splitCompoundSchemaKey(name);+ return halves === null+ ? name+ : `${normalizeSchemaToken(halves[0])}.${normalizeSchemaToken(halves[1])}`;+}++/**+ * The register an entry of list setting `key` belongs to: the entry in+ * the normalised form the pack parser reads, so two spellings of one+ * token are one register. A `name=value` line is split at its first `=`+ * and each side normalised on its own (each side of a `->` pair too); for+ * a keyed list the slot is the normalised name alone. Quotes around the+ * whole line count as quotes around the line, not around its first and+ * last part, unless the quote character also stands inside it.+ */+export function settingEntrySlot(key: string, entry: string): string {+ const text = unquotedLine(entry.normalize("NFC").trim());+ const eq = text.indexOf("=");+ if (eq < 0) return canonicalPart(text);+ if (KEYED_LIST_SETTINGS.has(key)) return keyedName(key, text.slice(0, eq));+ const name = canonicalPart(text.slice(0, eq));+ const value = text+ .slice(eq + 1)+ .split("->")+ .map(canonicalPart)+ .join("->");+ return `${name}=${value}`;+}++/** `text` without the quotes around the whole of it, when they are its only ones. */+function unquotedLine(text: string): string {+ const quote = text[0];+ if (text.length < 2 || (quote !== '"' && quote !== "'") || !text.endsWith(quote)) return text;+ const inner = text.slice(1, -1);+ return inner.includes(quote) ? text : inner.trim();+}++/**+ * The body field of a conditional removal: the id of the record that+ * decided the entry's slot when the removal was judged. The removal+ * applies only while that record still decides the slot among the key's+ * unconditional records, so a replacement written meanwhile on another+ * device, delivered before or after the removal, is never removed by it.+ */+export const IF_DECIDED_BY_FIELD = "if_decided_by";++/** The condition of a `setting-entry` removal, or `undefined` for an unconditional record. */+function conditionOf(record: StoreRecord): string | undefined {+ const value = record.body[IF_DECIDED_BY_FIELD];+ return record.kind === SETTING_ENTRY_KIND && typeof value === "string" ? value : undefined;+}++/**+ * Why a `setting-entry` record is malformed, or `null`. Authority and key+ * are checked by the settings fold first.+ */+export function settingEntryRejection(record: StoreRecord): string | null {+ const entry = record.body["entry"];+ if (typeof entry !== "string" || entry.trim() === "" || /[\r\n]/.test(entry)) {+ return "entry must be one non-empty line";+ }+ if (typeof record.body["present"] !== "boolean") return "present must be true or false";+ const condition = record.body[IF_DECIDED_BY_FIELD];+ if (condition !== undefined) {+ if (typeof condition !== "string" || condition === "") {+ return `${IF_DECIDED_BY_FIELD} must be a record id`;+ }+ if (record.body["present"] !== false) return `${IF_DECIDED_BY_FIELD} is for a removal only`;+ }+ const key = String(record.body["key"]);+ if (!isListSettingKey(key)) return `${key} is not a list setting`;+ return null;+}++function isOneLineScalar(item: unknown): boolean {+ return ["string", "number", "boolean"].includes(typeof item) && !/[\r\n]/.test(String(item));+}++/**+ * Why the whole-list value of list setting `key` is malformed, or `null`:+ * it must be a list of one-line scalars, each an entry as a `setting-entry`+ * record would hold it. A scalar or an object in its place would hide the+ * stored list from every reader of the pack while deciding no entry of it,+ * and an element with a line break would read as several pack lines.+ */+export function listValueRejection(key: string, value: unknown): string | null {+ if (!Array.isArray(value)) return `${key} is a list setting: its value must be a list`;+ if (!value.every(isOneLineScalar))+ return `${key} must list one-line strings, numbers or booleans`;+ return null;+}++/** One list key folded per entry. */+export interface ListSettingFold {+ /** The key's register as the settings fold reports it; its value is the list. */+ readonly state: RegisterState<unknown>;+ /** Each slot's register: the entry text, or removed. */+ readonly slots: ReadonlyMap<string, RegisterState<string>>;+}++/**+ * Fold one list key from its valid `setting-set`, `setting-unset` and+ * `setting-entry` records, given in fold order; `null` when no record is+ * left once the conditional removals whose condition fails are dropped.+ * A conditional removal ({@link IF_DECIDED_BY_FIELD}) is judged against+ * the key's unconditional records alone, so the result does not depend on+ * the order removals arrive in.+ */+export function foldListSetting(+ key: string,+ records: ReadonlyArray<StoreRecord>,+): ListSettingFold | null {+ const plain = records.filter((record) => conditionOf(record) === undefined);+ if (plain.length === records.length) return foldSlots(key, records);+ const decided = plain.length === 0 ? null : foldSlots(key, plain);+ const kept = records.filter((record) => {+ const condition = conditionOf(record);+ if (condition === undefined) return true;+ const slot = settingEntrySlot(key, record.body["entry"] as string);+ return decided?.slots.get(slot)?.winner.record.id === condition;+ });+ return kept.length === 0 ? null : foldSlots(key, kept);+}++function foldSlots(key: string, records: ReadonlyArray<StoreRecord>): ListSettingFold {+ const universe = new Set<string>();+ for (const record of records) {+ for (const text of entriesOf(record)) universe.add(settingEntrySlot(key, text));+ }+ const registers = new RegisterSet<string>();+ /** `<record index>\u0000<slot>` -> place of the entry in that record. */+ const place = new Map<string, number>();+ const index = new Map<string, number>();+ records.forEach((record, i) => {+ index.set(record.id, i);+ const authority = recordAuthority(record)!;+ if (record.kind === SETTING_ENTRY_KIND) {+ const text = record.body["entry"] as string;+ const slot = settingEntrySlot(key, text);+ registers.offer(slot, {+ record,+ authority,+ value: record.body["present"] === true ? text : TOMBSTONE,+ });+ place.set(`${i}\u0000${slot}`, 0);+ return;+ }+ const listed = new Map<string, string>();+ entriesOf(record).forEach((text, at) => {+ const slot = settingEntrySlot(key, text);+ listed.set(slot, text);+ place.set(`${i}\u0000${slot}`, at);+ });+ for (const slot of universe) {+ registers.offer(slot, { record, authority, value: listed.get(slot) ?? TOMBSTONE });+ }+ });+ const slots = new Map<string, RegisterState<string>>();+ for (const slot of registers.keys()) slots.set(slot, registers.get(slot)!);+ const decided = (slot: string, state: RegisterState<string>): [number, number] => {+ const i = index.get(state.winner.record.id)!;+ return [i, place.get(`${i}\u0000${slot}`) ?? 0];+ };+ const present = [...slots.entries()]+ .filter(([, state]) => !state.removed)+ .map(([slot, state]) => ({ text: state.value!, at: decided(slot, state) }))+ .toSorted((a, b) => a.at[0] - b.at[0] || a.at[1] - b.at[1]);+ // The record that decided last stands for the whole key (its authority+ // and id are what a settings view reports). Records that name no entry+ // at all (empty whole lists, unsets) decide no slot: the key is then won+ // as the whole-list register would win it.+ let winner = slots.size === 0 ? wholeListWinner(records) : [...slots.values()][0]!.winner;+ for (const state of slots.values()) {+ if (index.get(state.winner.record.id)! > index.get(winner.record.id)!) winner = state.winner;+ }+ const history = records.map((record) => ({+ record,+ authority: recordAuthority(record)!,+ value: record.kind === SETTING_UNSET_KIND ? TOMBSTONE : rawValue(record),+ }));+ const removed = present.length === 0;+ return {+ state: {+ value: removed ? null : present.map((p) => p.text),+ removed,+ winner: { record: winner.record, authority: winner.authority, value: winner.value },+ history,+ },+ slots,+ };+}++/** The winner of `records` (in fold order) read as one register over the whole list. */+function wholeListWinner(records: ReadonlyArray<StoreRecord>): RegisterState<string>["winner"] {+ const register = new RegisterSet<string>();+ for (const record of records) {+ register.offer("", { record, authority: recordAuthority(record)!, value: TOMBSTONE });+ }+ return register.get("")!.winner;+}++/** The entries a record names: a whole list's elements, or its one entry. */+function entriesOf(record: StoreRecord): string[] {+ if (record.kind === SETTING_ENTRY_KIND) return [record.body["entry"] as string];+ if (record.kind === SETTING_UNSET_KIND) return [];+ const value = record.body["value"];+ return Array.isArray(value) ? value.map(String) : [];+}++function rawValue(record: StoreRecord): unknown {+ if (record.kind === SETTING_ENTRY_KIND) {+ return { entry: record.body["entry"], present: record.body["present"] };+ }+ return record.body["value"];+}diff --git a/src/core/store/fold-settings.ts b/src/core/store/fold-settings.tsindex 51568855..e0c95644 100644--- a/src/core/store/fold-settings.ts+++ b/src/core/store/fold-settings.ts@@ -10,8 +10,14 @@* - `setting-set` `{ authority, key, value }`: proposes `value` for `key`.* - `setting-unset` `{ authority, key }`: a tombstone; the key falls back* to its default.+ * - `setting-entry` `{ authority, key, entry, present }`: adds or removes+ * one entry of a schema field list `brain.schema.<field>` (task 3.12);+ * a key that holds one folds per entry (`./fold-setting-lists.ts`). An+ * entry record for any other key is rejected, and so is a `setting-set`+ * of a list key whose value is no list: it would hide the stored list+ * from every reader of the pack while deciding no entry of it.*- * Each key is one register, resolved by authority, then clock. Keys this+ * Each other key is one register, resolved by authority, then clock. Keys this* version knows are validated (see {@link SETTING_RULES}); a record with* an invalid value is rejected and does not take effect. Unknown keys* (written by a newer release) fold as plain JSON values, so an older@@ -19,6 +25,7 @@*/import {+ SETTING_ENTRY_KIND,SETTING_SET_KIND,SETTING_TOPIC_MIN_RECORDS,SETTING_TOPIC_MIN_SESSIONS,@@ -28,6 +35,12 @@ import {type SettingKey,} from "./constants.ts";import type { StoreRecord } from "./envelope.ts";+import {+ foldListSetting,+ isListSettingKey,+ listValueRejection,+ settingEntryRejection,+} from "./fold-setting-lists.ts";import {prepareRecords,recordAuthority,@@ -72,6 +85,8 @@ function ruleOf(key: string): SettingRule | undefined {export interface SettingsFold {/** Resolved keys, inserted in code unit order. */readonly settings: ReadonlyMap<string, RegisterState<unknown>>;+ /** Per-entry registers of the list keys that hold a `setting-entry` record. */+ readonly lists: ReadonlyMap<string, ReadonlyMap<string, RegisterState<string>>>;readonly rejected: ReadonlyArray<RejectedRecord>;}@@ -80,7 +95,9 @@ function rejection(record: StoreRecord): string | null {const key = record.body["key"];if (typeof key !== "string" || !SETTING_KEY_RE.test(key)) return "missing or invalid key";if (record.kind === SETTING_UNSET_KIND) return null;+ if (record.kind === SETTING_ENTRY_KIND) return settingEntryRejection(record);if (!("value" in record.body) || record.body["value"] === null) return "value is required";+ if (isListSettingKey(key)) return listValueRejection(key, record.body["value"]);const rule = ruleOf(key);if (rule !== undefined && !rule.valid(record.body["value"])) {return `${key} must be ${rule.expected}`;@@ -91,25 +108,49 @@ function rejection(record: StoreRecord): string | null {/** Fold setting records (any order, duplicates allowed). */export function foldSettings(input: Iterable<StoreRecord>): SettingsFold {const { records } = prepareRecords(input);- const registers = new RegisterSet<unknown>();+ const valid: StoreRecord[] = [];+ const listKeys = new Set<string>();const rejected: RejectedRecord[] = [];for (const record of records) {- if (record.kind !== SETTING_SET_KIND && record.kind !== SETTING_UNSET_KIND) continue;+ if (+ record.kind !== SETTING_SET_KIND &&+ record.kind !== SETTING_UNSET_KIND &&+ record.kind !== SETTING_ENTRY_KIND+ )+ continue;const reason = rejection(record);if (reason !== null) {rejected.push({ id: record.id, kind: record.kind, reason });continue;}+ valid.push(record);+ if (record.kind === SETTING_ENTRY_KIND) listKeys.add(record.body["key"] as string);+ }+ const registers = new RegisterSet<unknown>();+ const byListKey = new Map<string, StoreRecord[]>();+ for (const record of valid) {+ const key = record.body["key"] as string;+ if (listKeys.has(key)) {+ const list = byListKey.get(key) ?? [];+ list.push(record);+ byListKey.set(key, list);+ continue;+ }const value = record.kind === SETTING_UNSET_KIND ? TOMBSTONE : record.body["value"];- registers.offer(record.body["key"] as string, {- record,- authority: recordAuthority(record)!,- value,- });+ registers.offer(key, { record, authority: recordAuthority(record)!, value });+ }+ const resolved = new Map<string, RegisterState<unknown>>();+ for (const key of registers.keys()) resolved.set(key, registers.get(key)!);+ const lists = new Map<string, ReadonlyMap<string, RegisterState<string>>>();+ for (const [key, list] of byListKey) {+ const folded = foldListSetting(key, list);+ if (folded === null) continue;+ resolved.set(key, folded.state);+ lists.set(key, folded.slots);}const settings = new Map<string, RegisterState<unknown>>();- for (const key of registers.keys()) settings.set(key, registers.get(key)!);- return { settings, rejected };+ for (const key of [...resolved.keys()].toSorted()) settings.set(key, resolved.get(key)!);+ return { settings, lists, rejected };}/**diff --git a/src/core/store/lane-dual-moves.ts b/src/core/store/lane-dual-moves.tsnew file mode 100644index 00000000..3de9b52e--- /dev/null+++ b/src/core/store/lane-dual-moves.ts@@ -0,0 +1,374 @@+/**+ * The records of a lane removal or move, and their dual write on an+ * unswitched machine (task 3.12 review, design D19 item 2).+ *+ * With the phase 1 dual write on, a lane page an unswitched machine writes+ * is also a `lane-entry` record in the store (`./lane-pages.ts`, through+ * `routeWrite`). A removal or a move of that page used to touch the file+ * alone, so the record stayed live: once the machine switched, the removed+ * page (a persona, a pledge, a staged page) was back, and a moved page was+ * in both places. A removal now dual-writes the stored entry's tombstone.+ *+ * A move is the write of the destination plus the removal of the source,+ * judged as those two writes would be judged, in both modes and for a page+ * and a folder alike (`tests/core/lanes/lane-move-table.test.ts` checks+ * the rule over every state a move can start from):+ *+ * - the destination ({@link destinationRecords}): the entry a write of+ * the content that moved leaves there, or, when nothing the store would+ * import arrived (a link), the removal of the entry held there;+ * - the source: its entry's tombstone where that entry lives, last.+ *+ * A move that involves no stored entry (an unswitched rename of a page the+ * store never saw onto a path it holds nothing at) writes nothing: the+ * legacy-only path is unchanged, and the migration imports the page.+ *+ * A move whose source is a lane page (or a folder holding stored entries)+ * and whose destination is no lane entry, outside `Brain/` or not in a+ * lane folder, is refused before anything moves, in both modes+ * ({@link assertLaneDestination}): a store-only machine has no file to+ * leave there, so its tombstones alone would lose the page.+ *+ * Every record of one call is built from one fold of the lane store, so an+ * entry's value and the partition it lives in cannot come from two+ * different reads. The destination write:+ *+ * - writes nothing when the source and the destination are one entry+ * (the same path, or one that normalises to it): a rename onto itself+ * changes nothing, and a tombstone after the entry would remove it;+ * - builds the new entry from the content that moved: on an unswitched+ * machine the page as it is on disk after the rename (a legacy-only+ * edit the store never saw included), on a store-only one the stored+ * value;+ * - puts the new entry in `shared` unless the source entry lives in+ * `private`, the moved page is private by its own content, the entry+ * or legacy page it replaces is private (it lives in `private` or its+ * fields reserve it: nothing lowers an entry's partition, as for a write+ * over it) or the fold may miss a record (an unreadable shard, or a+ * body blob in `private` that has not arrived: `laneFoldIncomplete`, fail closed as+ * every lane privacy verdict); each entry is placed by itself, so one private page+ * of a folder does not raise the others;+ * - when it replaces a destination entry that lives in `shared` with a+ * private one, writes that entry's shared tombstone first, so a device+ * that holds only `shared` does not keep reading the replaced page.+ *+ * A folder move whose folders overlap, the destination inside the source+ * or the source inside the destination, is refused before anything moves,+ * in both modes ({@link assertFoldersApart}): the rename an unswitched+ * machine runs refuses both (a folder cannot move into itself, and the+ * folder that holds the source is never empty), so a store-only machine+ * refuses them too and both modes end in the same state. Within one+ * folder move, therefore, no destination entry is the source entry of+ * another, and no tombstone of the batch can remove an entry the batch+ * wrote. A folder moved onto itself changes nothing.+ *+ * All the records of one call are one store operation (`dualWrite`,+ * `storeWrite`): the partitions are checked before the first append.+ */++import { closeSync, readFileSync } from "node:fs";+import { isAbsolute, join, relative, sep } from "node:path";++import { BRAIN_ROOT_REL } from "../brain/path-constants.ts";+import { containedRecordPath } from "../record-path.ts";+import type { StorePartition } from "./constants.ts";+import { dualWrite, type DualWriteContext, type DualWriteRecord } from "./dual-write.ts";+import {+ brainRelOf,+ entryRecord,+ insideBrain,+ isPrivateEntry,+ storeEntries,+ tombstoneOf,+} from "./lane-entry-store.ts";+import { laneFoldIncomplete, laneStoreFold, type LaneFolded } from "./lane-fold-cache.ts";+import { brainRelOfEntry, lanePlaceOf } from "./lane-paths.ts";+import { laneTextParts, rawToKeep } from "./lane-raw.ts";+import { laneEntryKey, type LaneEntryValue } from "./lane-records.ts";+import { isPrivateFrontmatter } from "./private-page.ts";+import { openRegularFileForRead } from "./safe-fs.ts";++/**+ * The absolute paths of every live store entry below `dir`, at any depth,+ * sorted, whatever the machine's mode (the store has no directories; a+ * folder is a path prefix). A path outside `Brain/` is never listed.+ */+export function storedLanePathsBelow(vault: string, dir: string, ext: string): string[] {+ const prefix = brainRelOf(vault, dir);+ const brain = join(vault, BRAIN_ROOT_REL);+ const out: string[] = [];+ for (const value of storeEntries().values()) {+ const rel = brainRelOfEntry(value.lane, value.entry);+ if (!rel.startsWith(prefix === "" ? "" : `${prefix}/`) || !rel.endsWith(ext)) continue;+ // The fold already refuses an entry id that is no record path; this+ // keeps a path outside `Brain/` out of every reader (and remover) of+ // the list even if that check is ever loosened (FORMAT.md, Trust).+ const path = containedRecordPath(brain, rel);+ if (path !== null) out.push(path);+ }+ return out.toSorted();+}++/** A live entry of one fold, with its key and the partition it lives in. */+interface FoldedEntry {+ readonly key: string;+ readonly value: LaneEntryValue;+ readonly partition: StorePartition;+}++/** The entry key of the lane path `path`, or `null` for a path that is no lane entry. */+function laneKeyAt(vault: string, path: string): string | null {+ if (!insideBrain(vault, path)) return null;+ const place = lanePlaceOf(brainRelOf(vault, path));+ return place === null ? null : laneEntryKey(place.lane, place.entry);+}++/** The live entry of `fold` at the lane path `path`, or `null`. */+function foldedAt(fold: LaneFolded, vault: string, path: string): FoldedEntry | null {+ const key = laneKeyAt(vault, path);+ if (key === null) return null;+ const value = fold.entries.get(key);+ if (value === undefined) return null;+ return { key, value, partition: fold.privateKeys.has(key) ? "private" : "shared" };+}++/** One move of a lane page: `destinationPrivate` when the legacy page it renamed over was private. */+export interface LaneMove {+ readonly from: string;+ readonly to: string;+ readonly destinationPrivate?: boolean;+}++/** Whether `a` and `b` name one lane entry (the same path, or one that normalises to it). */+export function sameLaneEntry(vault: string, a: string, b: string): boolean {+ const key = laneKeyAt(vault, a);+ return key !== null && key === laneKeyAt(vault, b);+}++/**+ * Refuse, before anything moves, a move whose source is a lane entry and+ * whose destination is none (see the module docblock). For a folder,+ * `moves` are the entries it carries (or a page path below it standing for+ * them); a folder moved out of `Brain/` is refused whatever it holds.+ */+export function assertLaneDestination(+ vault: string,+ from: string,+ to: string,+ moves: ReadonlyArray<LaneMove> = [{ from, to }],+): void {+ const leaves =+ (insideBrain(vault, from) && !insideBrain(vault, to)) ||+ moves.some(+ (move) => laneKeyAt(vault, move.from) !== null && laneKeyAt(vault, move.to) === null,+ );+ if (leaves)+ throw new Error(`cannot move a lane page or folder out of the Brain lanes: ${from} -> ${to}`);+}++/** Whether `inner` lies strictly below the folder `outer`. */+function strictlyBelow(outer: string, inner: string): boolean {+ const rel = relative(outer, inner);+ return rel !== "" && rel !== ".." && !rel.startsWith(`..${sep}`) && !isAbsolute(rel);+}++/** Refuse a folder move whose folders overlap, one inside the other (see the module docblock). */+export function assertFoldersApart(from: string, to: string): void {+ if (strictlyBelow(from, to)) {+ throw new Error(`cannot move a lane folder into itself: ${from} -> ${to}`);+ }+ if (strictlyBelow(to, from)) {+ throw new Error(`cannot move a lane folder into a folder that holds it: ${from} -> ${to}`);+ }+}++/** Whether a write over `entry` keeps it private: it lives in `private`, or its fields reserve it. */+function keepsPrivate(entry: FoldedEntry | null): boolean {+ return (+ entry !== null && (entry.partition === "private" || isPrivateFrontmatter(entry.value.fields))+ );+}++/** What a move carries to its destination. */+export interface MovedContent {+ readonly fields: Readonly<Record<string, unknown>>;+ readonly text: string;+ readonly raw?: string;+ /** Judge the content's own privacy (a page read from disk), not only where it lived. */+ readonly judged: boolean;+}++/** The content a store-only move carries: the stored value as it is. */+export function storedContent(value: LaneEntryValue): MovedContent {+ return {+ fields: value.fields,+ text: value.text,+ ...(value.raw !== undefined ? { raw: value.raw } : {}),+ judged: false,+ };+}++/**+ * The content an unswitched move carried: the page at `to` as it is on+ * disk after the rename, or `null` when there is none there (the page was+ * removed without the store seeing it) or what is there is no page the+ * store would import. As the migration's legacy scan, a symbolic link or+ * special file is never followed or read, nor is a file reached through a+ * linked folder below `Brain/` (`openRegularFileForRead`), so a link moved+ * as a lane page cannot carry a file from outside the vault into the store.+ */+function movedFile(vault: string, to: string): MovedContent | null {+ let fd: number | null;+ try {+ fd = openRegularFileForRead(to, join(vault, BRAIN_ROOT_REL));+ } catch (err) {+ if ((err as NodeJS.ErrnoException).code === "ENOENT") return null;+ throw err;+ }+ if (fd === null) return null;+ let text: string;+ try {+ text = readFileSync(fd, "utf8");+ } finally {+ closeSync(fd);+ }+ const { fields, body } = laneTextParts(to, text);+ const raw = rawToKeep(to, text, fields, body);+ return { fields, text: body, ...(raw !== undefined ? { raw } : {}), judged: true };+}++/**+ * The records of the write of what is now at `move.to` over the entry+ * `fold` holds there: the write of the moved page, judged as a write over+ * that entry is (see the module docblock), with `sourcePrivate` carrying+ * the moved entry's own partition; or, when nothing the store would import+ * is there, the held entry's tombstone, as a removal of it.+ */+function destinationRecords(+ vault: string,+ fold: LaneFolded,+ move: LaneMove,+ moved: MovedContent | null,+ sourcePrivate: boolean,+): DualWriteRecord[] {+ const replaced = foldedAt(fold, vault, move.to);+ if (moved === null) {+ return replaced === null ? [] : [tombstoneOf(replaced.value, false, replaced.partition)];+ }+ const isPrivate =+ sourcePrivate ||+ laneFoldIncomplete(fold) ||+ move.destinationPrivate === true ||+ keepsPrivate(replaced) ||+ (moved.judged && isPrivateEntry(vault, moved.fields, moved.text, moved.raw));+ const partition: StorePartition = isPrivate ? "private" : "shared";+ return [+ ...(isPrivate && replaced?.partition === "shared"+ ? [tombstoneOf(replaced.value, false, "shared")]+ : []),+ {+ ...entryRecord(vault, move.to, moved.fields, moved.text, isPrivate, "located", moved.raw),+ located: { partition, content: true },+ },+ ];+}++/**+ * The records of moving the entry at `from` to `to`, read from `fold`+ * (see the module docblock): the write of the destination+ * ({@link destinationRecords}) and the removal of the source. `content`+ * gives what moved (`null`: nothing importable). A destination that is no+ * lane entry writes the removal alone (callers refuse one first,+ * {@link assertLaneDestination}).+ */+export function laneMoveRecords(+ vault: string,+ fold: LaneFolded,+ move: LaneMove,+ content: (value: LaneEntryValue) => MovedContent | null,+): DualWriteRecord[] {+ const source = foldedAt(fold, vault, move.from);+ if (source === null) return [];+ const target = laneKeyAt(vault, move.to);+ if (target === source.key) return [];+ const gone = tombstoneOf(source.value, false, source.partition);+ if (target === null) return [gone];+ const sourcePrivate = source.partition === "private";+ return [...destinationRecords(vault, fold, move, content(source.value), sourcePrivate), gone];+}++/**+ * The dual-written records of one unswitched rename: a stored source moves+ * ({@link laneMoveRecords}, the content from disk); a page the store never+ * saw is the write of the destination alone, and nothing at all when the+ * store holds no entry there either (the legacy-only path is unchanged).+ */+function dualMoveRecords(vault: string, fold: LaneFolded, move: LaneMove): DualWriteRecord[] {+ if (foldedAt(fold, vault, move.from) !== null) {+ return laneMoveRecords(vault, fold, move, () => movedFile(vault, move.to));+ }+ if (foldedAt(fold, vault, move.to) === null) return [];+ return destinationRecords(vault, fold, move, movedFile(vault, move.to), false);+}++/** The tombstones of the live entries of `fold` at `paths`, each where its entry lives. */+export function laneRemovalRecords(+ vault: string,+ fold: LaneFolded,+ paths: ReadonlyArray<string>,+): DualWriteRecord[] {+ return paths.flatMap((path) => {+ const entry = foldedAt(fold, vault, path);+ return entry === null ? [] : [tombstoneOf(entry.value, false, entry.partition)];+ });+}++/** Dual-write the tombstones of the stored entries at `paths` (an unswitched removal). */+export function dualWriteLaneRemoval(+ vault: string,+ paths: () => ReadonlyArray<string>,+ context: DualWriteContext,+): void {+ dualWrite(() => laneRemovalRecords(vault, laneStoreFold(), paths()), context);+}++/** Dual-write an unswitched rename of one lane page. */+export function dualWriteLaneMove(vault: string, move: LaneMove, context: DualWriteContext): void {+ dualWrite(() => dualMoveRecords(vault, laneStoreFold(), move), context);+}++/**+ * The paths below `dir`, relative to it with `/` separators, of the live+ * entries of `fold`. Judged from the entry ids alone, never from the disk:+ * a link where an entry's file was must not hide the entry.+ */+function foldedBelow(fold: LaneFolded, vault: string, dir: string): string[] {+ const rel = brainRelOf(vault, dir);+ const prefix = rel === "" ? "" : `${rel}/`;+ const out: string[] = [];+ for (const value of fold.entries.values()) {+ const path = brainRelOfEntry(value.lane, value.entry);+ if (path.startsWith(prefix)) out.push(path.slice(prefix.length));+ }+ return out;+}++/**+ * Dual-write an unswitched rename of the folder `from` to `to`: each stored+ * entry below `from` moved, and each stored entry below `to` overwritten+ * by what the rename put there, one move per path below either folder.+ */+export function dualWriteLaneTreeMove(+ vault: string,+ from: string,+ to: string,+ context: DualWriteContext,+): void {+ dualWrite(() => {+ const fold = laneStoreFold();+ const below = new Set([...foldedBelow(fold, vault, from), ...foldedBelow(fold, vault, to)]);+ return [...below]+ .toSorted()+ .flatMap((rel) => dualMoveRecords(vault, fold, { from: join(from, rel), to: join(to, rel) }));+ }, context);+}diff --git a/src/core/store/lane-entry-store.ts b/src/core/store/lane-entry-store.tsindex 37f22af9..441011fa 100644--- a/src/core/store/lane-entry-store.ts+++ b/src/core/store/lane-entry-store.ts@@ -14,7 +14,7 @@ import { isAbsolute, join, relative, sep } from "node:path";import { BRAIN_ROOT_REL } from "../brain/path-constants.ts";import { canonicalJson } from "../integrity/digest.ts";import type { FrontmatterMap } from "../types.ts";-import { laneStoreFold } from "./lane-fold-cache.ts";+import { laneFoldIncomplete, laneStoreFold } from "./lane-fold-cache.ts";import { laneNameRefusal, lanePlaceOf } from "./lane-paths.ts";import {laneEntryKey,@@ -101,12 +101,14 @@ export function storeEntry(vault: string, path: string): LaneEntryValue | null {}/**- * Whether the lane fold misses an `events` shard (unreadable, or gone- * since this machine read it): an entry's private records may be in it,- * so every privacy verdict over the fold fails closed (design D14).+ * Whether the lane fold may miss a record ({@link laneFoldIncomplete}: an+ * `events` shard unreadable or gone since this machine read it, or a+ * record in `private` whose body blob has not arrived): an entry's private records may+ * be among them, so every privacy verdict over the fold fails closed+ * (design D14).*/export function foldIncomplete(): boolean {- return laneStoreFold().unreadable.length > 0;+ return laneFoldIncomplete(laneStoreFold());}/** Whether the live entry `value` was written to the `private` partition. */@@ -119,8 +121,9 @@ export function livesInPrivate(value: LaneEntryValue): boolean {* `private` partition. An entry can be private by partition alone, with no* `visibility:` in its fields, so an exporter that turns entries back into* pages asks here to keep the privacy (review finding F333). A path the- * store does not hold is no private entry. While the fold misses a shard,- * every lane entry reads as private (fail closed).+ * store does not hold is no private entry. While the fold may miss a+ * record ({@link foldIncomplete}), every lane entry reads as private (fail+ * closed).*/export function laneEntryInPrivate(vault: string, path: string): boolean {if (!heldInStore(vault, path)) return false;@@ -140,10 +143,14 @@ export function partitionOf(value: LaneEntryValue): StorePartition {* whatever the rest of its operation is (`Located`, content-free): a* device that holds only `shared` must see a shared entry's tombstone.* `onlyIfApart`: the tombstone of an entry a write replaces, needed only- * when the replacement lands in another partition.+ * when the replacement lands in another partition. `partition`: where the+ * entry lives, when the caller read it from the same fold as `value`.*/-export function tombstoneOf(value: LaneEntryValue, onlyIfApart = false): DualWriteRecord {- const partition = partitionOf(value);+export function tombstoneOf(+ value: LaneEntryValue,+ onlyIfApart = false,+ partition: StorePartition = partitionOf(value),+): DualWriteRecord {return {...laneEntryRemoveRecord({lane: value.lane,@@ -275,8 +282,8 @@ export function legacyPagePrivate(vault: string, path: string): boolean {}/**- * See {@link legacyPagePrivate}. While the fold misses a shard, the entry- * may be private in it: the write is judged private (on a machine without+ * See {@link legacyPagePrivate}. While the fold may miss a record+ * ({@link foldIncomplete}), the entry may be private in it: the write is judged private (on a machine without* the `private` partition, refused).*/export function storedEntryPrivate(vault: string, path: string): boolean {diff --git a/src/core/store/lane-fold-cache.ts b/src/core/store/lane-fold-cache.tsindex e2508774..ed6ae292 100644--- a/src/core/store/lane-fold-cache.ts+++ b/src/core/store/lane-fold-cache.ts@@ -62,6 +62,28 @@ export interface LaneFolded {readonly pendingBlobs: ReadonlyArray<string>;}+/**+ * Whether `fold` may miss a record that makes a lane entry private: an+ * `events` shard it could not see (in either partition), or a record in+ * `private` whose body blob has not arrived (quarantined on this machine+ * or not; a quarantine lets the shard read as whole, yet the record is+ * still left out). Every lane privacy verdict over such a fold fails+ * closed: the one test of "the fold is complete enough to place in+ * `shared`" for every lane write, move and privacy read.+ *+ * A pending blob in `shared` is no such gap. A record in `shared` never+ * carries a value a write would place in `private` (every writer judges+ * the fields first), and leaving one record out changes an entry's+ * winner only when it is that winner, so the fold without it is at least+ * as private as the fold with it. So a quarantined blob in `shared` does+ * not block lane writes on a machine that holds only `shared`.+ */+export function laneFoldIncomplete(fold: LaneFolded): boolean {+ return (+ fold.unreadable.length > 0 || fold.pendingBlobs.some((blob) => blob.startsWith("private/"))+ );+}+const EMPTY_FOLD: LaneFolded = {fingerprint: "",entries: new Map(),diff --git a/src/core/store/lane-pages.ts b/src/core/store/lane-pages.tsindex f47f8ef1..103bb8e9 100644--- a/src/core/store/lane-pages.ts+++ b/src/core/store/lane-pages.ts@@ -15,7 +15,10 @@* byte is written under `<vault>/Brain/`, and no vault is needed.* - **Unswitched machine**: the page is the legacy file, written exactly* as before, with the phase 1 dual write behind the flag- * ({@link routeWrite}).+ * ({@link routeWrite}). A removal or a move dual-writes too: the+ * stored entry's tombstone, and its entry at the new path+ * (`./lane-dual-moves.ts`), so a page removed before the switch is not+ * live again after it.** Privacy (phase 1 rules): a page is private when its frontmatter* reserves it, or when its fields or text name a private page or one that@@ -41,7 +44,6 @@ import {renameWithRetry,} from "../fs-atomic.ts";import { noVaultRoot } from "../platform-dirs.ts";-import { containedRecordPath } from "../record-path.ts";import type { FrontmatterMap } from "../types.ts";import {parseFrontmatter,@@ -49,13 +51,13 @@ import {writeFrontmatterAtomic,type WriteFrontmatterAtomicOptions,} from "../vault.ts";+import { assertStoreContentWrite } from "./content-write-gates.ts";import { type DualWriteContext } from "./dual-write.ts";import { laneLockTarget, laneStoreFold } from "./lane-fold-cache.ts";import {brainRelOf,entryRecord,fieldsCopy,- foldIncomplete,heldInStore,insideBrain,isPrivateEntry,@@ -63,7 +65,6 @@ import {legacyPagePrivate,livesInPrivate,lanePagesInStore,- partitionOf,replacedShared,sameEntry,storeEntries,@@ -71,6 +72,19 @@ import {storedEntryPrivate,tombstoneOf,} from "./lane-entry-store.ts";+import {+ assertFoldersApart,+ assertLaneDestination,+ dualWriteLaneMove,+ dualWriteLaneRemoval,+ dualWriteLaneTreeMove,+ laneMoveRecords,+ sameLaneEntry,+ type LaneMove,+ laneRemovalRecords,+ storedContent,+ storedLanePathsBelow,+} from "./lane-dual-moves.ts";import { rawParsesBack, rawToKeep, renderLaneText } from "./lane-raw.ts";import { brainRelOfEntry, lanePlaceOf } from "./lane-paths.ts";import { acquireDirLock, STORE_LOCK_STALE_MS, withLocalLock } from "./local-lock.ts";@@ -92,12 +106,15 @@ export function laneVaultRoot(vault: string | null | undefined): string {/*** The write guard of a lane write: the vault identity guard (freeze,- * migration gate, identity marker) on an unswitched machine; nothing on a- * store-only one, where the store write checks the store identity itself- * (design D14 item 13).+ * migration gate, identity marker) on an unswitched machine. On a+ * store-only one the store write checks the store identity in place of+ * the vault marker (design D14 item 13), and the freeze still holds+ * (`./content-write-gates.ts`); a migration gate left naming the vault is+ * a leftover there and refuses nothing.*/export function assertLaneWrite(vault: string): void {- if (!lanePagesInStore(vault)) assertVaultIdentityForWrite(vault);+ if (lanePagesInStore(vault)) assertStoreContentWrite(vault);+ else assertVaultIdentityForWrite(vault);}/** `routeWrite` for a lane path; a path outside `Brain/` is written as a file. */@@ -312,19 +329,7 @@ export function vaultOfBrainPath(path: string): string {*/export function listLaneTree(vault: string, dir: string, ext = ".md"): string[] {if (!heldInStore(vault, dir)) return [];- const prefix = brainRelOf(vault, dir);- const brain = join(laneVaultRoot(vault), BRAIN_ROOT_REL);- const out: string[] = [];- for (const value of storeEntries().values()) {- const rel = brainRelOfEntry(value.lane, value.entry);- if (!rel.startsWith(prefix === "" ? "" : `${prefix}/`) || !rel.endsWith(ext)) continue;- // The fold already refuses an entry id that is no record path; this- // keeps a path outside `Brain/` out of every reader (and remover) of- // the list even if that check is ever loosened (FORMAT.md, Trust).- const path = containedRecordPath(brain, rel);- if (path !== null) out.push(path);- }- return out.toSorted();+ return storedLanePathsBelow(laneVaultRoot(vault), dir, ext);}/**@@ -483,13 +488,16 @@ export function removeLanePage(vault: string, path: string): boolean {return true;}guardUserVaultWrite(vault, path);+ let removed = true;try {rmSync(path);- return true;} catch (err) {- if ((err as NodeJS.ErrnoException).code === "ENOENT") return false;- throw err;+ if ((err as NodeJS.ErrnoException).code !== "ENOENT") throw err;+ removed = false;}+ // The entry the dual write put in the store goes too (`./lane-dual-moves.ts`).+ dualWriteLaneRemoval(vault, () => [path], context(vault));+ return removed;}/**@@ -508,6 +516,7 @@ export function moveLanePage(to: string,rewrite?: { readonly frontmatter: FrontmatterMap; readonly body: string },): void {+ assertLaneDestination(vault, from, to);if (!heldInStore(vault, from)) {if (rewrite !== undefined) {writeLanePage(vault, to, rewrite.frontmatter, rewrite.body, {@@ -518,23 +527,31 @@ export function moveLanePage(return;}guardUserVaultWrite(vault, from);+ // Read before the rename replaces it, as a write over the page reads it.+ const destinationPrivate = legacyPagePrivate(vault, to);mkdirSync(dirname(to), { recursive: true });renameWithRetry(from, to);+ dualWriteLaneMove(vault, { from, to, destinationPrivate }, context(vault));return;}const value = storeEntry(vault, from);if (value === null) throw new Error(`no lane page to move: ${from}`);+ // A move onto the entry it moves changes nothing.+ if (rewrite === undefined && sameLaneEntry(vault, from, to)) return;// The tombstone stays where the entry lives: it must reach every device// that saw the entry. The new entry stays there too, unless a rewrite- // whose page is private raises it to `private`, as does a fold that- // misses a shard (the entry may have been raised in it).+ // whose page is private raises it to `private`, as does an entry it+ // replaces that is private (nothing lowers an entry's partition) and a+ // fold that may miss a record (the entry may have been raised in it).const isPrivate =livesInPrivate(value) ||- foldIncomplete() ||+ storedEntryPrivate(vault, to) ||(rewrite !== undefined && isPrivateEntry(vault, rewrite.frontmatter, rewrite.body));withOperationPrivacy(isPrivate, () =>storeWrite(() => [+ // A shared entry this move replaces with a private one goes first.+ ...replacedShared(vault, to),tombstoneOf(value),entryRecord(vault,@@ -578,46 +595,35 @@ export function listLaneDirs(vault: string, dir: string): string[] {* Move the folder `from` (every lane file below it) to `to`. A rename on an* unswitched machine; on a store-only one each entry's tombstone and its* new entry, all in one store write, each in the entry's own partition.+ * Folders that overlap, one inside the other, are refused in both modes+ * before anything moves; a folder moved onto itself changes nothing.*/export function moveLaneTree(vault: string, from: string, to: string): void {+ const movedTo = (path: string): LaneMove => ({ from: path, to: join(to, relative(from, path)) });+ // Judged from the paths alone, so a legacy move reads no store first.+ assertLaneDestination(vault, from, to, [movedTo(join(from, "page.md"))]);+ if (relative(from, to) === "") return;+ assertFoldersApart(from, to);if (!heldInStore(vault, from)) {guardUserVaultWrite(vault, from);mkdirSync(dirname(to), { recursive: true });renameWithRetry(from, to);+ dualWriteLaneTreeMove(vault, from, to, context(vault));return;}- const moves = listLaneTree(vault, from, "").map((path) => {- const value = storeEntry(vault, path)!;- return { value, target: join(to, relative(from, path)) };- });- if (moves.length === 0) return;- // One store write, but every entry is placed by itself (design D14 item- // 18): a shared entry and its tombstone stay in `shared`, so a device- // that holds only `shared` sees it leave the folder, and a private- // entry's text never reaches `shared`. An enclosing private operation- // still raises the moved entries, never their tombstones.- storeWrite(- () =>- moves.flatMap(({ value, target }) => {- const partition = partitionOf(value);- return [- {- ...entryRecord(- vault,- target,- value.fields,- value.text,- partition === "private",- "located",- value.raw,- ),- located: { partition, content: true },- },- tombstoneOf(value),- ];- }),- context(vault),- );+ const paths = listLaneTree(vault, from, "");+ if (paths.length === 0) return;+ assertLaneDestination(vault, from, to, paths.map(movedTo));+ // One store write, every record built from one fold (`laneMoveRecords`),+ // each entry placed by itself (design D14 item 18): a shared entry and+ // its tombstone stay in `shared`, so a device that holds only `shared`+ // sees it leave the folder, and a private entry's text never reaches+ // `shared`. An enclosing private operation still raises the moved+ // entries, never their tombstones.+ storeWrite(() => {+ const fold = laneStoreFold();+ return paths.flatMap((path) => laneMoveRecords(vault, fold, movedTo(path), storedContent));+ }, context(vault));}/** Remove the folder `dir` and every lane file below it; absent is a no-op. */@@ -625,12 +631,13 @@ export function removeLaneTree(vault: string, dir: string): void {if (!heldInStore(vault, dir)) {guardUserVaultWrite(vault, dir);rmSync(dir, { recursive: true, force: true });+ dualWriteLaneRemoval(vault, () => storedLanePathsBelow(vault, dir, ""), context(vault));return;}// One store write, so a folder is never half removed; each tombstone in// the partition its entry lives in, so a device that holds only `shared`// sees the folder's shared entries go.- const values = listLaneTree(vault, dir, "").map((path) => storeEntry(vault, path)!);- if (values.length === 0) return;- storeWrite(() => values.map((value) => tombstoneOf(value)), context(vault));+ const paths = listLaneTree(vault, dir, "");+ if (paths.length === 0) return;+ storeWrite(() => laneRemovalRecords(vault, laneStoreFold(), paths), context(vault));}diff --git a/src/mcp/brain/feedback-tool.ts b/src/mcp/brain/feedback-tool.tsindex cc7335f9..ef817583 100644--- a/src/mcp/brain/feedback-tool.ts+++ b/src/mcp/brain/feedback-tool.ts@@ -560,7 +560,7 @@ export async function toolBrainFeedback(const mirror =sharedNamespace === null || vault === null? undefined- : mirrorSignal(sharedNamespace, vault, signalInput, writeOpts);+ : mirrorSignal(sharedNamespace, vault, signalInput, writeOpts, ctx.configPath);try {timeStageSync(ROUTE_STAGE.logAppend, () =>diff --git a/tests/core/brain/schema-pack-ontology.test.ts b/tests/core/brain/schema-pack-ontology.test.tsindex 3501aebc..c870c490 100644--- a/tests/core/brain/schema-pack-ontology.test.ts+++ b/tests/core/brain/schema-pack-ontology.test.ts@@ -16,6 +16,7 @@ import {type SchemaMutation,} from "../../../src/core/brain/schema-mutate.ts";import {+ dropDanglingReferences,FRONTMATTER_TIERS,parseSchemaPack,renderSchemaBlock,@@ -214,6 +215,14 @@ describe("ontology mutations", () => {).toThrow(/link_constraints\.mystery.*not declared/);});+ test("add_link_constraint requires declared endpoint tokens", () => {+ expect(() =>+ applyMutationsToPack(base, [+ { op: "add_link_constraint", link_type: "depends_on", source: "memo", target: "person" },+ ]),+ ).toThrow(/link_constraints\.depends_on.*memo->person.*not declared/);+ });+test("remove_link_constraint drops one pair and prunes empty entries", () => {const added = applyMutationsToPack(base, [{ op: "add_link_constraint", link_type: "depends_on", source: "paper", target: "person" },@@ -290,6 +299,39 @@ describe("ontology mutations", () => {]);expect(removed.attributes).toEqual({});});++ test("remove_type drops the constraint pairs that name the type, then an emptied link type", () => {+ const added = applyMutationsToPack(base, [+ { op: "add_type", category: "page_types", token: "book" },+ { op: "add_link_type", token: "cites" },+ { op: "add_link_constraint", link_type: "depends_on", source: "paper", target: "person" },+ { op: "add_link_constraint", link_type: "depends_on", source: "book", target: "person" },+ { op: "add_link_constraint", link_type: "cites", source: "paper", target: "book" },+ ]);+ const removed = applyMutationsToPack(added, [+ { op: "remove_type", category: "page_types", token: "book" },+ ]);+ expect(removed.link_constraints).toEqual({ depends_on: ["paper->person"] });+ });+});++describe("dropDanglingReferences on link_constraints", () => {+ test("drops a pair with an undeclared endpoint, then a link type left with none", () => {+ const pack = parseSchemaPack(+ [+ BASE,+ " - cites",+ " link_constraints:",+ " - depends_on=paper->person",+ " - depends_on=memo->person",+ " - cites=paper->memo",+ " - mystery=paper->person",+ ].join("\n") + "\n",+ );+ expect(dropDanglingReferences(pack).link_constraints).toEqual({+ depends_on: ["paper->person"],+ });+ });});describe("coerceSchemaMutations (CLI/MCP boundary)", () => {diff --git a/tests/core/brain/schema-pack-store-roundtrip.test.ts b/tests/core/brain/schema-pack-store-roundtrip.test.tsindex fa61fdc9..5902ebee 100644--- a/tests/core/brain/schema-pack-store-roundtrip.test.ts+++ b/tests/core/brain/schema-pack-store-roundtrip.test.ts@@ -6,9 +6,11 @@import { afterEach, expect, test } from "bun:test";import {+ SCHEMA_PACK_FIELDS,storeSchemaFields,writeSchemaPackToStore,} from "../../../src/core/brain/schema-pack-store.ts";+import { parseSchemaPack, renderSchemaBlock } from "../../../src/core/brain/schema-pack.ts";import { makeStoreEnv, type StoreEnv } from "../store/fixtures/store-env.ts";let env: StoreEnv | undefined;@@ -22,9 +24,37 @@ test("an entry holding a comment marker is stored as rendered", () => {writeSchemaPackToStore({ vault: null, authority: "human", agent: "@t" },"schema:\n expert_routing:\n - review=docs #style\n link_types:\n - cites\n",+ "schema:\n {}\n",+ { fields: {}, decidedBy: {} },);expect(storeSchemaFields()).toEqual({expert_routing: ["review=docs #style"],link_types: ["cites"],});});++test("the pack fields the store renders and attests are the fields the pack parser reads", () => {+ const entries: Record<string, string> = {+ preference_types: "taste",+ signal_types: "hint",+ page_types: "paper",+ log_event_kinds: "audit",+ aliases: "paper=article",+ prefixes: "paper=pa",+ link_types: "cites",+ extractable: "paper",+ expert_routing: "paper=reviewer",+ labels: "paper=reading",+ link_constraints: "cites=paper->paper",+ attributes: "paper.year=Publication year",+ frontmatter_tiers: "paper.year=user",+ };+ const text = [+ "schema:",+ ...Object.entries(entries).flatMap(([field, entry]) => [` ${field}:`, ` - ${entry}`]),+ ].join("\n");+ const rendered = renderSchemaBlock(parseSchemaPack(`${text}\n`));+ const fields = [...rendered.matchAll(/^ {2}([a-z_]+):/gm)].map((m) => m[1]);+ expect(fields.toSorted()).toEqual([...SCHEMA_PACK_FIELDS].toSorted());+ expect(Object.keys(entries).toSorted()).toEqual([...SCHEMA_PACK_FIELDS].toSorted());+});diff --git a/tests/core/brain/schema-store-attestation.test.ts b/tests/core/brain/schema-store-attestation.test.tsnew file mode 100644index 00000000..0bd09280--- /dev/null+++ b/tests/core/brain/schema-store-attestation.test.ts@@ -0,0 +1,316 @@+/**+ * What an audit row of a store-only schema apply attests (task 3.12,+ * design D19 item 1): the records its own apply wrote, plus records rows+ * it has seen already attest, never every record that happens to decide+ * the pack. A deciding record no row attests is a finding the apply+ * refuses over, unless the operator adopts it by name. The migration's+ * import attests its records with a baseline row, and rows written before+ * rows named records keep the digest comparison.+ */+import { afterEach, describe, expect, setDefaultTimeout, setSystemTime, test } from "bun:test";+import { appendFileSync, readdirSync, readFileSync, writeFileSync } from "node:fs";+import { join } from "node:path";++import { brainConfigPath } from "../../../src/core/brain/paths.ts";+import { appendAuditRecord } from "../../../src/core/reliability/audit.ts";+import { applySchemaAdminMutations } from "../../../src/core/brain/schema-admin.ts";+import {+ assessSchemaPackIntegrity,+ computeSchemaPackDigest,+ schemaMutationAuditDir,+} from "../../../src/core/brain/schema-integrity.ts";+import type { SchemaMutation } from "../../../src/core/brain/schema-mutate.ts";+import { loadSchemaPack } from "../../../src/core/brain/schema-pack.ts";+import { appendRecord } from "../../../src/core/store/appender.ts";+import { SETTING_ENTRY_KIND, SETTING_SET_KIND } from "../../../src/core/store/constants.ts";+import { bodyChecksum, type StoreRecord } from "../../../src/core/store/envelope.ts";+import { openStore } from "../../../src/core/store/store-open.ts";+import { importBrainSettings } from "../../../src/core/store-migration/brain-settings.ts";+import { runCli } from "../../helpers/run-cli.ts";+import { CLI_SPAWN_BUDGET_MS } from "../../helpers/cli-timeout.ts";+import { makeStoreEnv, type StoreEnv } from "../store/fixtures/store-env.ts";+import { ledgerRows, partitionRecords, switchDevice } from "./store-only/fixtures.ts";+import { sync, twoMachines, use, type Machine } from "./store-only/two-machines.ts";++setDefaultTimeout(CLI_SPAWN_BUDGET_MS);++let env: StoreEnv | null = null;+afterEach(() => {+ env?.cleanup();+ env = null;+ setSystemTime();+});++let step = 0;+/** A fixed clock past the fixtures' own wall time, one second per step. */+function tick(): Date {+ step += 1;+ const now = new Date(Date.UTC(2030, 0, 1, 12, 0, step));+ setSystemTime(now);+ return now;+}++const addType = (token: string): SchemaMutation => ({+ op: "add_type",+ category: "page_types",+ token,+});++async function apply(+ m: Machine | StoreEnv,+ mutations: SchemaMutation[],+ extra: { adoptUnattested?: boolean; authority?: "human" | "agent"; reason?: string } = {},+) {+ if ("device" in m) use(m);+ return await applySchemaAdminMutations(m.vault, mutations, {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ ...extra,+ });+}++/** One shared settings record no apply wrote. */+function forge(m: Machine, kind: string, body: Record<string, unknown>): string {+ use(m);+ tick();+ return appendRecord(openStore({ root: m.storeRoot, device: m.device }), {+ partition: "shared",+ stream: "settings",+ kind,+ agent: "other-tool",+ body,+ }).record.id;+}++const forgedEntry = {+ authority: "agent",+ key: "brain.schema.page_types",+ entry: "forged",+ present: true,+};++type Row = { audit_dir?: string; record?: { action?: string; details?: Record<string, unknown> } };+const schemaRows = (root: string, partition: "shared" | "private") =>+ (ledgerRows(root, partition, "reliability-audit") as Row[]).filter(+ (row) => row.audit_dir === "schema-mutations",+ );++describe("an unattested record under an apply", () => {+ test("the apply refuses with a named reason, and another machine's apply attests nothing of it", async () => {+ env = makeStoreEnv("osb-schema-attest-forged-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper")]);+ sync(a, b);+ forge(b, SETTING_ENTRY_KIND, forgedEntry);+ await expect(apply(b, [addType("memo")])).rejects.toMatchObject({+ code: "unattested_schema_records",+ });+ // A never saw the forged record; its apply cannot attest it.+ await apply(a, [addType("person")]);+ sync(a, b);+ sync(b, a);+ for (const m of [a, b]) {+ use(m);+ expect(loadSchemaPack(m.vault).declarations.page_types).toContain("forged");+ expect(assessSchemaPackIntegrity(m.vault).status).toBe("modified");+ }+ });++ test("the operator adopts it by name: the row records the adoption and the pack is attested again", async () => {+ env = makeStoreEnv("osb-schema-attest-adopt-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper")]);+ sync(a, b);+ const forged = forge(b, SETTING_ENTRY_KIND, forgedEntry);+ await apply(b, [addType("memo")], { adoptUnattested: true, authority: "human" });+ const [row] = schemaRows(b.storeRoot, "shared").slice(-1);+ expect(row?.record?.details).toMatchObject({ adopted: [forged], adopted_by: "operator-ann" });+ sync(b, a);+ for (const m of [a, b]) {+ use(m);+ expect(assessSchemaPackIntegrity(m.vault).status).toBe("ok");+ }+ });++ test("an agent cannot adopt", async () => {+ env = makeStoreEnv("osb-schema-attest-agent-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper")]);+ sync(a, b);+ forge(b, SETTING_ENTRY_KIND, forgedEntry);+ await expect(+ apply(b, [addType("memo")], { adoptUnattested: true, authority: "agent" }),+ ).rejects.toMatchObject({ code: "unattested_schema_records" });+ });++ test("the CLI adopts with --adopt-unattested and names the operator", async () => {+ env = makeStoreEnv("osb-schema-attest-cli-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper")]);+ sync(a, b);+ const forged = forge(b, SETTING_ENTRY_KIND, forgedEntry);+ const args = ["brain", "schema", "apply", "--mutation", JSON.stringify(addType("memo"))];+ const opts = { env: { ...process.env, OPEN_SECOND_BRAIN_CONFIG: b.configPath } };+ const refused = await runCli([...args, "--actor", "ann"], opts);+ expect(refused.returncode).not.toBe(0);+ expect(refused.stderr).toContain("--adopt-unattested");+ const adopted = await runCli([...args, "--actor", "ann", "--adopt-unattested"], opts);+ expect(adopted.returncode).toBe(0);+ const [row] = schemaRows(b.storeRoot, "shared").slice(-1);+ expect(row?.record?.details).toMatchObject({ adopted: [forged], adopted_by: "ann" });+ });+});++describe("a record the attestation names", () => {+ test("is attested only in the version the row saw: another body under its id is a finding", async () => {+ env = makeStoreEnv("osb-schema-attest-reused-id-");+ const [a] = twoMachines(env);+ await apply(a, [addType("paper")]);+ expect(assessSchemaPackIntegrity(a.vault).status).toBe("ok");+ const original = partitionRecords(a.storeRoot, "shared").find(+ (r) => r.kind === SETTING_ENTRY_KIND && r.body["entry"] === "paper",+ )!;+ // Another version of the attested record: same id, another body, a+ // checksum that matches it. Ordered first among the versions of the id.+ const body = { ...original.body, authority: "agent", entry: "forged" };+ const forged: StoreRecord = { ...original, body, sum: bodyChecksum(body) };+ appendFileSync(shardHolding(a.storeRoot, original.id), `${JSON.stringify(forged)}\n`);+ use(a);+ expect(loadSchemaPack(a.vault).declarations.page_types).toEqual(["forged"]);+ expect(assessSchemaPackIntegrity(a.vault).status).toBe("modified");+ await expect(apply(a, [addType("memo")])).rejects.toMatchObject({+ code: "unattested_schema_records",+ });+ });+});++describe("a record under brain.schema. that no pack field reads", () => {+ test("is not attested and does not hold up the next apply", async () => {+ env = makeStoreEnv("osb-schema-attest-not-a-field-");+ const [a] = twoMachines(env);+ await apply(a, [addType("paper")]);+ forge(a, SETTING_SET_KIND, { authority: "human", key: "brain.schema.notes", value: ["x"] });+ use(a);+ expect(assessSchemaPackIntegrity(a.vault).status).toBe("ok");+ await apply(a, [addType("memo")]);+ expect(loadSchemaPack(a.vault).declarations.page_types).toEqual(["paper", "memo"]);+ });+});++/** The shard file of `root/shared/settings` that holds record `id`. */+function shardHolding(root: string, id: string): string {+ const dir = join(root, "shared", "settings");+ const found = (readdirSync(dir, { recursive: true }) as string[])+ .map((rel) => join(dir, rel))+ .find((path) => path.endsWith(".jsonl") && readFileSync(path, "utf8").includes(id));+ if (found === undefined) throw new Error(`no settings shard holds ${id}`);+ return found;+}++describe("adoption on its own", () => {+ test("o2b brain schema apply --adopt-unattested adopts with no mutation and changes no entry", async () => {+ env = makeStoreEnv("osb-schema-attest-adopt-only-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper")]);+ sync(a, b);+ const forged = forge(b, SETTING_ENTRY_KIND, forgedEntry);+ use(b);+ const before = loadSchemaPack(b.vault);+ const opts = { env: { ...process.env, OPEN_SECOND_BRAIN_CONFIG: b.configPath } };+ const adopted = await runCli(+ ["brain", "schema", "apply", "--adopt-unattested", "--actor", "ann"],+ opts,+ );+ expect(adopted.stderr).toBe("");+ expect(adopted.returncode).toBe(0);+ const [row] = schemaRows(b.storeRoot, "shared").slice(-1);+ expect(row?.record?.details).toMatchObject({+ applied: 0,+ adopted: [forged],+ adopted_by: "ann",+ });+ expect(loadSchemaPack(b.vault)).toEqual(before);+ expect(assessSchemaPackIntegrity(b.vault).status).toBe("ok");+ // Without the flag, an apply still needs a mutation.+ const bare = await runCli(["brain", "schema", "apply", "--actor", "ann"], opts);+ expect(bare.returncode).not.toBe(0);+ expect(bare.stderr).toContain("--mutation");+ });+});++describe("adoption on its own on an unswitched machine", () => {+ test("is refused, and _brain.yaml is left as it is", async () => {+ env = makeStoreEnv("osb-schema-attest-adopt-unswitched-", { dualWrite: false });+ const config = "schema_version: 1\nschema:\n page_types: [ Paper ]\n";+ writeFileSync(brainConfigPath(env.vault), config);+ const opts = { env: { ...process.env, OPEN_SECOND_BRAIN_CONFIG: env.configPath } };+ const refused = await runCli(+ ["brain", "schema", "apply", "--adopt-unattested", "--actor", "ann"],+ opts,+ );+ expect(refused.returncode).not.toBe(0);+ expect(refused.stderr).toContain("store-only machine");+ expect(readFileSync(brainConfigPath(env.vault), "utf8")).toBe(config);+ });+});++describe("the migration's import", () => {+ test("attests the records it wrote, so the first apply after the switch goes through", async () => {+ env = makeStoreEnv("osb-schema-attest-import-");+ writeFileSync(+ brainConfigPath(env.vault),+ "schema_version: 1\nschema:\n page_types:\n - paper\n - memo\n",+ );+ tick();+ importBrainSettings(env.vault, openStore({ root: env.storeRoot, device: "dev-a" }), "migrator");+ switchDevice(env.storeRoot);+ await apply(env, [addType("person")]);+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper", "memo", "person"]);+ expect(assessSchemaPackIntegrity(env.vault).status).toBe("ok");+ });+});++describe("audit rows written before rows named records", () => {+ test("keep the digest comparison, and an apply over the pack they attest goes through", async () => {+ env = makeStoreEnv("osb-schema-attest-legacy-");+ switchDevice(env.storeRoot);+ const [a] = twoMachines(env);+ forge(a, SETTING_SET_KIND, {+ authority: "human",+ key: "brain.schema.page_types",+ value: ["paper"],+ });+ use(a);+ // A row of that time: a digest, no `decided`.+ appendAuditRecord(+ schemaMutationAuditDir(a.vault),+ {+ timestamp: tick().toISOString(),+ actor: "old",+ action: "schema_apply_mutations",+ target: "store:shared/settings/brain.schema",+ ok: true,+ details: { applied: 1, pack_digest: computeSchemaPackDigest(loadSchemaPack(a.vault)) },+ },+ a.vault,+ { attestsShared: true },+ );+ expect(assessSchemaPackIntegrity(a.vault).status).toBe("ok");+ await apply(a, [addType("memo")]);+ expect(assessSchemaPackIntegrity(a.vault).status).toBe("ok");+ });+});++describe("a shared row whose mutations are withheld", () => {+ test("still names the records it attests", async () => {+ env = makeStoreEnv("osb-schema-attest-withheld-");+ switchDevice(env.storeRoot);+ await apply(env, [addType("paper")], { reason: "<private>why</private>" });+ const [row] = schemaRows(env.storeRoot, "shared");+ expect(row?.record?.details?.["withheld"]).toBeDefined();+ expect(row?.record?.details?.["mutations"]).toBeUndefined();+ const decided = row?.record?.details?.["decided"];+ expect(Array.isArray(decided) && decided.length > 0).toBe(true);+ });+});diff --git a/tests/core/brain/schema-store-authority.test.ts b/tests/core/brain/schema-store-authority.test.tsnew file mode 100644index 00000000..da069ce2--- /dev/null+++ b/tests/core/brain/schema-store-authority.test.ts@@ -0,0 +1,130 @@+/**+ * Who may change a schema field list on a store-only machine (task 3.12,+ * design D19 item 1): each entry folds as its own register, human over+ * agent, so the writers refuse an agent change the fold would ignore, per+ * entry for an apply and for every entry a whole-list settings write+ * would change, in the normalised form the pack parser reads.+ */+import { afterEach, describe, expect, test } from "bun:test";++import {+ BrainSettingError,+ getBrainSetting,+ setBrainSetting,+ unsetBrainSetting,+} from "../../../src/core/brain/policy/settings-write.ts";+import { applySchemaAdminMutations } from "../../../src/core/brain/schema-admin.ts";+import type { SchemaMutation } from "../../../src/core/brain/schema-mutate.ts";+import { loadSchemaPack } from "../../../src/core/brain/schema-pack.ts";+import { makeStoreEnv, type StoreEnv } from "../store/fixtures/store-env.ts";+import { switchDevice } from "./store-only/fixtures.ts";++let env: StoreEnv | null = null;+afterEach(() => {+ env?.cleanup();+ env = null;+});++const type = (op: "add_type" | "remove_type", token: string): SchemaMutation => ({+ op,+ category: "page_types",+ token,+});++async function apply(+ e: StoreEnv,+ authority: "human" | "agent",+ mutations: SchemaMutation[],+): Promise<void> {+ await applySchemaAdminMutations(e.vault, mutations, { actor: authority, authority });+}++const agentCtx = (e: StoreEnv) => ({ vault: e.vault, authority: "agent" as const, agent: "bot" });++function storeOnly(prefix: string): StoreEnv {+ const e = makeStoreEnv(prefix);+ switchDevice(e.storeRoot);+ return e;+}++/** The review's scenario: the operator removed `paper`, the agent owns the newest entry record. */+async function operatorRemovedPaper(e: StoreEnv): Promise<void> {+ await apply(e, "agent", [type("add_type", "memo")]);+ await apply(e, "human", [type("add_type", "paper")]);+ await apply(e, "human", [type("remove_type", "paper")]);+ await apply(e, "agent", [type("remove_type", "memo")]);+ await apply(e, "agent", [type("add_type", "memo")]);+}++function code(fn: () => unknown): string | null {+ try {+ fn();+ return null;+ } catch (err) {+ if (err instanceof BrainSettingError) return err.code;+ throw err;+ }+}++describe("an agent's whole-list write of a field folded per entry", () => {+ for (const list of [+ ["Paper", "memo"],+ [" paper", "memo"],+ ["paper", "memo"],+ ]) {+ test(`cannot bring back an entry the operator removed: ${JSON.stringify(list)}`, async () => {+ env = storeOnly("osb-schema-authority-list-");+ await operatorRemovedPaper(env);+ expect(code(() => setBrainSetting(agentCtx(env!), "schema.page_types", list))).toBe(+ "human_value_in_force",+ );+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["memo"]);+ });+ }++ test("an agent unset that would remove a human entry is refused", async () => {+ env = storeOnly("osb-schema-authority-unset-");+ await apply(env, "human", [type("add_type", "paper")]);+ await apply(env, "agent", [type("add_type", "memo")]);+ expect(code(() => unsetBrainSetting(agentCtx(env!), "schema.page_types"))).toBe(+ "human_value_in_force",+ );+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper", "memo"]);+ });++ test("an accepted write reports the value the fold holds, not the one requested", async () => {+ env = storeOnly("osb-schema-authority-report-");+ await apply(env, "agent", [type("add_type", "memo")]);+ const result = setBrainSetting(agentCtx(env), "schema.page_types", ["memo", "Memo", "person"]);+ expect(result.changed).toBe(true);+ const inForce = getBrainSetting({ vault: env.vault }, "schema.page_types").value;+ expect(result.value).toEqual(inForce);+ expect(inForce).toEqual(["Memo", "person"]);+ });+});++describe("an agent apply beside the operator's entries", () => {+ test("an agent may add an entry no human record decides", async () => {+ env = storeOnly("osb-schema-authority-add-");+ await apply(env, "human", [type("add_type", "paper")]);+ await apply(env, "agent", [type("add_type", "extra")]);+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper", "extra"]);+ });++ test("an agent may not remove an entry the operator added", async () => {+ env = storeOnly("osb-schema-authority-remove-");+ await apply(env, "human", [type("add_type", "paper")]);+ await apply(env, "agent", [type("add_type", "memo")]);+ await expect(apply(env, "agent", [type("remove_type", "paper")])).rejects.toMatchObject({+ code: "human_value_in_force",+ });+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper", "memo"]);+ });++ test("the operator may remove an entry an agent added", async () => {+ env = storeOnly("osb-schema-authority-human-");+ await apply(env, "agent", [type("add_type", "memo")]);+ await apply(env, "human", [type("remove_type", "memo")]);+ expect(loadSchemaPack(env.vault).declarations.page_types).toBeUndefined();+ });+});diff --git a/tests/core/brain/schema-store-merge.test.ts b/tests/core/brain/schema-store-merge.test.tsnew file mode 100644index 00000000..f543bdfc--- /dev/null+++ b/tests/core/brain/schema-store-merge.test.ts@@ -0,0 +1,250 @@+/**+ * Concurrent schema applies on two store-only machines (task 3.12, design+ * D19 item 1): each apply writes one `setting-entry` record per entry it+ * adds or removes, so edits two devices make before they sync both+ * survive, and both machines converge on one pack whatever order the sync+ * delivers the shards in. The merged pack stays attested.+ */+import { afterEach, describe, expect, setSystemTime, test } from "bun:test";+import { writeFileSync } from "node:fs";++import { brainConfigPath } from "../../../src/core/brain/paths.ts";+import { applySchemaAdminMutations } from "../../../src/core/brain/schema-admin.ts";+import { assessSchemaPackIntegrity } from "../../../src/core/brain/schema-integrity.ts";+import type { SchemaMutation } from "../../../src/core/brain/schema-mutate.ts";+import { storeSchemaFields } from "../../../src/core/brain/schema-pack-store.ts";+import { loadSchemaPack } from "../../../src/core/brain/schema-pack.ts";+import { appendRecord } from "../../../src/core/store/appender.ts";+import { SETTING_ENTRY_KIND, SETTING_SET_KIND } from "../../../src/core/store/constants.ts";+import { openStore } from "../../../src/core/store/store-open.ts";+import { importBrainSettings } from "../../../src/core/store-migration/brain-settings.ts";+import { makeStoreEnv, type StoreEnv } from "../store/fixtures/store-env.ts";+import { sync, twoMachines, use, type Machine } from "./store-only/two-machines.ts";++let env: StoreEnv | null = null;+afterEach(() => {+ env?.cleanup();+ env = null;+ setSystemTime();+});++/** A fixed clock past the fixtures' own wall time: each step is one second after the one before. */+let step = 0;+function tick(): Date {+ step += 1;+ const now = new Date(Date.UTC(2030, 0, 1, 12, 0, step));+ setSystemTime(now);+ return now;+}++async function apply(m: Machine, mutations: SchemaMutation[]): Promise<void> {+ use(m);+ const now = tick();+ await applySchemaAdminMutations(m.vault, mutations, {+ actor: m.device,+ now,+ authority: "human",+ });+}++const addType = (token: string): SchemaMutation => ({+ op: "add_type",+ category: "page_types",+ token,+});+const removeType = (token: string): SchemaMutation => ({+ op: "remove_type",+ category: "page_types",+ token,+});++function pageTypes(m: Machine): ReadonlyArray<string> | undefined {+ use(m);+ return loadSchemaPack(m.vault).declarations.page_types;+}++describe("schema field lists edited on two devices at once", () => {+ for (const order of ["a-first", "b-first"] as const) {+ test(`converge to one set on both machines (sync ${order})`, async () => {+ env = makeStoreEnv(`osb-schema-merge-${order}-`);+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper"), addType("report")]);+ sync(a, b);+ expect(pageTypes(b)).toEqual(["paper", "report"]);++ // Before either syncs: A adds one type and removes another, B adds a+ // third and removes `paper`, which A then removes and adds back later.+ await apply(a, [addType("person"), removeType("report")]);+ await apply(b, [addType("memo"), removeType("paper")]);+ await apply(a, [removeType("paper")]);+ await apply(a, [addType("paper")]);++ if (order === "a-first") {+ sync(a, b);+ sync(b, a);+ } else {+ sync(b, a);+ sync(a, b);+ }+ // `paper`: A's add is later than B's remove, so the add wins (later+ // clock within one authority, design D4).+ const expected = ["person", "memo", "paper"];+ expect(pageTypes(a)).toEqual(expected);+ expect(pageTypes(b)).toEqual(expected);++ // The merged pack is no single apply's output, and it is still attested.+ for (const m of [a, b]) {+ use(m);+ expect(assessSchemaPackIntegrity(m.vault).status).toBe("ok");+ }+ });+ }+});++/** Append one shared settings record as machine `m`, outside any schema apply. */+function rawSetting(m: Machine, kind: string, body: Record<string, unknown>): void {+ use(m);+ tick();+ appendRecord(openStore({ root: m.storeRoot, device: m.device }), {+ partition: "shared",+ stream: "settings",+ kind,+ agent: "other-tool",+ body,+ });+}++describe("schema field lists imported as whole lists", () => {+ test("the migration's whole list reads, and applies on two machines edit it per entry", async () => {+ env = makeStoreEnv("osb-schema-merge-import-");+ const [a, b] = twoMachines(env);+ use(a);+ tick();+ writeFileSync(+ brainConfigPath(a.vault),+ "schema_version: 1\nschema:\n page_types:\n - note\n - paper\n",+ );+ importBrainSettings(a.vault, openStore({ root: a.storeRoot, device: a.device }), "migrator");+ expect(pageTypes(a)).toEqual(["note", "paper"]);+ sync(a, b);+ await apply(b, [addType("memo"), removeType("note")]);+ await apply(a, [addType("person")]);+ sync(a, b);+ sync(b, a);+ expect(pageTypes(a)).toEqual(["paper", "memo", "person"]);+ expect(pageTypes(b)).toEqual(["paper", "memo", "person"]);+ use(a);+ expect(assessSchemaPackIntegrity(a.vault).status).toBe("ok");+ });+ test("an entry no audited apply wrote still reports the merged pack as modified", async () => {+ env = makeStoreEnv("osb-schema-merge-forged-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper")]);+ await apply(b, [addType("memo")]);+ sync(a, b);+ sync(b, a);+ use(a);+ expect(assessSchemaPackIntegrity(a.vault)).toMatchObject({ status: "ok", merged: true });+ rawSetting(a, SETTING_ENTRY_KIND, {+ authority: "agent",+ key: "brain.schema.page_types",+ entry: "forged",+ present: true,+ });+ expect(pageTypes(a)).toContain("forged");+ expect(assessSchemaPackIntegrity(a.vault).status).toBe("modified");+ });+});++describe("references across fields edited on two devices at once", () => {+ test("a removed type and a reference to it added elsewhere read as one consistent pack", async () => {+ env = makeStoreEnv("osb-schema-merge-refs-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper"), addType("note"), { op: "add_link_type", token: "cites" }]);+ sync(a, b);+ // A removes `paper`; B, before it syncs, adds an alias, a prefix and a+ // link constraint pair that name it.+ await apply(a, [removeType("paper")]);+ await apply(b, [+ { op: "add_alias", token: "paper", alias: "article" },+ { op: "add_prefix", prefix: "ppr", token: "paper" },+ { op: "add_link_constraint", link_type: "cites", source: "paper", target: "note" },+ ]);+ sync(a, b);+ sync(b, a);+ for (const m of [a, b]) {+ use(m);+ const pack = loadSchemaPack(m.vault);+ expect(pack.declarations.page_types).toEqual(["note"]);+ // The references to the undeclared type are dropped on read, the+ // cascade `remove_type` applies, on every machine alike.+ expect(pack.aliases).toEqual({});+ expect(pack.prefixes).toEqual({});+ // A pair with an undeclared endpoint would still match a page whose+ // frontmatter keeps `type: paper`, so it is dropped, and with it the+ // link type's key once no pair remains.+ expect(pack.link_constraints).toEqual({});+ }+ // A later unrelated apply works on either machine, and writes the+ // repair as entry removals, so the next machine reads it from records.+ await apply(a, [addType("memo")]);+ await apply(b, [addType("person")]);+ sync(a, b);+ sync(b, a);+ for (const m of [a, b]) {+ use(m);+ const pack = loadSchemaPack(m.vault);+ expect(pack.declarations.page_types).toEqual(["note", "memo", "person"]);+ expect(pack.aliases).toEqual({});+ expect(storeSchemaFields()["aliases"]).toBeUndefined();+ expect(storeSchemaFields()["prefixes"]).toBeUndefined();+ expect(storeSchemaFields()["link_constraints"]).toBeUndefined();+ expect(assessSchemaPackIntegrity(m.vault).status).toBe("ok");+ }+ });+});++describe("a keyed field edited on two devices at once", () => {+ test("a concurrent change of one prefix keeps the later value, other prefixes merge", async () => {+ env = makeStoreEnv("osb-schema-merge-keyed-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper"), addType("project"), addType("person")]);+ sync(a, b);+ await apply(a, [{ op: "add_prefix", prefix: "p", token: "paper" }]);+ await apply(b, [+ { op: "add_prefix", prefix: "p", token: "project" },+ { op: "add_prefix", prefix: "who", token: "person" },+ ]);+ sync(a, b);+ sync(b, a);+ for (const m of [a, b]) {+ use(m);+ expect(loadSchemaPack(m.vault).prefixes).toEqual({ p: "project", who: "person" });+ expect(assessSchemaPackIntegrity(m.vault).status).toBe("ok");+ }+ });+});++describe("records under brain.schema. that are no pack field", () => {+ test("do not turn a merged pack into modified", async () => {+ env = makeStoreEnv("osb-schema-merge-outside-");+ const [a, b] = twoMachines(env);+ await apply(a, [addType("paper")]);+ await apply(b, [addType("memo")]);+ sync(a, b);+ sync(b, a);+ use(a);+ expect(assessSchemaPackIntegrity(a.vault)).toMatchObject({ status: "ok", merged: true });+ rawSetting(a, SETTING_SET_KIND, {+ authority: "agent",+ key: "brain.schema.page_types.extra",+ value: ["x"],+ });+ rawSetting(a, SETTING_SET_KIND, {+ authority: "agent",+ key: "brain.schema.notes",+ value: "not a list",+ });+ expect(assessSchemaPackIntegrity(a.vault)).toMatchObject({ status: "ok", merged: true });+ });+});diff --git a/tests/core/brain/schema-store-only.test.ts b/tests/core/brain/schema-store-only.test.tsindex c6f88bc3..fbe59912 100644--- a/tests/core/brain/schema-store-only.test.ts+++ b/tests/core/brain/schema-store-only.test.ts@@ -9,14 +9,22 @@ import { afterEach, describe, expect, setDefaultTimeout, test } from "bun:test";import { cpSync, readFileSync, rmSync } from "node:fs";import { join } from "node:path";+import { freezeVault, unfreezeVault } from "../../../src/core/brain/freeze.ts";+import { VaultFrozenError } from "../../../src/core/brain/freeze-marker.ts";import { brainConfigPath } from "../../../src/core/brain/paths.ts";+import { setBrainSetting } from "../../../src/core/brain/policy/settings-write.ts";import { applySchemaAdminMutations } from "../../../src/core/brain/schema-admin.ts";import { assessSchemaPackIntegrity } from "../../../src/core/brain/schema-integrity.ts";import { SCHEMA_AUDIT_STORE_NAME } from "../../../src/core/brain/schema-mutate-store.ts";import type { SchemaMutation } from "../../../src/core/brain/schema-mutate.ts";import { STORE_SCHEMA_PACK_SOURCE } from "../../../src/core/brain/schema-pack-store.ts";import { loadSchemaPack, readSchemaPackSource } from "../../../src/core/brain/schema-pack.ts";-import { SETTING_SET_KIND, SETTING_UNSET_KIND } from "../../../src/core/store/constants.ts";+import {+ SETTING_ENTRY_KIND,+ SETTING_SET_KIND,+ SETTING_UNSET_KIND,+} from "../../../src/core/store/constants.ts";+import { holdMigrationGate, releaseMigrationGate } from "../../../src/core/migration-gate.ts";import { JSONRPC_VERSION, MCPServer, PROTOCOL_VERSION } from "../../../src/mcp/index.ts";import { runCli } from "../../helpers/run-cli.ts";import { makeStoreEnv, type StoreEnv } from "../store/fixtures/store-env.ts";@@ -43,7 +51,9 @@ const ALIAS: SchemaMutation = { op: "add_alias", token: "paper", alias: "articlefunction schemaRecords(e: StoreEnv) {return partitionRecords(e.storeRoot, "shared").filter((r) =>- (r.kind === SETTING_SET_KIND || r.kind === SETTING_UNSET_KIND) &&+ (r.kind === SETTING_SET_KIND ||+ r.kind === SETTING_UNSET_KIND ||+ r.kind === SETTING_ENTRY_KIND) &&String(r.body["key"]).startsWith("brain.schema."),);}@@ -68,10 +78,13 @@ describe("schema apply on a store-only machine", () => {expect(brainTreeDigest(env.vault)).toBe(before);const records = schemaRecords(env);- expect(records.map((r) => [r.body["key"], r.body["value"]])).toEqual([- ["brain.schema.aliases", ["paper=article"]],- ["brain.schema.page_types", ["paper"]],- ]);+ // One entry record per entry added, never the whole list (task 3.12).+ expect(records.map((r) => [r.kind, r.body["key"], r.body["entry"], r.body["present"]])).toEqual(+ [+ [SETTING_ENTRY_KIND, "brain.schema.aliases", "paper=article", true],+ [SETTING_ENTRY_KIND, "brain.schema.page_types", "paper", true],+ ],+ );// Not from the CLI: an agent wrote these.expect(records.every((r) => r.body["authority"] === "agent")).toBe(true);// The attestation in `shared`, the whole row in `private`.@@ -88,7 +101,7 @@ describe("schema apply on a store-only machine", () => {expect(integrity).toMatchObject({ audit_path: SCHEMA_AUDIT_STORE_NAME });});- test("a field the pack drops is unset; the next apply builds on the stored pack", async () => {+ test("an entry the pack drops is removed; the next apply builds on the stored pack", async () => {env = makeStoreEnv("osb-schema-store-unset-");switchDevice(env.storeRoot);await applySchemaAdminMutations(env.vault, [ADD_PAPER, ALIAS], { actor: "t" });@@ -98,14 +111,18 @@ describe("schema apply on a store-only machine", () => {{ actor: "t" },);const last = schemaRecords(env).at(-1)!;- expect(last.kind).toBe(SETTING_UNSET_KIND);- expect(last.body["key"]).toBe("brain.schema.aliases");+ expect(last.kind).toBe(SETTING_ENTRY_KIND);+ expect(last.body).toMatchObject({+ key: "brain.schema.aliases",+ entry: "paper=article",+ present: false,+ });const pack = loadSchemaPack(env.vault);expect(pack.aliases).toEqual({});expect(pack.declarations.page_types).toEqual(["paper"]);});- test("the CLI writes with human authority, and an agent cannot change that field", async () => {+ test("the CLI writes with human authority, and an agent cannot remove the operator's entry", async () => {env = makeStoreEnv("osb-schema-store-cli-");switchDevice(env.storeRoot);const cli = await runCli(@@ -133,7 +150,7 @@ describe("schema apply on a store-only machine", () => {params: {name: "schema_apply_mutations",arguments: {- mutations: [{ op: "add_type", category: "page_types", token: "memo" }],+ mutations: [{ op: "remove_type", category: "page_types", token: "paper" }],},},})) as { result?: { isError?: boolean; content: Array<{ text: string }> }; error?: unknown };@@ -154,6 +171,36 @@ function sharedOnlyPeer(from: StoreEnv): StoreEnv {return peer;}+describe("the content-write gates on a store-only machine", () => {+ test("a frozen machine refuses a schema apply and a settings write; a leftover gate does not", async () => {+ env = makeStoreEnv("osb-schema-store-gates-");+ switchDevice(env.storeRoot);+ const vault = env.vault;+ const settings = { vault, authority: "human" as const, agent: "test" };+ freezeVault(vault, { agent: "@operator", reason: "moving machines" });+ try {+ await expect(applySchemaAdminMutations(vault, [ADD_PAPER], { actor: "t" })).rejects.toThrow(+ VaultFrozenError,+ );+ expect(() => setBrainSetting(settings, "schema.page_types", ["paper"])).toThrow(+ VaultFrozenError,+ );+ } finally {+ unfreezeVault(vault);+ }+ expect(schemaRecords(env)).toHaveLength(0);+ expect(schemaAuditRows(env)).toHaveLength(0);+ // A migration gate left naming the vault refuses no store write.+ holdMigrationGate(vault, "run-schema-gates");+ try {+ await applySchemaAdminMutations(vault, [ADD_PAPER], { actor: "t" });+ } finally {+ releaseMigrationGate(vault, "run-schema-gates");+ }+ expect(schemaRecords(env)).toHaveLength(1);+ });+});+describe("a schema apply seen from a machine that holds only shared", () => {let other: StoreEnv | null = null;afterEach(() => {diff --git a/tests/core/brain/schema-store-snapshot.test.ts b/tests/core/brain/schema-store-snapshot.test.tsnew file mode 100644index 00000000..74686f76--- /dev/null+++ b/tests/core/brain/schema-store-snapshot.test.ts@@ -0,0 +1,398 @@+/**+ * One read of the store's schema records per apply (task 3.12, design D19+ * item 1): the pack an apply starts from, the records that decide it and+ * the entries a repair may remove all come from one fold of the shared+ * settings stream, so a record that syncs in while an apply runs can+ * neither be removed as a repair nor build a pack its audit row does not+ * attest. A record under a schema list key that holds no list is rejected+ * by the fold, so it can never hide the stored list.+ */+import { afterEach, describe, expect, setDefaultTimeout, setSystemTime, test } from "bun:test";+import { appendFileSync, readFileSync, writeFileSync } from "node:fs";++import { brainConfigPath } from "../../../src/core/brain/paths.ts";+import {+ BrainSettingError,+ setBrainSetting,+} from "../../../src/core/brain/policy/settings-write.ts";+import { applySchemaAdminMutations } from "../../../src/core/brain/schema-admin.ts";+import {+ assessSchemaPackIntegrity,+ computeSchemaPackDigest,+} from "../../../src/core/brain/schema-integrity.ts";+import { applySchemaMutationsToStore } from "../../../src/core/brain/schema-mutate-store.ts";+import type { SchemaMutation } from "../../../src/core/brain/schema-mutate.ts";+import {+ loadSchemaPack,+ parseSchemaPack,+ readStoreSchemaPack,+ renderSchemaBlock,+} from "../../../src/core/brain/schema-pack.ts";+import { appendRecord } from "../../../src/core/store/appender.ts";+import { SETTING_ENTRY_KIND, SETTING_SET_KIND } from "../../../src/core/store/constants.ts";+import { foldSettings } from "../../../src/core/store/fold-settings.ts";+import { buildRecord } from "../../../src/core/store/envelope.ts";+import { compareHlc, parseHlc } from "../../../src/core/store/hlc.ts";+import { shardMonth, shardPath } from "../../../src/core/store/store-paths.ts";+import { openStore } from "../../../src/core/store/store-open.ts";+import { importBrainSettings } from "../../../src/core/store-migration/brain-settings.ts";+import { runCli } from "../../helpers/run-cli.ts";+import { CLI_SPAWN_BUDGET_MS } from "../../helpers/cli-timeout.ts";+import { makeStoreEnv, type StoreEnv } from "../store/fixtures/store-env.ts";+import { partitionRecords, switchDevice } from "./store-only/fixtures.ts";++setDefaultTimeout(CLI_SPAWN_BUDGET_MS);++let env: StoreEnv | null = null;+afterEach(() => {+ env?.cleanup();+ env = null;+ setSystemTime();+});++let step = 0;+function tick(): Date {+ step += 1;+ const now = new Date(Date.UTC(2031, 0, 1, 12, 0, step));+ setSystemTime(now);+ return now;+}++const addType = (token: string): SchemaMutation => ({+ op: "add_type",+ category: "page_types",+ token,+});++/** A shared settings record another device wrote, delivered into `env`'s store. */+function peerRecord(e: StoreEnv, kind: string, body: Record<string, unknown>): string {+ tick();+ return appendRecord(openStore({ root: e.storeRoot, device: "dev-peer" }), {+ partition: "shared",+ stream: "settings",+ kind,+ agent: "peer",+ body,+ }).record.id;+}++const schemaEntries = (root: string) =>+ partitionRecords(root, "shared").filter((r) => r.kind === SETTING_ENTRY_KIND);++/** A store-only machine whose pack the migration imported: `page_types: [paper, memo]`. */+function importedStore(prefix: string): StoreEnv {+ const e = makeStoreEnv(prefix);+ writeFileSync(+ brainConfigPath(e.vault),+ "schema_version: 1\nschema:\n page_types:\n - paper\n - memo\n",+ );+ tick();+ importBrainSettings(e.vault, openStore({ root: e.storeRoot, device: "dev-a" }), "migrator");+ switchDevice(e.storeRoot);+ return e;+}++describe("the store's schema pack read once", () => {+ test("returns the pack and the records deciding it from one fold", () => {+ env = importedStore("osb-schema-snapshot-read-");+ const snapshot = readStoreSchemaPack();+ expect(snapshot.source.present).toBe(true);+ expect(snapshot.source.pack).toEqual(loadSchemaPack(env.vault));+ expect(snapshot.fields).toEqual({ page_types: ["paper", "memo"] });+ const imported = partitionRecords(env.storeRoot, "shared").filter(+ (r) => r.kind === SETTING_SET_KIND && r.body["key"] === "brain.schema.page_types",+ );+ expect(snapshot.deciding.map((r) => r.id)).toEqual(imported.map((r) => r.id));+ });+});++describe("an entry another device adds while an apply runs", () => {+ test("is never removed as a repair, and the apply's row does not attest it", async () => {+ env = importedStore("osb-schema-snapshot-repair-");+ const e = env;+ let peer = "";+ await applySchemaMutationsToStore(e.vault, [addType("person")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ next: (current) => {+ // The peer's entry syncs in after the apply read the pack.+ peer = peerRecord(e, SETTING_ENTRY_KIND, {+ authority: "human",+ key: "brain.schema.page_types",+ entry: "idea",+ present: true,+ });+ return parseSchemaPack(+ `schema_version: 1\n${renderSchemaBlock(current).replace(+ " - memo\n",+ " - memo\n - person\n",+ )}`,+ );+ },+ });+ const removals = schemaEntries(e.storeRoot).filter((r) => r.body["present"] === false);+ expect(removals).toEqual([]);+ expect(loadSchemaPack(e.vault).declarations.page_types?.toSorted()).toEqual([+ "idea",+ "memo",+ "paper",+ "person",+ ]);+ // The peer's record is no record this apply saw: still unattested.+ await expect(+ applySchemaAdminMutations(e.vault, [addType("memo2")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ }),+ ).rejects.toMatchObject({ code: "unattested_schema_records", records: [peer] });+ });+});++describe("a record under a schema list key that holds no list", () => {+ test("is rejected by the fold and cannot hide the stored list, imported or merged per entry", async () => {+ env = importedStore("osb-schema-scalar-");+ const scalar = peerRecord(env, SETTING_SET_KIND, {+ authority: "human",+ key: "brain.schema.page_types",+ value: "oops",+ });+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper", "memo"]);+ const records = partitionRecords(env.storeRoot, "shared");+ expect(foldSettings(records).rejected.map((r) => r.id)).toContain(scalar);+ // The same once the key folds per entry.+ await applySchemaAdminMutations(env.vault, [addType("person")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ });+ peerRecord(env, SETTING_SET_KIND, {+ authority: "human",+ key: "brain.schema.page_types",+ value: 7,+ });+ peerRecord(env, SETTING_SET_KIND, {+ authority: "human",+ key: "brain.schema.page_types",+ value: ["paper\n link_types:\n - forged"],+ });+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper", "memo", "person"]);+ expect(loadSchemaPack(env.vault).link_types).toEqual([]);+ expect(assessSchemaPackIntegrity(env.vault).status).toBe("ok");+ // Lists, removals and entries keep working.+ peerRecord(env, SETTING_SET_KIND, {+ authority: "human",+ key: "brain.schema.page_types",+ value: ["paper"],+ });+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper"]);+ });++ test("is refused by the settings writer, on a store-only and on an unswitched machine", () => {+ env = importedStore("osb-schema-scalar-write-");+ const ctx = { vault: env.vault, authority: "human" as const, agent: "ann" };+ expect(() => setBrainSetting(ctx, "schema.page_types", "oops")).toThrow(BrainSettingError);+ expect(loadSchemaPack(env.vault).declarations.page_types).toEqual(["paper", "memo"]);+ env.cleanup();+ env = makeStoreEnv("osb-schema-scalar-write-legacy-", { dualWrite: false });+ const config = "schema_version: 1\nschema:\n page_types:\n - paper\n";+ writeFileSync(brainConfigPath(env.vault), config);+ expect(() =>+ setBrainSetting({ ...ctx, vault: env!.vault }, "schema.page_types", "oops"),+ ).toThrow(BrainSettingError);+ expect(readFileSync(brainConfigPath(env.vault), "utf8")).toBe(config);+ });+});++describe("an empty batch with --adopt-unattested on an unswitched machine", () => {+ test("is refused when the batch is given as an empty list too, and _brain.yaml is left as it is", async () => {+ env = makeStoreEnv("osb-schema-adopt-empty-list-", { dualWrite: false });+ const config = "schema_version: 1\nschema:\n page_types: [ Paper ]\n";+ writeFileSync(brainConfigPath(env.vault), config);+ const opts = { env: { ...process.env, OPEN_SECOND_BRAIN_CONFIG: env.configPath } };+ const refused = await runCli(+ ["brain", "schema", "apply", "--adopt-unattested", "--mutation", "[]", "--actor", "ann"],+ opts,+ );+ expect(refused.returncode).not.toBe(0);+ expect(refused.stderr).toContain("store-only machine");+ expect(readFileSync(brainConfigPath(env.vault), "utf8")).toBe(config);+ });++ test("is refused by the lane itself, whoever calls it", async () => {+ env = makeStoreEnv("osb-schema-adopt-empty-lane-", { dualWrite: false });+ const config = "schema_version: 1\nschema:\n page_types: [ Paper ]\n";+ writeFileSync(brainConfigPath(env.vault), config);+ await expect(+ applySchemaAdminMutations(env.vault, [], {+ actor: "ann",+ now: tick(),+ authority: "human",+ adoptUnattested: true,+ }),+ ).rejects.toThrow("store-only machine");+ expect(readFileSync(brainConfigPath(env.vault), "utf8")).toBe(config);+ expect(computeSchemaPackDigest(loadSchemaPack(env.vault))).toBeString();+ });+});++describe("a record no apply wrote that leaves the pack as it was", () => {+ test("makes the integrity read report the pack as modified, as the next apply refuses it", async () => {+ env = importedStore("osb-schema-unattested-same-pack-");+ await applySchemaAdminMutations(env.vault, [addType("person")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ });+ expect(assessSchemaPackIntegrity(env.vault).status).toBe("ok");+ const before = computeSchemaPackDigest(loadSchemaPack(env.vault));+ // Another device sets an entry the pack already holds: same pack, new record.+ const forged = peerRecord(env, SETTING_ENTRY_KIND, {+ authority: "human",+ key: "brain.schema.page_types",+ entry: "person",+ present: true,+ });+ expect(computeSchemaPackDigest(loadSchemaPack(env.vault))).toBe(before);+ const integrity = assessSchemaPackIntegrity(env.vault);+ expect(integrity.status).toBe("modified");+ expect(JSON.stringify(integrity.mismatches)).toContain(forged);+ await expect(+ applySchemaAdminMutations(env.vault, [addType("memo2")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ }),+ ).rejects.toMatchObject({ code: "unattested_schema_records", records: [forged] });+ });+});++/** A dangling prefix (`p=ghost`, a type nobody declared) another device wrote. */+function danglingPrefix(e: StoreEnv): string {+ return peerRecord(e, SETTING_ENTRY_KIND, {+ authority: "human",+ key: "brain.schema.prefixes",+ entry: "p=ghost",+ present: true,+ });+}++describe("a repair of a dangling keyed entry", () => {+ test("never removes a newer replacement of the same slot that arrived while the apply ran", async () => {+ env = importedStore("osb-schema-repair-replaced-");+ const e = env;+ danglingPrefix(e);+ expect(loadSchemaPack(e.vault).prefixes).toEqual({});+ await applySchemaMutationsToStore(e.vault, [addType("person")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ adoptUnattested: true,+ next: (current) => {+ // The peer replaces the prefix with a valid one after the apply read the pack.+ peerRecord(e, SETTING_ENTRY_KIND, {+ authority: "human",+ key: "brain.schema.prefixes",+ entry: "p=paper",+ present: true,+ });+ return parseSchemaPack(+ `schema_version: 1\n${renderSchemaBlock(current).replace(+ " - memo\n",+ " - memo\n - person\n",+ )}`,+ );+ },+ });+ expect(loadSchemaPack(e.vault).prefixes).toEqual({ p: "paper" });+ });++ test("never removes a replacement written before the repair but delivered after it", async () => {+ env = importedStore("osb-schema-repair-late-replacement-");+ const e = env;+ const ghost = danglingPrefix(e);+ const ghostMs = Number(+ partitionRecords(e.storeRoot, "shared")+ .find((r) => r.id === ghost)!+ .hlc.split(".")[0],+ );+ await applySchemaAdminMutations(e.vault, [addType("person")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ adoptUnattested: true,+ });+ // Only now does sync deliver the peer's replacement, stamped before the+ // repair (half a second after `p=ghost`), so the repair never saw it:+ // its line lands in the peer's shard as a sync tool would put it there.+ const physical = ghostMs + 500;+ const line = buildRecord({+ hlc: { physical, counter: 0 },+ device: "dev-peer",+ agent: "peer",+ kind: SETTING_ENTRY_KIND,+ scope: null,+ topic: null,+ body: { authority: "human", key: "brain.schema.prefixes", entry: "p=paper", present: true },+ });+ appendFileSync(+ shardPath(e.storeRoot, "shared", { stream: "settings" }, "dev-peer", shardMonth(physical)),+ `${JSON.stringify(line)}\n`,+ );+ const repair = schemaEntries(e.storeRoot).find(+ (r) => r.body["present"] === false && r.body["entry"] === "p=ghost",+ );+ const replacement = schemaEntries(e.storeRoot).find((r) => r.body["entry"] === "p=paper");+ // The repair was written (no other record decided the slot when it+ // was) and orders after the replacement.+ expect(repair).toBeDefined();+ expect(compareHlc(parseHlc(replacement!.hlc)!, parseHlc(repair!.hlc)!)).toBeLessThan(0);+ expect(loadSchemaPack(e.vault).prefixes).toEqual({ p: "paper" });+ });+});++describe("an imported compound attribute key with spaces around its dot", () => {+ test("is no dangling reference: an unrelated apply keeps it", async () => {+ const e = makeStoreEnv("osb-schema-compound-spaced-");+ env = e;+ writeFileSync(+ brainConfigPath(e.vault),+ "schema_version: 1\nschema:\n page_types:\n - paper\n - memo\n" ++ " attributes:\n - paper . year=Publication year\n",+ );+ tick();+ importBrainSettings(e.vault, openStore({ root: e.storeRoot, device: "dev-a" }), "migrator");+ switchDevice(e.storeRoot);+ expect(loadSchemaPack(e.vault).attributes).toEqual({ paper: { year: "Publication year" } });++ await applySchemaAdminMutations(e.vault, [addType("person")], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ });++ expect(schemaEntries(e.storeRoot).filter((r) => r.body["present"] === false)).toEqual([]);+ expect(loadSchemaPack(e.vault).attributes).toEqual({ paper: { year: "Publication year" } });+ });+});++describe("an adoption-only apply on a store-only machine", () => {+ test("writes no entry record, a dangling one's repair included", async () => {+ env = importedStore("osb-schema-adopt-only-no-entries-");+ const dangling = danglingPrefix(env);+ const entries = schemaEntries(env.storeRoot).length;+ await applySchemaMutationsToStore(env.vault, [], {+ actor: "operator-ann",+ now: tick(),+ authority: "human",+ adoptUnattested: true,+ next: (current) => current,+ });+ expect(schemaEntries(env.storeRoot)).toHaveLength(entries);+ expect(readStoreSchemaPack().fields["prefixes"]).toEqual(["p=ghost"]);+ // The adoption itself is recorded: the dangling record is attested now.+ expect(assessSchemaPackIntegrity(env.vault).status).toBe("ok");+ expect(readStoreSchemaPack().deciding.map((r) => r.id)).toContain(dangling);+ });+});diff --git a/tests/core/brain/shared-namespace-store.test.ts b/tests/core/brain/shared-namespace-store.test.tsnew file mode 100644index 00000000..61c50fe8--- /dev/null+++ b/tests/core/brain/shared-namespace-store.test.ts@@ -0,0 +1,184 @@+/**+ * The shared namespace and the brain store (task 3.12, design D19 item 3):+ * the namespace gets no store of its own. Where it is store-only on this+ * machine it is this machine's brain store, the one every store-only agent+ * of every machine of the store reads. So the mirror writes there when the+ * origin record is not in that store already (the origin vault still+ * writes the legacy layout and dual writes are off), and refuses when it+ * is (a switched machine, a store-only origin, dual writes on) or when+ * there is no store at all.+ */+import { afterEach, describe, expect, test } from "bun:test";+import { mkdirSync, readdirSync, writeFileSync } from "node:fs";+import { join } from "node:path";++import { writeSignal } from "../../../src/core/brain/signal.ts";+import {+ MIRROR_OUTCOME,+ mirrorNote,+ mirrorSignal,+} from "../../../src/core/brain/shared-namespace.ts";+import { LOG_EVENT_KIND, SIGNAL_KIND } from "../../../src/core/store/constants.ts";+import type { StoreRecord } from "../../../src/core/store/envelope.ts";+import { makeStoreEnv, type StoreEnv } from "../store/fixtures/store-env.ts";+import { partitionRecords, switchDevice } from "./store-only/fixtures.ts";+import { sync, twoMachines, use } from "./store-only/two-machines.ts";++let env: StoreEnv | null = null;+afterEach(() => {+ env?.cleanup();+ env = null;+});++const SIGNAL = {+ topic: "mirror-topic",+ signal: "positive" as const,+ agent: "coding-agent",+ principle: "Mirrored facts stay attributed.",+ created_at: "2026-11-15T10:00:00Z",+ date: "2026-11-15",+ slug: "mirror-topic",+};++/** A shared namespace folder with no legacy tree: store-only on this machine. */+function storeOnlyNamespace(e: StoreEnv): string {+ const shared = join(e.parent, "shared-namespace");+ mkdirSync(shared, { recursive: true });+ return shared;+}++const storeRecords = (e: StoreEnv): StoreRecord[] => [+ ...partitionRecords(e.storeRoot, "shared"),+ ...partitionRecords(e.storeRoot, "private"),+];++const mirrored = (e: StoreEnv, kind: string) =>+ storeRecords(e).filter((r) => r.kind === kind && JSON.stringify(r.body).includes("origin_vault"));++describe("a store-only shared namespace on an unswitched machine", () => {+ test("receives the mirror in this machine's store when the origin wrote the legacy layout", () => {+ env = makeStoreEnv("osb-shared-ns-store-", { dualWrite: false });+ const shared = storeOnlyNamespace(env);+ // The origin write goes to the origin vault's legacy tree only.+ writeSignal(env.vault, SIGNAL);+ expect(storeRecords(env).filter((r) => r.kind === SIGNAL_KIND)).toHaveLength(0);++ expect(mirrorSignal(shared, env.vault, SIGNAL)).toEqual({ outcome: MIRROR_OUTCOME.ok });+ expect(mirrorNote(shared, env.vault, { text: "shared fact", agent: "a" })).toEqual({+ outcome: MIRROR_OUTCOME.ok,+ });+ const [signal] = mirrored(env, SIGNAL_KIND);+ expect(JSON.stringify(signal?.body)).toContain('"origin_vault":"vault"');+ expect(mirrored(env, LOG_EVENT_KIND)).toHaveLength(1);+ // Nothing lands in the namespace folder itself.+ expect(readdirSync(shared)).toEqual([]);+ });++ test("with dual writes on, the origin record reaches the store once and the mirror adds none", () => {+ env = makeStoreEnv("osb-shared-ns-dual-", { dualWrite: true });+ const shared = storeOnlyNamespace(env);+ writeSignal(env.vault, SIGNAL);+ const report = mirrorSignal(shared, env.vault, SIGNAL);+ expect(report.outcome).toBe(MIRROR_OUTCOME.misconfigured);+ expect(storeRecords(env).filter((r) => r.kind === SIGNAL_KIND)).toHaveLength(1);+ expect(mirrored(env, SIGNAL_KIND)).toHaveLength(0);+ });++ test("refuses when this machine has no store to write into, and says how to make one", () => {+ env = makeStoreEnv("osb-shared-ns-nostore-", { dualWrite: false, partitions: [] });+ const shared = storeOnlyNamespace(env);+ const report = mirrorSignal(shared, env.vault, SIGNAL);+ expect(report.outcome).toBe(MIRROR_OUTCOME.misconfigured);+ expect(report.reason).toContain("o2b store init");+ expect(readdirSync(shared)).toEqual([]);+ });++ test("refuses for an origin vault that is store-only too, before any switch", () => {+ env = makeStoreEnv("osb-shared-ns-origin-unswitched-", {+ dualWrite: false,+ legacyTree: false,+ });+ const shared = storeOnlyNamespace(env);+ const report = mirrorSignal(shared, env.vault, SIGNAL);+ expect(report.outcome).toBe(MIRROR_OUTCOME.misconfigured);+ expect(report.reason).toContain("already in this machine's brain store");+ expect(mirrored(env, SIGNAL_KIND)).toHaveLength(0);+ });++ test("a private signal on a machine whose store holds only shared fails cleanly", () => {+ env = makeStoreEnv("osb-shared-ns-private-", { dualWrite: false, partitions: ["shared"] });+ const shared = storeOnlyNamespace(env);+ const secret = { ...SIGNAL, principle: "Keep <private>the client</private> out." };+ const report = mirrorSignal(shared, env.vault, secret);+ expect(report.outcome).toBe(MIRROR_OUTCOME.failed);+ expect(mirrored(env, SIGNAL_KIND)).toHaveLength(0);+ expect(readdirSync(shared)).toEqual([]);+ });+});++describe("a shared namespace path that is no directory", () => {+ for (const shape of ["missing", "file"] as const) {+ test(`(${shape}) is reported failed, and nothing is mirrored into this machine's store`, () => {+ env = makeStoreEnv(`osb-shared-ns-${shape}-`, { dualWrite: false });+ const shared = join(env.parent, "shared-namespace");+ if (shape === "file") writeFileSync(shared, "not a vault\n");+ writeSignal(env.vault, SIGNAL);+ const signal = mirrorSignal(shared, env.vault, SIGNAL);+ const note = mirrorNote(shared, env.vault, { text: "shared fact", agent: "a" });+ for (const report of [signal, note]) {+ expect(report.outcome).toBe(MIRROR_OUTCOME.failed);+ expect(report.reason).toContain("not a directory");+ }+ expect(mirrored(env, SIGNAL_KIND)).toHaveLength(0);+ expect(mirrored(env, LOG_EVENT_KIND)).toHaveLength(0);+ });+ }+});++describe("a switched machine", () => {+ test("still refuses the mirror: the origin record is already in the one store every agent reads", () => {+ env = makeStoreEnv("osb-shared-ns-switched-", { dualWrite: false });+ const [a, b] = twoMachines(env);+ const shared = storeOnlyNamespace(env);+ use(a);+ writeSignal(a.vault, SIGNAL);+ const report = mirrorSignal(shared, a.vault, SIGNAL);+ expect(report.outcome).toBe(MIRROR_OUTCOME.misconfigured);+ expect(mirrored(env, SIGNAL_KIND)).toHaveLength(0);+ // No second store is needed: an agent on another machine of the store+ // reads the origin record once the sync delivers it.+ sync(a, b);+ use(b);+ const atB = [+ ...partitionRecords(b.storeRoot, "shared"),+ ...partitionRecords(b.storeRoot, "private"),+ ].filter((r) => r.kind === SIGNAL_KIND);+ expect(JSON.stringify(atB)).toContain(SIGNAL.principle);+ });++ test("refuses for a store-only origin vault too", () => {+ env = makeStoreEnv("osb-shared-ns-origin-", { dualWrite: false, legacyTree: false });+ switchDevice(env.storeRoot);+ const shared = storeOnlyNamespace(env);+ expect(mirrorSignal(shared, env.vault, SIGNAL).outcome).toBe(MIRROR_OUTCOME.misconfigured);+ });+});++describe("a caller that names its machine config", () => {+ test("is refused by that config's dual-write flag, not by the default config's", () => {+ env = makeStoreEnv("osb-shared-ns-config-", { dualWrite: false });+ const shared = storeOnlyNamespace(env);+ // The default config has dual writes off; the caller's config turns+ // them on, so the origin record already reached the store there.+ const callerConfig = join(env.parent, "caller-config.yaml");+ writeFileSync(callerConfig, `vault: ${env.vault}\nstore_dual_write: "true"\n`);+ const note = mirrorNote(shared, env.vault, { text: "shared fact", agent: "a" }, callerConfig);+ const signal = mirrorSignal(shared, env.vault, SIGNAL, {}, callerConfig);+ for (const report of [note, signal]) {+ expect(report.outcome).toBe(MIRROR_OUTCOME.misconfigured);+ expect(report.reason).toContain("already in this machine's brain store");+ }+ expect(mirrored(env, LOG_EVENT_KIND)).toHaveLength(0);+ expect(mirrored(env, SIGNAL_KIND)).toHaveLength(0);+ });+});diff --git a/tests/core/brain/store-only/two-machines.ts b/tests/core/brain/store-only/two-machines.tsnew file mode 100644index 00000000..9a5370f5--- /dev/null+++ b/tests/core/brain/store-only/two-machines.ts@@ -0,0 +1,81 @@+/**+ * Two store-only machines of one store in one test process (task 3.12):+ * machine A is the store env, machine B a second machine with its own+ * config, vault, device id, device-local state and copy of the store.+ * `use` points the process at one of them; `sync` delivers the shards one+ * machine wrote to the other, as a file-sync tool does.+ */+import { cpSync, existsSync, mkdirSync, readdirSync, statSync, writeFileSync } from "node:fs";+import { join, relative } from "node:path";++import { bootstrapBrain } from "../../../../src/core/brain/init.ts";+import { STORE_ROOT_ENV } from "../../../../src/core/store/constants.ts";+import type { StoreEnv } from "../../store/fixtures/store-env.ts";+import { switchDevice } from "./fixtures.ts";++export interface Machine {+ readonly device: string;+ readonly root: string;+ readonly vault: string;+ readonly storeRoot: string;+ readonly configPath: string;+}++/** Point this process at one machine: its config, store, device and local state. */+export function use(m: Machine): void {+ process.env["OPEN_SECOND_BRAIN_CONFIG"] = m.configPath;+ process.env[STORE_ROOT_ENV] = m.storeRoot;+ process.env["O2B_DEVICE_ID"] = m.device;+ process.env["HOME"] = join(m.root, "home");+ process.env["XDG_STATE_HOME"] = join(m.root, "state");+ process.env["LOCALAPPDATA"] = join(m.root, "state");+}++/** Machine A from the store env, machine B a second machine of the same store. */+export function twoMachines(e: StoreEnv): [Machine, Machine] {+ const a: Machine = {+ device: "dev-a",+ root: e.parent,+ vault: e.vault,+ storeRoot: e.storeRoot,+ configPath: e.configPath,+ };+ const rootB = join(e.parent, "machine-b");+ const b: Machine = {+ device: "dev-b",+ root: rootB,+ vault: join(rootB, "vault"),+ storeRoot: join(rootB, "store"),+ configPath: join(rootB, "config.yaml"),+ };+ mkdirSync(b.vault, { recursive: true });+ writeFileSync(b.configPath, `vault: ${b.vault}\n`);+ // The same store, as the sync tool delivers it before anything is written.+ cpSync(a.storeRoot, b.storeRoot, { recursive: true });+ use(b);+ bootstrapBrain(b.vault);+ switchDevice(b.storeRoot, b.device);+ use(a);+ switchDevice(a.storeRoot, a.device);+ return [a, b];+}++/** Deliver every shard `from` wrote into `to`'s store, as a file sync does. */+export function sync(from: Machine, to: Machine): void {+ const walk = (dir: string): void => {+ for (const name of readdirSync(dir)) {+ const path = join(dir, name);+ if (!statSync(path).isDirectory()) continue;+ if (name === from.device) {+ cpSync(path, join(to.storeRoot, relative(from.storeRoot, path)), {+ recursive: true,+ force: true,+ });+ } else walk(path);+ }+ };+ for (const partition of ["shared", "private"]) {+ const dir = join(from.storeRoot, partition);+ if (existsSync(dir)) walk(dir);+ }+}diff --git a/tests/core/brain/write-session/personas-store.test.ts b/tests/core/brain/write-session/personas-store.test.tsnew file mode 100644index 00000000..5d703829--- /dev/null+++ b/tests/core/brain/write-session/personas-store.test.ts@@ -0,0 +1,291 @@+/**+ * Panel personas are records (task 3.12, design D19 item 2): on a+ * store-only machine a persona is a `lane-entry` of the lane `personas`,+ * written and removed by the persona verbs and read from the fold, so a+ * persona the operator curates on one machine is the panel's lens on every+ * machine of the store, with or without a vault. An unswitched machine+ * keeps the persona notes under `Brain/personas/`.+ */+import { afterEach, describe, expect, setDefaultTimeout, test } from "bun:test";+import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";+import { join } from "node:path";++import {+ DEFAULT_PERSONAS,+ listPersonas,+ loadPersonas,+ PersonaError,+ removePersona,+ setPersona,+} from "../../../../src/core/brain/write-session/personas.ts";+import { freezeVault, unfreezeVault } from "../../../../src/core/brain/freeze.ts";+import { VaultFrozenError } from "../../../../src/core/brain/freeze-marker.ts";+import {+ holdMigrationGate,+ MigrationInProgressError,+ releaseMigrationGate,+} from "../../../../src/core/migration-gate.ts";+import { STORE_DUAL_WRITE_ENV } from "../../../../src/core/store/constants.ts";+import { writeLanePage } from "../../../../src/core/store/lane-pages.ts";+import { runCli } from "../../../helpers/run-cli.ts";+import { CLI_SPAWN_BUDGET_MS } from "../../../helpers/cli-timeout.ts";+import { makeStoreEnv, type StoreEnv } from "../../store/fixtures/store-env.ts";+import { brainTreeDigest, partitionRecords, switchDevice } from "../store-only/fixtures.ts";+import { sync, twoMachines, use } from "../store-only/two-machines.ts";++setDefaultTimeout(CLI_SPAWN_BUDGET_MS);++let env: StoreEnv | null = null;+afterEach(() => {+ env?.cleanup();+ env = null;+});++const LEGAL = {+ slug: "legal",+ lens: "legal exposure",+ prompt: "Assess the topic through legal exposure: contracts, licences and liability.",+};++const personaRecords = (root: string) =>+ partitionRecords(root, "shared").filter(+ (r) => r.kind.startsWith("lane-entry") && r.body["lane"] === "personas",+ );++describe("personas on a store-only machine", () => {+ test("a persona set on one machine is the panel's lens on another, and no file is written", () => {+ env = makeStoreEnv("osb-personas-store-");+ const [a, b] = twoMachines(env);+ const before = brainTreeDigest(a.vault);+ use(a);+ expect(setPersona(a.vault, LEGAL)).toEqual({ slug: "legal", changed: true });+ expect(brainTreeDigest(a.vault)).toBe(before);+ expect(personaRecords(a.storeRoot)).toHaveLength(1);+ expect(loadPersonas(a.vault)).toEqual([LEGAL]);++ use(b);+ expect(loadPersonas(b.vault)).toEqual(DEFAULT_PERSONAS);+ sync(a, b);+ expect(loadPersonas(b.vault)).toEqual([LEGAL]);+ expect(listPersonas(b.vault)).toEqual({ source: "custom", personas: [LEGAL] });+ });++ test("removing the last persona brings the default set back on every machine", () => {+ env = makeStoreEnv("osb-personas-remove-");+ const [a, b] = twoMachines(env);+ use(a);+ setPersona(a.vault, LEGAL);+ sync(a, b);+ use(b);+ expect(removePersona(b.vault, "legal")).toBe(true);+ expect(removePersona(b.vault, "legal")).toBe(false);+ sync(b, a);+ use(a);+ expect(loadPersonas(a.vault)).toEqual(DEFAULT_PERSONAS);+ expect(listPersonas(a.vault).source).toBe("default");+ });++ test("works with no vault at all", () => {+ env = makeStoreEnv("osb-personas-novault-", { configureVault: false, legacyTree: false });+ setPersona(null, LEGAL);+ expect(listPersonas(null)).toEqual({ source: "custom", personas: [LEGAL] });+ });++ test("a slug that is no plain lowercase name, or an empty prompt, is refused", () => {+ env = makeStoreEnv("osb-personas-refuse-");+ switchDevice(env.storeRoot);+ expect(() => setPersona(env!.vault, { ...LEGAL, slug: "../x" })).toThrow(PersonaError);+ expect(() => setPersona(env!.vault, { ...LEGAL, slug: "Legal" })).toThrow(PersonaError);+ expect(() => setPersona(env!.vault, { ...LEGAL, prompt: " " })).toThrow(PersonaError);+ expect(personaRecords(env.storeRoot)).toHaveLength(0);+ });+});++describe("personas on an unswitched machine", () => {+ test("stay notes under Brain/personas/", () => {+ env = makeStoreEnv("osb-personas-legacy-", { dualWrite: false });+ setPersona(env.vault, LEGAL);+ const path = join(env.vault, "Brain", "personas", "legal.md");+ expect(readFileSync(path, "utf8")).toContain("kind: persona");+ expect(loadPersonas(env.vault)).toEqual([LEGAL]);+ expect(removePersona(env.vault, "legal")).toBe(true);+ expect(existsSync(path)).toBe(false);+ });+});++describe("personas on an unswitched machine with dual writes", () => {+ test("a persona removed before the switch stays removed after it", () => {+ env = makeStoreEnv("osb-personas-dual-remove-");+ const path = join(env.vault, "Brain", "personas", "legal.md");+ setPersona(env.vault, LEGAL);+ expect(personaRecords(env.storeRoot)).toHaveLength(1);+ expect(removePersona(env.vault, "legal")).toBe(true);+ expect(existsSync(path)).toBe(false);+ switchDevice(env.storeRoot);+ expect(loadPersonas(env.vault)).toEqual(DEFAULT_PERSONAS);+ expect(removePersona(env.vault, "legal")).toBe(false);+ });++ test("a persona that never reached the store is removed as a file alone", () => {+ env = makeStoreEnv("osb-personas-dual-legacy-only-", { dualWrite: false });+ setPersona(env.vault, LEGAL);+ process.env[STORE_DUAL_WRITE_ENV] = "1";+ expect(removePersona(env.vault, "legal")).toBe(true);+ expect(personaRecords(env.storeRoot)).toEqual([]);+ });+});++describe("the persona verbs of o2b brain panel", () => {+ test("write a human persona record and list it", async () => {+ env = makeStoreEnv("osb-personas-cli-");+ switchDevice(env.storeRoot);+ const promptFile = join(env.parent, "prompt.txt");+ writeFileSync(promptFile, `${LEGAL.prompt}\n`);+ const cliEnv = { env: { OPEN_SECOND_BRAIN_CONFIG: env.configPath } };+ const set = await runCli(+ ["brain", "panel", "persona-set", "legal", "--lens", LEGAL.lens, "--file", promptFile],+ cliEnv,+ );+ expect(set.returncode).toBe(0);+ const [record] = personaRecords(env.storeRoot);+ expect(record?.body["authority"]).toBe("human");+ const list = await runCli(["brain", "panel", "personas", "--json"], cliEnv);+ expect(list.returncode).toBe(0);+ expect(JSON.parse(list.stdout)).toMatchObject({ source: "custom", personas: [LEGAL] });+ const removed = await runCli(["brain", "panel", "persona-remove", "legal"], cliEnv);+ expect(removed.returncode).toBe(0);+ expect(loadPersonas(env.vault)).toEqual(DEFAULT_PERSONAS);+ });+});++describe("the persona verbs refuse", () => {+ test("in a frozen vault, and while a store migration runs on it", () => {+ env = makeStoreEnv("osb-personas-frozen-", { dualWrite: false });+ const vault = env.vault;+ setPersona(vault, LEGAL);+ const path = join(vault, "Brain", "personas", "legal.md");+ const before = readFileSync(path, "utf8");+ freezeVault(vault, { agent: "@operator", reason: "moving machines" });+ expect(() => setPersona(vault, { ...LEGAL, prompt: "changed" })).toThrow(VaultFrozenError);+ expect(() => setPersona(vault, { ...LEGAL, slug: "new" })).toThrow(VaultFrozenError);+ expect(() => removePersona(vault, "legal")).toThrow(VaultFrozenError);+ expect(readFileSync(path, "utf8")).toBe(before);+ unfreezeVault(vault);+ holdMigrationGate(vault, "run-personas");+ try {+ expect(() => removePersona(vault, "legal")).toThrow(MigrationInProgressError);+ expect(existsSync(path)).toBe(true);+ } finally {+ releaseMigrationGate(vault, "run-personas");+ }+ });++ test("on a switched machine too, in a frozen vault; a leftover migration gate refuses nothing", () => {+ env = makeStoreEnv("osb-personas-frozen-switched-");+ switchDevice(env.storeRoot);+ const vault = env.vault;+ setPersona(vault, LEGAL);+ expect(personaRecords(env.storeRoot)).toHaveLength(1);+ freezeVault(vault, { agent: "@operator", reason: "moving machines" });+ try {+ expect(() => setPersona(vault, { ...LEGAL, prompt: "changed" })).toThrow(VaultFrozenError);+ expect(() => removePersona(vault, "legal")).toThrow(VaultFrozenError);+ } finally {+ unfreezeVault(vault);+ }+ expect(personaRecords(env.storeRoot)).toHaveLength(1);+ expect(loadPersonas(vault)).toEqual([LEGAL]);+ // The migration that switched the machine is over: a gate it left+ // naming the vault refuses no store write (design D14 item 13).+ holdMigrationGate(vault, "run-personas-switched");+ try {+ expect(removePersona(vault, "legal")).toBe(true);+ } finally {+ releaseMigrationGate(vault, "run-personas-switched");+ }+ });++ test("a replacement of a persona that lives in private, by name, and keeps it", () => {+ env = makeStoreEnv("osb-personas-private-target-");+ switchDevice(env.storeRoot);+ const path = join(env.vault, "Brain", "personas", "legal.md");+ writeLanePage(+ env.vault,+ path,+ { kind: "persona", lens: LEGAL.lens, visibility: "private" },+ `${LEGAL.prompt}\n`,+ );+ const privateEntries = () =>+ partitionRecords(env!.storeRoot, "private").filter((r) => r.kind.startsWith("lane-entry"));+ expect(privateEntries()).toHaveLength(1);+ expect(() => setPersona(env!.vault, { ...LEGAL, prompt: "A shared prompt." })).toThrow(+ /lives in private.*persona-remove legal/,+ );+ expect(privateEntries()).toHaveLength(1);+ expect(personaRecords(env.storeRoot)).toHaveLength(0);+ // Removed first, it can be recreated as shared.+ expect(removePersona(env.vault, "legal")).toBe(true);+ setPersona(env.vault, { ...LEGAL, prompt: "A shared prompt." });+ expect(personaRecords(env.storeRoot)).toHaveLength(1);+ });++ test("a persona whose text is private, by name", () => {+ env = makeStoreEnv("osb-personas-private-");+ switchDevice(env.storeRoot);+ for (const input of [+ { ...LEGAL, prompt: "Assess it. <private>the client's name</private>" },+ { ...LEGAL, lens: "<private>secret</private> lens" },+ ]) {+ expect(() => setPersona(env!.vault, input)).toThrow(/private/);+ }+ expect(personaRecords(env.storeRoot)).toHaveLength(0);+ expect(+ partitionRecords(env.storeRoot, "private").filter((r) => r.kind.startsWith("lane-entry")),+ ).toHaveLength(0);+ });++ test("a prompt over 16 KB, a lens over 200 characters, and a lens with a line separator", () => {+ env = makeStoreEnv("osb-personas-bounds-");+ switchDevice(env.storeRoot);+ const refused = [+ { ...LEGAL, prompt: "x".repeat(16 * 1024 + 1) },+ { ...LEGAL, prompt: "é".repeat(8 * 1024 + 1) },+ { ...LEGAL, lens: "l".repeat(201) },+ { ...LEGAL, lens: "legal
exposure" },+ { ...LEGAL, lens: "legal
exposure" },+ { ...LEGAL, lens: "legal\u0085exposure" },+ ];+ for (const input of refused) expect(() => setPersona(env!.vault, input)).toThrow(PersonaError);+ expect(personaRecords(env.storeRoot)).toHaveLength(0);+ // At the limits it is accepted.+ setPersona(env.vault, { ...LEGAL, lens: "l".repeat(200), prompt: "x".repeat(16 * 1024) });+ expect(personaRecords(env.storeRoot)).toHaveLength(1);+ });++ test("on a note at that path that is no persona", () => {+ env = makeStoreEnv("osb-personas-other-note-", { dualWrite: false });+ const path = join(env.vault, "Brain", "personas", "readme.md");+ mkdirSync(join(env.vault, "Brain", "personas"), { recursive: true });+ const note = "---\nkind: note\n---\nHow to write personas.\n";+ writeFileSync(path, note);+ expect(() => removePersona(env!.vault, "readme")).toThrow(PersonaError);+ expect(() => setPersona(env!.vault, { ...LEGAL, slug: "readme" })).toThrow(PersonaError);+ expect(readFileSync(path, "utf8")).toBe(note);+ });+});++describe("the persona verbs with no vault configured", () => {+ test("write and list a persona record", async () => {+ env = makeStoreEnv("osb-personas-cli-novault-", { configureVault: false, legacyTree: false });+ const promptFile = join(env.parent, "prompt.txt");+ writeFileSync(promptFile, `${LEGAL.prompt}\n`);+ const cliEnv = { env: { OPEN_SECOND_BRAIN_CONFIG: env.configPath } };+ const set = await runCli(+ ["brain", "panel", "persona-set", "legal", "--lens", LEGAL.lens, "--file", promptFile],+ cliEnv,+ );+ expect(set.returncode).toBe(0);+ const list = await runCli(["brain", "panel", "personas", "--json"], cliEnv);+ expect(JSON.parse(list.stdout)).toMatchObject({ source: "custom", personas: [LEGAL] });+ });+});diff --git a/tests/core/lanes/lane-dual-moves.test.ts b/tests/core/lanes/lane-dual-moves.test.tsnew file mode 100644index 00000000..de7eabcb--- /dev/null+++ b/tests/core/lanes/lane-dual-moves.test.ts@@ -0,0 +1,402 @@+/**+ * A lane removal or move on an unswitched machine with the dual write on+ * (task 3.12 review): the page's store entry, which the dual write put+ * there, is removed or moved with it, so after the switch the store holds+ * what the legacy tree held. A page that never reached the store is moved+ * or removed as a file alone.+ */+import { afterEach, describe, expect, test } from "bun:test";+import {+ existsSync,+ mkdirSync,+ mkdtempSync,+ readdirSync,+ rmSync,+ symlinkSync,+ writeFileSync,+} from "node:fs";+import { tmpdir } from "node:os";+import { dirname, join } from "node:path";++import { STORE_DUAL_WRITE_ENV } from "../../../src/core/store/constants.ts";+import {+ moveLanePage,+ moveLaneTree,+ removeLanePage,+ removeLaneTree,+ writeLanePage,+ writeLaneText,+} from "../../../src/core/store/lane-pages.ts";+import { foldIncomplete } from "../../../src/core/store/lane-entry-store.ts";+import { foldLaneEntries } from "../../../src/core/store/lane-records.ts";+import type { StoreRecord } from "../../../src/core/store/envelope.ts";+import { makeLaneEnv, type LaneEnv } from "./fixtures/lane-env.ts";++let lane: LaneEnv | null = null;+afterEach(() => {+ lane?.cleanup();+ lane = null;+});++/** The live entries of `records` (default: the whole store), by `<lane>/<entry>`, with their trimmed text. */+const liveTexts = (env: LaneEnv, records: StoreRecord[] = env.laneRecords()) =>+ Object.fromEntries(+ [...foldLaneEntries(records).entries]+ .filter(([, state]) => !state.removed)+ .map(([key, state]) => [key, state.value!.text.trim()])+ .toSorted(([a], [b]) => a!.localeCompare(b!)),+ );++/** The live entries of the store, by `<lane>/<entry>`. */+const live = (env: LaneEnv) => Object.keys(liveTexts(env));++/** Run `fn` with the dual write off: a write the store never sees. */+function legacyOnly(fn: () => void): void {+ delete process.env[STORE_DUAL_WRITE_ENV];+ try {+ fn();+ } finally {+ process.env[STORE_DUAL_WRITE_ENV] = "1";+ }+}++describe("an unswitched machine with the dual write on", () => {+ test("removes and moves the stored entries with the pages, one by one and by folder", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-moves-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ writeLanePage(vault, join(deadEnds, "de-a.md"), { kind: "brain-dead-end" }, "A.\n");+ writeLanePage(vault, join(deadEnds, "de-b.md"), { kind: "brain-dead-end" }, "B.\n");+ const staged = join(vault, "Brain", "dream", "staged");+ writeLaneText(vault, join(staged, "s1", "manifest.json"), '{"schema":"x"}\n');+ writeLaneText(vault, join(staged, "s2", "manifest.json"), '{"schema":"y"}\n');++ removeLanePage(vault, join(deadEnds, "de-a.md"));+ moveLanePage(vault, join(deadEnds, "de-b.md"), join(deadEnds, "archive", "de-b.md"));+ moveLaneTree(vault, join(staged, "s1"), join(vault, "Brain", "dream", "applied", "s1"));+ removeLaneTree(vault, join(staged, "s2"));++ expect(live(lane)).toEqual(["dead-ends/archive/de-b", "dream/applied/s1/manifest.json"]);+ });++ test("leaves the store alone for a page that never reached it", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-legacy-only-");+ const { vault } = lane;+ const page = join(vault, "Brain", "dead-ends", "de-old.md");+ delete process.env[STORE_DUAL_WRITE_ENV];+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "Old.\n");+ process.env[STORE_DUAL_WRITE_ENV] = "1";+ expect(removeLanePage(vault, page)).toBe(true);+ expect(lane.laneRecords()).toEqual([]);+ });+});++describe("an unswitched machine's dual-written move", () => {+ test("onto its own path writes nothing that removes the entry, page or folder", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-self-");+ const { vault } = lane;+ const page = join(vault, "Brain", "dead-ends", "de-a.md");+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "A.\n");+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "manifest.json"), '{"schema":"x"}\n');+ const before = lane.laneRecords().length;++ moveLanePage(vault, page, page);+ moveLanePage(vault, page, join(vault, "Brain", "dead-ends", ".", "de-a.md"));+ moveLaneTree(vault, staged, staged);++ expect(live(lane)).toEqual(["dead-ends/de-a", "dream/staged/s1/manifest.json"]);+ expect(lane.laneRecords()).toHaveLength(before);+ });++ test("builds the moved entry from the page as it is on disk after the move", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-moved-content-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ const page = join(deadEnds, "de-a.md");+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "Old.\n");+ // A legacy-only edit the store never saw.+ legacyOnly(() =>+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "New.\n", { overwrite: true }),+ );+ moveLanePage(vault, page, join(deadEnds, "archive", "de-a.md"));+ expect(liveTexts(lane)).toEqual({ "dead-ends/archive/de-a": "New." });++ // A stored entry whose page was removed legacy-only moves nowhere: its+ // tombstone alone, so the store does not keep a page the tree lost.+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "a.json"), '{"a":1}\n');+ writeLaneText(vault, join(staged, "b.json"), '{"b":1}\n');+ legacyOnly(() => rmSync(join(staged, "b.json")));+ moveLaneTree(vault, staged, join(vault, "Brain", "dream", "applied", "s1"));+ expect(live(lane)).toEqual(["dead-ends/archive/de-a", "dream/applied/s1/a.json"]);+ });++ test("a private page moved over a shared one leaves no shared copy behind", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-over-shared-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ const target = join(deadEnds, "de-target.md");+ writeLanePage(vault, target, { kind: "brain-dead-end" }, "Public.\n");+ const secret = join(deadEnds, "de-secret.md");+ writeLanePage(vault, secret, { kind: "brain-dead-end", visibility: "private" }, "Secret.\n");++ moveLanePage(vault, secret, target);++ // A device that holds only `shared` no longer reads the replaced page.+ expect(liveTexts(lane, lane.laneRecords("shared"))).toEqual({});+ expect(liveTexts(lane)).toEqual({ "dead-ends/de-target": "Secret." });+ expect(JSON.stringify(lane.laneRecords("shared").map((r) => r.body))).not.toContain("Secret");+ });++ test("a shared page moved over a private one stays private, as a write over it does", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-over-private-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ const target = join(deadEnds, "de-target.md");+ writeLanePage(vault, target, { kind: "brain-dead-end", visibility: "private" }, "Secret.\n");+ const open = join(deadEnds, "de-open.md");+ writeLanePage(vault, open, { kind: "brain-dead-end" }, "Open.\n");+ // A private legacy page the store never saw keeps its path private too.+ const legacyTarget = join(deadEnds, "de-legacy.md");+ legacyOnly(() =>+ writeLanePage(vault, legacyTarget, { kind: "brain-dead-end", visibility: "private" }, "L.\n"),+ );+ const other = join(deadEnds, "de-other.md");+ writeLanePage(vault, other, { kind: "brain-dead-end" }, "Other.\n");++ moveLanePage(vault, open, target);+ moveLanePage(vault, other, legacyTarget);++ expect(liveTexts(lane)).toEqual({+ "dead-ends/de-legacy": "Other.",+ "dead-ends/de-target": "Open.",+ });+ expect(liveTexts(lane, lane.laneRecords("shared"))).toEqual({});+ expect(JSON.stringify(lane.laneRecords("shared").map((r) => r.body))).not.toContain("Secret");+ });++ test("a page the store never saw, moved over one it holds, replaces the stored entry", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-untracked-over-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ const target = join(deadEnds, "de-target.md");+ writeLanePage(vault, target, { kind: "brain-dead-end" }, "Stale.\n");+ const fresh = join(deadEnds, "de-fresh.md");+ legacyOnly(() => writeLanePage(vault, fresh, { kind: "brain-dead-end" }, "Fresh.\n"));+ moveLanePage(vault, fresh, target);+ expect(liveTexts(lane)).toEqual({ "dead-ends/de-target": "Fresh." });++ // A private one is judged as a write of the page is: out of shared.+ const shared = join(deadEnds, "de-shared.md");+ writeLanePage(vault, shared, { kind: "brain-dead-end" }, "Shared.\n");+ const secret = join(deadEnds, "de-secret.md");+ legacyOnly(() =>+ writeLanePage(vault, secret, { kind: "brain-dead-end", visibility: "private" }, "Secret.\n"),+ );+ moveLanePage(vault, secret, shared);+ expect(liveTexts(lane)).toEqual({+ "dead-ends/de-shared": "Secret.",+ "dead-ends/de-target": "Fresh.",+ });+ expect(liveTexts(lane, lane.laneRecords("shared"))).toEqual({+ "dead-ends/de-target": "Fresh.",+ });+ expect(JSON.stringify(lane.laneRecords("shared").map((r) => r.body))).not.toContain("Secret");++ // In a folder: a file the store never saw lands where the store holds a stale entry.+ const applied = join(vault, "Brain", "dream", "applied", "s1");+ writeLaneText(vault, join(applied, "m.json"), '{"stale":1}\n');+ legacyOnly(() => rmSync(applied, { recursive: true }));+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ legacyOnly(() => writeLaneText(vault, join(staged, "m.json"), '{"fresh":1}\n'));+ moveLaneTree(vault, staged, applied);+ expect(liveTexts(lane)["dream/applied/s1/m.json"]).toBe("");+ const fields = foldLaneEntries(lane.laneRecords()).entries.get("dream/applied/s1/m.json");+ expect(fields?.value?.fields).toEqual({ fresh: 1 });+ });++ test("while the fold misses a shard, the moved entry stays out of shared", () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-incomplete-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ const page = join(deadEnds, "de-a.md");+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "Open.\n");+ writeLanePage(+ vault,+ join(deadEnds, "de-p.md"),+ { kind: "brain-dead-end", visibility: "private" },+ "Private.\n",+ );+ // The entry may have been raised in a shard this machine saw and has+ // lost since.+ expect(foldIncomplete()).toBe(false);+ const dir = join(lane.env.storeRoot, "private", "events");+ for (const device of readdirSync(dir)) rmSync(join(dir, device), { recursive: true });+ legacyOnly(() => writeFileSync(page, "---\nkind: brain-dead-end\n---\nRaised.\n"));+ expect(foldIncomplete()).toBe(true);+ const shared = lane.laneRecords("shared").length;++ moveLanePage(vault, page, join(deadEnds, "archive", "de-a.md"));++ // Only the old entry's tombstone reaches shared; the moved text does not.+ const added = lane.laneRecords("shared").slice(shared);+ expect(added.map((r) => r.kind)).toEqual(["lane-entry-remove"]);+ expect(JSON.stringify(lane.laneRecords("private").map((r) => r.body))).toContain("Raised.");+ });+});++/** Whether this host lets the test create a symlink (Windows needs a privilege). */+const CAN_SYMLINK = ((): boolean => {+ const probe = mkdtempSync(join(tmpdir(), "osb-lane-symlink-probe-"));+ try {+ symlinkSync(probe, join(probe, "link"), "dir");+ return true;+ } catch {+ return false;+ } finally {+ rmSync(probe, { recursive: true, force: true });+ }+})();++describe("an unswitched machine's dual-written move of a symbolic link", () => {+ test.skipIf(!CAN_SYMLINK)(+ "never reads the link's target into the store, as a page or through a folder",+ () => {+ lane = makeLaneEnv("legacy", "osb-lane-dual-symlink-");+ const { vault } = lane;+ const outside = join(dirname(vault), "outside");+ mkdirSync(join(outside, "s1"), { recursive: true });+ writeFileSync(join(outside, "note.md"), "---\nkind: brain-dead-end\n---\nOutside text.\n");+ writeFileSync(join(outside, "s1", "a.json"), '{"outside":"text"}\n');++ const deadEnds = join(vault, "Brain", "dead-ends");+ const page = join(deadEnds, "de-a.md");+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "A.\n");+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "a.json"), '{"a":1}\n');+ // Legacy-only, the page and the folder become links to files outside the vault.+ legacyOnly(() => {+ rmSync(page);+ symlinkSync(join(outside, "note.md"), page, "file");+ rmSync(staged, { recursive: true });+ symlinkSync(join(outside, "s1"), staged, "dir");+ });++ moveLanePage(vault, page, join(deadEnds, "archive", "de-a.md"));+ moveLaneTree(vault, staged, join(vault, "Brain", "dream", "applied", "s1"));++ // Neither link is a page the store would import: the old entries go, nothing arrives.+ expect(live(lane)).toEqual([]);+ const bodies = JSON.stringify(lane.laneRecords().map((r) => r.body));+ expect(bodies).not.toContain("Outside text");+ expect(bodies).not.toContain("outside");+ },+ );+});++describe("a store-only machine's move", () => {+ test("of a folder onto itself keeps its entries", () => {+ lane = makeLaneEnv("store", "osb-lane-store-self-");+ const { vault } = lane;+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "manifest.json"), '{"schema":"x"}\n');+ moveLaneTree(vault, staged, staged);+ expect(live(lane)).toEqual(["dream/staged/s1/manifest.json"]);+ });++ test("of a folder into its own subfolder is refused, as a rename is, and keeps its entries", () => {+ lane = makeLaneEnv("store", "osb-lane-store-into-self-");+ const { vault } = lane;+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "a.json"), '{"a":1}\n');+ writeLaneText(vault, join(staged, "sub", "a.json"), '{"b":1}\n');+ const before = lane.laneRecords().length;+ expect(() => moveLaneTree(vault, staged, join(staged, "sub"))).toThrow(/into itself/);+ expect(lane.laneRecords()).toHaveLength(before);+ expect(live(lane)).toEqual(["dream/staged/s1/a.json", "dream/staged/s1/sub/a.json"]);+ });++ test("of a private page over a shared one leaves no shared copy behind", () => {+ lane = makeLaneEnv("store", "osb-lane-store-over-shared-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ const target = join(deadEnds, "de-target.md");+ writeLanePage(vault, target, { kind: "brain-dead-end" }, "Public.\n");+ writeLanePage(+ vault,+ join(deadEnds, "de-secret.md"),+ { kind: "brain-dead-end", visibility: "private" },+ "Secret.\n",+ );+ moveLanePage(vault, join(deadEnds, "de-secret.md"), target);+ expect(liveTexts(lane, lane.laneRecords("shared"))).toEqual({});++ const applied = join(vault, "Brain", "dream", "applied", "s1");+ writeLaneText(vault, join(applied, "m.json"), '{"public":1}\n');+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "m.json"), '{"visibility":"private","s":1}\n');+ moveLaneTree(vault, staged, applied);+ expect(liveTexts(lane, lane.laneRecords("shared"))).toEqual({});+ expect(Object.keys(liveTexts(lane))).toEqual([+ "dead-ends/de-target",+ "dream/applied/s1/m.json",+ ]);+ });++ test("of a shared page or folder over a private entry keeps the entry private", () => {+ lane = makeLaneEnv("store", "osb-lane-store-over-private-");+ const { vault } = lane;+ const deadEnds = join(vault, "Brain", "dead-ends");+ const target = join(deadEnds, "de-target.md");+ writeLanePage(vault, target, { kind: "brain-dead-end", visibility: "private" }, "Secret.\n");+ writeLanePage(vault, join(deadEnds, "de-open.md"), { kind: "brain-dead-end" }, "Open.\n");+ moveLanePage(vault, join(deadEnds, "de-open.md"), target);++ const applied = join(vault, "Brain", "dream", "applied", "s1");+ writeLaneText(vault, join(applied, "m.json"), '{"visibility":"private","s":1}\n');+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "m.json"), '{"public":1}\n');+ moveLaneTree(vault, staged, applied);++ expect(live(lane)).toEqual(["dead-ends/de-target", "dream/applied/s1/m.json"]);+ expect(liveTexts(lane, lane.laneRecords("shared"))).toEqual({});+ });++ test("onto its own page writes nothing, a trailing slash included", () => {+ lane = makeLaneEnv("store", "osb-lane-store-self-page-");+ const { vault } = lane;+ const page = join(vault, "Brain", "dead-ends", "de-a.md");+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "A.\n");+ const before = lane.laneRecords().length;+ moveLanePage(vault, page, page);+ moveLanePage(vault, page, `${page}/`);+ expect(lane.laneRecords()).toHaveLength(before);+ expect(live(lane)).toEqual(["dead-ends/de-a"]);+ });+});++describe("a lane move out of Brain/", () => {+ for (const mode of ["legacy", "store"] as const) {+ test(`is refused before anything moves (${mode})`, () => {+ lane = makeLaneEnv(mode, `osb-lane-out-${mode}-`);+ const { vault } = lane;+ const staged = join(vault, "Brain", "dream", "staged", "s1");+ writeLaneText(vault, join(staged, "a.json"), '{"a":1}\n');+ const page = join(vault, "Brain", "dead-ends", "de-a.md");+ writeLanePage(vault, page, { kind: "brain-dead-end" }, "A.\n");+ const before = lane.laneRecords().length;++ expect(() => moveLaneTree(vault, staged, join(vault, "outside", "s1"))).toThrow(+ /out of the Brain lanes/,+ );+ expect(() => moveLanePage(vault, page, join(vault, "outside", "de-a.md"))).toThrow(+ /out of the Brain lanes/,+ );++ expect(lane.laneRecords()).toHaveLength(before);+ expect(live(lane)).toEqual(["dead-ends/de-a", "dream/staged/s1/a.json"]);+ expect(existsSync(join(vault, "outside"))).toBe(false);+ });+ }+});diff --git a/tests/core/lanes/lane-move-table.test.ts b/tests/core/lanes/lane-move-table.test.tsnew file mode 100644index 00000000..04016853--- /dev/null+++ b/tests/core/lanes/lane-move-table.test.ts@@ -0,0 +1,638 @@+/**+ * A lane move over its whole state space (task 3.12 review, design D19+ * item 2). The rule under test: a move is the write of the destination+ * page plus the removal of the source, judged as those two writes would be+ * judged. One oracle ({@link expected}) computes what every cell must+ * leave, so no cell carries a hand-written expectation.+ *+ * The table is the product of:+ *+ * - the source: tracked in `shared`, tracked in `private`, or untracked+ * (a legacy page the store never saw);+ * - what is at the destination after the rename: a page with public+ * content, a page private by its content (`visibility: private`), or a+ * symbolic link (no page the store would import);+ * - the destination before the move: stored in `shared`, stored in+ * `private`, or not stored;+ * - the mode: an unswitched machine with the dual write on, or a+ * store-only machine;+ * - the shape: a single page (`moveLanePage`) or the same page inside a+ * moved folder (`moveLaneTree`).+ *+ * Cells that cannot exist, so the table leaves them out:+ *+ * - a store-only move of an untracked source: a store-only machine holds+ * no page outside the store, so there is nothing to move;+ * - a store-only move that carries a symbolic link: the store holds+ * records, never links;+ * - a store-only move of a source in `shared` whose content is private:+ * on a store-only machine the moved content is the stored value, and+ * every write judges its content, so a value private by its frontmatter+ * is never stored in `shared`.+ *+ * Cells that carry a symbolic link are skipped where the host cannot create+ * one (Windows without the privilege).+ *+ * Two more dimensions, each crossed with a representative subset of the+ * ones above so the table stays fast:+ *+ * - the completeness of the fold the move reads: complete (every cell+ * above), an `events` shard read only in part (a corrupt line in the+ * middle), or a record of another device in `private` whose body blob+ * has not arrived (quarantined on this machine, so its shard reads as+ * whole), for the source entry or for the destination entry. Each+ * incomplete value is crossed with every cell above that carries a+ * public page (the cells where the verdict could be `shared`; a private+ * page or a link already decides it). The oracle: private whenever the+ * fold is incomplete, since the record it misses may make the entry+ * private. A record of another device in `shared` whose body blob has+ * not arrived, quarantined, for the source or the destination entry,+ * is no such gap: a record in `shared` never carries a value a write+ * would place in `private`, and the fold's winner of a key changes only+ * when the missing record is that winner, so the fold without it is at+ * least as private as the fold with it. These cells cross it with the+ * cells whose verdict over the visible fold is `shared` (a public page,+ * a source in `shared` or untracked, a destination in `shared` or not+ * stored, an untracked source onto an unstored path left out because+ * the missing record would store the destination there). The oracle:+ * what the complete fold gives, `shared`. A hidden newer value of an+ * entry the visible fold holds in `private` is left out: there the+ * complete fold would read `shared` and the move keeps `private`, the+ * side that leaks nothing;+ * - for a folder, the overlap of the source and the destination folders:+ * disjoint, the destination below the source, the destination an+ * ancestor of the source, and the same folder, in both modes, over a+ * folder holding a shared entry and a private one in a nested folder of+ * the same name. The oracle: refused with nothing written and every+ * entry still live, or the exact live set after the move; on an+ * unswitched machine the live set is also what the disk shows.+ *+ * The unswitched folder shape removes the destination folder legacy-only+ * before the move (the store keeps its entries there): a rename refuses a+ * non-empty destination folder, so the stored destination entry is one+ * whose file the tree lost.+ */+import { afterEach, describe, expect, test } from "bun:test";+import {+ existsSync,+ mkdirSync,+ mkdtempSync,+ readdirSync,+ readFileSync,+ rmSync,+ symlinkSync,+ writeFileSync,+} from "node:fs";+import { tmpdir } from "node:os";+import { dirname, join, relative } from "node:path";++import { appendRecord } from "../../../src/core/store/appender.ts";+import { BLOB_REF_KEY, STORE_DUAL_WRITE_ENV } from "../../../src/core/store/constants.ts";+import type { StoreRecord } from "../../../src/core/store/envelope.ts";+import {+ dualWriteLaneMove,+ dualWriteLaneTreeMove,+} from "../../../src/core/store/lane-dual-moves.ts";+import { laneEntryInPrivate } from "../../../src/core/store/lane-entry-store.ts";+import {+ moveLanePage,+ moveLaneTree,+ readLanePage,+ writeLanePage,+} from "../../../src/core/store/lane-pages.ts";+import { lanePlaceOf } from "../../../src/core/store/lane-paths.ts";+import { foldLaneEntries, laneEntryRecord } from "../../../src/core/store/lane-records.ts";+import { readShard } from "../../../src/core/store/reader.ts";+import { addQuarantine } from "../../../src/core/store/shard-quarantine-write.ts";+import { lineSha256At } from "../../../src/core/store/shard-quarantine.ts";+import { openStore } from "../../../src/core/store/store-open.ts";+import { partitionDir } from "../../../src/core/store/store-paths.ts";+import type { FrontmatterMap } from "../../../src/core/types.ts";+import { makeLaneEnv, type LaneEnv } from "./fixtures/lane-env.ts";++type Source = "shared" | "private" | "untracked";+type Disk = "public" | "private" | "link";+type Destination = "shared" | "private" | "none";+type Mode = "legacy" | "store";+type Shape = "page" | "folder";+type Fold =+ | "complete"+ | "unreadable"+ | "pending-source"+ | "pending-destination"+ | "pending-shared-source"+ | "pending-shared-destination";++/** The folds that may miss a record: the move fails closed over them. */+const INCOMPLETE: ReadonlySet<Fold> = new Set([+ "unreadable",+ "pending-source",+ "pending-destination",+]);++interface Cell {+ readonly source: Source;+ readonly disk: Disk;+ readonly destination: Destination;+ readonly mode: Mode;+ readonly shape: Shape;+ readonly fold: Fold;+}++/** Why a cell cannot exist (see the module docblock), or `null` when it can. */+function impossible(cell: Cell): string | null {+ if (cell.mode !== "store") return null;+ if (cell.source === "untracked") return "a store-only machine has no untracked page";+ if (cell.disk === "link") return "the store holds no links";+ if (cell.source === "shared" && cell.disk === "private") return "no write stores it in shared";+ return null;+}++const KIND = { kind: "brain-dead-end" };+const PRIVATE = { kind: "brain-dead-end", visibility: "private" };++/** The page that moves: its fields and text (every private text says "secret"). */+const MOVED: Record<Exclude<Disk, "link">, { fields: FrontmatterMap; text: string }> = {+ public: { fields: KIND, text: "Moved public.\n" },+ private: { fields: PRIVATE, text: "Moved secret.\n" },+};++/** What one cell must leave at the destination, computed from the cell alone. */+interface Expected {+ /** The live destination entry, or `null` when none may be live. */+ readonly entry: {+ readonly fields: FrontmatterMap;+ readonly text: string;+ readonly partition: "shared" | "private";+ } | null;+ /** The move must append no record at all. */+ readonly untouched: boolean;+}++/**+ * The oracle. The destination entry is what a fresh write of the page now+ * there would leave: its content, private when the source entry, the+ * destination entry or the content is private (nothing lowers an entry's+ * partition, and the source's privacy moves with it, and a fold that misses+ * a record in `private` may miss the one that makes it private); no live entry when+ * nothing importable is there. A move that involves no stored entry at all+ * (an untracked page onto an unstored path, unswitched) writes nothing:+ * the dual write mirrors what the store holds, and the migration imports+ * the rest.+ */+function expected(cell: Cell): Expected {+ if (cell.source === "untracked" && cell.destination === "none") {+ return { entry: null, untouched: true };+ }+ if (cell.disk === "link") return { entry: null, untouched: false };+ const moved = MOVED[cell.disk];+ const isPrivate =+ cell.source === "private" ||+ cell.destination === "private" ||+ cell.disk === "private" ||+ INCOMPLETE.has(cell.fold);+ return {+ entry: { ...moved, partition: isPrivate ? "private" : "shared" },+ untouched: false,+ };+}++/** Whether this host lets the test create a symlink (Windows needs a privilege). */+const CAN_SYMLINK = ((): boolean => {+ const probe = mkdtempSync(join(tmpdir(), "osb-lane-table-symlink-probe-"));+ try {+ symlinkSync(probe, join(probe, "link"), "dir");+ return true;+ } catch {+ return false;+ } finally {+ rmSync(probe, { recursive: true, force: true });+ }+})();++let lane: LaneEnv | null = null;+afterEach(() => {+ lane?.cleanup();+ lane = null;+});++/** Run `fn` with the dual write off: a write the store never sees. */+function legacyOnly(fn: () => void): void {+ delete process.env[STORE_DUAL_WRITE_ENV];+ try {+ fn();+ } finally {+ process.env[STORE_DUAL_WRITE_ENV] = "1";+ }+}++/** The live entries of `records`, by key, with fields and text. */+function liveOf(records: StoreRecord[]): Map<string, { fields: unknown; text: string }> {+ const out = new Map<string, { fields: unknown; text: string }>();+ // A record whose body blob has not arrived is left out, as every fold leaves it out.+ const known = records.filter((r) => r.body[BLOB_REF_KEY] === undefined);+ for (const [key, state] of foldLaneEntries(known).entries) {+ // Trimmed: a page read back from disk loses the newline that ends its body.+ if (!state.removed) {+ out.set(key, { fields: state.value!.fields, text: state.value!.text.trim() });+ }+ }+ return out;+}++/** Every live entry with the partition it is live in, as one comparable value. */+function liveState(env: LaneEnv): Record<string, unknown> {+ const all = liveOf(env.laneRecords());+ const inShared = liveOf(env.laneRecords("shared"));+ const inPrivate = liveOf(env.laneRecords("private"));+ return Object.fromEntries(+ [...all.keys()]+ .toSorted()+ .map((key) => [+ key,+ { ...all.get(key), shared: inShared.has(key), private: inPrivate.has(key) },+ ]),+ );+}++/** The paths of one cell: the source and destination pages, and the moved folders. */+function pathsOf(vault: string, shape: Shape) {+ if (shape === "page") {+ const dir = join(vault, "Brain", "dead-ends");+ return {+ from: join(dir, "de-src.md"),+ to: join(dir, "de-dst.md"),+ fromKey: "dead-ends/de-src",+ toKey: "dead-ends/de-dst",+ };+ }+ const fromDir = join(vault, "Brain", "dream", "staged", "s1");+ const toDir = join(vault, "Brain", "dream", "applied", "s1");+ return {+ fromDir,+ toDir,+ from: join(fromDir, "page.md"),+ to: join(toDir, "page.md"),+ fromKey: "dream/staged/s1/page",+ toKey: "dream/applied/s1/page",+ };+}++/** Build the state of `cell` before the move. */+function arrange(env: LaneEnv, cell: Cell): void {+ const { vault } = env;+ const paths = pathsOf(vault, cell.shape);+ if (cell.destination !== "none") {+ const own = cell.destination === "private" ? PRIVATE : KIND;+ const text = cell.destination === "private" ? "Dest secret.\n" : "Dest public.\n";+ writeLanePage(vault, paths.to, own, text);+ if (cell.mode === "legacy" && paths.toDir !== undefined) {+ legacyOnly(() => rmSync(paths.toDir!, { recursive: true }));+ }+ }+ if (cell.mode === "store") {+ // The stored value is what moves; a private entry keeps its partition+ // when overwritten with public content.+ const moved = MOVED[cell.disk as Exclude<Disk, "link">];+ if (cell.source === "private") writeLanePage(vault, paths.from, PRIVATE, "Source secret.\n");+ writeLanePage(vault, paths.from, moved.fields, moved.text, { overwrite: true });+ return;+ }+ if (cell.source !== "untracked") {+ const own = cell.source === "private" ? PRIVATE : KIND;+ const text = cell.source === "private" ? "Source secret.\n" : "Source public.\n";+ writeLanePage(vault, paths.from, own, text);+ }+ // Legacy-only, the source becomes what the move carries.+ legacyOnly(() => {+ rmSync(paths.from, { force: true });+ if (cell.disk === "link") {+ const outside = join(dirname(vault), "outside.md");+ writeFileSync(outside, "---\nkind: brain-dead-end\n---\nOutside secret.\n");+ mkdirSync(dirname(paths.from), { recursive: true });+ symlinkSync(outside, paths.from, "file");+ return;+ }+ const moved = MOVED[cell.disk];+ writeLanePage(vault, paths.from, moved.fields, moved.text);+ });+}++/** Every file below `dir`, recursively, partition-relative paths. */+function filesBelow(dir: string): string[] {+ if (!existsSync(dir)) return [];+ return readdirSync(dir, { recursive: true, withFileTypes: true })+ .filter((d) => d.isFile())+ .map((d) => join(d.parentPath, d.name));+}++/**+ * Append `record` as another device's record in `partition` whose body+ * blob never arrived, quarantined here so its shard reads as whole.+ */+function appendPendingQuarantined(+ env: LaneEnv,+ partition: "shared" | "private",+ record: { readonly kind: string; readonly body: Record<string, unknown> },+): void {+ const root = env.env.storeRoot;+ const partitionRoot = partitionDir(root, partition);+ const other = openStore({ root, device: "dev-b" });+ const blobsBefore = new Set(filesBelow(join(partitionRoot, "blobs")));+ const path = appendRecord(other, {+ partition,+ stream: "events",+ kind: record.kind,+ agent: "t",+ body: record.body,+ }).path;+ const blobs = filesBelow(join(partitionRoot, "blobs")).filter((f) => !blobsBefore.has(f));+ expect(blobs).toHaveLength(1);+ rmSync(blobs[0]!);+ const offset = readShard(path, { partitionRoot }).records.find(+ (r) => r.bodyState === "pending",+ )!.offset;+ addQuarantine(root, {+ partition,+ shard: relative(partitionRoot, path).split("\\").join("/"),+ offset,+ sha256: lineSha256At(path, offset, partitionRoot)!,+ kind: "blob-pending",+ reason: "test",+ });+}++/**+ * Make the fold the move reads lack a record, as `cell.fold` says, with+ * records of another device: in `private`, a shard with a corrupt line in+ * its middle, or a newer value of the source or destination entry whose+ * body blob never arrived; in `shared`, a newer public value of the source+ * or destination entry whose body blob never arrived. A pending blob is+ * quarantined here, so its shard reads as whole.+ */+function makeIncomplete(env: LaneEnv, cell: Cell): void {+ if (cell.fold === "complete") return;+ if (cell.fold === "unreadable") {+ const other = openStore({ root: env.env.storeRoot, device: "dev-b" });+ const note = { partition: "private", stream: "events", kind: "note", agent: "t" } as const;+ const path = appendRecord(other, { ...note, body: { text: "one" } }).path;+ appendRecord(other, { ...note, body: { text: "two" } });+ const [first, ...rest] = readFileSync(path, "utf8").split("\n");+ writeFileSync(path, [first, "not a record", ...rest].join("\n"));+ return;+ }+ const paths = pathsOf(env.vault, cell.shape);+ const onSource = cell.fold === "pending-source" || cell.fold === "pending-shared-source";+ const inShared =+ cell.fold === "pending-shared-source" || cell.fold === "pending-shared-destination";+ const at = onSource ? paths.from : paths.to;+ const place = lanePlaceOf(relative(join(env.vault, "Brain"), at).split("\\").join("/"))!;+ const record = laneEntryRecord({+ ...place,+ fields: KIND,+ text: (inShared ? "Written elsewhere. " : "Raised elsewhere. ").repeat(2_000),+ authority: "human",+ private: !inShared,+ });+ appendPendingQuarantined(env, inShared ? "shared" : "private", record);+}++/** The cell's move, through the lane API. */+function act(env: LaneEnv, cell: Cell): void {+ const paths = pathsOf(env.vault, cell.shape);+ if (paths.fromDir !== undefined) moveLaneTree(env.vault, paths.fromDir, paths.toDir!);+ else moveLanePage(env.vault, paths.from, paths.to);+}++/** Assert the invariants of `cell` after its move. */+function assertCell(env: LaneEnv, cell: Cell, recordsBefore: number, sharedBefore: number): void {+ const want = expected(cell);+ const { fromKey, toKey } = pathsOf(env.vault, cell.shape);+ const all = liveOf(env.laneRecords());+ const inShared = liveOf(env.laneRecords("shared"));+ const inPrivate = liveOf(env.laneRecords("private"));++ if (want.untouched) expect(env.laneRecords()).toHaveLength(recordsBefore);+ // (b) nothing live at the source; (a) the destination as a fresh write leaves it.+ expect(all.has(fromKey)).toBe(false);+ expect([...all.keys()]).toEqual(want.entry === null ? [] : [toKey]);+ if (want.entry !== null) {+ expect(all.get(toKey)).toEqual({ fields: want.entry.fields, text: want.entry.text.trim() });+ // (c) live in exactly the partition the rule gives.+ expect({ shared: inShared.has(toKey), private: inPrivate.has(toKey) }).toEqual({+ shared: want.entry.partition === "shared",+ private: want.entry.partition === "private",+ });+ }+ // (c) nothing private reaches a shared record the move wrote, and no link is followed.+ const added = JSON.stringify(env.laneRecords("shared").slice(sharedBefore));+ expect(added.toLowerCase()).not.toContain("secret");+ expect(JSON.stringify(env.laneRecords())).not.toContain("Outside");+}++const CELLS: Cell[] = [];+const FOLDS = [+ "complete",+ "unreadable",+ "pending-source",+ "pending-destination",+ "pending-shared-source",+ "pending-shared-destination",+] as const;++/** Whether `cell` belongs to the table for its fold (see the module docblock). */+function crossed(cell: Cell): boolean {+ if (cell.fold === "complete") return true;+ // An incomplete fold is crossed with the public-page cells only.+ if (cell.disk !== "public") return false;+ if (INCOMPLETE.has(cell.fold)) return true;+ // A pending blob in shared: the cells whose verdict over the visible fold is shared.+ if (cell.source === "private" || cell.destination === "private") return false;+ if (cell.fold === "pending-shared-source") return cell.source === "shared";+ return !(cell.source === "untracked" && cell.destination === "none");+}++for (const fold of FOLDS)+ for (const mode of ["legacy", "store"] as const)+ for (const shape of ["page", "folder"] as const)+ for (const source of ["shared", "private", "untracked"] as const)+ for (const disk of ["public", "private", "link"] as const)+ for (const destination of ["shared", "private", "none"] as const) {+ const cell = { source, disk, destination, mode, shape, fold };+ if (crossed(cell) && impossible(cell) === null) CELLS.push(cell);+ }++describe("a lane move over its state space", () => {+ for (const cell of CELLS) {+ const name = `${cell.mode} ${cell.shape}: source ${cell.source}, ${cell.disk} on disk, destination ${cell.destination}, fold ${cell.fold}`;+ test.skipIf(cell.disk === "link" && !CAN_SYMLINK)(name, () => {+ lane = makeLaneEnv(cell.mode, "osb-lane-table-");+ arrange(lane, cell);+ makeIncomplete(lane, cell);+ const before = lane.laneRecords().length;+ const sharedBefore = lane.laneRecords("shared").length;++ act(lane, cell);+ assertCell(lane, cell, before, sharedBefore);++ // (d) The dual write again, over the same move, changes nothing live+ // and still writes nothing private to shared.+ if (cell.mode === "legacy") {+ const state = liveState(lane);+ const sharedAfter = lane.laneRecords("shared").length;+ const paths = pathsOf(lane.vault, cell.shape);+ const context = { vault: lane.vault };+ if (paths.fromDir !== undefined) {+ dualWriteLaneTreeMove(lane.vault, paths.fromDir, paths.toDir!, context);+ } else {+ const destinationPrivate = cell.destination === "private";+ dualWriteLaneMove(+ lane.vault,+ { from: paths.from, to: paths.to, destinationPrivate },+ context,+ );+ }+ expect(liveState(lane)).toEqual(state);+ assertCell(lane, cell, before, sharedAfter);+ }+ });+ }++ test("covers every cell that can exist", () => {+ // Complete fold: 108 cells in the product; 36 store-only cells cannot+ // exist (see above). Each of the three incomplete folds: the 30 of+ // those 72 that carry a public page. A pending blob in shared: for the+ // source, the 8 cells with a source in shared and a destination in+ // shared or not stored; for the destination, those 8 and the 2+ // unswitched cells of an untracked source onto a destination in shared.+ expect(CELLS).toHaveLength(72 + 3 * 30 + 8 + 10);+ });+});++type Overlap = "disjoint" | "below" | "ancestor" | "same";++/** The folder each overlap moves `dream/staged/s1` to, `Brain/`-relative. */+const OVERLAP_TO: Record<Overlap, string> = {+ disjoint: "dream/applied/s1",+ below: "dream/staged/s1/s1",+ ancestor: "dream/staged",+ same: "dream/staged/s1",+};++/**+ * The entries before every overlap move: two below the moved folder, one+ * of them in a nested folder of the same name (so the destination of one+ * is the source of the other when the folders overlap), and a sibling+ * folder's entry no move may touch.+ */+const OVERLAP_BEFORE: Record<string, { text: string; partition: "shared" | "private" }> = {+ "dream/staged/s1/z": { text: "Outer.", partition: "shared" },+ "dream/staged/s1/s1/z": { text: "Inner secret.", partition: "private" },+ "dream/staged/s2/y": { text: "Sibling.", partition: "shared" },+};++/**+ * The overlap oracle: a move whose folders overlap (one inside the other)+ * is refused before anything moves, as a rename refuses it; a folder moved+ * onto itself changes nothing; a disjoint move carries each entry to the+ * same path below the destination, in the partition it lived in.+ */+function overlapExpected(overlap: Overlap): {+ readonly refused: boolean;+ readonly live: Record<string, { text: string; partition: "shared" | "private" }>;+} {+ if (overlap === "below" || overlap === "ancestor") return { refused: true, live: OVERLAP_BEFORE };+ if (overlap === "same") return { refused: false, live: OVERLAP_BEFORE };+ const from = "dream/staged/s1/";+ return {+ refused: false,+ live: Object.fromEntries(+ Object.entries(OVERLAP_BEFORE).map(([key, value]) => [+ key.startsWith(from) ? `${OVERLAP_TO.disjoint}/${key.slice(from.length)}` : key,+ value,+ ]),+ ),+ };+}++/** The live entries with their text and partition, keyed by entry. */+function liveWithPartition(+ env: LaneEnv,+): Record<string, { text: string; partition: "shared" | "private" }> {+ const inPrivate = liveOf(env.laneRecords("private"));+ const all = liveOf(env.laneRecords());+ return Object.fromEntries(+ [...all.entries()]+ .toSorted(([a], [b]) => a.localeCompare(b))+ .map(([key, value]) => [+ key,+ { text: value.text, partition: inPrivate.has(key) ? "private" : "shared" },+ ]),+ );+}++/** The lane pages on disk below `Brain/dream`, as entry keys. */+function diskEntries(vault: string): string[] {+ const brain = join(vault, "Brain");+ return filesBelow(join(brain, "dream"))+ .filter((f) => f.endsWith(".md"))+ .map((f) => relative(brain, f).split("\\").join("/").slice(0, -".md".length))+ .toSorted();+}++describe("a lane folder move over the overlap of its folders", () => {+ for (const mode of ["legacy", "store"] as const)+ for (const overlap of ["disjoint", "below", "ancestor", "same"] as const) {+ test(`${mode}: destination ${overlap}`, () => {+ lane = makeLaneEnv(mode, "osb-lane-overlap-");+ const brain = join(lane.vault, "Brain");+ const at = (rel: string): string => join(brain, ...rel.split("/"));+ for (const [key, { text, partition }] of Object.entries(OVERLAP_BEFORE)) {+ writeLanePage(+ lane.vault,+ at(`${key}.md`),+ partition === "private" ? PRIVATE : KIND,+ `${text}\n`,+ );+ }+ const want = overlapExpected(overlap);+ const before = lane.laneRecords().length;+ const sharedBefore = lane.laneRecords("shared").length;++ const move = (): void =>+ moveLaneTree(lane!.vault, at("dream/staged/s1"), at(OVERLAP_TO[overlap]));+ if (want.refused) expect(move).toThrow();+ else move();++ if (want.refused || overlap === "same") expect(lane.laneRecords()).toHaveLength(before);+ expect(liveWithPartition(lane)).toEqual(want.live);+ // No private text reaches a shared record the move wrote.+ const added = JSON.stringify(lane.laneRecords("shared").slice(sharedBefore));+ expect(added.toLowerCase()).not.toContain("secret");+ // Unswitched: the store ends where the rename left the disk.+ if (mode === "legacy")+ expect(diskEntries(lane.vault)).toEqual(Object.keys(want.live).toSorted());+ });+ }++ test("covers every overlap in both modes", () => {+ expect(Object.keys(OVERLAP_TO)).toHaveLength(4);+ });+});++describe("a store-only machine that holds only shared, with a quarantined pending blob in shared", () => {+ test("still writes a lane page to shared", () => {+ lane = makeLaneEnv("store", "osb-lane-shared-only-", { partitions: ["shared"] });+ // A record of another kind, unrelated to the page, whose blob was lost.+ appendPendingQuarantined(lane, "shared", {+ kind: "note",+ body: { authority: "agent", text: "Lost elsewhere. ".repeat(2_000) },+ });+ const page = join(lane.vault, "Brain", "obligations", "obl-q.md");+ writeLanePage(lane.vault, page, { kind: "brain-obligation" }, "Open.\n");++ expect(readLanePage(lane.vault, page)?.body.trim()).toBe("Open.");+ expect(laneEntryInPrivate(lane.vault, page)).toBe(false);+ expect(lane.laneRecords("shared").map((r) => r.body["text"])).toEqual(["Open.\n"]);+ });+});diff --git a/tests/core/store/fold-setting-lists.test.ts b/tests/core/store/fold-setting-lists.test.tsnew file mode 100644index 00000000..56bf23c2--- /dev/null+++ b/tests/core/store/fold-setting-lists.test.ts@@ -0,0 +1,466 @@+/**+ * The set merge of list settings (task 3.12, design D19 item 1): a list+ * setting such as a schema field folds per entry, not as one register+ * over the whole list, so concurrent edits on two devices both survive+ * and every delivery order gives the same list.+ */+import { describe, expect, test } from "bun:test";++import { canonicalJson } from "../../../src/core/integrity/digest.ts";+import { SETTING_ENTRY_KIND } from "../../../src/core/store/constants.ts";+import {+ foldSettings,+ settingsFingerprint,+ settingValue,+} from "../../../src/core/store/fold-settings.ts";+import type { StoreRecord } from "../../../src/core/store/envelope.ts";+import { settingEntrySlot } from "../../../src/core/store/fold-setting-lists.ts";+import { authored, pick, rec, seededRandom, shuffled } from "./fixtures/fold-records.ts";++const T = Date.UTC(2026, 10, 15, 12, 0, 0);+const KEY = "brain.schema.page_types";++const base = (value: string[], physical = T, authority: "human" | "agent" = "agent") =>+ rec({+ device: "dev-a",+ physical,+ kind: "setting-set",+ body: authored(authority, { key: KEY, value }),+ });++const entry = (+ device: string,+ physical: number,+ value: string,+ present: boolean,+ authority: "human" | "agent" | "model" = "agent",+ key = KEY,+) =>+ rec({+ device,+ physical,+ kind: SETTING_ENTRY_KIND,+ body: authored(authority, { key, entry: value, present }),+ });++/** Every delivery order of `records` folds to one list. */+function foldsInAnyOrder(records: ReadonlyArray<StoreRecord>, key = KEY): unknown {+ const rng = seededRandom(312);+ const expected = canonicalJson(settingValue(foldSettings(records), key));+ for (let round = 0; round < 25; round++) {+ const order = shuffled(rng, [...records, ...records.slice(0, round % records.length)]);+ expect(canonicalJson(settingValue(foldSettings(order), key))).toBe(expected);+ }+ return settingValue(foldSettings(records), key);+}++describe("list settings fold per entry", () => {+ test("two devices add different entries at once: both stay, in any delivery order", () => {+ const records = [+ base(["note"]),+ entry("dev-a", T + 10, "paper", true),+ entry("dev-b", T + 5, "person", true),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["note", "person", "paper"]);+ });++ test("an add on one device and a remove on another of different entries both apply", () => {+ const records = [+ base(["note", "paper"]),+ entry("dev-a", T + 10, "person", true),+ entry("dev-b", T + 5, "paper", false),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["note", "person"]);+ });++ test("add and remove of the same entry on two devices: the later clock wins", () => {+ const addLater = [+ base(["note"]),+ entry("dev-b", T + 5, "paper", false),+ entry("dev-a", T + 10, "paper", true),+ ];+ expect(foldsInAnyOrder(addLater)).toEqual(["note", "paper"]);+ const removeLater = [+ base(["note"]),+ entry("dev-a", T + 5, "paper", true),+ entry("dev-b", T + 10, "paper", false),+ ];+ expect(foldsInAnyOrder(removeLater)).toEqual(["note"]);+ });++ test("a human entry beats a later agent entry for the same entry", () => {+ const records = [+ base(["note"]),+ entry("dev-a", T + 5, "paper", true, "human"),+ entry("dev-b", T + 10, "paper", false, "agent"),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["note", "paper"]);+ });++ test("a whole-list record later than an entry decides that entry too", () => {+ // A release before the set merge writes the whole list: what it leaves+ // out is removed, as it was before.+ const records = [+ base(["note"]),+ entry("dev-b", T + 5, "paper", true),+ base(["note", "person"], T + 10),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["note", "person"]);+ });++ test("keyed entries hold one value per key: a concurrent change keeps only the later", () => {+ const key = "brain.schema.prefixes";+ expect(settingEntrySlot(key, "proj=project")).toBe("proj");+ expect(settingEntrySlot(KEY, "paper")).toBe("paper");+ const records = [+ rec({+ physical: T,+ kind: "setting-set",+ body: authored("agent", { key, value: ["proj=project"] }),+ }),+ entry("dev-a", T + 5, "proj=initiative", true, "agent", key),+ entry("dev-b", T + 10, "proj=programme", true, "agent", key),+ ];+ expect(foldsInAnyOrder(records, key)).toEqual(["proj=programme"]);+ });++ test("removing every entry removes the setting", () => {+ const fold = foldSettings([base(["note"]), entry("dev-a", T + 5, "note", false)]);+ expect(fold.settings.get(KEY)?.removed).toBe(true);+ expect(settingValue(fold, KEY)).toBeNull();+ });++ test("a key with no entry record folds exactly as before", () => {+ const records = [base(["b", "a", "b"]), base(["c", "a"], T + 1)];+ const fold = foldSettings(records);+ expect(settingValue(fold, KEY)).toEqual(["c", "a"]);+ expect(fold.settings.get(KEY)?.winner.record.id).toBe(records[1]!.id);+ });++ test("malformed entry records are rejected, and only schema list keys take entries", () => {+ const malformed = [+ entry("dev-a", T, "", true),+ entry("dev-a", T + 1, "two\nlines", true),+ rec({+ physical: T + 2,+ kind: SETTING_ENTRY_KIND,+ body: authored("agent", { key: KEY, entry: "x", present: "yes" }),+ }),+ entry("dev-a", T + 3, "ru", true, "human", "view.language"),+ ];+ const fold = foldSettings(malformed);+ expect(fold.rejected.map((r) => r.id).toSorted()).toEqual(+ malformed.map((r) => r.id).toSorted(),+ );+ expect(fold.settings.size).toBe(0);+ });+});++describe("entry records on a key that is not a schema list", () => {+ for (const [key, value] of [+ ["permissions.document", "deny:\n - Private/**\n"],+ ["brain.dream.enabled", true],+ ["brain.example.block", { a: 1 }],+ ["brain.example.numbers", [1, 2]],+ ] as const) {+ test(`${key}: the entry record is rejected and the human value stays in force`, () => {+ const set = rec({+ physical: T,+ kind: "setting-set",+ body: authored("human", { key, value }),+ });+ const stray = entry("dev-b", T + 10, "x", false, "model", key);+ const fold = foldSettings([set, stray]);+ expect(fold.rejected.map((r) => r.id)).toEqual([stray.id]);+ expect(fold.settings.get(key)).toMatchObject({ removed: false, value });+ expect(fold.lists.has(key)).toBe(false);+ });+ }+});++describe("one entry spelled several ways is one slot", () => {+ test("the slot is the form the schema pack parser reads", () => {+ expect(settingEntrySlot(KEY, " Paper ")).toBe("paper");+ expect(settingEntrySlot(KEY, '"paper"')).toBe("paper");+ expect(settingEntrySlot(KEY, "Cafe\u0301")).toBe(settingEntrySlot(KEY, "caf\u00e9"));+ expect(settingEntrySlot("brain.schema.prefixes", " PROJ = project")).toBe("proj");+ expect(settingEntrySlot("brain.schema.aliases", "Paper = Article")).toBe("paper=article");+ expect(settingEntrySlot("brain.schema.link_constraints", "cites=Paper -> Note")).toBe(+ "cites=paper->note",+ );+ });++ test("each side of a name=value line is normalised on its own, quotes included", () => {+ const aliases = "brain.schema.aliases";+ expect(settingEntrySlot(aliases, '"Paper"="Article"')).toBe("paper=article");+ expect(settingEntrySlot(aliases, "'Paper' = 'Article'")).toBe("paper=article");+ expect(settingEntrySlot(aliases, '"Paper=Article"')).toBe("paper=article");+ expect(settingEntrySlot("brain.schema.prefixes", '"Proj"="project"')).toBe("proj");+ expect(settingEntrySlot("brain.schema.link_constraints", '"cites"="Paper"->"Note"')).toBe(+ "cites=paper->note",+ );+ // A removal in the plain spelling removes the quoted entry.+ const records = [+ entry("dev-a", T + 5, '"Paper"="Article"', true, "human", aliases),+ entry("dev-b", T + 10, "paper=article", false, "human", aliases),+ ];+ expect(foldSettings(records).settings.get(aliases)).toMatchObject({ removed: true });+ });++ test("a compound type.field key is one slot however its dot is spaced, as the parser reads it", () => {+ for (const key of ["brain.schema.attributes", "brain.schema.frontmatter_tiers"]) {+ expect(settingEntrySlot(key, "Paper . Year=Publication year")).toBe("paper.year");+ expect(settingEntrySlot(key, "paper.year=other")).toBe("paper.year");+ }+ // Only the first dot splits the key, as in the parser.+ expect(settingEntrySlot("brain.schema.attributes", "paper . year . x=y")).toBe(+ "paper.year . x",+ );+ });++ test("an agent entry in another spelling cannot bring back an entry the operator removed", () => {+ const records = [+ base(["note"]),+ entry("dev-a", T + 5, "paper", true, "agent"),+ entry("dev-a", T + 10, "paper", false, "human"),+ entry("dev-b", T + 20, " Paper", true, "agent"),+ base(["note", "PAPER"], T + 30, "agent"),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["note"]);+ });+});++describe("whole lists, unsets and entries together", () => {+ test("a setting-unset between entries removes what came before it, and a later entry adds back", () => {+ const records = [+ base(["note", "paper"]),+ entry("dev-a", T + 5, "person", true),+ rec({+ device: "dev-b",+ physical: T + 10,+ kind: "setting-unset",+ body: authored("agent", { key: KEY }),+ }),+ entry("dev-a", T + 15, "memo", true),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["memo"]);+ });++ test("a later agent whole list does not undo human entries", () => {+ const records = [+ base(["note"]),+ entry("dev-a", T + 5, "paper", true, "human"),+ entry("dev-a", T + 6, "note", false, "human"),+ base(["note", "memo"], T + 10, "agent"),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["paper", "memo"]);+ });++ test("a later human whole list decides agent entries", () => {+ const records = [+ base(["note"]),+ entry("dev-a", T + 5, "paper", true, "agent"),+ entry("dev-b", T + 6, "memo", true, "agent"),+ base(["note", "memo"], T + 10, "human"),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["note", "memo"]);+ });++ test("a model entry loses to an earlier agent entry", () => {+ const records = [+ base(["note"]),+ entry("dev-a", T + 5, "paper", true, "agent"),+ entry("dev-b", T + 10, "paper", false, "model"),+ ];+ expect(foldsInAnyOrder(records)).toEqual(["note", "paper"]);+ });++ test("a clock tie between two devices resolves the same way in every order", () => {+ const records = [+ base(["note"]),+ entry("dev-a", T + 5, "paper", true),+ entry("dev-b", T + 5, "paper", false),+ ];+ const value = foldsInAnyOrder(records);+ expect([["note"], ["note", "paper"]]).toContainEqual(value as string[]);+ });++ test("the digest of a store with no entry record is the plain register's", () => {+ const records = [base(["b", "a"]), base(["c"], T + 1, "human")];+ expect(canonicalJson(settingsFingerprint(foldSettings(records)))).toBe(+ canonicalJson({+ settings: { [KEY]: { value: ["c"], removed: false, winner: records[1]!.id } },+ rejected: [],+ }),+ );+ });+});++describe("a conditional removal (if_decided_by)", () => {+ const PREFIXES = "brain.schema.prefixes";+ const conditional = (device: string, physical: number, text: string, decidedBy: string) =>+ rec({+ device,+ physical,+ kind: SETTING_ENTRY_KIND,+ body: authored("human", {+ key: PREFIXES,+ entry: text,+ present: false,+ if_decided_by: decidedBy,+ }),+ });++ test("applies while the record it names decides the slot", () => {+ const ghost = entry("dev-b", T + 5, "p=ghost", true, "human", PREFIXES);+ const records = [ghost, conditional("dev-a", T + 10, "p=ghost", ghost.id)];+ expect(foldsInAnyOrder(records, PREFIXES)).toBeNull();+ });++ test("never removes a replacement it did not judge, whatever the clocks and delivery order", () => {+ const ghost = entry("dev-b", T + 5, "p=ghost", true, "human", PREFIXES);+ const paper = entry("dev-c", T + 7, "p=paper", true, "human", PREFIXES);+ const records = [ghost, paper, conditional("dev-a", T + 10, "p=ghost", ghost.id)];+ expect(foldsInAnyOrder(records, PREFIXES)).toEqual(["p=paper"]);+ });++ test("whose record is not there leaves the key as its other records decide it", () => {+ const lone = conditional("dev-a", T + 10, "p=ghost", "01AAAAAAAAAAAAAAAAAAAAAAAA");+ expect(foldSettings([lone]).settings.has(PREFIXES)).toBe(false);+ const other = entry("dev-b", T + 5, "q=note", true, "human", PREFIXES);+ expect(foldsInAnyOrder([other, lone], PREFIXES)).toEqual(["q=note"]);+ });++ test("is malformed on an addition or without a record id", () => {+ const added = rec({+ device: "dev-a",+ physical: T,+ kind: SETTING_ENTRY_KIND,+ body: authored("human", { key: PREFIXES, entry: "p=x", present: true, if_decided_by: "x" }),+ });+ const empty = conditional("dev-a", T, "p=x", "");+ // The fold lists rejections in fold order, which ties on the clock+ // break by generated id: compare by record, not by position.+ const reasons = new Map(foldSettings([added, empty]).rejected.map((r) => [r.id, r.reason]));+ expect(reasons).toEqual(+ new Map([+ [added.id, "if_decided_by is for a removal only"],+ [empty.id, "if_decided_by must be a record id"],+ ]),+ );+ });+});++describe("if_decided_by on a record that is no setting-entry", () => {+ test("is ignored, as any field a kind does not define: the record applies as written", () => {+ const set = rec({+ physical: T,+ kind: "setting-set",+ body: authored("human", {+ key: KEY,+ value: ["note"],+ if_decided_by: "01AAAAAAAAAAAAAAAAAAAAAAAA",+ }),+ });+ const fold = foldSettings([set]);+ expect(fold.rejected).toEqual([]);+ expect(settingValue(fold, KEY)).toEqual(["note"]);+ });+});++describe("a list key whose records name no entry", () => {+ const PREFIXES = "brain.schema.prefixes";+ const ABSENT = "01AAAAAAAAAAAAAAAAAAAAAAAA";+ const whole = (value: string[], physical: number, authority: "human" | "agent" = "human") =>+ rec({ physical, kind: "setting-set", body: authored(authority, { key: PREFIXES, value }) });+ const unset = (physical: number) =>+ rec({ physical, kind: "setting-unset", body: authored("human", { key: PREFIXES }) });+ const removal = (physical: number, text: string, decidedBy?: string) =>+ rec({+ device: "dev-b",+ physical,+ kind: SETTING_ENTRY_KIND,+ body: authored("human", {+ key: PREFIXES,+ entry: text,+ present: false,+ ...(decidedBy === undefined ? {} : { if_decided_by: decidedBy }),+ }),+ });++ test("an empty whole list with a removal whose named record is absent is a removed key", () => {+ const records = [whole([], T), removal(T + 10, "p=ghost", ABSENT)];+ expect(foldsInAnyOrder(records, PREFIXES)).toBeNull();+ const state = foldSettings(records).settings.get(PREFIXES);+ expect(state?.removed).toBe(true);+ expect(state?.winner.record.id).toBe(records[0]!.id);+ });++ test("an unset followed by conditional removals is a removed key", () => {+ const records = [unset(T), removal(T + 10, "p=a", ABSENT), removal(T + 20, "q=b", ABSENT)];+ expect(foldsInAnyOrder(records, PREFIXES)).toBeNull();+ expect(foldSettings(records).settings.get(PREFIXES)?.removed).toBe(true);+ });++ test("the key that names no entry is won as the whole-list register would win it", () => {+ const human = whole([], T);+ const records = [human, whole([], T + 5, "agent"), removal(T + 10, "p=a", ABSENT)];+ expect(foldSettings(records).settings.get(PREFIXES)?.winner.record.id).toBe(human.id);+ });+});++describe("the list fold never throws and does not depend on delivery order", () => {+ const PREFIXES = "brain.schema.prefixes";+ const ENTRIES = ["p=a", "p=b", "q=a", "r=c"];+ const AUTHORITIES = ["human", "agent", "model"] as const;++ /** A small random record set of one list key: every record shape the fold takes, and broken ones. */+ function recordSet(rng: () => number): StoreRecord[] {+ const out: StoreRecord[] = [];+ const count = 1 + Math.floor(rng() * 6);+ for (let i = 0; i < count; i++) {+ const physical = T + Math.floor(rng() * 4) * 10;+ const device = pick(rng, ["dev-a", "dev-b", "dev-c"]);+ const authority = pick(rng, AUTHORITIES);+ const shape = Math.floor(rng() * 6);+ const body =+ shape === 0+ ? { key: PREFIXES, value: ENTRIES.filter(() => rng() < 0.4) }+ : shape === 1+ ? { key: PREFIXES }+ : {+ key: PREFIXES,+ entry: pick(rng, ENTRIES),+ present: shape === 2,+ ...(shape === 4+ ? { if_decided_by: pick(rng, [...out.map((r) => r.id), "x"]) }+ : {}),+ ...(shape === 5 ? { if_decided_by: "" } : {}),+ };+ const kind = shape === 0 ? "setting-set" : shape === 1 ? "setting-unset" : SETTING_ENTRY_KIND;+ out.push(rec({ device, physical, kind, body: authored(authority, body), rng }));+ }+ return out;+ }++ const digest = (records: ReadonlyArray<StoreRecord>) => {+ const fold = foldSettings(records);+ return canonicalJson({+ fingerprint: settingsFingerprint(fold),+ lists: [...(fold.lists.get(PREFIXES) ?? new Map()).entries()]+ .map(([slot, state]) => [slot, state.value, state.winner.record.id])+ .toSorted(),+ });+ };++ test("over seeded small record sets", () => {+ const rng = seededRandom(3120);+ for (let round = 0; round < 400; round++) {+ const records = recordSet(rng);+ const expected = digest(records);+ for (let order = 0; order < 6; order++) {+ expect(digest(shuffled(rng, [...records, ...records.slice(0, order)]))).toBe(expected);+ }+ }+ });+});diff --git a/tests/core/store/format-doc.test.ts b/tests/core/store/format-doc.test.tsindex 668262b7..7da086d2 100644--- a/tests/core/store/format-doc.test.ts+++ b/tests/core/store/format-doc.test.ts@@ -29,6 +29,17 @@ import {} from "../../../src/core/store/constants.ts";import { DUAL_WRITE_LEDGERS } from "../../../src/core/store/dual-write-records.ts";import { formatDocVersion, shippedFormatDoc } from "../../../src/core/store/format-doc.ts";+import {+ IF_DECIDED_BY_FIELD,+ KEYED_LIST_SETTINGS,+} from "../../../src/core/store/fold-setting-lists.ts";+import {+ SCHEMA_DECIDED_RECORDS_FIELD,+ SCHEMA_IMPORT_BASELINE_ACTION,+ SCHEMA_MUTATION_AUDIT_ACTION,+ SCHEMA_MUTATION_AUDIT_DIR,+} from "../../../src/core/brain/schema-integrity.ts";+import { SCHEMA_PACK_FIELDS } from "../../../src/core/brain/schema-pack-store.ts";const doc = shippedFormatDoc();@@ -102,6 +113,102 @@ describe("FORMAT.md", () => {expect(doc).toContain(`Format version: ${STORE_FORMAT_VERSION}`);});+ test("states which keys are list settings and how an entry's register is named", () => {+ expect(doc).toContain(+ "a `setting-entry` record for any other key is not applied, so a key that holds one value stays one register",+ );+ for (const key of KEYED_LIST_SETTINGS) expect(codeSpans.has(key)).toBe(true);+ expect(doc).toContain("NFC-normalised");+ });++ test("names the schema attestation rows the code writes", () => {+ for (const name of [+ SCHEMA_MUTATION_AUDIT_ACTION,+ SCHEMA_IMPORT_BASELINE_ACTION,+ SCHEMA_DECIDED_RECORDS_FIELD,+ SCHEMA_MUTATION_AUDIT_DIR,+ "adopted",+ "adopted_by",+ ]) {+ expect(codeSpans.has(name)).toBe(true);+ }+ });++ test("binds each attested record to its content and names the fields the pack reads", () => {+ expect(codeSpans.has("<id>:<sha256>")).toBe(true);+ expect(doc).toContain("the sha256 of the record's canonical JSON");+ for (const field of SCHEMA_PACK_FIELDS) expect(codeSpans.has(field)).toBe(true);+ expect(doc).toContain("each side of a `name=value` line normalised on its own");+ });++ test("rejects a whole-list value of a list setting that is no list, and reads the schema once", () => {+ expect(doc).toContain(+ "A `setting-set` of a list setting whose value is not a list of one-line strings, numbers or booleans is not applied",+ );+ expect(doc).not.toContain("a `setting-set` of any other value and a `setting-unset` offer");+ expect(doc).toContain("from one read of the `settings` stream");+ });++ test("drops a link constraint pair whose endpoint is undeclared", () => {+ expect(doc).toContain(+ "a link constraint pair whose source or target the vocabulary does not hold, and the constraints of an undeclared link type or of one left with no pair",+ );+ });++ test("dual-writes the tombstone of a lane page an unswitched machine removes or moves", () => {+ expect(doc).toContain(+ "writes it beside a removed or moved legacy page whose entry the store holds",+ );+ });++ test("a deciding record no row attests is a finding even when the pack digests as recorded", () => {+ expect(doc).toContain("even when the pack matches the newest row's digest");+ expect(doc).toContain("still decides the entry's slot when the removal is written");+ });++ test("a lane move replaces a shared destination with a private one by its tombstone first", () => {+ expect(doc).toContain("a move that replaces a shared entry with a private one");+ expect(doc).toContain("A move onto the entry it moves writes nothing");+ expect(doc).toContain("built from the page as it is on disk after the move");+ });++ test("a repair's removal is conditional on the record it judged, and compound keys split like the parser", () => {+ expect(codeSpans.has(IF_DECIDED_BY_FIELD)).toBe(true);+ expect(doc).toContain("applies only while the record it names decides the entry's slot");+ expect(doc).toContain("a compound `type.field` name is split at its first `.`");+ });++ test("a moved lane page that is a link is not read, and a folder is never moved into itself", () => {+ expect(doc).toContain("never read through a symbolic link");+ expect(doc).toContain("that of a destination entry the rename overwrote");+ expect(doc).toContain(+ "A folder is never moved into its own subfolder, nor into a folder that holds it",+ );+ expect(doc).toContain(+ "a record in `private` whose body blob has not arrived, quarantined on this machine or not",+ );+ });++ test("if_decided_by is ignored on other kinds, and a key that names no entry is removed", () => {+ expect(doc).toContain("on a `setting-set` or `setting-unset` it is ignored");+ expect(doc).toContain("name no entry at all once the conditional removals");+ });++ test("a move keeps a private destination private, stays in the lanes, and replaces an overwritten entry", () => {+ expect(doc).toContain(+ "A move lands in `private` when the entry it replaces at its destination is private",+ );+ expect(doc).toContain(+ "to a path that is no lane entry (outside `Brain/`, or not in a lane folder) is refused",+ );+ expect(doc).toContain("so the store never keeps the content the rename replaced");+ });++ test("describes no earlier release that misreads setting-entry", () => {+ expect(doc).not.toContain("from before `setting-entry`");+ expect(doc).toContain("Format version 1 includes `setting-entry` from the first release");+ });+test("follows the documentation style rules", () => {expect(doc).toContain("Open Second Brain");const prose = doc.replaceAll(/<!--.*?-->/g, "");