Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.
Rust ai-nativedescription-logicsformal-verificationinfrastructure-as-codeisabelle-languageknowledge-graphleanlean4mcpmcp-serverontologyowlowl2reasoningrustshaclsparqltheorem-proving
Measure the concurrency, decline Iris, and find a gate that cannot fire (#182)
Fabio Rovai75d5b55df3b0
diff --git a/docs/concurrency-inventory.md b/docs/concurrency-inventory.mdnew file mode 100644index 00000000..ce8c8c68--- /dev/null+++ b/docs/concurrency-inventory.md@@ -0,0 +1,282 @@+# The concurrent state in this codebase, measured++Every piece of shared mutable state in the Rust engine, where it lives, who touches it+concurrently, and whether anything can go wrong. It was written to answer a narrower question+(whether Iris has any application here, which is+[decision 0012](decisions/0012-concurrency-lives-below-the-certificate.md)), and it outlived that+question, because two of the things it found are defects and one of them is live.++The method was to read the code rather than the design notes. Every claim below carries a file and+a line. Counts are as of commit `32029c5`.++## Is this program actually concurrent?++Yes, and more so than the stdio transport suggests.++`rmcp` spawns every incoming JSON-RPC request as its own tokio task+(`rmcp-1.4.0/src/service.rs:963`, `spawn_service_task(async move { service.handle_request(...) })`,+where `spawn_service_task` is `tokio::spawn` without the `local` feature, which this crate does not+enable). Tool calls therefore run concurrently on a multi-threaded runtime **in stdio mode too**,+not only over HTTP. No tool method in `src/server.rs` takes a lock at its entry, so any two tools+can be inside the engine at the same time.++`serve-http` adds a second and stronger form of it. One `Arc<GraphStore>` is shared across every+MCP session and every REST route (`src/main.rs:1932`, `src/main.rs:1941-1947`), and axum serves+those requests concurrently.++So "the MCP server is async" understates the position. The server is genuinely parallel, over a+store every request can write to.++## The inventory++### 1. Process-global configuration atomics++`src/runtime.rs:44-61`. Fourteen atomics and one `RwLock<Vec<String>>`, all `static`, all written by+`init_from_config` (`src/runtime.rs:66`) and read by accessors on every use.++| global | line | written by | read by |+|---|---|---|---|+| `TABLEAUX_MAX_DEPTH`, `TABLEAUX_MAX_NODES` | 44, 45 | `init_from_config` | `src/tableaux.rs:1728, 1729, 1814, 3313, 3314` |+| `TABLEAUX_TEST_TIMEOUT_MS` | 46 | `init_from_config`, `set_tableaux_test_timeout_ms:154` | `src/tableaux.rs` phase deadlines |+| `CLASSIFY_TIMEOUT_MS` | 48 | `init_from_config`, `set_classify_timeout_ms:192` | `classify_timeout_ms():185` |+| `REASONER_MAX_ITER` | 49 | `init_from_config`, `set_reasoner_max_iterations:206` | `src/reason.rs` |+| `CACHE_HASH_PREFIX` | 50 | `init_from_config` | `src/cache.rs:64` |+| `FB_SUPPRESS`, `FB_DOWNGRADE` | 51, 52 | `init_from_config` | `src/feedback.rs` |+| `REPO_LIST_LIMIT` | 53 | `init_from_config` | `src/server.rs:326` |+| `IMPORTS_MAX_DEPTH`, `IMPORTS_TIMEOUT`, `IMPORTS_FOLLOW_REMOTE` | 54-56 | `init_from_config` | `src/server.rs:648, 650` |+| `WEBHOOK_TIMEOUT` | 57 | `init_from_config` | `src/webhook.rs` |+| `PREFERRED_LANGUAGES` | 61 | `apply_language:78` | `preferred_languages():221` |++Every one of them uses `Ordering::Relaxed`, and that is correct rather than sloppy. The fourteen+values are independent scalars. No reader derives an invariant from two of them jointly, and no+write publishes a data structure whose initialisation a reader must not observe out of order.+Relaxed is the right ordering for a knob, and there is no happens-before edge for anyone to prove.++In production the write happens once, at startup (`src/main.rs:1702`, `1801`, `2160`), before any+request is served. The interesting case is therefore tests, which is where this inventory started.++**The test-side hazard is real in principle and handled in practice.** `cargo test` runs each+integration file as its own binary and runs the tests within a binary on several threads, so a test+that stores into one of these globals is mutating state its siblings read. Every live instance is+already guarded, by three different mechanisms, and the guard is a different one each time:++- `tests/reasoner_global_budget_test.rs` stores into `CLASSIFY_TIMEOUT_MS` and+ `TABLEAUX_TEST_TIMEOUT_MS` nine times (lines 97, 98, 131, 132, 148, 149, 162, 163, 178, 179, 267,+ 268) and contains exactly one `#[test]` (line 90). Nothing races because nothing else is running.+- `tests/owl2_conformance_test.rs:37-40` stores `0` into both clocks from a helper every one of its+ 21 tests calls, and never restores. Idempotent, so concurrent stores of the same value are benign,+ and the comment at line 35 says so.+- `tests/closure_diff_test.rs`, `tests/lean_projection_entailment_test.rs` and+ `tests/gate_demonstration_test.rs` mutate `REASONER_MAX_ITER` and each carry a `serial()` helper+ over a file-static `Mutex` (`closure_diff_test.rs:24-27`, `lean_projection_entailment_test.rs:59`,+ `gate_demonstration_test.rs:28-31`). Coverage is total rather than partial: all 14 `#[test]`+ functions in `closure_diff_test.rs` take the lock, and all 17 in+ `lean_projection_entailment_test.rs` do.++The one unguarded mutation left is the unit test in `src/runtime.rs:239-264`, which stores 250 into+`TABLEAUX_MAX_DEPTH`, 128 KiB into `CACHE_HASH_PREFIX` and `false` into `IMPORTS_FOLLOW_REMOTE`, in+the lib test binary, with no lock. Its own comment (lines 232-237) explains that the two assertions+were merged into one test for exactly this reason, which handles the race against itself and not+the race against the rest of the binary. The readers in that binary are `src/cache.rs:64` and+`src/tableaux.rs:1728`. It is unobservable today, for a reason that is luck rather than design:+`src/cache.rs:65` computes `vec![0u8; prefix_len.min(size as usize)]`, and every fixture the+`cache.rs` unit tests hash is far smaller than 64 KiB, so the prefix length is never the binding+term and moving it from 64 KiB to 128 KiB changes no digest. A fixture larger than 64 KiB would+make this flaky. It is a latent hazard, not a live defect, and the cheap fix is the `serial()`+pattern the three integration files already use.++`src/config.rs:340-346` is worth reading next to this. It records the same class of hazard for+environment variables, notes that `set_var` is `unsafe` in edition 2024 precisely because the+environment is unsynchronised shared mutable state, and resolves it by splitting a pure+`resolve_storage_mode_from` out of the reader. `src/embed_fingerprint.rs:176-221` resolves the same+hazard the other way, with an `ENV_LOCK` taken before the save so a sibling cannot have its scratch+state restored as if it were the guard's. The repository already understands this category.++### 2. The shared graph store++`src/graph.rs:81-83`. `GraphStore` holds an `oxigraph::store::Store` directly, not behind a lock,+and the comment at lines 74-80 is explicit that this is deliberate: Oxigraph's `Store` is+`Send + Sync` and synchronises internally, so an added mutex bought nothing but serialisation of+non-conflicting reads and fourteen poisoning sites.++This is the most important line in the inventory for the Iris question. **All synchronisation of the+one genuinely shared, genuinely mutable object in this program happens inside a dependency.** The+Rust here holds `&self` and calls `store.insert`, `store.clear`, `store.update`, `store.iter`.++Two consequences follow, and they point in opposite directions:++- There is no ordering or ownership property in *our* code to prove, because we never wrote one.+- There is no isolation either. `src/graph.rs:630-637` (`load_lines`) inserts quad by quad in a+ loop with no transaction, so a concurrent reader observes a partially loaded graph; and+ `src/graph.rs:607-611` (`clear`) can empty the store under an in-flight query.++The `Arc<GraphStore>` is cloned into: the per-session server (`src/server.rs:26`), every registry+(`src/registry.rs:64`), the seven REST handlers (`src/main.rs:1941-1947`), and the batch runner.++### 3. The ontology registry++`src/registry.rs`. Four locks, and the only hand-written lock protocol in the codebase:+`active: Mutex<Option<ActiveEntry>>` (line 69), `reload_lock: Mutex<()>` (line 71),+`last_access: Mutex<Instant>` (line 56) and `evicted: Mutex<bool>` (line 59), the last two nested+inside the first.++The protocol is not uniform, and the gaps are where the defects are.++- `load_file` calls `self.graph.clear()` at line 128 and loads at 134 or 139, and does not take+ `active` until line 158 or `reload_lock` at all.+- `ensure_loaded` takes `reload_lock` first (line 190), snapshots the fields it needs under `active`+ (lines 199-212), then releases `active` before touching the graph.+- `evictor_tick` takes `active` (line 283) and holds it across the serialise and the clear (lines+ 298-313). It never takes `reload_lock`.+- `unload`, `unload_named`, `recompile_named`, `list_cached` and `status` take `active` only.++So `load_file` and `evictor_tick` can both be inside `self.graph` at once, and neither excludes the+other. See defect B below.++### 4. Parallel classification++`src/tableaux.rs:2648`, `2712` and `src/claimcheck.rs:554`. Three rayon `par_iter` fan-outs.++All three are read-only over immutable data, and this is worth stating precisely because the brief+suspected otherwise. `DlReasoner` contains no `Mutex`, `RwLock`, `RefCell` or `Cell`: the grep for+interior mutability across `src/` returns no hit in `tableaux.rs` at all. `decide_satisfiable_within`+and `decide_subsumption_within` take `&self` and allocate their own tableau. Results are returned by+value and `collect`ed. `CompiledOntology::check_batch` fans out over an `Arc<Index>` that is built+once and never mutated (`src/claimcheck.rs:308-310` says so, and it is true).++The single piece of cross-thread mutable state in the parallel phases is+`subsumption_cut_short: AtomicBool` at `src/tableaux.rs:2707`, a monotone latch that is only ever+stored `true` (line 2716) and is read after the join. A one-way flag needs no ordering argument.++This is embarrassingly parallel work over frozen data. There is no invariant here that a type system+does not already enforce.++### 5. Lazy index with double-checked locking++`src/claimcheck.rs:339-351`. `CompiledOntology::index()` reads under `RwLock::read`, drops, takes+`RwLock::write`, re-checks, builds, stores. Textbook double-checked locking, correct because the+re-check is inside the write lock and because `Arc<Index>` is what escapes rather than a reference+into the lock.++There is a benign inefficiency rather than a bug: `invalidate()` (line 336) can run between another+thread's build and its store, so a stale index can be published and the next reader rebuilds. The+index is a cache of a pure function of `raw`, so a stale read is a lost invalidation, not a wrong+answer, and `known_compatible` is deliberately kept outside the index (lines 313-315) so tier-2+learning does not force a rebuild.++### 6. The rest++- `src/state.rs:424`, `conn: Arc<Mutex<Connection>>`. One SQLite connection behind one mutex, with+ WAL on (`src/state.rs:428`) and a busy retry (`is_busy:412`, backoff at `src/state.rs:398`).+ Coarse, and correct by coarseness.+- `src/align.rs:13`, `src/server.rs:37`, `vecstore: Arc<Mutex<VecStore>>`. Same shape.+- `src/registry.rs:526`, `spawn_evictor`, a detached tokio task. `src/monitor.rs:261`, the same.+- `src/vecstore.rs:585, 622`, `src/sqlsource.rs:230`, `src/server.rs:2590`, `src/main.rs:3219`:+ `spawn_blocking` for CPU-bound or blocking work. Owned data moved in, nothing shared.+- `src/main.rs:1259`, `JSON_MODE: OnceLock<bool>`, set once at `src/main.rs:1930` and read+ thereafter. Write-once is the one shape that needs no argument at all.+- `src/cache.rs:353` `TMP_SEQ: AtomicU64` and `src/lineage.rs:90` `COUNTER: AtomicU64`, both+ `fetch_add` unique-name counters. Relaxed is right; only uniqueness is wanted, not order.+- `src/main.rs:1548`, an explicit 8 MiB thread for the tokio root because Windows gives the main+ thread 1 MiB. Not concurrency, stack sizing.++## The defects++### Defect A: in `serve-http` and `daemon` mode the evictor can never evict. LIVE.++`src/main.rs:1913-1921` constructs a registry solely to drive eviction:++ let shared_registry = Arc::new(OntologyRegistry::new(shared_graph.clone(), evictor_db, ...)?);+ let _evictor = spawn_evictor(shared_registry);++`OntologyRegistry::new` sets `active: Mutex::new(None)` (`src/registry.rs:82`). Nothing ever calls+`load_file` on `shared_registry`: the binding is created inside a block, moved into `spawn_evictor`,+and the block ends at line 1921. The per-session registries that `load_file` does populate are+different objects, constructed by the service factory at `src/main.rs:1928-1936`.++`evictor_tick` bails on the third line of its body:++ let Some(entry) = active.as_ref() else { return Ok(false) }; // src/registry.rs:284++So the task at `src/registry.rs:528` wakes every `evictor_interval_secs`, calls `evictor_tick`, gets+`Ok(false)`, and sleeps again, forever. The comment above it (`src/main.rs:1910-1912`) claims "the+shared one drives memory cleanup". It drives nothing.++The consequence is that `[cache] idle_ttl_secs` is silently inert in HTTP and daemon mode. An+operator who sets it to 120, as `src/config.rs:780` shows a config doing, gets no eviction and+unbounded in-memory growth, with no warning and no error. The stdio arm is fine, because there+`spawn_evictor` is given `server.registry()` (`src/main.rs:1764`), the same object `onto_load`+populates.++This is a gate that cannot fail, in a repository whose stated culture is that a gate which cannot+fail is worse than no gate. `tests/registry_evictor_wiring_test.rs` pins it.++The fix is not to point the evictor at the shared graph. Eviction state (`evicted`, at+`src/registry.rs:59`) is per registry, and `ensure_loaded` reloads only when its own registry's flag+is set (`src/registry.rs:243`). Clearing the shared store from a registry no session consults would+leave every session believing its ontology is resident, with no path back. The honest options are to+say plainly that the TTL is not honoured in HTTP mode, or to move the active slot and its eviction+flag to where the store is, which is the shared object. Both are larger than this document.++### Defect B: concurrent `load_file` over a shared store interleaves. LATENT.++`src/registry.rs:128` clears the graph, lines 130 to 155 load into it, and `active` is not taken+until line 158. Under `serve-http` every session's registry wraps the same `Arc<GraphStore>`, and+rmcp runs each request in its own task. Two concurrent `onto_load` calls can interleave as:++ session 1: clear() store = {}+ session 2: clear() store = {}+ session 1: load A store = A+ session 2: load B store = A union B+ session 2: active := B, reports |B|+ session 1: active := A, reports |A|++The store then holds the union of two ontologies while both registries report their own triple+count, and `onto_cache_status` (`src/registry.rs:489`) reports `in_memory_triples` from the store,+which agrees with neither. The reasoner would then classify A union B and certify the result against+an `asserted.tsv` drawn from the same union, so the certificate would be internally consistent and+about a graph no user asked for.++The REST surface reaches the same state without any concurrency at all: `/api/load`+(`src/main.rs:1981-1992`) calls `g.load_file(&path)` directly on the shared graph, with no clear and+no registry update, so it appends into a store a session's registry believes holds only its own+ontology.++Marked latent rather than live only because defect A means the evictor is not also in there, and+because the multi-session HTTP path is not the common deployment. The missing lock is real.++### Defect C: there is no read lease, so a long tool can be evicted under itself. LATENT.++`touch()` (`src/registry.rs:179`) stamps `last_access` when a tool STARTS. `ensure_loaded` releases+`active` at line 212 and returns, and the caller then uses the graph with nothing held. The TTL clock+runs during the tool's execution. A run longer than `idle_ttl_secs` can therefore have+`evictor_tick` clear the store under it at `src/registry.rs:312`.++The default `classify_timeout_ms` is 180 000 (`src/runtime.rs:30`), so a classification is allowed to+run for three minutes; a TTL below that is an ordinary setting. The mitigation is accidental:+`idle_ttl_secs` defaults to 0 (`src/config.rs:282`), which disables eviction, and in HTTP mode defect+A disables it again.++### Defect D: the store can drift between hashing and reasoning. KNOWN, DOCUMENTED, OPEN.++`Reasoner::run_horn` takes `&Arc<GraphStore>` (`src/reason.rs:2915`) and reads the store at line 2935.+This is already+[decision 0010](decisions/0010-the-input-is-a-value-and-not-a-store.md), which names it, calls the+window a race in as many words, and specifies `CertifiedInput` to dissolve it by copying once. It is+accepted and not yet implemented. Recording it here so the four sit in one list.++## What would have caught these++None of A, B or C needs a program logic, and it is worth being concrete about what each does need,+because the answer is the same cheap answer three times.++A is not a race. The code is perfectly synchronised and does nothing; a two-line test that calls+`evictor_tick` on a registry nothing loaded into catches it, and now does.++B and C are missing critical sections. `loom` explores every interleaving of `std::sync` primitives+exhaustively and is the standard tool for exactly this; a two-thread test with a barrier finds B in+an afternoon without it. Neither needs a proof, because the property being violated is not subtle:+two writers are inside one object with nothing between them.++D is the one with real depth, and decision 0010 already solves it by removing the shared state+rather than by reasoning about it.diff --git a/docs/decisions/0012-concurrency-lives-below-the-certificate.md b/docs/decisions/0012-concurrency-lives-below-the-certificate.mdnew file mode 100644index 00000000..fd666d60--- /dev/null+++ b/docs/decisions/0012-concurrency-lives-below-the-certificate.md@@ -0,0 +1,164 @@+# 0012. Concurrency lives below the certificate, and a proof cannot follow it there++Status: **declined**. Written 18 September 2026.++Iris is the higher-order concurrent separation logic built in Rocq. The question put to it was+whether it has any real application in this repository, given that the Rust side is not pure: it has+a store behind an `Arc`, fourteen process-global atomics, three rayon fan-outs, and an MCP server+that turns out to be more parallel than it looks.++The answer is no, and the evidence is an inventory rather than an argument about Iris.+`docs/concurrency-inventory.md` is that inventory, and it should be read first. This record is the+ruling that follows from it.++## The failure this rule prevents++Adopting a program logic for a hazard that lives in a dependency, and calling the result assurance.++That is the same failure decision 0005 rules on for provers and decision 0006 rules on for+refutations, in a third place. The pattern is that a name with a strong reputation gets pointed at a+system, and nobody checks whether the thing the name is strong at is the thing this system does.++## What was measured++What reading the code established, at the level of detail the inventory carries.++**The program is genuinely concurrent, and more so than the design notes say.** `rmcp` spawns every+request as its own tokio task (`rmcp-1.4.0/src/service.rs:963`), so tool calls run in parallel in+stdio mode and not only over HTTP. No tool method takes a lock on entry. `serve-http` shares one+`Arc<GraphStore>` across every session and every REST route (`src/main.rs:1932`, `1941-1947`).++**The one shared mutable object synchronises itself, inside a dependency.** `GraphStore` holds an+`oxigraph::store::Store` directly, and `src/graph.rs:74-80` says why: Oxigraph's `Store` is+`Send + Sync` and synchronises internally, so a mutex bought only the serialisation of+non-conflicting reads. Every `insert`, `clear`, `update` and `iter` in this codebase is a call into+that engine, which is backed by RocksDB, which is C++.++**The concurrency we do write has no invariant to prove.** All fourteen atomics in `src/runtime.rs`+are independent scalars read with `Ordering::Relaxed`, with no reader deriving anything from two of+them jointly, so there is no ordering property to establish. The three rayon fan-outs+(`src/tableaux.rs:2648`, `2712`, `src/claimcheck.rs:554`) are read-only over frozen data:+`DlReasoner` contains no interior mutability at all, and the only cross-thread mutable cell in the+parallel phases is `subsumption_cut_short`, a monotone latch that is only ever stored `true`+(`src/tableaux.rs:2707, 2716`). The double-checked lock at `src/claimcheck.rs:339-351` is textbook+and correct, and its one imperfection loses a cache invalidation rather than an answer.++**The certificate layer has no concurrency by construction.** `docs/trusted-computing-base.md`+states it plainly: the checkers are pure functions of files on disk. Decision 0010 goes further and+makes the reasoner's input an immutable value precisely so that the one window where the store could+drift under a certified run stops existing. Iris reasons about stateful concurrent programs. The+part of this system that carries the assurance is neither, on purpose, and decision 0010 is the+repository choosing to keep it that way.++## Why Iris in particular does not apply++The reasons below run in increasing order of how hard they are to argue with.++**The subset problem, which this repository has already ruled on twice.**+`docs/reasoning-systems-inventory.md` declines Aeneas with Charon, and separately Verus, Creusot and+Prusti, because each needs the code inside its own subset and this codebase is not inside any of+them. The Iris-based route to Rust is RefinedRust (PLDI 2024), and its subset is narrower than+Aeneas's, not wider. Declining the broader tools and adopting the narrower one would be incoherent.++**The part of our concurrency that is delicate is already proved, by someone else, and we get it for+free.** RustBelt (Jung, Jourdan, Krebbers and Dreyer, POPL 2018) used Iris to verify the soundness of+exactly the standard library types this codebase leans on: `Arc`, `Mutex`, `RwLock`, `Cell`,+`RefCell`. The unsafe code that makes `Arc<GraphStore>` sound to share is not our unsafe code. There+is no unsafe concurrent code anywhere in `src/`, and no hand-rolled lock-free structure. Pointing+Iris at a program whose only concurrency primitives are the ones Iris has already verified is+re-proving a published theorem about someone else's crate.++**The state that matters is not reachable.** Suppose the subset problem were solved. The property an+operator actually wants is that a query and a load do not corrupt each other, and that property is a+statement about Oxigraph's transactional behaviour. An Iris proof of it would need a formal model of+Oxigraph's MVCC layer, which nobody has written, and that model would still bottom out in RocksDB,+which is C++ and outside Rocq entirely. The proof would be larger than everything in `lean/`+combined and its conclusion would be conditional on an unverified model of a dependency. That is the+same shape as trusting a prover's refutation, which decision 0005 refuses.++**The defects that are actually here are not the defects Iris prevents.** This is the decisive one,+because it is empirical rather than architectural. The inventory found three defects while looking+for a property worth proving, and not one of them is a race:++- Defect A, live: the evictor spawned in `serve-http` and `daemon` mode is wired to a registry+ nothing loads into (`src/main.rs:1913-1921`), so `evictor_tick` returns `Ok(false)` forever on its+ third line (`src/registry.rs:284`) and `[cache] idle_ttl_secs` is silently inert. The code is+ perfectly synchronised. It simply does nothing. An Iris proof would establish the evictor's+ specification, and the specification is satisfied.+- Defect B, latent: `load_file` clears the store at `src/registry.rs:128` and does not take a lock+ until line 158, so two concurrent loads over the shared store leave it holding the union of two+ ontologies while each registry reports its own count. This is a missing critical section, visible+ by reading thirty lines.+- Defect C, latent: `touch()` stamps `last_access` when a tool starts and nothing holds a read lease+ while it runs (`src/registry.rs:179`, `212`), so a tool outliving `idle_ttl_secs` can have the+ store cleared under it at `src/registry.rs:312`.++A logic adopted to find concurrency defects that finds none of the three concurrency-adjacent+defects present is not being declined on taste.++## If it were attempted anyway, the size++Asked for honestly, because "it would be a lot of work" is the kind of claim this repository does not+accept without a number.++The only candidate property is the registry's lock protocol: that `active`, `reload_lock`,+`last_access` and `evicted` together maintain the invariant that the in-memory store agrees with the+active entry's `evicted` flag. `src/registry.rs` is 540 lines. A RefinedRust proof of it would need,+in order: the file rewritten into RefinedRust's subset, including its `anyhow` error paths and its+`serde_json` output; a specification of `tokio::spawn` and of task detachment, which RefinedRust does+not have; a specification of `std::fs` for the eviction snapshot, which is where the interesting+failure modes live; and a model of `GraphStore::clear` and `serialize`, which is the Oxigraph problem+above. Three of those four are new research rather than new proof. The fourth, the Oxigraph model, is+the largest single formalisation anyone has proposed for this project.++And the theorem at the end would be conditional on that model, so it would be an assurance claim+resting on an unverified translation of a dependency, which is the exact shape of claim decision 0005+exists to refuse.++## What is adopted instead++Nothing exotic, which is the point.++`loom` explores every interleaving of `std::sync` primitives exhaustively and is the standard tool+for defects B and C. It costs a dev-dependency and a `cfg` attribute, it needs no rewrite, and it+fails a test rather than emitting a proof obligation. A two-thread test with a barrier finds defect B+without even that.++Defect A needs no concurrency tooling at all, and+`tests/registry_evictor_wiring_test.rs` is the test that pins it. It asserts the current wrong+behaviour deliberately, in both directions, so that closing the defect breaks one assertion and the+tempting wrong repair breaks the other.++Kani stays where `docs/reasoning-systems-inventory.md` already puts it, on the trusted boundary, and+decision 0010's `CertifiedInput` stays the answer to defect D. Removing shared state beats reasoning+about it.++## What would have to become true to revisit this++Named concretely, so that a future reader can check them rather than re-argue the question.++1. **Hand-written unsafe concurrent code in the trusted path.** Today there is none. If someone+ writes a lock-free structure whose invariant no type system enforces, and that structure sits+ between the store and a certificate, the calculation changes.+2. **A concurrent phase inside the certified reasoner.** Decision 0010 makes the input a value+ precisely to avoid this. If a certified run ever acquires genuinely concurrent access to shared+ mutable state, there is a property worth proving and it is on the assurance path.+3. **A verified store.** If the RDF store were replaced by something with a machine-checked+ concurrency model, the third objection above dissolves and the second and fourth would have to be+ re-weighed on their own.+4. **RefinedRust's subset growing to contain async Rust and this codebase.** This is the least likely+ of the four and the easiest to check.++None of the four is true today. Two of them, 1 and 2, are things this project is actively arranging+not to be true.++## Consequences++The inventory is the deliverable and it stands on its own: `docs/concurrency-inventory.md` names+every piece of shared mutable state in the engine with a file and a line, which nothing did before.+One live defect, A, is recorded and pinned by a test. Two latent ones, B and C, are recorded with the+interleaving that reaches them.++`docs/reasoning-systems-inventory.md` gains a row, because a system considered and declined belongs+in that table rather than in a commit message. That is the whole purpose of the table: a reader+should be able to see that Iris was looked at, and why the answer was no, without reading this file.diff --git a/docs/decisions/README.md b/docs/decisions/README.mdindex dcdac3f9..a8c92123 100644--- a/docs/decisions/README.md+++ b/docs/decisions/README.md@@ -12,6 +12,7 @@ and whose benefit is not gets dropped the first time it is inconvenient.| [0006](0006-a-model-is-a-certificate-and-a-refutation-is-not.md) | A model is a certificate, and a refutation is not | Treating an exhausted bounded search as unsatisfiability || [0007](0007-a-slice-preserves-a-conclusion-or-it-does-not.md) | A slice preserves a conclusion, or it does not | A retrieval slice quietly dropping the conclusion it was asked about || [0008](0008-a-binding-is-data-and-evidence-admits-one-reading.md) | A binding is data, and evidence admits one reading | A certificate format admitting two readings, so two checkers disagree |+| [0012](0012-concurrency-lives-below-the-certificate.md) | Concurrency lives below the certificate, and a proof cannot follow it there | Adopting a program logic for a hazard that lives in a dependency, and calling the result assurance |There is no 0004. The decision now numbered 0006 was drafted as 0004 on a branch that never merged,so the number never reached `main`. It is left as a hole rather than reused, because a reused numberdiff --git a/docs/reasoning-systems-inventory.md b/docs/reasoning-systems-inventory.mdindex c743f11c..e4ce15f2 100644--- a/docs/reasoning-systems-inventory.md+++ b/docs/reasoning-systems-inventory.md@@ -36,7 +36,9 @@ about the file we gave it. The addendum to decision 0005 draws the new line.| Duper, lean-smt | Lean automation | Declined. Both require Mathlib. || Aeneas with Charon | Rust to Lean | Declined. Subset does not contain this codebase. || Verus, Creusot, Prusti | Rust verification | Declined. Each needs the code rewritten in its subset. |+| Iris, RefinedRust | Concurrent separation logic | Declined. The shared state is inside Oxigraph, and the certificate layer is pure. Below. || Kani | Rust bounded model checker | Being applied to the trusted boundary only. |+| loom | Rust interleaving explorer | The right tool for the two latent lock defects. Not yet wired in. || TPTP and TSTP | Interchange | Implemented, in both directions. TPTP out, TSTP back in and checked. || CLIF, ISO/IEC 24707 | Interchange | Implemented. The conformance format. || SMT-LIB 2 | Interchange | Under construction. |@@ -243,6 +245,34 @@ it is a serialiser, a parser and an interner rather than fifty thousand lines. Tis being property-tested and, where it pays, model-checked with Kani. What remains trusted after thatwork will be named explicitly rather than left for a reader to infer.+## Iris, and the concurrency that is not ours++Iris is the higher-order concurrent separation logic built in Rocq, and the obvious reason to want it+here is that the Rust side is plainly not pure: a store behind an `Arc`, fourteen process-global+atomics in `src/runtime.rs`, three rayon fan-outs, and an MCP server that spawns every request as its+own tokio task. That last fact is worth knowing on its own, since it means tool calls run in parallel+in stdio mode and not only over HTTP.++It is declined, and the argument is measured rather than asserted.+[docs/concurrency-inventory.md](concurrency-inventory.md) names every piece of shared mutable state in+the engine with a file and a line, and+[decision 0012](decisions/0012-concurrency-lives-below-the-certificate.md) is the ruling.++The short version has three parts. The one genuinely shared mutable object is+`oxigraph::store::Store`, which synchronises itself inside a dependency backed by RocksDB, so there is+no ordering property in our code to prove and no way for a Rocq proof to reach the one that matters.+The concurrency we do write has nothing to establish: all fourteen atomics are independent scalars+read relaxed, the rayon phases are read-only over frozen data with one monotone latch, and `Arc`,+`Mutex` and `RwLock` were already verified in Iris by RustBelt, so we consume that theorem by using+`std` rather than by re-proving it. And the subset objection that declined Aeneas, Verus, Creusot and+Prusti in the section above applies harder to RefinedRust, whose subset is narrower still.++The decisive evidence is empirical. The inventory went looking for a property worth proving and came+back with three defects instead, none of which is a race: an evictor wired to a registry nothing loads+into, so it can never fire; a missing critical section in `load_file`; and a missing read lease around+long-running tools. `loom` and a two-thread test find the second and third. The first needs no+concurrency tooling at all and is pinned by `tests/registry_evictor_wiring_test.rs`.+## Rule languagesThe Horn certificate layer is generic over rule tables, with one soundness theorem covering everydiff --git a/tests/registry_evictor_wiring_test.rs b/tests/registry_evictor_wiring_test.rsnew file mode 100644index 00000000..13cea406--- /dev/null+++ b/tests/registry_evictor_wiring_test.rs@@ -0,0 +1,185 @@+//! The evictor the HTTP server spawns is wired to a registry nothing loads into, so it+//! can never evict anything. This file pins that.+//!+//! `src/main.rs:1913-1921` builds a registry for the sole purpose of driving eviction,+//! over the graph every session shares, and hands it to `spawn_evictor`. The comment+//! above it says "the shared one drives memory cleanup". It drives nothing.+//! `OntologyRegistry::new` starts with `active: None` (`src/registry.rs:82`), only+//! `load_file` ever fills that slot (`src/registry.rs:158`), nothing calls `load_file` on+//! this registry, and `evictor_tick` returns `Ok(false)` on its third line when the slot+//! is empty (`src/registry.rs:284`). So `[cache] idle_ttl_secs` is silently inert in+//! `serve-http` and `daemon` mode however it is configured.+//!+//! This is the shape `docs/ci-gates.md` exists to attack, one level down: not a test that+//! skips and reports `ok`, but a background task that runs, does nothing, and reports+//! nothing. A green process is not a working evictor.+//!+//! These tests are characterisation tests. They assert what the code does TODAY, which is+//! wrong, so that the wrongness is visible in the suite rather than in a reader's+//! inference from two files. They are written so that either direction of change breaks+//! them: closing the defect properly makes `an_unloaded_registry_never_evicts` fail, and+//! "fixing" it by pointing the evictor at the shared graph regardless of the active slot+//! makes `the_dead_evictor_does_not_touch_the_shared_store` fail, which is the fix that+//! must not be made (eviction state is per registry, so clearing a store no session's+//! registry has flagged leaves every session believing its ontology is resident, with no+//! path back through `ensure_loaded`).+//!+//! Full analysis in `docs/concurrency-inventory.md`, defect A.+//!+//! Nothing here can skip: no external toolchain, no fixture, no network. It needs only a+//! temporary directory.++use std::path::PathBuf;+use std::sync::Arc;++use open_ontologies::config::CacheConfig;+use open_ontologies::graph::GraphStore;+use open_ontologies::registry::OntologyRegistry;+use open_ontologies::state::StateDb;++const TTL_SECS: u64 = 1;++/// A scratch directory unique to one test, removed when the test ends.+struct Scratch(PathBuf);++impl Scratch {+ fn new(name: &str) -> Self {+ let dir = std::env::temp_dir().join(format!(+ "oo-evictor-{}-{}-{}",+ name,+ std::process::id(),+ std::time::SystemTime::now()+ .duration_since(std::time::UNIX_EPOCH)+ .map(|d| d.as_nanos())+ .unwrap_or(0)+ ));+ std::fs::create_dir_all(&dir).expect("create scratch dir");+ Self(dir)+ }+}++impl Drop for Scratch {+ fn drop(&mut self) {+ let _ = std::fs::remove_dir_all(&self.0);+ }+}++/// Exactly what `src/main.rs:1913-1921` builds: a registry over a shared graph, with+/// eviction enabled and a TTL that has already elapsed by the time the tick runs, and+/// with nothing ever loaded through it.+fn evictor_registry(scratch: &Scratch, graph: Arc<GraphStore>) -> Arc<OntologyRegistry> {+ let db = StateDb::open(&scratch.0.join("state.db")).expect("open state db");+ let cfg = CacheConfig {+ enabled: true,+ dir: scratch.0.to_string_lossy().into_owned(),+ idle_ttl_secs: TTL_SECS,+ evictor_interval_secs: 1,+ auto_refresh: false,+ ..CacheConfig::default()+ };+ Arc::new(OntologyRegistry::new(graph, db, cfg).expect("build registry"))+}++fn some_triples(graph: &GraphStore) -> usize {+ graph+ .load_turtle(+ "@prefix : <https://example.org/> .\n\+ :A a <http://www.w3.org/2002/07/owl#Class> .\n\+ :B a <http://www.w3.org/2002/07/owl#Class> .\n\+ :A <http://www.w3.org/2000/01/rdf-schema#subClassOf> :B .\n",+ None,+ )+ .expect("load turtle")+}++/// The defect itself. The store is full, the TTL has elapsed, eviction is enabled, and+/// the tick still reports that it evicted nothing, because the active slot of THIS+/// registry was never filled.+///+/// When defect A is closed this assertion must be inverted, and that is the point of+/// writing it down: the fix cannot land silently.+#[test]+fn an_unloaded_registry_never_evicts() {+ let scratch = Scratch::new("never-evicts");+ let graph = Arc::new(GraphStore::new());+ let loaded = some_triples(&graph);+ assert!(loaded > 0, "the fixture must put something in the store");++ let registry = evictor_registry(&scratch, graph.clone());++ // Well past `idle_ttl_secs`, so nothing below can be explained by the entry still+ // being fresh. There is no entry at all.+ std::thread::sleep(std::time::Duration::from_millis(1_100 * TTL_SECS));++ let evicted = registry.evictor_tick().expect("tick must not error");+ assert!(+ !evicted,+ "DEFECT A IS FIXED: the evictor now reports an eviction. Invert this test and \+ update docs/concurrency-inventory.md, which records it as live."+ );+}++/// The other half, and the one that guards against the wrong fix. Because the tick does+/// nothing, the shared store is untouched, so an operator who set `idle_ttl_secs` gets+/// unbounded growth rather than the eviction they configured.+///+/// This assertion is what must NOT change. Making the evictor clear a store whose active+/// slot it does not own is the tempting one-line repair and it is worse than the defect:+/// `ensure_loaded` reloads only when its OWN registry's `evicted` flag is set+/// (`src/registry.rs:243`), and this registry is not the one any session consults.+#[test]+fn the_dead_evictor_does_not_touch_the_shared_store() {+ let scratch = Scratch::new("untouched");+ let graph = Arc::new(GraphStore::new());+ let loaded = some_triples(&graph);++ let registry = evictor_registry(&scratch, graph.clone());+ std::thread::sleep(std::time::Duration::from_millis(1_100 * TTL_SECS));+ let _ = registry.evictor_tick().expect("tick must not error");++ assert_eq!(+ graph.triple_count(),+ loaded,+ "a registry whose active slot is empty must never clear the shared store: \+ eviction state is per registry, so clearing a store no session's registry has \+ flagged strands every session on a store it believes is resident"+ );+}++/// The control, and the reason the two tests above are about wiring rather than about+/// eviction being broken in general. Give a registry an entry of its own and the same+/// tick evicts, which is what the stdio arm gets at `src/main.rs:1764` because there+/// `spawn_evictor` is handed the very registry `onto_load` fills.+#[test]+fn a_registry_with_its_own_entry_does_evict() {+ let scratch = Scratch::new("does-evict");+ let graph = Arc::new(GraphStore::new());+ let registry = evictor_registry(&scratch, graph.clone());++ let source = scratch.0.join("ontology.ttl");+ std::fs::write(+ &source,+ "@prefix : <https://example.org/> .\n\+ :A a <http://www.w3.org/2002/07/owl#Class> .\n",+ )+ .expect("write source");++ registry+ .load_file(&source.to_string_lossy(), Default::default())+ .expect("load through the registry");+ assert!(graph.triple_count() > 0, "the load must populate the store");++ std::thread::sleep(std::time::Duration::from_millis(1_100 * TTL_SECS));++ let evicted = registry.evictor_tick().expect("tick must not error");+ assert!(+ evicted,+ "a registry that owns an idle entry must evict it; if this fails the defect is \+ not the wiring and the other two tests in this file are measuring the wrong thing"+ );+ assert_eq!(+ graph.triple_count(),+ 0,+ "eviction must actually clear the in-memory store"+ );+}