Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.
Rust ai-nativedescription-logicsformal-verificationinfrastructure-as-codeisabelle-languageknowledge-graphleanlean4mcpmcp-serverontologyowlowl2reasoningrustshaclsparqltheorem-proving
A model of a weaker export is not a model of the ontology (#269)
Fabio Rovai747a89e7871f
diff --git a/CLAUDE.md b/CLAUDE.mdindex f69e997d..99fec653 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -115,7 +115,7 @@ Claude dynamically decides the next tool call based on what the previous tool re| `onto_dl_explain` | To explain why a class is unsatisfiable using DL tableaux reasoning — returns clash trace || `onto_dl_check` | To check if one class is subsumed by another using DL tableaux reasoning || `onto_fol_export` | To hand the ontology to the automated-theorem-proving ecosystem. Writes TPTP FOF (what E and Vampire read), ISO/IEC 24707 CLIF (the standards-track interchange syntax, and what ISO/IEC 21838-2 publishes BFO in), restricted both to the first-order-equivalent fragment of Common Logic and to the subdialect ISO/IEC 24707 A.4.2 calls exactly semantically conformant, each checked by a test rather than asserted, and ISO/IEC 24707 CGIF, the second of Common Logic's three dialects, in CORE CGIF and in the compact sub-dialect clause 7.1.1 names (no sequence markers, which is what clause 6.5 says takes Common Logic past first order; also no `#?` type label and no actor). CGIF conformance is pinned by a lexer, parser and checker written from Annex B's EBNF IN THE TEST, not by an independent parser, because no installable CGIF parser exists; the CLIF claims rest on two external parsers and the CGIF claims do not, and the report names XCL as the dialect still not emitted. Both are serialisers over ONE translation: the one `OwlLean.adequacy` in the sibling owl-lean project is machine-checked about, including the background axioms and the individual typing axioms that theorem requires. The correspondence between this emitter and that Lean is PINNED BY TESTS AND NOT PROVED, and the emitted file says so. Constructs outside the fragment are named with counts and reasons in `constructs_not_exported`, never dropped silently. CLIF sentences are emitted BARE with standalone `(cl:comment '...')` labels, because the two CLIF parsers that exist both return an EMPTY theory from the wrapped `(cl:comment '...' SENTENCE)` form that ISO/IEC 21838-2's own BFO files use; `clif_comments: "wrapped"` restores that form and loses the content. `smtlib` (SMT-LIB 2, what Z3 reads) and `ladr` (what Mace4 reads, every symbol MANGLED with the table in `symbols.tsv`, because LADR reads a name beginning with u, v, w, x, y or z as a VARIABLE) write the same translation for MODEL FINDERS, and assert the NEGATED goal because a countermodel to `G |= phi` is a model of `G + {not phi}`. Every run also writes `problem.tsv` with its digest, the format the verified checker reads. Pass `goals_file` to also write one problem per conjecture. A PROVER'S VERDICT ON THE OUTPUT IS AN ORACLE OPINION AND NEVER A CERTIFICATE: use tools/fol_differential.py, which reports disagreement between this engine and an ATP and does not adjudicate it. See docs/first-order-export.md and decision 0005 |-| `onto_fol_model` | To find a FINITE MODEL of the loaded ontology and CHECK it, so the answer names what it rests on. The SAT/SMT family, and deliberately not symmetric: a refutation cannot be replayed in core Lean, so a solver's `unsat` is an ORACLE OPINION for ever, while a MODEL is a finite object and `Fol.satisfiable_of_check` in `lean/Fol/` turns an accepted structure into satisfiability of exactly the formulas it was checked against. With a goal, `Fol.not_entails_of_check` gives a machine-checked NON-ENTAILMENT, the one sentence no prover can produce. FIVE FIELDS THAT ARE NEVER COLLAPSED: `solver_verdict`, `encoding`, `checker_exit`, `verdict`, `owl_reading`. Five verdict words: `model_checked` is the ONLY certified one and requires `checker_exit: 0`; `no_model_up_to_size_k` is an exhausted BOUNDED search and IS NOT unsatisfiability; `unsatisfiable_oracle` may come only from a run with no cardinality constraint. A solver answering `sat` whose model the checker REJECTS is a STOP_THE_LINE disagreement, never `rejected` and never `model_checked`. Needs z3 or mace4 and `oo-folmodel` (`cd lean && lake build`); the absence of either is reported in `skipped`. See docs/decisions/0006 |+| `onto_fol_model` | To find a FINITE MODEL of the loaded ontology and CHECK it, so the answer names what it rests on. The SAT/SMT family, and deliberately not symmetric: a refutation cannot be replayed in core Lean, so a solver's `unsat` is an ORACLE OPINION for ever, while a MODEL is a finite object and `Fol.satisfiable_of_check` in `lean/Fol/` turns an accepted structure into satisfiability of exactly the formulas it was checked against. With a goal, `Fol.not_entails_of_check` gives a machine-checked NON-ENTAILMENT, the one sentence no prover can produce. FIVE FIELDS THAT ARE NEVER COLLAPSED: `solver_verdict`, `encoding`, `checker_exit`, `verdict`, `owl_reading`. Five verdict words: `model_checked` is the ONLY certified one and requires `checker_exit: 0`; `no_model_up_to_size_k` is an exhausted BOUNDED search and IS NOT unsatisfiability; `unsatisfiable_oracle` may come only from a run with no cardinality constraint. A solver answering `sat` whose model the checker REJECTS is a STOP_THE_LINE disagreement, never `rejected` and never `model_checked`. Needs z3 or mace4 and `oo-folmodel` (`cd lean && lake build`); the absence of either is reported in `skipped`. A construct the reader cannot translate makes the export WEAKER than the ontology, and a model of a weaker theory is not a model of the ontology: `exports_a_weaker_axiom_set` and `constructs_not_exported` name every dropped construct, `owl_reading` is withheld, and `transfers_to_the_ontology` is false for `model_checked` and `satisfiable_oracle` (the unsatisfiable-direction verdicts still carry over). See docs/decisions/0006 || `onto_embed` | After loading an ontology — generates text + Poincaré structural embeddings for all classes. The default local model is **multilingual** (`paraphrase-multilingual-MiniLM-L12-v2`), so labels in different natural languages embed into a shared space. Honours `[embeddings] provider = "local" \| "openai"` in `config.toml`; OpenAI-compatible gateways (Azure, Ollama, vLLM, LocalAI, …) are supported via `OPEN_ONTOLOGIES_EMBEDDINGS_*` env vars || `onto_search` | To find classes by natural language description — requires onto_embed first || `onto_similarity` | To compute embedding similarity between two specific IRIs |diff --git a/src/fol_solve.rs b/src/fol_solve.rsindex c1945364..8fb022c6 100644--- a/src/fol_solve.rs+++ b/src/fol_solve.rs@@ -826,6 +826,32 @@ pub fn verdict_means(v: &str) -> &'static str {}}+/// Does a verdict about the exported problem hold of the ontology too?+///+/// The export drops constructs and never adds any, so the ontology's+/// translation entails the export's. A verdict that there is NO model (or no+/// model up to k) therefore carries over, and a verdict that there IS one does+/// not. A goal problem is the axioms plus the negated goal, so the same holds+/// for it: a countermodel of the weaker theory is not a non-entailment.+fn transfers(o: &Outcome, weaker: bool) -> bool {+ !weaker+ || !matches!(+ o.verdict,+ FolVerdict::ModelChecked(_) | FolVerdict::SatisfiableOracle+ )+}++/// `owl_reading` is the OWL-level sentence "not entailed". Over a weaker+/// export it can be false of the ontology, so it is not minted into the+/// report. The verdict itself stays: it is a true, checked statement about+/// the problem the checker was handed, and `transfers_to_the_ontology` says+/// what it is not.+fn withhold_owl_reading(o: &mut Outcome, weaker: bool) {+ if weaker {+ o.owl_reading = None;+ }+}+/// Run the pipeline over the loaded ontology and, optionally, one problem per/// goal. Returns the report JSON.pub fn solve_export(@@ -839,8 +865,19 @@ pub fn solve_export(let read = crate::tptp::read_graph(triples);std::fs::create_dir_all(dir)?;+ // What the reader could not translate. `tptp::export` has always put this+ // ledger in its report; this pipeline did not, and so a model of the+ // WEAKER exported theory came back as `model_checked` with `owl_reading`+ // set and nothing anywhere to say the checked problem was not the+ // ontology. It was found by running inconsistent ontologies through here:+ // `owl:AllDisjointProperties`, `owl:NegativePropertyAssertion`,+ // `owl:hasKey` and a functional datatype property each produced a+ // certified model of a graph with no models at all.+ let weaker = !read.dropped.is_empty();+let base = FolProblem::build(&read.axioms, None)?;- let ontology = solve(&base, opts, &dir.join("ontology"))?;+ let mut ontology = solve(&base, opts, &dir.join("ontology"))?;+ withhold_owl_reading(&mut ontology, weaker);let mut goal_reports = Vec::new();// A goal the fragment cannot express is NAMED, not skipped. `tptp::export`@@ -886,24 +923,42 @@ pub fn solve_export(}};let gp = FolProblem::build(&read.axioms, Some(&ax))?;- let g = solve(&gp, opts, &dir.join(format!("goal_{i:05}")))?;+ let mut g = solve(&gp, opts, &dir.join(format!("goal_{i:05}")))?;+ withhold_owl_reading(&mut g, weaker);*counts.entry(g.verdict.word()).or_default() += 1;if g.disagreement.is_some() {stop_the_line += 1;}+ let mut outcome = outcome_json(&g);+ outcome["transfers_to_the_ontology"] = transfers(&g, weaker).into();goal_reports.push(serde_json::json!({"triple": [s, p, o],- "outcome": outcome_json(&g),+ "outcome": outcome,}));}}+ let mut ontology_json = outcome_json(&ontology);+ ontology_json["transfers_to_the_ontology"] = transfers(&ontology, weaker).into();+Ok(serde_json::json!({"solver": opts.solver.name(),"max_domain": opts.max_domain,"timeout_secs": opts.timeout_secs,"dir": dir.display().to_string(),- "ontology": outcome_json(&ontology),+ "ontology": ontology_json,+ "exports_a_weaker_axiom_set": weaker,+ "constructs_not_exported": read.dropped,+ "reduced_to_fragment": read.reduced,+ "transfers_to_the_ontology_means": "whether the verdict, which is about the EXPORTED \+ problem, also holds of the ontology. Dropping a \+ construct only removes constraints, so every model \+ of the ontology is a model of the export and not \+ the reverse: when exports_a_weaker_axiom_set is \+ true, `model_checked` and `satisfiable_oracle` \+ say NOTHING about the ontology and `owl_reading` \+ is withheld, while `unsatisfiable_oracle` and \+ `no_model_up_to_size_k` still carry over","goals": goal_reports,"goals_not_asked": not_asked.len(),"not_asked": not_asked,diff --git a/src/server.rs b/src/server.rsindex 94c89147..b8a7a197 100644--- a/src/server.rs+++ b/src/server.rs@@ -2400,7 +2400,7 @@ impl OpenOntologiesServer {.unwrap_or_else(Self::err_json)}- #[tool(name = "onto_fol_model", description = "Find a finite model of the loaded ontology and CHECK IT, so the answer names what it rests on. This is the SAT/SMT family, and it is deliberately not symmetric. A REFUTATION cannot be replayed in core Lean, so a solver saying `unsat` is an ORACLE OPINION for ever (decision 0005). A MODEL is a finite object, checking a formula against it is decidable, and `lean/Fol/` holds a checker whose soundness is machine-checked: `Fol.satisfiable_of_check` turns an accepted structure into satisfiability of exactly the formulas it was checked against, and `Fol.not_entails_of_check` turns a checked model of the NEGATED goal into a machine-checked NON-ENTAILMENT, which is the one sentence no theorem prover can produce. The pipeline exports SMT-LIB 2 (for Z3) or LADR (for Mace4) and `problem.tsv` from ONE representation, climbs a cardinality ladder, reads the structure back, and runs `oo-folmodel`. FIVE FIELDS THAT ARE NEVER COLLAPSED: `solver_verdict` (sat/unsat/unknown, what the oracle said), `encoding` (unbounded or finite(k), what it was asked), `checker_exit` (null means the checker never ran), `verdict`, and `owl_reading`. The verdict is one of FIVE WORDS: `model_checked` is the ONLY certified one and requires checker_exit 0; `satisfiable_oracle` is sat with nothing checked; `no_model_up_to_size_k` is an exhausted BOUNDED search and IS NOT UNSATISFIABILITY (a theory with no model of size k can have one of size k+1, and SHIQ has no finite model property at all, so a satisfiable ontology can have only infinite models and will never receive a certificate here); `unsatisfiable_oracle` may be produced ONLY by a run with no cardinality constraint of any kind and can never be more than an opinion; `unknown_oracle` is a timeout or a give-up. A solver answering sat whose model the checker REJECTS is a STOP-THE-LINE disagreement reported in its own block with severity STOP_THE_LINE, never `rejected` as though the ontology were at fault and never `model_checked`. The OWL-level reading carries its own word, `not_entailed_under_unproved_translation`, because it rides on OwlLean.adequacy in a sibling project AND on the Rust-to-Lean correspondence that is PINNED BY TESTS AND NOT PROVED. Needs z3 or mace4 on PATH and `oo-folmodel` built (cd lean && lake build); the absence of either is reported loudly in `skipped` and never worked around. See docs/decisions/0006.")]+ #[tool(name = "onto_fol_model", description = "Find a finite model of the loaded ontology and CHECK IT, so the answer names what it rests on. This is the SAT/SMT family, and it is deliberately not symmetric. A REFUTATION cannot be replayed in core Lean, so a solver saying `unsat` is an ORACLE OPINION for ever (decision 0005). A MODEL is a finite object, checking a formula against it is decidable, and `lean/Fol/` holds a checker whose soundness is machine-checked: `Fol.satisfiable_of_check` turns an accepted structure into satisfiability of exactly the formulas it was checked against, and `Fol.not_entails_of_check` turns a checked model of the NEGATED goal into a machine-checked NON-ENTAILMENT, which is the one sentence no theorem prover can produce. The pipeline exports SMT-LIB 2 (for Z3) or LADR (for Mace4) and `problem.tsv` from ONE representation, climbs a cardinality ladder, reads the structure back, and runs `oo-folmodel`. FIVE FIELDS THAT ARE NEVER COLLAPSED: `solver_verdict` (sat/unsat/unknown, what the oracle said), `encoding` (unbounded or finite(k), what it was asked), `checker_exit` (null means the checker never ran), `verdict`, and `owl_reading`. The verdict is one of FIVE WORDS: `model_checked` is the ONLY certified one and requires checker_exit 0; `satisfiable_oracle` is sat with nothing checked; `no_model_up_to_size_k` is an exhausted BOUNDED search and IS NOT UNSATISFIABILITY (a theory with no model of size k can have one of size k+1, and SHIQ has no finite model property at all, so a satisfiable ontology can have only infinite models and will never receive a certificate here); `unsatisfiable_oracle` may be produced ONLY by a run with no cardinality constraint of any kind and can never be more than an opinion; `unknown_oracle` is a timeout or a give-up. A solver answering sat whose model the checker REJECTS is a STOP-THE-LINE disagreement reported in its own block with severity STOP_THE_LINE, never `rejected` as though the ontology were at fault and never `model_checked`. The OWL-level reading carries its own word, `not_entailed_under_unproved_translation`, because it rides on OwlLean.adequacy in a sibling project AND on the Rust-to-Lean correspondence that is PINNED BY TESTS AND NOT PROVED. Needs z3 or mace4 on PATH and `oo-folmodel` built (cd lean && lake build); the absence of either is reported loudly in `skipped` and never worked around. A construct the reader cannot translate makes the export WEAKER than the ontology, and a model of a weaker theory is not a model of the ontology: `exports_a_weaker_axiom_set` and `constructs_not_exported` name every dropped construct, `owl_reading` is withheld, and `transfers_to_the_ontology` is false for `model_checked` and `satisfiable_oracle` (the unsatisfiable-direction verdicts still carry over). See docs/decisions/0006.")]async fn onto_fol_model(&self, Parameters(input): Parameters<OntoFolModelInput>) -> String {use crate::fol_solve::{SolveOptions, Solver, solve_export};let solver = match Solver::parse(input.solver.as_deref().unwrap_or("z3")) {diff --git a/tests/dl_consistency_differential_test.rs b/tests/dl_consistency_differential_test.rsindex b1b9b679..93d75716 100644--- a/tests/dl_consistency_differential_test.rs+++ b/tests/dl_consistency_differential_test.rs@@ -153,3 +153,117 @@ fn an_inconsistent_ontology_never_gets_a_checked_model() {);}}++/// Inconsistent ontologies whose inconsistency rests on a construct the FOL+/// reader cannot translate. Each is the smallest witness from the 29 Sep 2026+/// reproduction: the export is a strictly weaker theory, and it HAS a model.+fn out_of_fragment_inconsistent_cases() -> Vec<(&'static str, &'static str)> {+ vec![+ ("owl:AllDisjointProperties", "ex:p a owl:ObjectProperty . ex:q a owl:ObjectProperty . \+ [] a owl:AllDisjointProperties ; owl:members ( ex:p ex:q ) . \+ ex:a ex:p ex:b ; ex:q ex:b ."),+ ("owl:NegativePropertyAssertion", "ex:p a owl:ObjectProperty . ex:a ex:p ex:b . \+ [] a owl:NegativePropertyAssertion ; \+ owl:sourceIndividual ex:a ; owl:assertionProperty ex:p ; \+ owl:targetIndividual ex:b ."),+ ("a functional datatype property with two values", "ex:d a owl:DatatypeProperty , \+ owl:FunctionalProperty . ex:a ex:d 1 , 2 ."),+ ]+}++/// The report is about the EXPORT, and must say when that is not the ontology.+///+/// Before this, `solve_export` never read the reader's drop ledger. Each case+/// below came back `model_checked`, with `not_asked: []` and no flag, for a+/// graph with no models at all: a certified verdict about a different problem.+/// The verdict word may stay, because it is a true statement about the problem+/// the checker was handed. What may not happen is a report that lets it be+/// read as a statement about the ontology.+#[test]+fn a_model_of_a_weaker_export_is_never_reported_as_a_model_of_the_ontology() {+ if skip() {+ return;+ }+ for (name, body) in out_of_fragment_inconsistent_cases() {+ let g = store(body);+ let dir = std::env::temp_dir().join(format!(+ "oo-dl-weaker-{}-{}",+ std::process::id(),+ name.len()+ ));+ let _ = std::fs::remove_dir_all(&dir);+ std::fs::create_dir_all(&dir).expect("scratch dir");+ let opts = SolveOptions {+ solver: Solver::Z3,+ max_domain: 4,+ timeout_secs: 20,+ unbounded_probe: false,+ checker: None,+ };+ let report = solve_export(&g, &dir, &opts, None, 0).expect("solve_export");+ let _ = std::fs::remove_dir_all(&dir);+ let v: serde_json::Value = serde_json::from_str(&report).expect("report is JSON");+ let o = &v["ontology"];+ assert_eq!(+ v["exports_a_weaker_axiom_set"], true,+ "{name}: the reader dropped the construct the inconsistency rests on, and the \+ report does not say so: {v}"+ );+ assert!(+ v["constructs_not_exported"]+ .as_array()+ .is_some_and(|a| !a.is_empty()),+ "{name}: the flag without the list names nothing a reader can act on: {v}"+ );+ // The case is only a test of the report if the weaker export really+ // does have a model. If a later change makes the reader translate the+ // construct, the verdict stops being model_checked and this case has+ // to be replaced rather than silently passing.+ assert_eq!(+ o["verdict"], "model_checked",+ "{name}: expected the weaker export to have a checked model. If the construct is \+ now translated, move this case to inconsistent_cases(): {v}"+ );+ assert_eq!(+ o["transfers_to_the_ontology"], false,+ "{name}: a model of a weaker theory was reported as carrying over to an \+ ontology that has no model: {v}"+ );+ assert_eq!(+ o["owl_reading"],+ serde_json::Value::Null,+ "{name}: an OWL-level reading was minted over a weaker export: {v}"+ );+ }+}++/// The control: in the fragment, nothing is dropped and a checked model does+/// transfer. Without this, `transfers_to_the_ontology: false` everywhere would+/// pass the test above.+#[test]+fn a_model_of_a_faithful_export_transfers() {+ if skip() {+ return;+ }+ let (name, body) = consistent_cases()[1];+ let g = store(body);+ let dir = std::env::temp_dir().join(format!("oo-dl-faithful-{}", std::process::id()));+ let _ = std::fs::remove_dir_all(&dir);+ std::fs::create_dir_all(&dir).expect("scratch dir");+ let opts = SolveOptions {+ solver: Solver::Z3,+ max_domain: 4,+ timeout_secs: 20,+ unbounded_probe: false,+ checker: None,+ };+ let report = solve_export(&g, &dir, &opts, None, 0).expect("solve_export");+ let _ = std::fs::remove_dir_all(&dir);+ let v: serde_json::Value = serde_json::from_str(&report).expect("report is JSON");+ assert_eq!(v["exports_a_weaker_axiom_set"], false, "{name}: {v}");+ assert_eq!(v["ontology"]["verdict"], "model_checked", "{name}: {v}");+ assert_eq!(+ v["ontology"]["transfers_to_the_ontology"], true,+ "{name}: {v}"+ );+}