Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.
Rust ai-nativedescription-logicsformal-verificationinfrastructure-as-codeisabelle-languageknowledge-graphleanlean4mcpmcp-serverontologyowlowl2reasoningrustshaclsparqltheorem-proving
Wire oo-fores into onto_fol_prove (CNF first) and amend decision 0005 to match (#227)
Fabio Rovai5caafd3a86b3
diff --git a/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md b/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.mdindex bcab5af8..ea281629 100644--- a/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md+++ b/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md@@ -6,6 +6,11 @@`OwlLean/Translation.lean` in the sibling `owl-lean` project, whose `OwlLean.adequacy` ismachine-checked with axioms `propext`, `Classical.choice`, `Quot.sound` · **the Rust-to-Leancorrespondence is pinned by tests and is NOT proved**+- **Amended**: 2026-09-20 · the resolution half of the missing calculus now exists in core Lean+ (`lean/Fo`, `Fo.unsat_of_check`), a prover's derivation is translated into it+ (`tstp::to_fo_certificate`), and `fol --format cnf` keeps the prover from clausifying, so on+ the clausal fragment a refutation is CHECKED BY A THEOREM. Outside that fragment, and for+ equality, everything below still holds. See the second addendum.- **Written**: 2026-09-14- **Related**: decision 0002 (an inference carries a certificate), which this decision is theboundary of; `tools/shacl_differential.py`, whose treatment of pyshacl this copies exactly@@ -369,3 +374,137 @@ is visible rather than absorbed.- **The variant check is backtracking with a budget.** A clause larger than the budget is reportedas unchecked with that reason, which is honest and is also a place a forger could aim at. Thebudget is 200,000 pairings and no real step has come near it.++## Addendum, 20 September 2026 · The resolution half is a theorem, and the clausal fragment is certified end to end++The problem statement above says a superposition refutation cannot be certified because checking+one "needs a verified first-order calculus with unification, which does not exist in core Lean."+Half of that sentence was avoidable, and avoiding it is what changed.++### What was avoidable++A checker does not need an algorithm that FINDS a most general unifier. It needs to confirm that+a substitution it was HANDED makes two literals complementary, and that is an equality on terms.+The prover searches; the certificate carries the substitution it used; the checker applies it and+compares. Resolving on a unifier that is not most general is still sound, it merely proves less,+and proving less is the prover's problem. So there is no `mgu` in `lean/Fo` and no theorem about+one.++`Fo.unsat_of_check`: a refutation the checker accepts is a proof that the clause set it was handed+has no model, over any carrier, finite or infinite. Axioms `propext`, `Classical.choice`,+`Quot.sound`; no `sorry`; core Lean, no Mathlib, like everything else in `lean/`. Three choices in+it are load-bearing. Two substitutions per step, one per parent, which removes standardising apart+entirely. Covering rather than deletion for the remainder, because soundness needs no more and+exact deletion put list arithmetic in every proof. And a domain element carried in the structure,+because over an EMPTY carrier there are no environments, the empty clause is vacuously satisfied,+and every refutation would prove nothing.++`oo-fores REFUTATION.cert` exits 0, 1, 2 on the usual convention and exit 1 names no theorem.+`Fo/Demo.lean` runs the textbook refutation of `∀x. P(x) → Q(x)`, `P(a)`, `¬Q(a)`, the smallest one+that actually needs unification, six forgeries that each fail for a different reason, and+`no_refutation_of_a_satisfiable_set`, a theorem over EVERY refutation rather than an example.++### A real prover's derivation is now checked by it++`tstp::to_fo_certificate` turns the part of a TSTP derivation that IS resolution into that+certificate. The rule NAME is a hint and the witness is the check: E calls every inference `spm`,+including ones that are plain binary resolution, and a translator keyed on names translated+nothing from E; every two-parent step is offered to the resolvent witness instead, and a step is+resolution exactly when a witness exists. What does not translate is named with its rule and+counted, never skipped, and a certificate that does not reach the empty clause is refused.++Measured, the day it was written, with Vampire 5.1.0 and E 3.2.5:++| input | translated | `oo-fores` |+|---|---|---|+| Vampire, pure CNF, modus ponens | 2 of 2 | exit 0 |+| Vampire, pure CNF, four-link chain | 4 of 4 | exit 0 |+| Vampire, the same content as FOF | 0 of 8 (3 `cnf_transformation`, `ennf_transformation`, `flattening`, `negated_conjecture`) | refused |+| Vampire, with equality | 0 of 2 (`definition_unfolding`) | refused |+| E, pure CNF, four-link chain | 2 of 3 | refused |++E's last step is `cn(rw(spm(c_0_13, c_0_14), c_0_15))`, three inferences in one node with NO+intermediate formulas printed. That is the `Parent::Inline` case the first addendum already calls+unreconstructable; it is a limit of E's output and not of the translation.++### Clausification was the gap, and the fix is not to clausify better++A prover handed `fof` clausifies before it resolves, and clausification is not resolution. The+FOF row above is the whole problem in one line. The fix is to stop making the prover do it, and to+say exactly when that is free. Measured over three exported ontologies, 1,905 formulas, an+existential occurs in exactly two shapes and they are nothing alike:++- `? [X0] : thing(X0)`, the non-empty-domain axiom: one per export, CLOSED, witnessed by a single+ constant.+- `someValuesFrom` in SUPERCLASS position, `A ⊑ ∃r.B`: 126 of 426 formulas in an OWL DL ontology+ (pizza), and ZERO in both OWL 2 RL ones (1,422 of 1,423 formulas in `ies-building-extension`+ have no existential at all).++That zero is not luck: OWL 2 RL forbids an existential restriction in superclass position, because+it is not clausal. And a third shape only the refutation revealed: a NEGATED universal conjecture+is itself a closed existential, `¬∀X.(A ⇒ B)` being `∃X.(A ∧ ¬B)`; it is the one `skolemize` in+Vampire's FOF proof.++So the line is **constant versus function**. `fol --format cnf` strips a closed existential prefix+with one fresh constant per variable, `$domain_witness` and `$sk_<goal>_X<n>`, and the argument is+one sentence: any model interprets the constant as the witness. An existential UNDER a universal+needs a function of that variable, nothing here proves that Skolemisation, and the export REFUSES+it by name rather than clausifying it:++```+this ontology is outside the clausal fragment: 1 of its 1424 formulas need a Skolem FUNCTION+(owl_2_subClassOf), an existential under a universal. ... Export as `tptp` instead; a prover's+refutation of it is an oracle opinion, and `cnf` will not pretend otherwise.+```++On the fragment, the chain closes. `ies-building-extension.ttl`, 1,423 axioms; goal+`Building ⊑ ies:Entity`, two subclass steps deep, so derived and not asserted; `--format cnf`;+Vampire, 5 steps, all resolution; `to_fo_certificate`, 5 of 5; `oo-fores` exit 0,+`Fo.unsat_of_check`. The FOF encoding of the SAME goal is 18 steps, 13 of them clausification, and+translates nothing. Vampire says `Unsatisfiable` of the one and `Theorem` of the other, one verdict+in two SZS dialects. `tests/cnf_end_to_end_test.rs` is that run and its control.++### The rule, amended rather than reversed++Decision 5 above said an ATP verdict is an oracle opinion everywhere it appears. It now reads:++> An ATP verdict is an oracle opinion, **unless** the problem reached the prover as clauses+> (`onto_fol_prove` tries `to_cnf` first and falls back to FOF only outside the clausal fragment),+> every step of the derivation translated, and `oo-fores` exited 0. In that case, and only that+> case, the verdict is **`refutation_certified`**, resting on `Fo.unsat_of_check`, with the theorem's+> name read from the checker's own stdout. Otherwise the verdict is one of the first addendum's+> eight words, and the report's `certificate` field says which of five things stopped it:+> `outside_clausal_fragment` (naming the axiom), `steps_untranslated` (naming the rules),+> `does_not_reach_false`, `checker_absent`, or `checker_refused`.++`refutation_certified` is the ninth word and the only one that rests on a theorem.+`refutation_fully_replayed` is still the strongest word the Rust replayer can earn and is still not+a certificate. The ninth word is set only when a `Certified` token exists, and that token is minted+only by `CheckerRun::accepted_naming`, so it cannot be printed by any path that did not run the+checker and read `Fo.unsat_of_check` back from it. `unsatisfiable`, the word `oo-fores` and+`oo-lrat` print for themselves, joined `CHECKER_OWNED_WORDS`: Rust never says it. `problem_form` in+the run report says whether the prover got `cnf` or `fof`, and no user action selects between them.++### What remains an oracle, in full++- **Equality reasoning.** Paramodulation, demodulation, `definition_unfolding`. `Fo` has no+ equality; the next piece is to interpret the equality predicate as Lean's `=` on the carrier, at+ which point congruence is free and paramodulation's soundness is `subst`. `owl:sameAs` makes+ this unavoidable for real data.+- **Superclass existentials**, and with them every OWL DL ontology. The export refuses them; the+ FOF path and the first addendum's replayer are what they get.+- **Compressed derivations**, E's inline inferences. Prefer Vampire's format.+- **The TSTP parser and the translator**, which are Rust. What is proved is that the CERTIFICATE+ is a proof of the clause set in its leaves. That those leaves are the exported clauses is the+ first addendum's leaf-match check, still Rust and still pinned by tests rather than proved. A+ wrong translation cannot make an accepted certificate unsound; it can only fail to produce one.+- **The correspondence between `src/tptp.rs` and `OwlLean.adequacy`**, exactly as before.++### Measured, in the negative++The first version of the translator keyed on rule names and translated 0 of E's steps; the first+propositional checker consumed hints in order and failed 40 of 53 real picosat traces; the first+CNF export refused every goal because a negated universal is a closed existential. Each is a+test now. None of them could have made an accepted certificate unsound, since soundness is the+theorem's and not the pipeline's; each of them would have made the checker refuse correct proofs,+and a checker that refuses correct proofs teaches its users to stop running it.diff --git a/docs/decisions/README.md b/docs/decisions/README.mdindex f9ce9296..8831becd 100644--- a/docs/decisions/README.md+++ b/docs/decisions/README.md@@ -8,7 +8,7 @@ and whose benefit is not gets dropped the first time it is inconvenient.| [0001](0001-an-inference-is-not-an-assertion.md) | An inference is not an assertion | A materialised conclusion becoming the next run's premise, so the engine cites itself || [0002](0002-an-inference-carries-a-certificate.md) | An inference carries a certificate, and the certificate has a proof | Trusting the engine's report of its own work || [0003](0003-a-rule-is-data-and-an-assumption-is-not-a-fact.md) | A rule is data, and an assumption is not a fact | A user-supplied rule table earning the word reserved for the checked one |-| [0005](0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md) | A prover is an oracle, and a translation is a theorem | Reading a theorem prover's confident answer as evidence |+| [0005](0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md) | A prover is an oracle, and a translation is a theorem (amended 20 Sep 2026: certified on the clausal fragment) | Reading a theorem prover's confident answer as evidence || [0006](0006-a-model-is-a-certificate-and-a-refutation-is-not.md) | A model is a certificate, and a refutation is not | Treating an exhausted bounded search as unsatisfiability || [0007](0007-a-slice-preserves-a-conclusion-or-it-does-not.md) | A slice preserves a conclusion, or it does not | A retrieval slice quietly dropping the conclusion it was asked about || [0008](0008-a-binding-is-data-and-evidence-admits-one-reading.md) | A binding is data, and evidence admits one reading | A certificate format admitting two readings, so two checkers disagree |diff --git a/src/tstp.rs b/src/tstp.rsindex 7fc9c7f2..57a56381 100644--- a/src/tstp.rs+++ b/src/tstp.rs@@ -1383,6 +1383,13 @@ pub enum LeafMatch {Identical,AlphaEquivalent,AssociativityNormalised,+ /// Both sides are clauses and are the same clause: the same SET of+ /// literals up to a renaming of variables. Needed the day CNF input+ /// arrived: a prover echoes a `cnf(` axiom back in ITS literal order and+ /// under an explicit `![X]:`, so `~thing | ~A | B` comes back as+ /// `![X0]: (~A | ~thing | B)`. That is the same clause, and it is exact+ /// under clause semantics, not a leniency: a clause IS a set.+ ClauseVariant,}impl LeafMatch {@@ -1391,6 +1398,7 @@ impl LeafMatch {LeafMatch::Identical => "identical",LeafMatch::AlphaEquivalent => "alpha_equivalent",LeafMatch::AssociativityNormalised => "associativity_normalised",+ LeafMatch::ClauseVariant => "clause_variant",}}}@@ -1405,6 +1413,14 @@ fn leaf_match(proof: &Formula, problem: &Formula) -> Option<LeafMatch> {if alpha_eq(&proof.assoc_normalised(), &problem.assoc_normalised()) {return Some(LeafMatch::AssociativityNormalised);}+ // Last: both read as clauses, and are the same clause as a SET of+ // literals up to variable renaming. `variant` returns Err when its+ // backtracking budget runs out; that is "not established", not "no".+ if let (Ok(a), Ok(b)) = (clause_view(proof), clause_view(problem))+ && variant(&a, &b).unwrap_or(false)+ {+ return Some(LeafMatch::ClauseVariant);+ }None}@@ -1508,6 +1524,39 @@ pub struct Report {/// The one word. See [`verdict_means`].pub verdict: &'static str,+ /// What became of the attempt to have the derivation CHECKED BY A THEOREM+ /// rather than replayed. `None` when no attempt was made because the+ /// structure did not hold or the leaves were not ours; otherwise says+ /// either which theorem accepted it or exactly why none did.+ pub certificate: Option<CertificateOutcome>,+}++/// The outcome of offering a derivation to `oo-fores`.+///+/// `Certified` can only be built from a [`crate::verdict::Certified`] token,+/// and that token only exists after a checker exited zero naming the theorem,+/// so the word `refutation_certified` cannot be printed by any path that did+/// not run the checker.+#[derive(Clone, Debug, serde::Serialize)]+#[serde(tag = "status", rename_all = "snake_case")]+pub enum CertificateOutcome {+ /// `oo-fores` accepted. `theorem` is read from the token, which read it+ /// from the checker's stdout.+ Certified { theorem: &'static str, checker: String, steps: usize, certificate: String },+ /// The problem was exported as FOF because it is outside the clausal+ /// fragment, so the prover clausified and no resolution-only derivation+ /// exists to translate. `why` is `to_cnf`'s refusal, naming the axiom.+ OutsideClausalFragment { why: String },+ /// Some step of the derivation is not resolution or factoring. Named, with+ /// the rule, so a reader knows what kind of reasoning cost the certificate.+ StepsUntranslated { untranslated: Vec<(String, String)>, translated: usize },+ /// Every step translated but the chain does not reach the empty clause.+ DoesNotReachFalse { translated: usize },+ /// The certificate was written and `oo-fores` is not installed to read it.+ CheckerAbsent { certificate: String },+ /// `oo-fores` REFUSED a certificate this translator produced. Either a+ /// defect in the translation or in the derivation; both stop the line.+ CheckerRefused { exit: i32, output: String, certificate: String },}/// One sentence per word, so a report is readable without this file open@@ -1544,6 +1593,12 @@ pub fn verdict_means(v: &str) -> &'static str {from their premises. The unchecked rules are named \and counted. This is the normal outcome, because \clausification and AVATAR splitting are not checked",+ "refutation_certified" => "the derivation was translated into lean/Fo's certificate format \+ and oo-fores ACCEPTED it, discharging Fo.unsat_of_check: the \+ clause set in the derivation's leaves has no model, over any \+ carrier. The leaves are the formulas this engine emitted \+ (leaves_match_problem), so that is a refutation of OUR problem. \+ This is the only word in this vocabulary that rests on a theorem","refutation_fully_replayed" => "the structure holds and EVERY step was recomputed. This \is the strongest word here and it is still NOT a proof \of unsatisfiability: the calculus's soundness is not \@@ -1606,6 +1661,7 @@ pub fn check(problem_text: &str, proof_text: &str) -> Report {steps_unchecked: Vec::new(),steps_not_reconstructed: Vec::new(),verdict: "problem_unparsed",+ certificate: None,};// ── The problem ────────────────────────────────────────────────────────@@ -2512,10 +2568,22 @@ pub fn report_json(r: &Report) -> serde_json::Value {"steps_not_reconstructed": r.steps_not_reconstructed,"verdict": r.verdict,"verdict_means": verdict_means(r.verdict),- "checked_by": "src/tstp.rs, an UNVERIFIED Rust replayer. lean/ is not involved and no \- theorem is cited. A refutation remains an ORACLE OPINION (decision \- 0005); what this adds is that the opinion is now about a derivation \- somebody can read, over the problem this engine emitted",+ "certificate": r.certificate,+ "checked_by": if matches!(r.certificate, Some(CertificateOutcome::Certified { .. })) {+ "lean/Fo via oo-fores, whose acceptance discharges Fo.unsat_of_check (axioms propext, \+ Classical.choice, Quot.sound; no sorry). The replay in src/tstp.rs ran too and is \+ reported above, but the verdict rests on the theorem and not on the replay. What is \+ still Rust: the TSTP parser, the translation into the certificate, and the \+ leaf-match that ties the certificate's clauses to the problem this engine emitted; \+ a defect in any of them can fail to produce a certificate, never make an accepted \+ one unsound"+ } else {+ "src/tstp.rs, an UNVERIFIED Rust replayer. lean/ is not involved and no \+ theorem is cited. A refutation remains an ORACLE OPINION (decision \+ 0005); what this adds is that the opinion is now about a derivation \+ somebody can read, over the problem this engine emitted. The `certificate` field \+ says why no theorem was reached"+ },})}@@ -2625,8 +2693,20 @@ fn run_prover(prover: Prover, file: &Path, out: &Path, secs: u32) -> std::io::Re}/// One problem: write it, run the prover, check what comes back.+/// How the problem reached the prover, which decides whether a certificate+/// can even be attempted.+#[derive(Clone, Debug)]+pub enum ProblemForm {+ /// `to_cnf` succeeded: clauses only, so the prover's derivation is+ /// resolution end to end and can be translated.+ Cnf,+ /// `to_cnf` refused, naming the axiom; the prover got FOF and clausified.+ Fof { why: String },+}+fn prove_one(problem_text: &str,+ form: &ProblemForm,dir: &Path,stem: &str,opts: &ProveOptions,@@ -2636,7 +2716,75 @@ fn prove_one(std::fs::write(&p, problem_text)?;let out = dir.join(format!("{stem}.tstp"));let text = run_prover(opts.prover, &p, &out, opts.timeout_secs)?;- Ok(check(problem_text, &text))+ let mut r = check(problem_text, &text);+ r.certificate = certify(&r, form, &text, dir, stem);+ if matches!(r.certificate, Some(CertificateOutcome::Certified { .. })) {+ r.verdict = "refutation_certified";+ }+ Ok(r)+}++fn find_fores() -> Option<PathBuf> {+ // An explicit OO_FORES is an instruction. If it points nowhere the answer+ // is "absent", not "fall back to whatever else is lying around": the same+ // rule `shacl_verified::resolve_checker` applies, for the same reason. A+ // test that sets it to a missing path is asking to see the absent branch.+ if let Ok(p) = std::env::var("OO_FORES") {+ let p = PathBuf::from(p);+ return p.exists().then_some(p);+ }+ let built = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("lean/.lake/build/bin/oo-fores");+ if built.exists() {+ return Some(built);+ }+ std::env::split_paths(&std::env::var_os("PATH")?).map(|d| d.join("oo-fores")).find(|p| p.is_file())+}++/// Offer the derivation to `oo-fores`, and say exactly what happened.+///+/// Only when the replay found a well-founded refutation whose leaves are OUR+/// formulas. A certificate over some other clause set would be a proof of+/// something, and `leaves_match_problem` is what makes it a proof of the+/// problem this engine emitted; that check is Rust, and the first addendum to+/// decision 0005 says so.+fn certify(r: &Report, form: &ProblemForm, proof_text: &str, dir: &Path, stem: &str) -> Option<CertificateOutcome> {+ if !(r.root_is_false && r.leaves_match_problem && r.derivation_wellformed) {+ return None;+ }+ if let ProblemForm::Fof { why } = form {+ return Some(CertificateOutcome::OutsideClausalFragment { why: why.clone() });+ }+ let cert = match to_fo_certificate(proof_text) {+ Ok(c) => c,+ Err(e) => return Some(CertificateOutcome::CheckerRefused { exit: -1, output: e, certificate: String::new() }),+ };+ if !cert.untranslated.is_empty() {+ return Some(CertificateOutcome::StepsUntranslated { untranslated: cert.untranslated, translated: cert.translated });+ }+ if !cert.reaches_false {+ return Some(CertificateOutcome::DoesNotReachFalse { translated: cert.translated });+ }+ let path = dir.join(format!("{stem}.fo.cert"));+ let _ = std::fs::write(&path, &cert.text);+ let shown = path.display().to_string();+ let Some(bin) = find_fores() else {+ return Some(CertificateOutcome::CheckerAbsent { certificate: shown });+ };+ let mut cmd = Command::new(&bin);+ cmd.arg(&path);+ let run = match crate::verdict::CheckerRun::spawn(&crate::verdict::CheckerBinary::found_at(bin.clone()), cmd) {+ Ok(run) => run,+ Err(e) => return Some(CertificateOutcome::CheckerRefused { exit: -1, output: e.to_string(), certificate: shown }),+ };+ match run.accepted_naming(&["Fo.unsat_of_check"]) {+ Some(tok) => Some(CertificateOutcome::Certified {+ theorem: tok.theorem(),+ checker: bin.display().to_string(),+ steps: cert.translated,+ certificate: shown,+ }),+ None => Some(CertificateOutcome::CheckerRefused { exit: run.exit(), output: run.output(), certificate: shown }),+ }}/// Run the prover over the loaded ontology and, optionally, one problem per@@ -2668,8 +2816,20 @@ pub fn prove_export(let read = crate::tptp::read_graph(triples);std::fs::create_dir_all(dir)?;+ // CNF when the fragment allows it, FOF otherwise, and the report says+ // which. A prover handed FOF clausifies before it resolves, and that+ // derivation cannot be translated; handed clauses, it resolves from the+ // first step and can be. Measured on one derived goal: FOF 18 steps, 13+ // of them clausification, 0 translated; CNF 5 steps, 5 translated.+ let render = |p: &crate::tptp::FolProblem| -> (String, ProblemForm) {+ match p.to_cnf() {+ Ok(t) => (t, ProblemForm::Cnf),+ Err(e) => (p.to_tptp(), ProblemForm::Fof { why: e.to_string() }),+ }+ };let base = crate::tptp::FolProblem::build(&read.axioms, None)?;- let ontology = prove_one(&base.to_tptp(), &dir.join("ontology"), "ontology", opts)?;+ let (btext, bform) = render(&base);+ let ontology = prove_one(&btext, &bform, &dir.join("ontology"), "ontology", opts)?;let mut counts: BTreeMap<&'static str, usize> = BTreeMap::new();*counts.entry(ontology.verdict).or_default() += 1;@@ -2703,8 +2863,10 @@ pub fn prove_export(}};let gp = crate::tptp::FolProblem::build(&read.axioms, Some(&ax))?;+ let (gtext, gform) = render(&gp);let rep = prove_one(- &gp.to_tptp(),+ >ext,+ &gform,&dir.join(format!("goal_{i:05}")),&format!("goal_{i:05}"),opts,@@ -2719,6 +2881,7 @@ pub fn prove_export(let rejected = counts.get("derivation_rejected").copied().unwrap_or(0);let unreconstructed = counts.get("refutation_step_not_reconstructed").copied().unwrap_or(0);+ let certified = counts.get("refutation_certified").copied().unwrap_or(0);Ok(serde_json::json!({"prover": opts.prover.name(),@@ -2739,14 +2902,19 @@ pub fn prove_export(than what it was given; the second is either a defect in the \derivation or a gap in this checker and needs a human. The \command exits non-zero on either",- "not_certified": "NOTHING HERE IS CERTIFIED. A refutation is an ORACLE OPINION \- (decision 0005) and stays one: the calculus's soundness is not \- machine-checked anywhere in lean/, and this replayer is ordinary \- Rust. What the check adds is that the opinion is now about a \- derivation that was read, whose leaves are the formulas this engine \- emitted, and some of whose steps were recomputed. The MODEL \- direction is the one that can be certified; see decision 0006 and \- onto_fol_model",+ "problem_form": match &bform { ProblemForm::Cnf => "cnf", ProblemForm::Fof { .. } => "fof" },+ "certified": certified,+ "certified_means": if certified > 0 {+ "this many refutations were CHECKED BY A THEOREM: translated into lean/Fo's \+ certificate format and accepted by oo-fores (Fo.unsat_of_check). Every other \+ refutation in this run is an oracle opinion and its `certificate` field says why"+ } else {+ "NOTHING HERE IS CERTIFIED. A refutation is an ORACLE OPINION (decision 0005) \+ unless it was exported in the clausal fragment, every step translated, and \+ oo-fores exited 0. None did; each report's `certificate` field says which of \+ those failed. The MODEL direction is the other one that can be certified; see \+ decision 0006 and onto_fol_model"+ },}).to_string())}diff --git a/src/verdict.rs b/src/verdict.rsindex 58e6bf0f..07e4e5ed 100644--- a/src/verdict.rs+++ b/src/verdict.rs@@ -405,6 +405,10 @@ pub const CHECKER_OWNED_WORDS: &[&str] = &["entailed_under_supplied_rules",// `oo-refute`, lean/OOCert/Refute.lean."unsatisfiable_under_disjointness",+ // `oo-fores`, lean/FoMain.lean, and `oo-lrat`, lean/LratMain.lean. Both+ // print this for an accepted refutation. The Rust side says+ // `refutation_certified` and `refuted`, never this.+ "unsatisfiable",];/// The word a SUPPLIED Horn table can earn from `oo-horn`, for the one placediff --git a/tests/fol_prove_certified_test.rs b/tests/fol_prove_certified_test.rsnew file mode 100644index 00000000..0165de75--- /dev/null+++ b/tests/fol_prove_certified_test.rs@@ -0,0 +1,94 @@+//! `onto_fol_prove`'s certified path: CNF first, `oo-fores` after the replay,+//! and the word `refutation_certified` only from a minted token.++use std::path::PathBuf;+use std::sync::atomic::{AtomicU64, Ordering};++use open_ontologies::graph::GraphStore;+use open_ontologies::tstp::{prove_export, ProveOptions, Prover};++static NEXT: AtomicU64 = AtomicU64::new(0);++fn scratch() -> PathBuf {+ let d = std::env::temp_dir().join(format!("oo-prove-cert-{}-{}", std::process::id(), NEXT.fetch_add(1, Ordering::Relaxed)));+ std::fs::create_dir_all(&d).unwrap();+ d+}++fn tools() -> bool {+ let fores = std::env::var("OO_FORES").map(|p| PathBuf::from(p).exists()).unwrap_or(false)+ || PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("lean/.lake/build/bin/oo-fores").exists();+ fores && Prover::Vampire.available()+}++fn graph_of(path: &str) -> std::sync::Arc<GraphStore> {+ let ttl = std::fs::read_to_string(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(path)).expect("fixture");+ let g = std::sync::Arc::new(GraphStore::new());+ g.load_turtle(&ttl, None).expect("turtle parses");+ g+}++fn run(path: &str, goal: &str) -> serde_json::Value {+ let dir = scratch();+ let goals = dir.join("goals.tsv");+ std::fs::write(&goals, goal).unwrap();+ let opts = ProveOptions { prover: Prover::Vampire, timeout_secs: 20 };+ let out = prove_export(&graph_of(path), &dir, &opts, Some(goals.as_path()), 0).expect("prove runs");+ serde_json::from_str(&out).expect("json")+}++const RL: &str = "benchmark/generated/ies-building-extension.ttl";+const RL_GOAL: &str = "<http://example.org/ontology/ies-building#Building>\t<http://www.w3.org/2000/01/rdf-schema#subClassOf>\t<http://ies.data.gov.uk/ontology/ies4#Entity>\n";++#[test]+fn an_rl_goal_is_certified_by_the_theorem_with_no_user_action() {+ if !tools() { return; }+ let j = run(RL, RL_GOAL);+ assert_eq!(j["problem_form"], "cnf", "an RL ontology must go to the prover as clauses: {j}");+ let g = &j["goals"][0]["report"];+ assert_eq!(g["verdict"], "refutation_certified", "{g}");+ assert_eq!(g["certificate"]["status"], "certified", "{g}");+ assert_eq!(g["certificate"]["theorem"], "Fo.unsat_of_check", "the theorem name comes from the token: {g}");+ assert!(g["checked_by"].as_str().unwrap().contains("Fo.unsat_of_check"), "{g}");+ assert!(!g["checked_by"].as_str().unwrap().contains("lean/ is not involved"), "{g}");+ assert_eq!(j["certified"], 1, "{j}");+ assert!(j["certified_means"].as_str().unwrap().starts_with("this many"), "{j}");+}++const DL: &str = "benchmark/generated/pizza-ai.ttl";+const DL_GOAL: &str = "<http://www.co-ode.org/ontologies/pizza/pizza.owl#American>\t<http://www.w3.org/2000/01/rdf-schema#subClassOf>\t<http://www.co-ode.org/ontologies/pizza/pizza.owl#Pizza>\n";++#[test]+fn a_dl_ontology_stays_an_opinion_and_the_report_names_the_axiom_that_cost_it() {+ if !Prover::Vampire.available() { return; }+ let j = run(DL, DL_GOAL);+ assert_eq!(j["problem_form"], "fof", "a superclass existential forces FOF: {j}");+ let g = &j["goals"][0]["report"];+ assert_ne!(g["verdict"], "refutation_certified", "{g}");+ assert_eq!(g["certificate"]["status"], "outside_clausal_fragment", "{g}");+ let why = g["certificate"]["why"].as_str().unwrap_or("");+ assert!(why.contains("Skolem FUNCTION") && why.contains("owl_"), "the refusal names the axiom: {why}");+ assert!(g["checked_by"].as_str().unwrap().contains("ORACLE OPINION"), "{g}");+ assert_eq!(j["certified"], 0, "{j}");+ assert!(j["certified_means"].as_str().unwrap().starts_with("NOTHING HERE IS CERTIFIED"), "{j}");+}++#[test]+fn the_certified_word_cannot_appear_without_the_checker() {+ // Same RL goal, checker pointed at a path that does not exist. An explicit+ // OO_FORES is an instruction, not a hint, so there is no fallback: the+ // replay still runs, the certificate field says the checker was absent,+ // and the WORD is not printed. This test is the sole writer of OO_FORES in+ // its process.+ if !Prover::Vampire.available() { return; }+ let had = std::env::var("OO_FORES").ok();+ unsafe { std::env::set_var("OO_FORES", "/nonexistent/oo-fores") };+ let j = run(RL, RL_GOAL);+ match had { Some(v) => unsafe { std::env::set_var("OO_FORES", v) }, None => unsafe { std::env::remove_var("OO_FORES") } }+ let g = &j["goals"][0]["report"];+ assert_eq!(j["problem_form"], "cnf", "{j}");+ assert_ne!(g["verdict"], "refutation_certified", "no checker ran, so the word may not appear: {g}");+ assert_eq!(g["certificate"]["status"], "checker_absent", "{g}");+ assert!(g["checked_by"].as_str().unwrap().contains("ORACLE OPINION"), "{g}");+ assert_eq!(j["certified"], 0, "{j}");+}