Skip to content

Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.

Wire oo-fores into onto_fol_prove (CNF first) and amend decision 0005 to match (#227)

Fabio Rovai5caafd3a86b3
diff --git a/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md b/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md
index bcab5af8..ea281629 100644
--- a/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md
+++ b/docs/decisions/0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md
@@ -6,6 +6,11 @@
`OwlLean/Translation.lean` in the sibling `owl-lean` project, whose `OwlLean.adequacy` is
machine-checked with axioms `propext`, `Classical.choice`, `Quot.sound` · **the Rust-to-Lean
correspondence is pinned by tests and is NOT proved**
+- **Amended**: 2026-09-20 · the resolution half of the missing calculus now exists in core Lean
+ (`lean/Fo`, `Fo.unsat_of_check`), a prover's derivation is translated into it
+ (`tstp::to_fo_certificate`), and `fol --format cnf` keeps the prover from clausifying, so on
+ the clausal fragment a refutation is CHECKED BY A THEOREM. Outside that fragment, and for
+ equality, everything below still holds. See the second addendum.
- **Written**: 2026-09-14
- **Related**: decision 0002 (an inference carries a certificate), which this decision is the
boundary of; `tools/shacl_differential.py`, whose treatment of pyshacl this copies exactly
@@ -369,3 +374,137 @@ is visible rather than absorbed.
- **The variant check is backtracking with a budget.** A clause larger than the budget is reported
as unchecked with that reason, which is honest and is also a place a forger could aim at. The
budget is 200,000 pairings and no real step has come near it.
+
+## Addendum, 20 September 2026 · The resolution half is a theorem, and the clausal fragment is certified end to end
+
+The problem statement above says a superposition refutation cannot be certified because checking
+one "needs a verified first-order calculus with unification, which does not exist in core Lean."
+Half of that sentence was avoidable, and avoiding it is what changed.
+
+### What was avoidable
+
+A checker does not need an algorithm that FINDS a most general unifier. It needs to confirm that
+a substitution it was HANDED makes two literals complementary, and that is an equality on terms.
+The prover searches; the certificate carries the substitution it used; the checker applies it and
+compares. Resolving on a unifier that is not most general is still sound, it merely proves less,
+and proving less is the prover's problem. So there is no `mgu` in `lean/Fo` and no theorem about
+one.
+
+`Fo.unsat_of_check`: a refutation the checker accepts is a proof that the clause set it was handed
+has no model, over any carrier, finite or infinite. Axioms `propext`, `Classical.choice`,
+`Quot.sound`; no `sorry`; core Lean, no Mathlib, like everything else in `lean/`. Three choices in
+it are load-bearing. Two substitutions per step, one per parent, which removes standardising apart
+entirely. Covering rather than deletion for the remainder, because soundness needs no more and
+exact deletion put list arithmetic in every proof. And a domain element carried in the structure,
+because over an EMPTY carrier there are no environments, the empty clause is vacuously satisfied,
+and every refutation would prove nothing.
+
+`oo-fores REFUTATION.cert` exits 0, 1, 2 on the usual convention and exit 1 names no theorem.
+`Fo/Demo.lean` runs the textbook refutation of `∀x. P(x) → Q(x)`, `P(a)`, `¬Q(a)`, the smallest one
+that actually needs unification, six forgeries that each fail for a different reason, and
+`no_refutation_of_a_satisfiable_set`, a theorem over EVERY refutation rather than an example.
+
+### A real prover's derivation is now checked by it
+
+`tstp::to_fo_certificate` turns the part of a TSTP derivation that IS resolution into that
+certificate. The rule NAME is a hint and the witness is the check: E calls every inference `spm`,
+including ones that are plain binary resolution, and a translator keyed on names translated
+nothing from E; every two-parent step is offered to the resolvent witness instead, and a step is
+resolution exactly when a witness exists. What does not translate is named with its rule and
+counted, never skipped, and a certificate that does not reach the empty clause is refused.
+
+Measured, the day it was written, with Vampire 5.1.0 and E 3.2.5:
+
+| input | translated | `oo-fores` |
+|---|---|---|
+| Vampire, pure CNF, modus ponens | 2 of 2 | exit 0 |
+| Vampire, pure CNF, four-link chain | 4 of 4 | exit 0 |
+| Vampire, the same content as FOF | 0 of 8 (3 `cnf_transformation`, `ennf_transformation`, `flattening`, `negated_conjecture`) | refused |
+| Vampire, with equality | 0 of 2 (`definition_unfolding`) | refused |
+| E, pure CNF, four-link chain | 2 of 3 | refused |
+
+E's last step is `cn(rw(spm(c_0_13, c_0_14), c_0_15))`, three inferences in one node with NO
+intermediate formulas printed. That is the `Parent::Inline` case the first addendum already calls
+unreconstructable; it is a limit of E's output and not of the translation.
+
+### Clausification was the gap, and the fix is not to clausify better
+
+A prover handed `fof` clausifies before it resolves, and clausification is not resolution. The
+FOF row above is the whole problem in one line. The fix is to stop making the prover do it, and to
+say exactly when that is free. Measured over three exported ontologies, 1,905 formulas, an
+existential occurs in exactly two shapes and they are nothing alike:
+
+- `? [X0] : thing(X0)`, the non-empty-domain axiom: one per export, CLOSED, witnessed by a single
+ constant.
+- `someValuesFrom` in SUPERCLASS position, `A ⊑ ∃r.B`: 126 of 426 formulas in an OWL DL ontology
+ (pizza), and ZERO in both OWL 2 RL ones (1,422 of 1,423 formulas in `ies-building-extension`
+ have no existential at all).
+
+That zero is not luck: OWL 2 RL forbids an existential restriction in superclass position, because
+it is not clausal. And a third shape only the refutation revealed: a NEGATED universal conjecture
+is itself a closed existential, `¬∀X.(A ⇒ B)` being `∃X.(A ∧ ¬B)`; it is the one `skolemize` in
+Vampire's FOF proof.
+
+So the line is **constant versus function**. `fol --format cnf` strips a closed existential prefix
+with one fresh constant per variable, `$domain_witness` and `$sk_<goal>_X<n>`, and the argument is
+one sentence: any model interprets the constant as the witness. An existential UNDER a universal
+needs a function of that variable, nothing here proves that Skolemisation, and the export REFUSES
+it by name rather than clausifying it:
+
+```
+this ontology is outside the clausal fragment: 1 of its 1424 formulas need a Skolem FUNCTION
+(owl_2_subClassOf), an existential under a universal. ... Export as `tptp` instead; a prover's
+refutation of it is an oracle opinion, and `cnf` will not pretend otherwise.
+```
+
+On the fragment, the chain closes. `ies-building-extension.ttl`, 1,423 axioms; goal
+`Building ⊑ ies:Entity`, two subclass steps deep, so derived and not asserted; `--format cnf`;
+Vampire, 5 steps, all resolution; `to_fo_certificate`, 5 of 5; `oo-fores` exit 0,
+`Fo.unsat_of_check`. The FOF encoding of the SAME goal is 18 steps, 13 of them clausification, and
+translates nothing. Vampire says `Unsatisfiable` of the one and `Theorem` of the other, one verdict
+in two SZS dialects. `tests/cnf_end_to_end_test.rs` is that run and its control.
+
+### The rule, amended rather than reversed
+
+Decision 5 above said an ATP verdict is an oracle opinion everywhere it appears. It now reads:
+
+> An ATP verdict is an oracle opinion, **unless** the problem reached the prover as clauses
+> (`onto_fol_prove` tries `to_cnf` first and falls back to FOF only outside the clausal fragment),
+> every step of the derivation translated, and `oo-fores` exited 0. In that case, and only that
+> case, the verdict is **`refutation_certified`**, resting on `Fo.unsat_of_check`, with the theorem's
+> name read from the checker's own stdout. Otherwise the verdict is one of the first addendum's
+> eight words, and the report's `certificate` field says which of five things stopped it:
+> `outside_clausal_fragment` (naming the axiom), `steps_untranslated` (naming the rules),
+> `does_not_reach_false`, `checker_absent`, or `checker_refused`.
+
+`refutation_certified` is the ninth word and the only one that rests on a theorem.
+`refutation_fully_replayed` is still the strongest word the Rust replayer can earn and is still not
+a certificate. The ninth word is set only when a `Certified` token exists, and that token is minted
+only by `CheckerRun::accepted_naming`, so it cannot be printed by any path that did not run the
+checker and read `Fo.unsat_of_check` back from it. `unsatisfiable`, the word `oo-fores` and
+`oo-lrat` print for themselves, joined `CHECKER_OWNED_WORDS`: Rust never says it. `problem_form` in
+the run report says whether the prover got `cnf` or `fof`, and no user action selects between them.
+
+### What remains an oracle, in full
+
+- **Equality reasoning.** Paramodulation, demodulation, `definition_unfolding`. `Fo` has no
+ equality; the next piece is to interpret the equality predicate as Lean's `=` on the carrier, at
+ which point congruence is free and paramodulation's soundness is `subst`. `owl:sameAs` makes
+ this unavoidable for real data.
+- **Superclass existentials**, and with them every OWL DL ontology. The export refuses them; the
+ FOF path and the first addendum's replayer are what they get.
+- **Compressed derivations**, E's inline inferences. Prefer Vampire's format.
+- **The TSTP parser and the translator**, which are Rust. What is proved is that the CERTIFICATE
+ is a proof of the clause set in its leaves. That those leaves are the exported clauses is the
+ first addendum's leaf-match check, still Rust and still pinned by tests rather than proved. A
+ wrong translation cannot make an accepted certificate unsound; it can only fail to produce one.
+- **The correspondence between `src/tptp.rs` and `OwlLean.adequacy`**, exactly as before.
+
+### Measured, in the negative
+
+The first version of the translator keyed on rule names and translated 0 of E's steps; the first
+propositional checker consumed hints in order and failed 40 of 53 real picosat traces; the first
+CNF export refused every goal because a negated universal is a closed existential. Each is a
+test now. None of them could have made an accepted certificate unsound, since soundness is the
+theorem's and not the pipeline's; each of them would have made the checker refuse correct proofs,
+and a checker that refuses correct proofs teaches its users to stop running it.
diff --git a/docs/decisions/README.md b/docs/decisions/README.md
index f9ce9296..8831becd 100644
--- a/docs/decisions/README.md
+++ b/docs/decisions/README.md
@@ -8,7 +8,7 @@ and whose benefit is not gets dropped the first time it is inconvenient.
| [0001](0001-an-inference-is-not-an-assertion.md) | An inference is not an assertion | A materialised conclusion becoming the next run's premise, so the engine cites itself |
| [0002](0002-an-inference-carries-a-certificate.md) | An inference carries a certificate, and the certificate has a proof | Trusting the engine's report of its own work |
| [0003](0003-a-rule-is-data-and-an-assumption-is-not-a-fact.md) | A rule is data, and an assumption is not a fact | A user-supplied rule table earning the word reserved for the checked one |
-| [0005](0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md) | A prover is an oracle, and a translation is a theorem | Reading a theorem prover's confident answer as evidence |
+| [0005](0005-a-prover-is-an-oracle-and-a-translation-is-a-theorem.md) | A prover is an oracle, and a translation is a theorem (amended 20 Sep 2026: certified on the clausal fragment) | Reading a theorem prover's confident answer as evidence |
| [0006](0006-a-model-is-a-certificate-and-a-refutation-is-not.md) | A model is a certificate, and a refutation is not | Treating an exhausted bounded search as unsatisfiability |
| [0007](0007-a-slice-preserves-a-conclusion-or-it-does-not.md) | A slice preserves a conclusion, or it does not | A retrieval slice quietly dropping the conclusion it was asked about |
| [0008](0008-a-binding-is-data-and-evidence-admits-one-reading.md) | A binding is data, and evidence admits one reading | A certificate format admitting two readings, so two checkers disagree |
diff --git a/src/tstp.rs b/src/tstp.rs
index 7fc9c7f2..57a56381 100644
--- a/src/tstp.rs
+++ b/src/tstp.rs
@@ -1383,6 +1383,13 @@ pub enum LeafMatch {
Identical,
AlphaEquivalent,
AssociativityNormalised,
+ /// Both sides are clauses and are the same clause: the same SET of
+ /// literals up to a renaming of variables. Needed the day CNF input
+ /// arrived: a prover echoes a `cnf(` axiom back in ITS literal order and
+ /// under an explicit `![X]:`, so `~thing | ~A | B` comes back as
+ /// `![X0]: (~A | ~thing | B)`. That is the same clause, and it is exact
+ /// under clause semantics, not a leniency: a clause IS a set.
+ ClauseVariant,
}
impl LeafMatch {
@@ -1391,6 +1398,7 @@ impl LeafMatch {
LeafMatch::Identical => "identical",
LeafMatch::AlphaEquivalent => "alpha_equivalent",
LeafMatch::AssociativityNormalised => "associativity_normalised",
+ LeafMatch::ClauseVariant => "clause_variant",
}
}
}
@@ -1405,6 +1413,14 @@ fn leaf_match(proof: &Formula, problem: &Formula) -> Option<LeafMatch> {
if alpha_eq(&proof.assoc_normalised(), &problem.assoc_normalised()) {
return Some(LeafMatch::AssociativityNormalised);
}
+ // Last: both read as clauses, and are the same clause as a SET of
+ // literals up to variable renaming. `variant` returns Err when its
+ // backtracking budget runs out; that is "not established", not "no".
+ if let (Ok(a), Ok(b)) = (clause_view(proof), clause_view(problem))
+ && variant(&a, &b).unwrap_or(false)
+ {
+ return Some(LeafMatch::ClauseVariant);
+ }
None
}
@@ -1508,6 +1524,39 @@ pub struct Report {
/// The one word. See [`verdict_means`].
pub verdict: &'static str,
+ /// What became of the attempt to have the derivation CHECKED BY A THEOREM
+ /// rather than replayed. `None` when no attempt was made because the
+ /// structure did not hold or the leaves were not ours; otherwise says
+ /// either which theorem accepted it or exactly why none did.
+ pub certificate: Option<CertificateOutcome>,
+}
+
+/// The outcome of offering a derivation to `oo-fores`.
+///
+/// `Certified` can only be built from a [`crate::verdict::Certified`] token,
+/// and that token only exists after a checker exited zero naming the theorem,
+/// so the word `refutation_certified` cannot be printed by any path that did
+/// not run the checker.
+#[derive(Clone, Debug, serde::Serialize)]
+#[serde(tag = "status", rename_all = "snake_case")]
+pub enum CertificateOutcome {
+ /// `oo-fores` accepted. `theorem` is read from the token, which read it
+ /// from the checker's stdout.
+ Certified { theorem: &'static str, checker: String, steps: usize, certificate: String },
+ /// The problem was exported as FOF because it is outside the clausal
+ /// fragment, so the prover clausified and no resolution-only derivation
+ /// exists to translate. `why` is `to_cnf`'s refusal, naming the axiom.
+ OutsideClausalFragment { why: String },
+ /// Some step of the derivation is not resolution or factoring. Named, with
+ /// the rule, so a reader knows what kind of reasoning cost the certificate.
+ StepsUntranslated { untranslated: Vec<(String, String)>, translated: usize },
+ /// Every step translated but the chain does not reach the empty clause.
+ DoesNotReachFalse { translated: usize },
+ /// The certificate was written and `oo-fores` is not installed to read it.
+ CheckerAbsent { certificate: String },
+ /// `oo-fores` REFUSED a certificate this translator produced. Either a
+ /// defect in the translation or in the derivation; both stop the line.
+ CheckerRefused { exit: i32, output: String, certificate: String },
}
/// One sentence per word, so a report is readable without this file open
@@ -1544,6 +1593,12 @@ pub fn verdict_means(v: &str) -> &'static str {
from their premises. The unchecked rules are named \
and counted. This is the normal outcome, because \
clausification and AVATAR splitting are not checked",
+ "refutation_certified" => "the derivation was translated into lean/Fo's certificate format \
+ and oo-fores ACCEPTED it, discharging Fo.unsat_of_check: the \
+ clause set in the derivation's leaves has no model, over any \
+ carrier. The leaves are the formulas this engine emitted \
+ (leaves_match_problem), so that is a refutation of OUR problem. \
+ This is the only word in this vocabulary that rests on a theorem",
"refutation_fully_replayed" => "the structure holds and EVERY step was recomputed. This \
is the strongest word here and it is still NOT a proof \
of unsatisfiability: the calculus's soundness is not \
@@ -1606,6 +1661,7 @@ pub fn check(problem_text: &str, proof_text: &str) -> Report {
steps_unchecked: Vec::new(),
steps_not_reconstructed: Vec::new(),
verdict: "problem_unparsed",
+ certificate: None,
};
// ── The problem ────────────────────────────────────────────────────────
@@ -2512,10 +2568,22 @@ pub fn report_json(r: &Report) -> serde_json::Value {
"steps_not_reconstructed": r.steps_not_reconstructed,
"verdict": r.verdict,
"verdict_means": verdict_means(r.verdict),
- "checked_by": "src/tstp.rs, an UNVERIFIED Rust replayer. lean/ is not involved and no \
- theorem is cited. A refutation remains an ORACLE OPINION (decision \
- 0005); what this adds is that the opinion is now about a derivation \
- somebody can read, over the problem this engine emitted",
+ "certificate": r.certificate,
+ "checked_by": if matches!(r.certificate, Some(CertificateOutcome::Certified { .. })) {
+ "lean/Fo via oo-fores, whose acceptance discharges Fo.unsat_of_check (axioms propext, \
+ Classical.choice, Quot.sound; no sorry). The replay in src/tstp.rs ran too and is \
+ reported above, but the verdict rests on the theorem and not on the replay. What is \
+ still Rust: the TSTP parser, the translation into the certificate, and the \
+ leaf-match that ties the certificate's clauses to the problem this engine emitted; \
+ a defect in any of them can fail to produce a certificate, never make an accepted \
+ one unsound"
+ } else {
+ "src/tstp.rs, an UNVERIFIED Rust replayer. lean/ is not involved and no \
+ theorem is cited. A refutation remains an ORACLE OPINION (decision \
+ 0005); what this adds is that the opinion is now about a derivation \
+ somebody can read, over the problem this engine emitted. The `certificate` field \
+ says why no theorem was reached"
+ },
})
}
@@ -2625,8 +2693,20 @@ fn run_prover(prover: Prover, file: &Path, out: &Path, secs: u32) -> std::io::Re
}
/// One problem: write it, run the prover, check what comes back.
+/// How the problem reached the prover, which decides whether a certificate
+/// can even be attempted.
+#[derive(Clone, Debug)]
+pub enum ProblemForm {
+ /// `to_cnf` succeeded: clauses only, so the prover's derivation is
+ /// resolution end to end and can be translated.
+ Cnf,
+ /// `to_cnf` refused, naming the axiom; the prover got FOF and clausified.
+ Fof { why: String },
+}
+
fn prove_one(
problem_text: &str,
+ form: &ProblemForm,
dir: &Path,
stem: &str,
opts: &ProveOptions,
@@ -2636,7 +2716,75 @@ fn prove_one(
std::fs::write(&p, problem_text)?;
let out = dir.join(format!("{stem}.tstp"));
let text = run_prover(opts.prover, &p, &out, opts.timeout_secs)?;
- Ok(check(problem_text, &text))
+ let mut r = check(problem_text, &text);
+ r.certificate = certify(&r, form, &text, dir, stem);
+ if matches!(r.certificate, Some(CertificateOutcome::Certified { .. })) {
+ r.verdict = "refutation_certified";
+ }
+ Ok(r)
+}
+
+fn find_fores() -> Option<PathBuf> {
+ // An explicit OO_FORES is an instruction. If it points nowhere the answer
+ // is "absent", not "fall back to whatever else is lying around": the same
+ // rule `shacl_verified::resolve_checker` applies, for the same reason. A
+ // test that sets it to a missing path is asking to see the absent branch.
+ if let Ok(p) = std::env::var("OO_FORES") {
+ let p = PathBuf::from(p);
+ return p.exists().then_some(p);
+ }
+ let built = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("lean/.lake/build/bin/oo-fores");
+ if built.exists() {
+ return Some(built);
+ }
+ std::env::split_paths(&std::env::var_os("PATH")?).map(|d| d.join("oo-fores")).find(|p| p.is_file())
+}
+
+/// Offer the derivation to `oo-fores`, and say exactly what happened.
+///
+/// Only when the replay found a well-founded refutation whose leaves are OUR
+/// formulas. A certificate over some other clause set would be a proof of
+/// something, and `leaves_match_problem` is what makes it a proof of the
+/// problem this engine emitted; that check is Rust, and the first addendum to
+/// decision 0005 says so.
+fn certify(r: &Report, form: &ProblemForm, proof_text: &str, dir: &Path, stem: &str) -> Option<CertificateOutcome> {
+ if !(r.root_is_false && r.leaves_match_problem && r.derivation_wellformed) {
+ return None;
+ }
+ if let ProblemForm::Fof { why } = form {
+ return Some(CertificateOutcome::OutsideClausalFragment { why: why.clone() });
+ }
+ let cert = match to_fo_certificate(proof_text) {
+ Ok(c) => c,
+ Err(e) => return Some(CertificateOutcome::CheckerRefused { exit: -1, output: e, certificate: String::new() }),
+ };
+ if !cert.untranslated.is_empty() {
+ return Some(CertificateOutcome::StepsUntranslated { untranslated: cert.untranslated, translated: cert.translated });
+ }
+ if !cert.reaches_false {
+ return Some(CertificateOutcome::DoesNotReachFalse { translated: cert.translated });
+ }
+ let path = dir.join(format!("{stem}.fo.cert"));
+ let _ = std::fs::write(&path, &cert.text);
+ let shown = path.display().to_string();
+ let Some(bin) = find_fores() else {
+ return Some(CertificateOutcome::CheckerAbsent { certificate: shown });
+ };
+ let mut cmd = Command::new(&bin);
+ cmd.arg(&path);
+ let run = match crate::verdict::CheckerRun::spawn(&crate::verdict::CheckerBinary::found_at(bin.clone()), cmd) {
+ Ok(run) => run,
+ Err(e) => return Some(CertificateOutcome::CheckerRefused { exit: -1, output: e.to_string(), certificate: shown }),
+ };
+ match run.accepted_naming(&["Fo.unsat_of_check"]) {
+ Some(tok) => Some(CertificateOutcome::Certified {
+ theorem: tok.theorem(),
+ checker: bin.display().to_string(),
+ steps: cert.translated,
+ certificate: shown,
+ }),
+ None => Some(CertificateOutcome::CheckerRefused { exit: run.exit(), output: run.output(), certificate: shown }),
+ }
}
/// Run the prover over the loaded ontology and, optionally, one problem per
@@ -2668,8 +2816,20 @@ pub fn prove_export(
let read = crate::tptp::read_graph(triples);
std::fs::create_dir_all(dir)?;
+ // CNF when the fragment allows it, FOF otherwise, and the report says
+ // which. A prover handed FOF clausifies before it resolves, and that
+ // derivation cannot be translated; handed clauses, it resolves from the
+ // first step and can be. Measured on one derived goal: FOF 18 steps, 13
+ // of them clausification, 0 translated; CNF 5 steps, 5 translated.
+ let render = |p: &crate::tptp::FolProblem| -> (String, ProblemForm) {
+ match p.to_cnf() {
+ Ok(t) => (t, ProblemForm::Cnf),
+ Err(e) => (p.to_tptp(), ProblemForm::Fof { why: e.to_string() }),
+ }
+ };
let base = crate::tptp::FolProblem::build(&read.axioms, None)?;
- let ontology = prove_one(&base.to_tptp(), &dir.join("ontology"), "ontology", opts)?;
+ let (btext, bform) = render(&base);
+ let ontology = prove_one(&btext, &bform, &dir.join("ontology"), "ontology", opts)?;
let mut counts: BTreeMap<&'static str, usize> = BTreeMap::new();
*counts.entry(ontology.verdict).or_default() += 1;
@@ -2703,8 +2863,10 @@ pub fn prove_export(
}
};
let gp = crate::tptp::FolProblem::build(&read.axioms, Some(&ax))?;
+ let (gtext, gform) = render(&gp);
let rep = prove_one(
- &gp.to_tptp(),
+ &gtext,
+ &gform,
&dir.join(format!("goal_{i:05}")),
&format!("goal_{i:05}"),
opts,
@@ -2719,6 +2881,7 @@ pub fn prove_export(
let rejected = counts.get("derivation_rejected").copied().unwrap_or(0);
let unreconstructed = counts.get("refutation_step_not_reconstructed").copied().unwrap_or(0);
+ let certified = counts.get("refutation_certified").copied().unwrap_or(0);
Ok(serde_json::json!({
"prover": opts.prover.name(),
@@ -2739,14 +2902,19 @@ pub fn prove_export(
than what it was given; the second is either a defect in the \
derivation or a gap in this checker and needs a human. The \
command exits non-zero on either",
- "not_certified": "NOTHING HERE IS CERTIFIED. A refutation is an ORACLE OPINION \
- (decision 0005) and stays one: the calculus's soundness is not \
- machine-checked anywhere in lean/, and this replayer is ordinary \
- Rust. What the check adds is that the opinion is now about a \
- derivation that was read, whose leaves are the formulas this engine \
- emitted, and some of whose steps were recomputed. The MODEL \
- direction is the one that can be certified; see decision 0006 and \
- onto_fol_model",
+ "problem_form": match &bform { ProblemForm::Cnf => "cnf", ProblemForm::Fof { .. } => "fof" },
+ "certified": certified,
+ "certified_means": if certified > 0 {
+ "this many refutations were CHECKED BY A THEOREM: translated into lean/Fo's \
+ certificate format and accepted by oo-fores (Fo.unsat_of_check). Every other \
+ refutation in this run is an oracle opinion and its `certificate` field says why"
+ } else {
+ "NOTHING HERE IS CERTIFIED. A refutation is an ORACLE OPINION (decision 0005) \
+ unless it was exported in the clausal fragment, every step translated, and \
+ oo-fores exited 0. None did; each report's `certificate` field says which of \
+ those failed. The MODEL direction is the other one that can be certified; see \
+ decision 0006 and onto_fol_model"
+ },
})
.to_string())
}
diff --git a/src/verdict.rs b/src/verdict.rs
index 58e6bf0f..07e4e5ed 100644
--- a/src/verdict.rs
+++ b/src/verdict.rs
@@ -405,6 +405,10 @@ pub const CHECKER_OWNED_WORDS: &[&str] = &[
"entailed_under_supplied_rules",
// `oo-refute`, lean/OOCert/Refute.lean.
"unsatisfiable_under_disjointness",
+ // `oo-fores`, lean/FoMain.lean, and `oo-lrat`, lean/LratMain.lean. Both
+ // print this for an accepted refutation. The Rust side says
+ // `refutation_certified` and `refuted`, never this.
+ "unsatisfiable",
];
/// The word a SUPPLIED Horn table can earn from `oo-horn`, for the one place
diff --git a/tests/fol_prove_certified_test.rs b/tests/fol_prove_certified_test.rs
new file mode 100644
index 00000000..0165de75
--- /dev/null
+++ b/tests/fol_prove_certified_test.rs
@@ -0,0 +1,94 @@
+//! `onto_fol_prove`'s certified path: CNF first, `oo-fores` after the replay,
+//! and the word `refutation_certified` only from a minted token.
+
+use std::path::PathBuf;
+use std::sync::atomic::{AtomicU64, Ordering};
+
+use open_ontologies::graph::GraphStore;
+use open_ontologies::tstp::{prove_export, ProveOptions, Prover};
+
+static NEXT: AtomicU64 = AtomicU64::new(0);
+
+fn scratch() -> PathBuf {
+ let d = std::env::temp_dir().join(format!("oo-prove-cert-{}-{}", std::process::id(), NEXT.fetch_add(1, Ordering::Relaxed)));
+ std::fs::create_dir_all(&d).unwrap();
+ d
+}
+
+fn tools() -> bool {
+ let fores = std::env::var("OO_FORES").map(|p| PathBuf::from(p).exists()).unwrap_or(false)
+ || PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("lean/.lake/build/bin/oo-fores").exists();
+ fores && Prover::Vampire.available()
+}
+
+fn graph_of(path: &str) -> std::sync::Arc<GraphStore> {
+ let ttl = std::fs::read_to_string(PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(path)).expect("fixture");
+ let g = std::sync::Arc::new(GraphStore::new());
+ g.load_turtle(&ttl, None).expect("turtle parses");
+ g
+}
+
+fn run(path: &str, goal: &str) -> serde_json::Value {
+ let dir = scratch();
+ let goals = dir.join("goals.tsv");
+ std::fs::write(&goals, goal).unwrap();
+ let opts = ProveOptions { prover: Prover::Vampire, timeout_secs: 20 };
+ let out = prove_export(&graph_of(path), &dir, &opts, Some(goals.as_path()), 0).expect("prove runs");
+ serde_json::from_str(&out).expect("json")
+}
+
+const RL: &str = "benchmark/generated/ies-building-extension.ttl";
+const RL_GOAL: &str = "<http://example.org/ontology/ies-building#Building>\t<http://www.w3.org/2000/01/rdf-schema#subClassOf>\t<http://ies.data.gov.uk/ontology/ies4#Entity>\n";
+
+#[test]
+fn an_rl_goal_is_certified_by_the_theorem_with_no_user_action() {
+ if !tools() { return; }
+ let j = run(RL, RL_GOAL);
+ assert_eq!(j["problem_form"], "cnf", "an RL ontology must go to the prover as clauses: {j}");
+ let g = &j["goals"][0]["report"];
+ assert_eq!(g["verdict"], "refutation_certified", "{g}");
+ assert_eq!(g["certificate"]["status"], "certified", "{g}");
+ assert_eq!(g["certificate"]["theorem"], "Fo.unsat_of_check", "the theorem name comes from the token: {g}");
+ assert!(g["checked_by"].as_str().unwrap().contains("Fo.unsat_of_check"), "{g}");
+ assert!(!g["checked_by"].as_str().unwrap().contains("lean/ is not involved"), "{g}");
+ assert_eq!(j["certified"], 1, "{j}");
+ assert!(j["certified_means"].as_str().unwrap().starts_with("this many"), "{j}");
+}
+
+const DL: &str = "benchmark/generated/pizza-ai.ttl";
+const DL_GOAL: &str = "<http://www.co-ode.org/ontologies/pizza/pizza.owl#American>\t<http://www.w3.org/2000/01/rdf-schema#subClassOf>\t<http://www.co-ode.org/ontologies/pizza/pizza.owl#Pizza>\n";
+
+#[test]
+fn a_dl_ontology_stays_an_opinion_and_the_report_names_the_axiom_that_cost_it() {
+ if !Prover::Vampire.available() { return; }
+ let j = run(DL, DL_GOAL);
+ assert_eq!(j["problem_form"], "fof", "a superclass existential forces FOF: {j}");
+ let g = &j["goals"][0]["report"];
+ assert_ne!(g["verdict"], "refutation_certified", "{g}");
+ assert_eq!(g["certificate"]["status"], "outside_clausal_fragment", "{g}");
+ let why = g["certificate"]["why"].as_str().unwrap_or("");
+ assert!(why.contains("Skolem FUNCTION") && why.contains("owl_"), "the refusal names the axiom: {why}");
+ assert!(g["checked_by"].as_str().unwrap().contains("ORACLE OPINION"), "{g}");
+ assert_eq!(j["certified"], 0, "{j}");
+ assert!(j["certified_means"].as_str().unwrap().starts_with("NOTHING HERE IS CERTIFIED"), "{j}");
+}
+
+#[test]
+fn the_certified_word_cannot_appear_without_the_checker() {
+ // Same RL goal, checker pointed at a path that does not exist. An explicit
+ // OO_FORES is an instruction, not a hint, so there is no fallback: the
+ // replay still runs, the certificate field says the checker was absent,
+ // and the WORD is not printed. This test is the sole writer of OO_FORES in
+ // its process.
+ if !Prover::Vampire.available() { return; }
+ let had = std::env::var("OO_FORES").ok();
+ unsafe { std::env::set_var("OO_FORES", "/nonexistent/oo-fores") };
+ let j = run(RL, RL_GOAL);
+ match had { Some(v) => unsafe { std::env::set_var("OO_FORES", v) }, None => unsafe { std::env::remove_var("OO_FORES") } }
+ let g = &j["goals"][0]["report"];
+ assert_eq!(j["problem_form"], "cnf", "{j}");
+ assert_ne!(g["verdict"], "refutation_certified", "no checker ran, so the word may not appear: {g}");
+ assert_eq!(g["certificate"]["status"], "checker_absent", "{g}");
+ assert!(g["checked_by"].as_str().unwrap().contains("ORACLE OPINION"), "{g}");
+ assert_eq!(j["certified"], 0, "{j}");
+}