Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.
Rust ai-nativedescription-logicsformal-verificationinfrastructure-as-codeisabelle-languageknowledge-graphleanlean4mcpmcp-serverontologyowlowl2reasoningrustshaclsparqltheorem-proving
fix(load): set the base IRI from the file path; bench: LUBM harness and results
FABIOTESSb8ecd71ad8ec
diff --git a/benchmark/lubm/.gitignore b/benchmark/lubm/.gitignorenew file mode 100644index 00000000..b047365a--- /dev/null+++ b/benchmark/lubm/.gitignore@@ -0,0 +1,9 @@+# Generated data and the generator itself: reproduce with the steps in+# RESULTS.md rather than committing 115MB of derived files.+data1/+data10/+classes/+src/+uba.zip+*.owl+log.txtdiff --git a/benchmark/lubm/RESULTS.md b/benchmark/lubm/RESULTS.mdnew file mode 100644index 00000000..ad3d3f88--- /dev/null+++ b/benchmark/lubm/RESULTS.md@@ -0,0 +1,114 @@+# LUBM results++Measured on the official LUBM data generator (UBA 1.7, Lehigh University),+single node, Apple M-series laptop, release build, in-memory store.++These numbers exist so that claims about this engine can be checked rather+than believed, including the ones that do not flatter it. The reproduction+steps are at the bottom; disagreement is welcome and cheap to settle.++## Load and materialisation++| Dataset | Files | Triples | Load | Rate | OWL-RL | Inferred |+|---|---|---|---|---|---|---|+| LUBM(1) | 15 | 100,866 | 0.2 s | ~649k triples/s | 0.2 s | +37,942 |+| LUBM(10) | 189 | 1,273,246 | 2.0 s | ~625k triples/s | 2.9 s | +472,155 |++Loading is linear and fast, and materialisation of 1.27M triples in under+three seconds is respectable for a single node with no tuning. Nothing here+is a scale claim: LUBM(100) and beyond, and any concurrent workload, remain+unmeasured.++## The 14 queries, LUBM(1), after OWL-RL materialisation++`expected` is the published answer count for LUBM(1) under complete OWL+inference.++| Query | Expected | Ours | Complete | Note |+|---|---|---|---|---|+| Q1 | 4 | 4 | yes | |+| Q2 | 0 | 0 | yes | genuinely empty at LUBM(1), one university |+| Q3 | 6 | 6 | yes | |+| Q4 | 34 | 34 | yes | subclass and property inference |+| Q5 | 719 | 719 | yes | `memberOf` subproperty inference |+| Q6 | 7,790 | 5,916 | **no** | needs `Student` definition |+| Q7 | 67 | 59 | **no** | needs `Student` definition |+| Q8 | 7,790 | 5,916 | **no** | needs `Student` definition |+| Q9 | 208 | 103 | **no** | needs `Student` definition |+| Q10 | 4 | 0 | **no** | needs `Student` definition |+| Q11 | 224 | 224 | yes | transitive `subOrganizationOf` |+| Q12 | 15 | 0 | **no** | needs `Chair` definition |+| Q13 | 1 | 1 | yes | inverse property (`hasAlumnus`) |+| Q14 | 5,916 | 5,916 | yes | |++**8 of 14 complete. The 6 incomplete answers share one cause.**++LUBM defines two classes by equivalence to a restriction:++```+Student ≡ Person ⊓ ∃takesCourse.Course+Chair ≡ Person ⊓ ∃headOf.Department+```++Recognising a `GraduateStudent` as a `Student` therefore requires reasoning+over an existential restriction on the *left* of an equivalence, which is+OWL 2 DL and outside the OWL-RL profile the materialiser implements. Every+incomplete row above is that single missing capability, not six separate+gaps: Q6, Q7, Q8, Q9 and Q10 all need `Student`, and Q12 needs `Chair`.++This is the honest statement of where the reasoner stands. It is a bounded,+nameable gap rather than a vague one, and closing it is a known piece of+work: recognise `C ≡ D ⊓ ∃P.E` definitions and add the corresponding+instance-recognition rule during materialisation.++## Query latency++The per-query timings from the harness are not reported here because they+are dominated by a measurement artifact: the CLI is stateless, so the+harness reloads and re-materialises the dataset for every query, and each+measurement is therefore ~190 ms of setup around a query that returns in+well under a millisecond. The unreasoned run, where no materialisation+happens, shows queries completing in 0.0 to 2.4 ms.++Measuring query latency properly needs the HTTP server with a warm store,+and concurrent-client throughput needs a load generator. Both are worth+doing before any performance claim is made against a commercial store, and+neither has been done.++## What benchmarking already fixed++Running this suite immediately found a defect that no unit test had:+`load` set no base IRI, so any RDF/XML document using relative IRIs failed+to parse at all. LUBM's generated data is exactly that shape, and loaded+zero triples before the fix. A document's own location is its base per RFC+3986; it is now set from the file path.++That is the argument for benchmarking in one paragraph: the suite paid for+itself before producing a single performance number.++## Reproducing++```bash+# 1. generator (Java), from the LUBM project+curl -sLO https://swat.cse.lehigh.edu/projects/lubm/uba1.7.zip+unzip -q uba1.7.zip+curl -sLO https://swat.cse.lehigh.edu/onto/univ-bench.owl++# 2. data. The generator writes names containing a literal backslash on+# non-Windows platforms; move them into place afterwards.+java -cp classes edu.lehigh.swat.bench.uba.Generator \+ -univ 1 -index 0 -seed 0 \+ -onto http://swat.cse.lehigh.edu/onto/univ-bench.owl++# 3. run+python3 run_lubm.py --data data1 --reason --runs 3 --out results-lubm1.json+```++## Not yet measured++- LUBM(100) and LUBM(1000): where the single-node ceiling actually is+- Query latency with a warm store, and throughput under concurrency+- BSBM, SP2Bench, WatDiv: query optimisation under other shapes+- Any comparison against another store on the same hardware++Until those exist, this file says what it says and no more.diff --git a/benchmark/lubm/results-lubm1-reasoned.json b/benchmark/lubm/results-lubm1-reasoned.jsonnew file mode 100644index 00000000..59608f4b--- /dev/null+++ b/benchmark/lubm/results-lubm1-reasoned.json@@ -0,0 +1,96 @@+{+ "dataset": "data1",+ "files": 15,+ "triples": 100866,+ "load_seconds": 0.16,+ "reasoned": true,+ "reason_seconds": 0.21,+ "inferred_triples": 37942,+ "runs_per_query": 1,+ "queries": [+ {+ "query": "Q1",+ "results": 4,+ "ms": 198.3,+ "needs_inference": false+ },+ {+ "query": "Q2",+ "results": 0,+ "ms": 207.8,+ "needs_inference": true+ },+ {+ "query": "Q3",+ "results": 6,+ "ms": 186.2,+ "needs_inference": false+ },+ {+ "query": "Q4",+ "results": 34,+ "ms": 188.0,+ "needs_inference": true+ },+ {+ "query": "Q5",+ "results": 719,+ "ms": 195.6,+ "needs_inference": true+ },+ {+ "query": "Q6",+ "results": 5916,+ "ms": 197.8,+ "needs_inference": true+ },+ {+ "query": "Q7",+ "results": 59,+ "ms": 187.9,+ "needs_inference": true+ },+ {+ "query": "Q8",+ "results": 5916,+ "ms": 245.5,+ "needs_inference": true+ },+ {+ "query": "Q9",+ "results": 103,+ "ms": 210.3,+ "needs_inference": true+ },+ {+ "query": "Q10",+ "results": 0,+ "ms": 198.5,+ "needs_inference": true+ },+ {+ "query": "Q11",+ "results": 224,+ "ms": 194.2,+ "needs_inference": true+ },+ {+ "query": "Q12",+ "results": 0,+ "ms": 195.0,+ "needs_inference": true+ },+ {+ "query": "Q13",+ "results": 1,+ "ms": 194.8,+ "needs_inference": true+ },+ {+ "query": "Q14",+ "results": 5916,+ "ms": 197.9,+ "needs_inference": false+ }+ ]+}\ No newline at end of filediff --git a/benchmark/lubm/results-lubm1.json b/benchmark/lubm/results-lubm1.jsonnew file mode 100644index 00000000..b66223c7--- /dev/null+++ b/benchmark/lubm/results-lubm1.json@@ -0,0 +1,96 @@+{+ "dataset": "data1",+ "files": 15,+ "triples": 100866,+ "load_seconds": 0.16,+ "reasoned": false,+ "reason_seconds": null,+ "inferred_triples": null,+ "runs_per_query": 1,+ "queries": [+ {+ "query": "Q1",+ "results": 4,+ "ms": 0.0,+ "needs_inference": false+ },+ {+ "query": "Q2",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q3",+ "results": 6,+ "ms": 0.0,+ "needs_inference": false+ },+ {+ "query": "Q4",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q5",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q6",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q7",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q8",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q9",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q10",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q11",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q12",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q13",+ "results": 0,+ "ms": 0.0,+ "needs_inference": true+ },+ {+ "query": "Q14",+ "results": 5916,+ "ms": 2.4,+ "needs_inference": false+ }+ ]+}\ No newline at end of filediff --git a/benchmark/lubm/results-lubm10.json b/benchmark/lubm/results-lubm10.jsonnew file mode 100644index 00000000..36c07aad--- /dev/null+++ b/benchmark/lubm/results-lubm10.json@@ -0,0 +1,8 @@+{+ "dataset": "data10",+ "files": 189,+ "triples": 1273246,+ "load_seconds": 2.04,+ "reason_seconds": 2.9,+ "total_after_reasoning": 1745401+}\ No newline at end of filediff --git a/benchmark/lubm/run_lubm.py b/benchmark/lubm/run_lubm.pynew file mode 100644index 00000000..e181a3df--- /dev/null+++ b/benchmark/lubm/run_lubm.py@@ -0,0 +1,186 @@+#!/usr/bin/env python3+"""LUBM harness: load, reason, and run the 14 standard queries, with timings.++LUBM (Guo, Pan and Heflin) is the benchmark RDF stores publish against, so it+is the honest way to find out where this engine actually stands rather than+asserting a position. The point is not to win: it is to replace opinion with+a table, including the rows where we lose.++What is measured+ - load time and the resulting triple count, cold, from the generated files+ - OWL-RL materialisation time and the inferred triple count+ - each of the 14 LUBM queries, best of N runs, with result counts++Queries 4, 6, 8, 10, 11, 12, 13 need inference to return complete answers+(subclass and subproperty entailment, transitive subOrganizationOf,+inverse properties). Running the suite before and after materialisation+therefore also shows which answers the reasoner is responsible for, which is+more informative than a single number.++Usage:+ python3 run_lubm.py --data data1 --binary ../../target/release/open-ontologies+ python3 run_lubm.py --data data10 --reason --runs 3 --out results-10.json+"""++from __future__ import annotations++import argparse+import glob+import json+import pathlib+import subprocess+import time++UB = "http://swat.cse.lehigh.edu/onto/univ-bench.owl#"+RDF = "http://www.w3.org/1999/02/22-rdf-syntax-ns#"+RDFS = "http://www.w3.org/2000/01/rdf-schema#"++# The 14 standard LUBM queries. Kept in SPARQL rather than the original+# KIF/description-logic notation, which is how every published comparison+# states them.+QUERIES: list[tuple[str, str, bool]] = [+ ("Q1", f"""SELECT ?x WHERE {{+ ?x <{RDF}type> <{UB}GraduateStudent> .+ ?x <{UB}takesCourse> <http://www.Department0.University0.edu/GraduateCourse0> }}""", False),+ ("Q2", f"""SELECT ?x ?y ?z WHERE {{+ ?x <{RDF}type> <{UB}GraduateStudent> . ?y <{RDF}type> <{UB}University> .+ ?z <{RDF}type> <{UB}Department> . ?x <{UB}memberOf> ?z .+ ?z <{UB}subOrganizationOf> ?y . ?x <{UB}undergraduateDegreeFrom> ?y }}""", True),+ ("Q3", f"""SELECT ?x WHERE {{+ ?x <{RDF}type> <{UB}Publication> .+ ?x <{UB}publicationAuthor> <http://www.Department0.University0.edu/AssistantProfessor0> }}""", False),+ ("Q4", f"""SELECT ?x ?y1 ?y2 ?y3 WHERE {{+ ?x <{RDF}type> <{UB}Professor> .+ ?x <{UB}worksFor> <http://www.Department0.University0.edu> .+ ?x <{UB}name> ?y1 . ?x <{UB}emailAddress> ?y2 . ?x <{UB}telephone> ?y3 }}""", True),+ ("Q5", f"""SELECT ?x WHERE {{+ ?x <{RDF}type> <{UB}Person> .+ ?x <{UB}memberOf> <http://www.Department0.University0.edu> }}""", True),+ ("Q6", f"""SELECT ?x WHERE {{ ?x <{RDF}type> <{UB}Student> }}""", True),+ ("Q7", f"""SELECT ?x ?y WHERE {{+ ?x <{RDF}type> <{UB}Student> . ?y <{RDF}type> <{UB}Course> .+ ?x <{UB}takesCourse> ?y .+ <http://www.Department0.University0.edu/AssociateProfessor0> <{UB}teacherOf> ?y }}""", True),+ ("Q8", f"""SELECT ?x ?y ?z WHERE {{+ ?x <{RDF}type> <{UB}Student> . ?y <{RDF}type> <{UB}Department> .+ ?x <{UB}memberOf> ?y .+ ?y <{UB}subOrganizationOf> <http://www.University0.edu> .+ ?x <{UB}emailAddress> ?z }}""", True),+ ("Q9", f"""SELECT ?x ?y ?z WHERE {{+ ?x <{RDF}type> <{UB}Student> . ?y <{RDF}type> <{UB}Faculty> .+ ?z <{RDF}type> <{UB}Course> . ?x <{UB}advisor> ?y .+ ?y <{UB}teacherOf> ?z . ?x <{UB}takesCourse> ?z }}""", True),+ ("Q10", f"""SELECT ?x WHERE {{+ ?x <{RDF}type> <{UB}Student> .+ ?x <{UB}takesCourse> <http://www.Department0.University0.edu/GraduateCourse0> }}""", True),+ ("Q11", f"""SELECT ?x WHERE {{+ ?x <{RDF}type> <{UB}ResearchGroup> .+ ?x <{UB}subOrganizationOf> <http://www.University0.edu> }}""", True),+ ("Q12", f"""SELECT ?x ?y WHERE {{+ ?x <{RDF}type> <{UB}Chair> . ?y <{RDF}type> <{UB}Department> .+ ?x <{UB}worksFor> ?y .+ ?y <{UB}subOrganizationOf> <http://www.University0.edu> }}""", True),+ ("Q13", f"""SELECT ?x WHERE {{+ ?x <{RDF}type> <{UB}Person> .+ <http://www.University0.edu> <{UB}hasAlumnus> ?x }}""", True),+ ("Q14", f"""SELECT ?x WHERE {{ ?x <{RDF}type> <{UB}UndergraduateStudent> }}""", False),+]+++def batch(binary: str, commands: list[str]) -> list[dict]:+ """Run a batch and return the parsed JSON line per command."""+ proc = subprocess.run(+ [binary, "batch", "-"],+ input="\n".join(commands) + "\n",+ capture_output=True,+ text=True,+ )+ out = []+ for line in proc.stdout.splitlines():+ line = line.strip()+ if line.startswith("{"):+ try:+ out.append(json.loads(line))+ except json.JSONDecodeError:+ pass+ return out+++def main() -> int:+ ap = argparse.ArgumentParser()+ ap.add_argument("--data", default="data1", help="directory of generated .owl files")+ ap.add_argument("--binary", default="../../target/release/open-ontologies")+ ap.add_argument("--ontology", default="univ-bench.owl")+ ap.add_argument("--reason", action="store_true", help="materialise OWL-RL before querying")+ ap.add_argument("--runs", type=int, default=3, help="runs per query, best taken")+ ap.add_argument("--out", default=None, help="write results as JSON")+ args = ap.parse_args()++ files = sorted(glob.glob(str(pathlib.Path(args.data) / "*.owl")))+ if not files:+ print(f"no .owl files in {args.data}: generate with the UBA generator first")+ return 1++ print(f"LUBM harness: {len(files)} files from {args.data}")++ # Load, cold. The ontology first so its schema is present for reasoning.+ loads = ["clear", f"load {args.ontology}"] + [f"load {f}" for f in files]+ t0 = time.time()+ results = batch(args.binary, loads + ["stats"])+ load_seconds = time.time() - t0+ stats = next((r["result"] for r in results if r.get("command") == "stats"), {})+ triples = stats.get("triples", 0)+ print(f" load: {load_seconds:.1f}s for {triples} triples "+ f"({triples / load_seconds:,.0f} triples/s)" if load_seconds else "")++ reason_seconds = None+ inferred = None+ if args.reason:+ t0 = time.time()+ r = batch(args.binary, loads + ["reason owl-rl", "stats"])+ reason_seconds = time.time() - t0 - load_seconds+ after = next((x["result"] for x in r if x.get("command") == "stats"), {})+ inferred = after.get("triples", 0) - triples+ print(f" reason (owl-rl): {reason_seconds:.1f}s, +{inferred} inferred triples")+ loads = loads + ["reason owl-rl"]++ rows = []+ for name, query, needs_inference in QUERIES:+ flat = " ".join(query.split())+ best = None+ count = 0+ for _ in range(args.runs):+ t0 = time.time()+ r = batch(args.binary, loads + [f"query {json.dumps(flat)}"])+ elapsed = (time.time() - t0 - load_seconds) * 1000+ res = next((x["result"] for x in r if x.get("command") == "query"), {})+ count = len(res.get("results", []))+ best = elapsed if best is None else min(best, elapsed)+ rows.append({+ "query": name,+ "results": count,+ "ms": round(max(best, 0.0), 1),+ "needs_inference": needs_inference,+ })+ flag = " (needs inference)" if needs_inference else ""+ print(f" {name:>4}: {count:>7} results {max(best, 0.0):>8.1f} ms{flag}")++ summary = {+ "dataset": args.data,+ "files": len(files),+ "triples": triples,+ "load_seconds": round(load_seconds, 2),+ "reasoned": bool(args.reason),+ "reason_seconds": round(reason_seconds, 2) if reason_seconds else None,+ "inferred_triples": inferred,+ "runs_per_query": args.runs,+ "queries": rows,+ }+ if args.out:+ pathlib.Path(args.out).write_text(json.dumps(summary, indent=2))+ print(f"wrote {args.out}")+ return 0+++if __name__ == "__main__":+ raise SystemExit(main())diff --git a/src/communities.rs b/src/communities.rsindex 9c9d3d9d..562a1ea5 100644--- a/src/communities.rs+++ b/src/communities.rs@@ -160,8 +160,9 @@ impl Communities {}} else if si || oi {let (from, to) = if si { (s, o) } else { (o, s) };- if let Some(&other) = community_of.get(to.as_str()) {- if other != id && bridges.len() < top_members {+ let other = community_of.get(to.as_str()).copied();+ if let Some(other) = other.filter(|&c| c != id && bridges.len() < top_members) {+ {bridges.push(Bridge {from: short(from),predicate: local_name(p).to_string(),@@ -244,7 +245,7 @@ impl Communities {};let mut best = current;let mut best_gain = gain(current);- for (&c, _) in &links {+ for &c in links.keys() {let g = gain(c);if g > best_gain + 1e-12 || (g > best_gain - 1e-12 && c < best) {best = c;@@ -328,8 +329,7 @@ fn strip(value: &str) -> String {if v.starts_with('<') && v.ends_with('>') {return v[1..v.len() - 1].to_string();}- if v.starts_with('"') {- let body = &v[1..];+ if let Some(body) = v.strip_prefix('"') {for cut in ["\"^^", "\"@", "\""] {if let Some(i) = body.find(cut) {return body[..i].to_string();diff --git a/src/graph.rs b/src/graph.rsindex 3e055c71..1d0356cf 100644--- a/src/graph.rs+++ b/src/graph.rs@@ -144,8 +144,20 @@ impl GraphStore {let format = Self::detect_format_sniffed(path, &content);let store = self.store.lock().unwrap();let reader = Cursor::new(content.as_bytes());++ // A document's own location is its default base, per RFC 3986. Without+ // it, any file using relative IRIs fails to parse at all, which is+ // most published RDF/XML: LUBM's generated data would not load a+ // single triple before this.+ let base = std::fs::canonicalize(path)+ .ok()+ .and_then(|abs| abs.to_str().map(|s| format!("file://{s}")));// All or nothing: see load_turtle (issue #93).- let quads: Vec<_> = RdfParser::from_format(format)+ let mut parser = RdfParser::from_format(format);+ if let Some(p) = base.as_ref().and_then(|b| parser.clone().with_base_iri(b).ok()) {+ parser = p;+ }+ let quads: Vec<_> = parser.for_reader(reader).collect::<Result<_, _>>()?;for quad in &quads {diff --git a/src/server.rs b/src/server.rsindex 19908321..487291cd 100644--- a/src/server.rs+++ b/src/server.rs@@ -859,10 +859,12 @@ impl OpenOntologiesServer {chrono::Utc::now().to_rfc3339()));let mut seen = std::collections::HashSet::new();for line in ntriples.lines() {- if let Some(subject) = line.split_whitespace().next() {- if subject.starts_with('<') && seen.insert(subject.to_string()) {- extra.push_str(&format!("{subject} {PROV_DERIVED} <{source_iri}> .\n"));- }+ if let Some(subject) = line+ .split_whitespace()+ .next()+ .filter(|s| s.starts_with('<') && seen.insert(s.to_string()))+ {+ extra.push_str(&format!("{subject} {PROV_DERIVED} <{source_iri}> .\n"));}}prov_triples = extra.lines().count();diff --git a/src/support.rs b/src/support.rsindex 08a45c36..a944f12b 100644--- a/src/support.rs+++ b/src/support.rs@@ -62,8 +62,7 @@ fn strip(value: &str) -> String {if v.starts_with('<') && v.ends_with('>') {return v[1..v.len() - 1].to_string();}- if v.starts_with('"') {- let body = &v[1..];+ if let Some(body) = v.strip_prefix('"') {for cut in ["\"^^", "\"@", "\""] {if let Some(i) = body.find(cut) {return body[..i].to_string();