Skip to content

Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.

fix(load): set the base IRI from the file path; bench: LUBM harness and results

FABIOTESSb8ecd71ad8ec
diff --git a/benchmark/lubm/.gitignore b/benchmark/lubm/.gitignore
new file mode 100644
index 00000000..b047365a
--- /dev/null
+++ b/benchmark/lubm/.gitignore
@@ -0,0 +1,9 @@
+# Generated data and the generator itself: reproduce with the steps in
+# RESULTS.md rather than committing 115MB of derived files.
+data1/
+data10/
+classes/
+src/
+uba.zip
+*.owl
+log.txt
diff --git a/benchmark/lubm/RESULTS.md b/benchmark/lubm/RESULTS.md
new file mode 100644
index 00000000..ad3d3f88
--- /dev/null
+++ b/benchmark/lubm/RESULTS.md
@@ -0,0 +1,114 @@
+# LUBM results
+
+Measured on the official LUBM data generator (UBA 1.7, Lehigh University),
+single node, Apple M-series laptop, release build, in-memory store.
+
+These numbers exist so that claims about this engine can be checked rather
+than believed, including the ones that do not flatter it. The reproduction
+steps are at the bottom; disagreement is welcome and cheap to settle.
+
+## Load and materialisation
+
+| Dataset | Files | Triples | Load | Rate | OWL-RL | Inferred |
+|---|---|---|---|---|---|---|
+| LUBM(1) | 15 | 100,866 | 0.2 s | ~649k triples/s | 0.2 s | +37,942 |
+| LUBM(10) | 189 | 1,273,246 | 2.0 s | ~625k triples/s | 2.9 s | +472,155 |
+
+Loading is linear and fast, and materialisation of 1.27M triples in under
+three seconds is respectable for a single node with no tuning. Nothing here
+is a scale claim: LUBM(100) and beyond, and any concurrent workload, remain
+unmeasured.
+
+## The 14 queries, LUBM(1), after OWL-RL materialisation
+
+`expected` is the published answer count for LUBM(1) under complete OWL
+inference.
+
+| Query | Expected | Ours | Complete | Note |
+|---|---|---|---|---|
+| Q1 | 4 | 4 | yes | |
+| Q2 | 0 | 0 | yes | genuinely empty at LUBM(1), one university |
+| Q3 | 6 | 6 | yes | |
+| Q4 | 34 | 34 | yes | subclass and property inference |
+| Q5 | 719 | 719 | yes | `memberOf` subproperty inference |
+| Q6 | 7,790 | 5,916 | **no** | needs `Student` definition |
+| Q7 | 67 | 59 | **no** | needs `Student` definition |
+| Q8 | 7,790 | 5,916 | **no** | needs `Student` definition |
+| Q9 | 208 | 103 | **no** | needs `Student` definition |
+| Q10 | 4 | 0 | **no** | needs `Student` definition |
+| Q11 | 224 | 224 | yes | transitive `subOrganizationOf` |
+| Q12 | 15 | 0 | **no** | needs `Chair` definition |
+| Q13 | 1 | 1 | yes | inverse property (`hasAlumnus`) |
+| Q14 | 5,916 | 5,916 | yes | |
+
+**8 of 14 complete. The 6 incomplete answers share one cause.**
+
+LUBM defines two classes by equivalence to a restriction:
+
+```
+Student ≡ Person ⊓ ∃takesCourse.Course
+Chair ≡ Person ⊓ ∃headOf.Department
+```
+
+Recognising a `GraduateStudent` as a `Student` therefore requires reasoning
+over an existential restriction on the *left* of an equivalence, which is
+OWL 2 DL and outside the OWL-RL profile the materialiser implements. Every
+incomplete row above is that single missing capability, not six separate
+gaps: Q6, Q7, Q8, Q9 and Q10 all need `Student`, and Q12 needs `Chair`.
+
+This is the honest statement of where the reasoner stands. It is a bounded,
+nameable gap rather than a vague one, and closing it is a known piece of
+work: recognise `C ≡ D ⊓ ∃P.E` definitions and add the corresponding
+instance-recognition rule during materialisation.
+
+## Query latency
+
+The per-query timings from the harness are not reported here because they
+are dominated by a measurement artifact: the CLI is stateless, so the
+harness reloads and re-materialises the dataset for every query, and each
+measurement is therefore ~190 ms of setup around a query that returns in
+well under a millisecond. The unreasoned run, where no materialisation
+happens, shows queries completing in 0.0 to 2.4 ms.
+
+Measuring query latency properly needs the HTTP server with a warm store,
+and concurrent-client throughput needs a load generator. Both are worth
+doing before any performance claim is made against a commercial store, and
+neither has been done.
+
+## What benchmarking already fixed
+
+Running this suite immediately found a defect that no unit test had:
+`load` set no base IRI, so any RDF/XML document using relative IRIs failed
+to parse at all. LUBM's generated data is exactly that shape, and loaded
+zero triples before the fix. A document's own location is its base per RFC
+3986; it is now set from the file path.
+
+That is the argument for benchmarking in one paragraph: the suite paid for
+itself before producing a single performance number.
+
+## Reproducing
+
+```bash
+# 1. generator (Java), from the LUBM project
+curl -sLO https://swat.cse.lehigh.edu/projects/lubm/uba1.7.zip
+unzip -q uba1.7.zip
+curl -sLO https://swat.cse.lehigh.edu/onto/univ-bench.owl
+
+# 2. data. The generator writes names containing a literal backslash on
+# non-Windows platforms; move them into place afterwards.
+java -cp classes edu.lehigh.swat.bench.uba.Generator \
+ -univ 1 -index 0 -seed 0 \
+ -onto http://swat.cse.lehigh.edu/onto/univ-bench.owl
+
+# 3. run
+python3 run_lubm.py --data data1 --reason --runs 3 --out results-lubm1.json
+```
+
+## Not yet measured
+
+- LUBM(100) and LUBM(1000): where the single-node ceiling actually is
+- Query latency with a warm store, and throughput under concurrency
+- BSBM, SP2Bench, WatDiv: query optimisation under other shapes
+- Any comparison against another store on the same hardware
+
+Until those exist, this file says what it says and no more.
diff --git a/benchmark/lubm/results-lubm1-reasoned.json b/benchmark/lubm/results-lubm1-reasoned.json
new file mode 100644
index 00000000..59608f4b
--- /dev/null
+++ b/benchmark/lubm/results-lubm1-reasoned.json
@@ -0,0 +1,96 @@
+{
+ "dataset": "data1",
+ "files": 15,
+ "triples": 100866,
+ "load_seconds": 0.16,
+ "reasoned": true,
+ "reason_seconds": 0.21,
+ "inferred_triples": 37942,
+ "runs_per_query": 1,
+ "queries": [
+ {
+ "query": "Q1",
+ "results": 4,
+ "ms": 198.3,
+ "needs_inference": false
+ },
+ {
+ "query": "Q2",
+ "results": 0,
+ "ms": 207.8,
+ "needs_inference": true
+ },
+ {
+ "query": "Q3",
+ "results": 6,
+ "ms": 186.2,
+ "needs_inference": false
+ },
+ {
+ "query": "Q4",
+ "results": 34,
+ "ms": 188.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q5",
+ "results": 719,
+ "ms": 195.6,
+ "needs_inference": true
+ },
+ {
+ "query": "Q6",
+ "results": 5916,
+ "ms": 197.8,
+ "needs_inference": true
+ },
+ {
+ "query": "Q7",
+ "results": 59,
+ "ms": 187.9,
+ "needs_inference": true
+ },
+ {
+ "query": "Q8",
+ "results": 5916,
+ "ms": 245.5,
+ "needs_inference": true
+ },
+ {
+ "query": "Q9",
+ "results": 103,
+ "ms": 210.3,
+ "needs_inference": true
+ },
+ {
+ "query": "Q10",
+ "results": 0,
+ "ms": 198.5,
+ "needs_inference": true
+ },
+ {
+ "query": "Q11",
+ "results": 224,
+ "ms": 194.2,
+ "needs_inference": true
+ },
+ {
+ "query": "Q12",
+ "results": 0,
+ "ms": 195.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q13",
+ "results": 1,
+ "ms": 194.8,
+ "needs_inference": true
+ },
+ {
+ "query": "Q14",
+ "results": 5916,
+ "ms": 197.9,
+ "needs_inference": false
+ }
+ ]
+}
\ No newline at end of file
diff --git a/benchmark/lubm/results-lubm1.json b/benchmark/lubm/results-lubm1.json
new file mode 100644
index 00000000..b66223c7
--- /dev/null
+++ b/benchmark/lubm/results-lubm1.json
@@ -0,0 +1,96 @@
+{
+ "dataset": "data1",
+ "files": 15,
+ "triples": 100866,
+ "load_seconds": 0.16,
+ "reasoned": false,
+ "reason_seconds": null,
+ "inferred_triples": null,
+ "runs_per_query": 1,
+ "queries": [
+ {
+ "query": "Q1",
+ "results": 4,
+ "ms": 0.0,
+ "needs_inference": false
+ },
+ {
+ "query": "Q2",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q3",
+ "results": 6,
+ "ms": 0.0,
+ "needs_inference": false
+ },
+ {
+ "query": "Q4",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q5",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q6",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q7",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q8",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q9",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q10",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q11",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q12",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q13",
+ "results": 0,
+ "ms": 0.0,
+ "needs_inference": true
+ },
+ {
+ "query": "Q14",
+ "results": 5916,
+ "ms": 2.4,
+ "needs_inference": false
+ }
+ ]
+}
\ No newline at end of file
diff --git a/benchmark/lubm/results-lubm10.json b/benchmark/lubm/results-lubm10.json
new file mode 100644
index 00000000..36c07aad
--- /dev/null
+++ b/benchmark/lubm/results-lubm10.json
@@ -0,0 +1,8 @@
+{
+ "dataset": "data10",
+ "files": 189,
+ "triples": 1273246,
+ "load_seconds": 2.04,
+ "reason_seconds": 2.9,
+ "total_after_reasoning": 1745401
+}
\ No newline at end of file
diff --git a/benchmark/lubm/run_lubm.py b/benchmark/lubm/run_lubm.py
new file mode 100644
index 00000000..e181a3df
--- /dev/null
+++ b/benchmark/lubm/run_lubm.py
@@ -0,0 +1,186 @@
+#!/usr/bin/env python3
+"""LUBM harness: load, reason, and run the 14 standard queries, with timings.
+
+LUBM (Guo, Pan and Heflin) is the benchmark RDF stores publish against, so it
+is the honest way to find out where this engine actually stands rather than
+asserting a position. The point is not to win: it is to replace opinion with
+a table, including the rows where we lose.
+
+What is measured
+ - load time and the resulting triple count, cold, from the generated files
+ - OWL-RL materialisation time and the inferred triple count
+ - each of the 14 LUBM queries, best of N runs, with result counts
+
+Queries 4, 6, 8, 10, 11, 12, 13 need inference to return complete answers
+(subclass and subproperty entailment, transitive subOrganizationOf,
+inverse properties). Running the suite before and after materialisation
+therefore also shows which answers the reasoner is responsible for, which is
+more informative than a single number.
+
+Usage:
+ python3 run_lubm.py --data data1 --binary ../../target/release/open-ontologies
+ python3 run_lubm.py --data data10 --reason --runs 3 --out results-10.json
+"""
+
+from __future__ import annotations
+
+import argparse
+import glob
+import json
+import pathlib
+import subprocess
+import time
+
+UB = "http://swat.cse.lehigh.edu/onto/univ-bench.owl#"
+RDF = "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+RDFS = "http://www.w3.org/2000/01/rdf-schema#"
+
+# The 14 standard LUBM queries. Kept in SPARQL rather than the original
+# KIF/description-logic notation, which is how every published comparison
+# states them.
+QUERIES: list[tuple[str, str, bool]] = [
+ ("Q1", f"""SELECT ?x WHERE {{
+ ?x <{RDF}type> <{UB}GraduateStudent> .
+ ?x <{UB}takesCourse> <http://www.Department0.University0.edu/GraduateCourse0> }}""", False),
+ ("Q2", f"""SELECT ?x ?y ?z WHERE {{
+ ?x <{RDF}type> <{UB}GraduateStudent> . ?y <{RDF}type> <{UB}University> .
+ ?z <{RDF}type> <{UB}Department> . ?x <{UB}memberOf> ?z .
+ ?z <{UB}subOrganizationOf> ?y . ?x <{UB}undergraduateDegreeFrom> ?y }}""", True),
+ ("Q3", f"""SELECT ?x WHERE {{
+ ?x <{RDF}type> <{UB}Publication> .
+ ?x <{UB}publicationAuthor> <http://www.Department0.University0.edu/AssistantProfessor0> }}""", False),
+ ("Q4", f"""SELECT ?x ?y1 ?y2 ?y3 WHERE {{
+ ?x <{RDF}type> <{UB}Professor> .
+ ?x <{UB}worksFor> <http://www.Department0.University0.edu> .
+ ?x <{UB}name> ?y1 . ?x <{UB}emailAddress> ?y2 . ?x <{UB}telephone> ?y3 }}""", True),
+ ("Q5", f"""SELECT ?x WHERE {{
+ ?x <{RDF}type> <{UB}Person> .
+ ?x <{UB}memberOf> <http://www.Department0.University0.edu> }}""", True),
+ ("Q6", f"""SELECT ?x WHERE {{ ?x <{RDF}type> <{UB}Student> }}""", True),
+ ("Q7", f"""SELECT ?x ?y WHERE {{
+ ?x <{RDF}type> <{UB}Student> . ?y <{RDF}type> <{UB}Course> .
+ ?x <{UB}takesCourse> ?y .
+ <http://www.Department0.University0.edu/AssociateProfessor0> <{UB}teacherOf> ?y }}""", True),
+ ("Q8", f"""SELECT ?x ?y ?z WHERE {{
+ ?x <{RDF}type> <{UB}Student> . ?y <{RDF}type> <{UB}Department> .
+ ?x <{UB}memberOf> ?y .
+ ?y <{UB}subOrganizationOf> <http://www.University0.edu> .
+ ?x <{UB}emailAddress> ?z }}""", True),
+ ("Q9", f"""SELECT ?x ?y ?z WHERE {{
+ ?x <{RDF}type> <{UB}Student> . ?y <{RDF}type> <{UB}Faculty> .
+ ?z <{RDF}type> <{UB}Course> . ?x <{UB}advisor> ?y .
+ ?y <{UB}teacherOf> ?z . ?x <{UB}takesCourse> ?z }}""", True),
+ ("Q10", f"""SELECT ?x WHERE {{
+ ?x <{RDF}type> <{UB}Student> .
+ ?x <{UB}takesCourse> <http://www.Department0.University0.edu/GraduateCourse0> }}""", True),
+ ("Q11", f"""SELECT ?x WHERE {{
+ ?x <{RDF}type> <{UB}ResearchGroup> .
+ ?x <{UB}subOrganizationOf> <http://www.University0.edu> }}""", True),
+ ("Q12", f"""SELECT ?x ?y WHERE {{
+ ?x <{RDF}type> <{UB}Chair> . ?y <{RDF}type> <{UB}Department> .
+ ?x <{UB}worksFor> ?y .
+ ?y <{UB}subOrganizationOf> <http://www.University0.edu> }}""", True),
+ ("Q13", f"""SELECT ?x WHERE {{
+ ?x <{RDF}type> <{UB}Person> .
+ <http://www.University0.edu> <{UB}hasAlumnus> ?x }}""", True),
+ ("Q14", f"""SELECT ?x WHERE {{ ?x <{RDF}type> <{UB}UndergraduateStudent> }}""", False),
+]
+
+
+def batch(binary: str, commands: list[str]) -> list[dict]:
+ """Run a batch and return the parsed JSON line per command."""
+ proc = subprocess.run(
+ [binary, "batch", "-"],
+ input="\n".join(commands) + "\n",
+ capture_output=True,
+ text=True,
+ )
+ out = []
+ for line in proc.stdout.splitlines():
+ line = line.strip()
+ if line.startswith("{"):
+ try:
+ out.append(json.loads(line))
+ except json.JSONDecodeError:
+ pass
+ return out
+
+
+def main() -> int:
+ ap = argparse.ArgumentParser()
+ ap.add_argument("--data", default="data1", help="directory of generated .owl files")
+ ap.add_argument("--binary", default="../../target/release/open-ontologies")
+ ap.add_argument("--ontology", default="univ-bench.owl")
+ ap.add_argument("--reason", action="store_true", help="materialise OWL-RL before querying")
+ ap.add_argument("--runs", type=int, default=3, help="runs per query, best taken")
+ ap.add_argument("--out", default=None, help="write results as JSON")
+ args = ap.parse_args()
+
+ files = sorted(glob.glob(str(pathlib.Path(args.data) / "*.owl")))
+ if not files:
+ print(f"no .owl files in {args.data}: generate with the UBA generator first")
+ return 1
+
+ print(f"LUBM harness: {len(files)} files from {args.data}")
+
+ # Load, cold. The ontology first so its schema is present for reasoning.
+ loads = ["clear", f"load {args.ontology}"] + [f"load {f}" for f in files]
+ t0 = time.time()
+ results = batch(args.binary, loads + ["stats"])
+ load_seconds = time.time() - t0
+ stats = next((r["result"] for r in results if r.get("command") == "stats"), {})
+ triples = stats.get("triples", 0)
+ print(f" load: {load_seconds:.1f}s for {triples} triples "
+ f"({triples / load_seconds:,.0f} triples/s)" if load_seconds else "")
+
+ reason_seconds = None
+ inferred = None
+ if args.reason:
+ t0 = time.time()
+ r = batch(args.binary, loads + ["reason owl-rl", "stats"])
+ reason_seconds = time.time() - t0 - load_seconds
+ after = next((x["result"] for x in r if x.get("command") == "stats"), {})
+ inferred = after.get("triples", 0) - triples
+ print(f" reason (owl-rl): {reason_seconds:.1f}s, +{inferred} inferred triples")
+ loads = loads + ["reason owl-rl"]
+
+ rows = []
+ for name, query, needs_inference in QUERIES:
+ flat = " ".join(query.split())
+ best = None
+ count = 0
+ for _ in range(args.runs):
+ t0 = time.time()
+ r = batch(args.binary, loads + [f"query {json.dumps(flat)}"])
+ elapsed = (time.time() - t0 - load_seconds) * 1000
+ res = next((x["result"] for x in r if x.get("command") == "query"), {})
+ count = len(res.get("results", []))
+ best = elapsed if best is None else min(best, elapsed)
+ rows.append({
+ "query": name,
+ "results": count,
+ "ms": round(max(best, 0.0), 1),
+ "needs_inference": needs_inference,
+ })
+ flag = " (needs inference)" if needs_inference else ""
+ print(f" {name:>4}: {count:>7} results {max(best, 0.0):>8.1f} ms{flag}")
+
+ summary = {
+ "dataset": args.data,
+ "files": len(files),
+ "triples": triples,
+ "load_seconds": round(load_seconds, 2),
+ "reasoned": bool(args.reason),
+ "reason_seconds": round(reason_seconds, 2) if reason_seconds else None,
+ "inferred_triples": inferred,
+ "runs_per_query": args.runs,
+ "queries": rows,
+ }
+ if args.out:
+ pathlib.Path(args.out).write_text(json.dumps(summary, indent=2))
+ print(f"wrote {args.out}")
+ return 0
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/src/communities.rs b/src/communities.rs
index 9c9d3d9d..562a1ea5 100644
--- a/src/communities.rs
+++ b/src/communities.rs
@@ -160,8 +160,9 @@ impl Communities {
}
} else if si || oi {
let (from, to) = if si { (s, o) } else { (o, s) };
- if let Some(&other) = community_of.get(to.as_str()) {
- if other != id && bridges.len() < top_members {
+ let other = community_of.get(to.as_str()).copied();
+ if let Some(other) = other.filter(|&c| c != id && bridges.len() < top_members) {
+ {
bridges.push(Bridge {
from: short(from),
predicate: local_name(p).to_string(),
@@ -244,7 +245,7 @@ impl Communities {
};
let mut best = current;
let mut best_gain = gain(current);
- for (&c, _) in &links {
+ for &c in links.keys() {
let g = gain(c);
if g > best_gain + 1e-12 || (g > best_gain - 1e-12 && c < best) {
best = c;
@@ -328,8 +329,7 @@ fn strip(value: &str) -> String {
if v.starts_with('<') && v.ends_with('>') {
return v[1..v.len() - 1].to_string();
}
- if v.starts_with('"') {
- let body = &v[1..];
+ if let Some(body) = v.strip_prefix('"') {
for cut in ["\"^^", "\"@", "\""] {
if let Some(i) = body.find(cut) {
return body[..i].to_string();
diff --git a/src/graph.rs b/src/graph.rs
index 3e055c71..1d0356cf 100644
--- a/src/graph.rs
+++ b/src/graph.rs
@@ -144,8 +144,20 @@ impl GraphStore {
let format = Self::detect_format_sniffed(path, &content);
let store = self.store.lock().unwrap();
let reader = Cursor::new(content.as_bytes());
+
+ // A document's own location is its default base, per RFC 3986. Without
+ // it, any file using relative IRIs fails to parse at all, which is
+ // most published RDF/XML: LUBM's generated data would not load a
+ // single triple before this.
+ let base = std::fs::canonicalize(path)
+ .ok()
+ .and_then(|abs| abs.to_str().map(|s| format!("file://{s}")));
// All or nothing: see load_turtle (issue #93).
- let quads: Vec<_> = RdfParser::from_format(format)
+ let mut parser = RdfParser::from_format(format);
+ if let Some(p) = base.as_ref().and_then(|b| parser.clone().with_base_iri(b).ok()) {
+ parser = p;
+ }
+ let quads: Vec<_> = parser
.for_reader(reader)
.collect::<Result<_, _>>()?;
for quad in &quads {
diff --git a/src/server.rs b/src/server.rs
index 19908321..487291cd 100644
--- a/src/server.rs
+++ b/src/server.rs
@@ -859,10 +859,12 @@ impl OpenOntologiesServer {
chrono::Utc::now().to_rfc3339()));
let mut seen = std::collections::HashSet::new();
for line in ntriples.lines() {
- if let Some(subject) = line.split_whitespace().next() {
- if subject.starts_with('<') && seen.insert(subject.to_string()) {
- extra.push_str(&format!("{subject} {PROV_DERIVED} <{source_iri}> .\n"));
- }
+ if let Some(subject) = line
+ .split_whitespace()
+ .next()
+ .filter(|s| s.starts_with('<') && seen.insert(s.to_string()))
+ {
+ extra.push_str(&format!("{subject} {PROV_DERIVED} <{source_iri}> .\n"));
}
}
prov_triples = extra.lines().count();
diff --git a/src/support.rs b/src/support.rs
index 08a45c36..a944f12b 100644
--- a/src/support.rs
+++ b/src/support.rs
@@ -62,8 +62,7 @@ fn strip(value: &str) -> String {
if v.starts_with('<') && v.ends_with('>') {
return v[1..v.len() - 1].to_string();
}
- if v.starts_with('"') {
- let body = &v[1..];
+ if let Some(body) = v.strip_prefix('"') {
for cut in ["\"^^", "\"@", "\""] {
if let Some(i) = body.find(cut) {
return body[..i].to_string();