Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.
Rust ai-nativedescription-logicsformal-verificationinfrastructure-as-codeisabelle-languageknowledge-graphleanlean4mcpmcp-serverontologyowlowl2reasoningrustshaclsparqltheorem-proving
fix(temporal): report the four scan caps instead of answering as if complete (#111)
Nicolasdce43993108f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 530054cb..fc73409f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -60,6 +60,34 @@ All notable changes to Open Ontologies are documented here.keep flattening, which is the only thing they can represent. TriG and N-Quadsround-trip tests in `tests/graph_named_graph_roundtrip_test.rs`.+- **The bi-temporal tools say when a scan was cut short.** Four queries behind+ `onto_temporal_snapshot`, `onto_temporal_query` and `onto_temporal_conflicts`+ are capped — 20,000 validity rows, 20,000 named graphs, 10,000 result rows,+ 5,000 disjointness pairs — and a store that reached one got a confident+ answer with nothing to say it was partial (issue #95). Every response now+ carries `complete`, and a cut run also carries `truncated`: one+ `{scan, limit, consequence}` entry per cap that bit.++ Truncating a result list gives you fewer rows. Truncating the validity scan+ gives you a WRONG answer, so those responses also carry a `warning`. A graph+ whose validity rows fell past the cap reads as having no validity at all, and+ an undescribed graph is timeless and always in scope — the opposite of the+ truth for a graph whose period had ended. In `onto_temporal_conflicts` the+ same gap is a false positive rather than a gap: a timeless period overlaps+ everything, so a superseded correction is republished as a live+ contradiction, which is the one thing that tool exists to prevent.+ `onto_temporal_query` inherits the verdict from the scope it ran over,+ including on the empty-scope path, where "no graphs in scope" may mean+ nothing among the graphs that were read.++ Detection is proof rather than inference: each scan is sent with `LIMIT n+1`+ and the extra row, if it arrives, is evidence that more exist. It is dropped+ before the response is built, so a store holding exactly the cap is reported+ as complete, and every untruncated response keeps its 1.2.0 values. 10 tests+ in `src/temporal.rs`, including one pinning an exactly-full scan as NOT+ truncated and one showing a correction turn into a false contradiction when+ the validity scan is cut.+### Added- **Optional eviction of float32 text vectors from memory**(`VecStore::with_text_vectors_evicted`, `turbovec` feature). Without it thediff --git a/src/server.rs b/src/server.rsindex b2545b6a..eb025a37 100644--- a/src/server.rs+++ b/src/server.rs@@ -1010,7 +1010,7 @@ impl OpenOntologiesServer {}}- #[tool(name = "onto_temporal_snapshot", description = "Which named graphs are in scope at a point in time, and which are excluded and why. Two independent clocks: valid_at asks what was TRUE then, as_of asks what was KNOWN then. Assertions live in named graphs described in the default graph with temporal:validFrom, validTo and recordedAt; a graph with no description is timeless and always in scope.")]+ #[tool(name = "onto_temporal_snapshot", description = "Which named graphs are in scope at a point in time, and which are excluded and why. Two independent clocks: valid_at asks what was TRUE then, as_of asks what was KNOWN then. Assertions live in named graphs described in the default graph with temporal:validFrom, validTo and recordedAt; a graph with no description is timeless and always in scope. The scans are capped: `complete` says whether they finished, and a run that was cut short also carries `truncated` and a `warning`, because a truncated validity scan makes this partition wrong rather than merely short.")]async fn onto_temporal_snapshot(&self, Parameters(input): Parameters<OntoTemporalSnapshotInput>) -> String {use crate::temporal::Temporal;match Temporal::new(self.graph.clone()).snapshot(input.valid_at.as_deref(), input.as_of.as_deref()) {@@ -1019,7 +1019,7 @@ impl OpenOntologiesServer {}}- #[tool(name = "onto_temporal_query", description = "Run a SPARQL graph pattern against only the graphs in temporal scope: what the graph said at a given valid time, as known at a given recorded time. Pass the body of a WHERE clause as `pattern`.")]+ #[tool(name = "onto_temporal_query", description = "Run a SPARQL graph pattern against only the graphs in temporal scope: what the graph said at a given valid time, as known at a given recorded time. Pass the body of a WHERE clause as `pattern`. `complete` is false when the result list was cut at its cap or when the scope it ran over was, and `truncated` says which; only the second case is a wrong answer, and it carries a `warning`.")]async fn onto_temporal_query(&self, Parameters(input): Parameters<OntoTemporalQueryInput>) -> String {use crate::temporal::Temporal;match Temporal::new(self.graph.clone()).query_at(@@ -1030,7 +1030,7 @@ impl OpenOntologiesServer {}}- #[tool(name = "onto_temporal_conflicts", description = "Disjointness violations that claim OVERLAPPING validity, separated from those that do not. Without valid time a correction reads as a contradiction: an entity recorded one way until May and another after trips every check. This reports genuine disagreement about the same period as contradictions, and everything else as superseded history.")]+ #[tool(name = "onto_temporal_conflicts", description = "Disjointness violations that claim OVERLAPPING validity, separated from those that do not. Without valid time a correction reads as a contradiction: an entity recorded one way until May and another after trips every check. This reports genuine disagreement about the same period as contradictions, and everything else as superseded history. The scans are capped: `complete` says whether they finished, and if the validity scan was cut a `warning` marks the classification unsound, because a pair compared without its periods can appear here as a contradiction when it is a correction.")]async fn onto_temporal_conflicts(&self) -> String {use crate::temporal::Temporal;match Temporal::new(self.graph.clone()).conflicts() {diff --git a/src/temporal.rs b/src/temporal.rsindex 2f889c59..34c771ae 100644--- a/src/temporal.rs+++ b/src/temporal.rs@@ -39,6 +39,19 @@//! Intervals are half-open, `[validFrom, validTo)`. Two facts that meet at a//! boundary do not overlap, which is what makes "adherent until May,//! suspension from May" a correction rather than a contradiction.+//!+//! ## Bounded scans+//!+//! Every query behind these tools is capped so a pathological store cannot+//! pull the whole dataset into memory. A cap that quietly changes the answer+//! is worse than a slow query, so each response carries `complete`, and a run+//! that was cut short says which scan was cut and at what limit.+//!+//! The two failures are not equally bad. Truncating a result list gives you+//! fewer rows. Truncating the validity scan gives you a WRONG scope: a graph+//! whose description fell past the cap reads as having no description at all,+//! and an undescribed graph is timeless and always in scope. Those responses+//! also carry a `warning`.use crate::graph::GraphStore;use std::collections::{BTreeMap, BTreeSet};@@ -48,6 +61,130 @@ pub const NS: &str = "https://open-ontologies.org/temporal#";pub struct Temporal {graph: Arc<GraphStore>,+ limits: Limits,+}++/// Validity ROWS, not graphs. The query is a three-way UNION, so a graph+/// carrying validFrom, validTo and recordedAt costs three rows: the cap is+/// reached at roughly 6,700 fully described graphs.+const VALIDITY_SCAN_LIMIT: usize = 20_000;+/// Distinct named graphs holding assertions.+const GRAPH_SCAN_LIMIT: usize = 20_000;+/// Result rows returned by a snapshot-scoped query.+const QUERY_ROW_LIMIT: usize = 10_000;+/// Candidate disjointness pairs examined for overlap.+const CONFLICT_PAIR_LIMIT: usize = 5_000;++/// The four scan caps. Only `Limits::default` ships; the field-by-field form+/// exists so a test can reach a truncated scan with three graphs instead of+/// twenty thousand.+#[derive(Clone, Copy, Debug, PartialEq, Eq)]+struct Limits {+ validity_scan: usize,+ graph_scan: usize,+ query_rows: usize,+ conflict_pairs: usize,+}++impl Default for Limits {+ fn default() -> Self {+ Self {+ validity_scan: VALIDITY_SCAN_LIMIT,+ graph_scan: GRAPH_SCAN_LIMIT,+ query_rows: QUERY_ROW_LIMIT,+ conflict_pairs: CONFLICT_PAIR_LIMIT,+ }+ }+}++/// Whether one capped scan was cut short, and at what cap. Carried apart from+/// the rows, so a caller that has already folded the rows into a map can still+/// say what the map was built from.+#[derive(Clone, Copy, Debug)]+struct Capped {+ hit: bool,+ cap: usize,+}++impl Capped {+ /// A machine-readable report, or `None` when the scan was complete: a+ /// response that was never cut gains no `truncated` entry at all.+ fn report(self, scan: &str, consequence: &str) -> Option<serde_json::Value> {+ self.hit.then(|| {+ serde_json::json!({+ "scan": scan,+ "limit": self.cap,+ "consequence": consequence,+ })+ })+ }+}++/// The rows one capped scan returned, and whether the cap cut it short.+struct Scan {+ rows: Vec<serde_json::Value>,+ capped: Capped,+}++/// The graphs a snapshot puts in and out of scope, and how trustworthy that+/// partition is. `snapshot` renders it; `query_at` runs against it.+struct Scope {+ in_scope: Vec<serde_json::Value>,+ excluded: Vec<serde_json::Value>,+ /// The in-scope graph IRIs, in the same order as `in_scope`.+ graphs: Vec<String>,+ validity_scan: Capped,+ graph_scan: Capped,+}++impl Scope {+ fn complete(&self) -> bool {+ !self.validity_scan.hit && !self.graph_scan.hit+ }++ /// One report per scan that was cut. Both scans decide the partition, so+ /// either one being cut makes it wrong rather than merely short.+ fn cuts(&self) -> Vec<serde_json::Value> {+ [+ self.validity_scan.report(+ "validities",+ "graphs whose validity rows fell past the limit were read as having no \+ validity at all, so they are listed in scope as timeless even where their \+ period excludes them",+ ),+ self.graph_scan.report(+ "all_graphs",+ "graphs past the limit are missing from both in_scope and excluded",+ ),+ ]+ .into_iter()+ .flatten()+ .collect()+ }++ /// The loud half, in SHACL's register (`src/shacl.rs`): a sentence that+ /// refuses to let a degraded answer read as a clean one. Callers add what+ /// it means for their own answer and the pointer to `truncated`.+ fn warning(&self) -> Option<String> {+ let mut parts: Vec<&str> = Vec::new();+ if self.validity_scan.hit {+ parts.push(+ "described graphs were read as timeless and put in scope, which is the \+ opposite of the truth for any graph whose period had ended",+ );+ }+ if self.graph_scan.hit {+ parts.push("graphs are missing from both in_scope and excluded");+ }+ if parts.is_empty() {+ return None;+ }+ Some(format!(+ "INCOMPLETE SCOPE: a scan hit its row limit, so {}. The scope is not merely short, \+ part of it is wrong.",+ parts.join(", and ")+ ))+ }}#[derive(Clone, Debug)]@@ -110,28 +247,56 @@ fn local(iri: &str) -> &str {impl Temporal {pub fn new(graph: Arc<GraphStore>) -> Self {- Self { graph }+ Self::with_limits(graph, Limits::default())+ }++ /// The same view over the same store with different scan caps.+ ///+ /// Private, and the only caller outside this module's tests is `new` with+ /// the shipped defaults: the caps are a safety bound, not a knob.+ fn with_limits(graph: Arc<GraphStore>, limits: Limits) -> Self {+ Self { graph, limits }}- fn rows(&self, query: &str) -> anyhow::Result<Vec<serde_json::Value>> {- let raw = self.graph.sparql_select(query)?;+ /// Run one capped scan.+ ///+ /// `query` must NOT carry its own LIMIT: the cap is appended here as+ /// `cap + 1`. That extra row is a probe rather than data — if it comes+ /// back, more rows exist, which is proof instead of the guess you get from+ /// comparing a returned count against the cap (a store holding exactly+ /// `cap` rows is complete and must not be reported as cut). The probe is+ /// dropped before returning, so an untruncated scan yields exactly the+ /// rows it yielded in 1.2.0.+ fn rows(&self, query: &str, cap: usize) -> anyhow::Result<Scan> {+ let probe = cap.saturating_add(1);+ let raw = self+ .graph+ .sparql_select(&format!("{query} LIMIT {probe}"))?;let parsed: serde_json::Value = serde_json::from_str(&raw)?;- Ok(parsed+ let mut rows: Vec<serde_json::Value> = parsed.get("results").and_then(|r| r.as_array()).cloned()- .unwrap_or_default())+ .unwrap_or_default();+ let hit = rows.len() > cap;+ rows.truncate(cap);+ Ok(Scan {+ rows,+ capped: Capped { hit, cap },+ })}- /// Every graph that carries validity metadata.- fn validities(&self) -> anyhow::Result<BTreeMap<String, Validity>> {+ /// Every graph that carries validity metadata, and whether the scan that+ /// found them was complete.+ fn validities(&self) -> anyhow::Result<(BTreeMap<String, Validity>, Capped)> {let query = format!("SELECT ?g ?from ?to ?rec WHERE {{ \{{ ?g <{NS}validFrom> ?from }} UNION {{ ?g <{NS}validTo> ?to }} \- UNION {{ ?g <{NS}recordedAt> ?rec }} }} LIMIT 20000"+ UNION {{ ?g <{NS}recordedAt> ?rec }} }}");+ let scan = self.rows(&query, self.limits.validity_scan)?;let mut out: BTreeMap<String, Validity> = BTreeMap::new();- for row in self.rows(&query)? {+ for row in &scan.rows {let Some(g) = row.get("g").and_then(|v| v.as_str()).map(plain) else {continue;};@@ -151,35 +316,51 @@ impl Temporal {entry.recorded_at = Some(plain(v));}}- Ok(out)+ Ok((out, scan.capped))}/// Named graphs holding assertions, whether or not they are described.- fn all_graphs(&self) -> anyhow::Result<BTreeSet<String>> {- Ok(self- .rows("SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 20000")?+ fn all_graphs(&self) -> anyhow::Result<(BTreeSet<String>, Capped)> {+ let scan = self.rows(+ "SELECT DISTINCT ?g WHERE { GRAPH ?g { ?s ?p ?o } }",+ self.limits.graph_scan,+ )?;+ let graphs = scan+ .rows.iter().filter_map(|r| r.get("g").and_then(|v| v.as_str()).map(plain))- .collect())+ .collect();+ Ok((graphs, scan.capped))}- /// Which graphs are in scope for a snapshot, and why.- pub fn snapshot(&self, valid_at: Option<&str>, as_of: Option<&str>) -> anyhow::Result<String> {- let validities = self.validities()?;- let graphs = self.all_graphs()?;+ /// Partition the store's named graphs into in scope and excluded.+ ///+ /// `snapshot` renders this and `query_at` runs against it, so the two+ /// tools agree by construction and the truncation verdict reaches+ /// `query_at` as a value rather than through its own JSON output.+ fn scope(&self, valid_at: Option<&str>, as_of: Option<&str>) -> anyhow::Result<Scope> {+ let (validities, validity_scan) = self.validities()?;+ let (graphs, graph_scan) = self.all_graphs()?;let mut in_scope = Vec::new();let mut excluded = Vec::new();+ let mut names = Vec::new();for g in &graphs {match validities.get(g) {// Undescribed graphs are timeless and always in scope, so// this vocabulary is additive to an existing store.- None => in_scope.push(serde_json::json!({"graph": g, "reason": "no validity recorded, timeless"})),+ None => {+ in_scope.push(+ serde_json::json!({"graph": g, "reason": "no validity recorded, timeless"}),+ );+ names.push(g.clone());+ }Some(v) => {let valid_ok = valid_at.is_none_or(|t| v.valid_at(t));let recorded_ok = as_of.is_none_or(|t| v.recorded_by(t));if valid_ok && recorded_ok {in_scope.push(serde_json::json!({"graph": g, "valid": v.describe()}));+ names.push(g.clone());} else {excluded.push(serde_json::json!({"graph": g,@@ -191,15 +372,33 @@ impl Temporal {}}- Ok(serde_json::json!({+ Ok(Scope {+ in_scope,+ excluded,+ graphs: names,+ validity_scan,+ graph_scan,+ })+ }++ /// Which graphs are in scope for a snapshot, and why.+ pub fn snapshot(&self, valid_at: Option<&str>, as_of: Option<&str>) -> anyhow::Result<String> {+ let scope = self.scope(valid_at, as_of)?;++ let mut out = serde_json::json!({"ok": true,"valid_at": valid_at,"as_of": as_of,- "in_scope": in_scope,- "excluded": excluded,+ "in_scope": scope.in_scope,+ "excluded": scope.excluded,+ "complete": scope.complete(),"note": "Graphs without validity metadata are timeless and always in scope.",- })- .to_string())+ });+ if let Some(warning) = scope.warning() {+ out["warning"] = serde_json::Value::String(format!("{warning} See truncated."));+ out["truncated"] = serde_json::Value::Array(scope.cuts());+ }+ Ok(out.to_string())}/// Run a query against only the graphs in temporal scope.@@ -213,24 +412,31 @@ impl Temporal {valid_at: Option<&str>,as_of: Option<&str>,) -> anyhow::Result<String> {- let snapshot: serde_json::Value = serde_json::from_str(&self.snapshot(valid_at, as_of)?)?;- let graphs: Vec<String> = snapshot- .get("in_scope")- .and_then(|v| v.as_array())- .map(|rows| {- rows.iter()- .filter_map(|r| r.get("graph").and_then(|g| g.as_str()).map(String::from))- .collect()- })- .unwrap_or_default();+ let scope = self.scope(valid_at, as_of)?;+ // The scope this query runs over is the snapshot's scope, so the+ // snapshot's truncation is this query's truncation. Saying nothing+ // here would hide a wrong scope behind a tool that never mentions+ // scans at all.+ let mut cuts = scope.cuts();+ let scope_warning = scope.warning();+ let scope_complete = scope.complete();+ let graphs = scope.graphs;if graphs.is_empty() {- return Ok(serde_json::json!({+ let mut out = serde_json::json!({"ok": true,"results": [],+ "complete": scope_complete,"note": "no graphs in scope at that instant",- })- .to_string());+ });+ if let Some(warning) = scope_warning {+ out["warning"] = serde_json::Value::String(format!(+ "{warning} An empty scope here may mean \"nothing among the graphs that \+ were read\" rather than \"nothing\". See truncated."+ ));+ out["truncated"] = serde_json::Value::Array(cuts);+ }+ return Ok(out.to_string());}let values = graphs@@ -243,19 +449,34 @@ impl Temporal {.strip_prefix('{').and_then(|b| b.strip_suffix('}')).unwrap_or(body);- let wrapped = format!(- "SELECT * WHERE {{ VALUES ?__g {{ {values} }} GRAPH ?__g {{ {body} }} }} LIMIT 10000"- );- let raw = self.graph.sparql_select(&wrapped)?;- let parsed: serde_json::Value = serde_json::from_str(&raw)?;- Ok(serde_json::json!({+ let wrapped =+ format!("SELECT * WHERE {{ VALUES ?__g {{ {values} }} GRAPH ?__g {{ {body} }} }}");+ let scan = self.rows(&wrapped, self.limits.query_rows)?;+ cuts.extend(scan.capped.report(+ "query_rows",+ "the result list is the first page only; rows past the limit were not returned",+ ));++ let mut out = serde_json::json!({"ok": true,"valid_at": valid_at,"as_of": as_of,"graphs_in_scope": graphs.len(),- "results": parsed.get("results").cloned().unwrap_or(serde_json::json!([])),- })- .to_string())+ "results": scan.rows,+ "complete": scope_complete && !scan.capped.hit,+ });+ if !cuts.is_empty() {+ out["truncated"] = serde_json::Value::Array(cuts);+ }+ // A short result list is fewer rows and stays a quiet key. A cut scope+ // is a wrong answer, so it keeps the warning it arrived with.+ if let Some(warning) = scope_warning {+ out["warning"] = serde_json::Value::String(format!(+ "{warning} These results were drawn from that scope, so they come from the \+ wrong set of graphs. See truncated."+ ));+ }+ Ok(out.to_string())}/// Disjointness violations, but only where the two assertions claim@@ -267,8 +488,8 @@ impl Temporal {/// the finding is noise. With it, a superseded statement is superseded,/// and only genuine disagreement about the same period survives.pub fn conflicts(&self) -> anyhow::Result<String> {- let validities = self.validities()?;- let rows = self.rows(+ let (validities, validity_scan) = self.validities()?;+ let scan = self.rows("PREFIX owl: <http://www.w3.org/2002/07/owl#> \PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> \SELECT DISTINCT ?s ?a ?b ?ga ?gb WHERE { \@@ -276,12 +497,13 @@ impl Temporal {FILTER(STR(?a) < STR(?b)) \?a rdfs:subClassOf* ?da . ?b rdfs:subClassOf* ?db . \{ ?da owl:disjointWith ?db } UNION { ?db owl:disjointWith ?da } \- } LIMIT 5000",+ }",+ self.limits.conflict_pairs,)?;let mut conflicts = Vec::new();let mut superseded = Vec::new();- for row in &rows {+ for row in &scan.rows {let get = |k: &str| row.get(k).and_then(|v| v.as_str()).map(plain);let (Some(s), Some(a), Some(b), Some(ga), Some(gb)) = (get("s"), get("a"), get("b"), get("ga"), get("gb"),@@ -314,16 +536,371 @@ impl Temporal {}}- Ok(serde_json::json!({+ let mut out = serde_json::json!({"ok": true,"contradictions": conflicts,"contradiction_count": conflicts.len(),"superseded": superseded,"superseded_count": superseded.len(),+ "complete": !validity_scan.hit && !scan.capped.hit,"note": "contradictions claim overlapping validity and genuinely disagree. \superseded pairs are corrections: one period ends where the other begins, \which is a history rather than a conflict.",- })- .to_string())+ });++ // A cut validity scan is not a smaller answer here, it is a wrong one:+ // a graph whose rows fell past the cap takes the `timeless` fallback+ // above, a timeless period has open ends, so `overlaps` returns true+ // against everything and a correction that should read as superseded+ // history is published as a live contradiction.+ let cuts: Vec<serde_json::Value> = [+ validity_scan.report(+ "validities",+ "pairs whose validity rows fell past the limit were compared as timeless, and \+ a timeless period overlaps everything: superseded corrections can appear here \+ as contradictions",+ ),+ scan.capped.report(+ "conflict_pairs",+ "candidate pairs past the limit were never examined, so the absence of a \+ contradiction is not evidence that there is none",+ ),+ ]+ .into_iter()+ .flatten()+ .collect();++ if !cuts.is_empty() {+ out["truncated"] = serde_json::Value::Array(cuts);+ }+ if validity_scan.hit {+ out["warning"] = serde_json::Value::String(+ "UNSOUND CLASSIFICATION: the validity scan hit its row limit, so some pairs \+ were compared without their periods. A correction can be reported here as a \+ contradiction, which is the one thing this tool exists to prevent; \+ contradiction_count is an upper bound over the pairs that were examined, and \+ says nothing about any pair the candidate scan did not reach. See truncated."+ .to_string(),+ );+ }+ Ok(out.to_string())+ }+}++#[cfg(test)]+mod tests {+ use super::*;+ use oxigraph::io::RdfFormat;++ /// Three graphs, one validity row each: a cap of 2 cuts a scan with a+ /// dataset that fits on a screen.+ const THREE: &str = r#"+@prefix ex: <http://example.org/> .+@prefix t: <https://open-ontologies.org/temporal#> .++ex:g1 { ex:X ex:p ex:one . }+ex:g2 { ex:X ex:p ex:two . }+ex:g3 { ex:X ex:p ex:three . }++{+ ex:g1 t:validFrom "2024-01-01" .+ ex:g2 t:validFrom "2025-01-01" .+ ex:g3 t:validFrom "2026-01-01" .+}+"#;++ /// A correction: one type up to a boundary, a disjoint one from it.+ /// Half-open, so this is superseded history and not a contradiction.+ const CORRECTION: &str = r#"+@prefix ex: <http://example.org/> .+@prefix owl: <http://www.w3.org/2002/07/owl#> .+@prefix t: <https://open-ontologies.org/temporal#> .++ex:g_before { ex:X a ex:Adherent . }+ex:g_after { ex:X a ex:Suspension . }++{+ ex:Adherent owl:disjointWith ex:Suspension .+ ex:g_before t:validFrom "2024-01-01" ; t:validTo "2026-05-01" .+ ex:g_after t:validFrom "2026-05-01" .+}+"#;++ fn store(trig: &str) -> Arc<GraphStore> {+ let store = Arc::new(GraphStore::new());+ store+ .load_content(trig, RdfFormat::TriG)+ .expect("TriG fixture parses");+ store+ }++ fn json(raw: String) -> serde_json::Value {+ serde_json::from_str(&raw).expect("tool output is JSON")+ }++ /// The overrides below are for tests. If one ever became the shipped+ /// value, this is the line that says so.+ #[test]+ fn shipped_caps_are_the_defaults() {+ let l = Limits::default();+ assert_eq!(+ (+ l.validity_scan,+ l.graph_scan,+ l.query_rows,+ l.conflict_pairs+ ),+ (+ VALIDITY_SCAN_LIMIT,+ GRAPH_SCAN_LIMIT,+ QUERY_ROW_LIMIT,+ CONFLICT_PAIR_LIMIT+ )+ );+ }++ /// The contract for every store that never reaches a cap: one honest+ /// `complete`, and not a key more than 1.2.0 emitted.+ #[test]+ fn a_run_that_was_never_cut_says_so_and_adds_nothing_else() {+ let t = Temporal::new(store(THREE));+ for out in [+ json(t.snapshot(None, None).unwrap()),+ json(t.query_at("{ ?s ?p ?o }", None, None).unwrap()),+ json(t.conflicts().unwrap()),+ ] {+ assert_eq!(out["complete"], true, "{out}");+ assert!(out.get("truncated").is_none(), "{out}");+ assert!(out.get("warning").is_none(), "{out}");+ }+ }++ /// Why the probe row exists. Three graphs, three validity rows, both caps+ /// set to exactly three: the pages are full and nothing is missing. A+ /// `len() == cap` detector reports two false truncations here.+ #[test]+ fn an_exactly_full_scan_is_not_reported_as_truncated() {+ let t = Temporal::with_limits(+ store(THREE),+ Limits {+ validity_scan: 3,+ graph_scan: 3,+ ..Limits::default()+ },+ );+ let snap = json(t.snapshot(None, None).unwrap());+ assert_eq!(snap["in_scope"].as_array().unwrap().len(), 3);+ assert_eq!(+ snap["complete"], true,+ "a full page is not a cut one: {snap}"+ );+ assert!(snap.get("truncated").is_none(), "{snap}");+ }++ /// The case that has to be loud. Every graph in the fixture is described,+ /// so a graph the snapshot calls timeless is provably a misclassification.+ #[test]+ fn a_cut_validity_scan_makes_the_snapshot_loudly_incomplete() {+ let t = Temporal::with_limits(+ store(THREE),+ Limits {+ validity_scan: 2,+ ..Limits::default()+ },+ );+ let snap = json(t.snapshot(Some("2024-06-01"), None).unwrap());++ assert_eq!(snap["complete"], false, "{snap}");+ assert!(+ snap["warning"]+ .as_str()+ .expect("a cut scope carries a warning")+ .contains("INCOMPLETE SCOPE"),+ "loud, not a quiet key: {snap}"+ );+ assert_eq!(snap["truncated"][0]["scan"], "validities");+ assert_eq!(snap["truncated"][0]["limit"], 2);++ // SPARQL does not order an unordered LIMIT, so WHICH graph lost its+ // description is not fixed — but exactly one did, whichever it was.+ let misclassified = snap["in_scope"]+ .as_array()+ .unwrap()+ .iter()+ .filter(|row| row.get("reason").is_some())+ .count();+ assert_eq!(+ misclassified, 1,+ "a described graph was published as timeless and in scope, which is what the \+ warning is for: {snap}"+ );+ }++ #[test]+ fn a_cut_graph_scan_drops_graphs_from_both_buckets() {+ let t = Temporal::with_limits(+ store(THREE),+ Limits {+ graph_scan: 2,+ ..Limits::default()+ },+ );+ let snap = json(t.snapshot(None, None).unwrap());+ let seen =+ snap["in_scope"].as_array().unwrap().len() + snap["excluded"].as_array().unwrap().len();+ assert_eq!(seen, 2, "the third graph is in neither bucket: {snap}");+ assert_eq!(snap["complete"], false);+ assert_eq!(snap["truncated"][0]["scan"], "all_graphs");+ }++ #[test]+ fn query_at_inherits_the_scope_truncation() {+ let t = Temporal::with_limits(+ store(THREE),+ Limits {+ validity_scan: 2,+ ..Limits::default()+ },+ );+ let out = json(+ t.query_at("{ ?s ?p ?o }", Some("2024-06-01"), None)+ .unwrap(),+ );+ assert_eq!(+ out["complete"], false,+ "a query over a wrong scope is a wrong query: {out}"+ );+ assert!(+ out["warning"]+ .as_str()+ .unwrap()+ .contains("INCOMPLETE SCOPE"),+ "{out}"+ );+ assert!(+ out["truncated"]+ .as_array()+ .unwrap()+ .iter()+ .any(|c| c["scan"] == "validities"),+ "{out}"+ );+ }++ /// The other half of the distinction: a short result list is fewer rows,+ /// not a wrong answer, so it is reported without the warning.+ #[test]+ fn query_at_reports_its_own_row_cap_without_calling_the_scope_wrong() {+ let t = Temporal::with_limits(+ store(THREE),+ Limits {+ query_rows: 2,+ ..Limits::default()+ },+ );+ let out = json(t.query_at("{ ?s ?p ?o }", None, None).unwrap());+ assert_eq!(+ out["results"].as_array().unwrap().len(),+ 2,+ "the probe row is dropped, not returned: {out}"+ );+ assert_eq!(out["complete"], false);+ assert_eq!(out["truncated"][0]["scan"], "query_rows");+ assert_eq!(out["truncated"][0]["limit"], 2);+ assert!(+ out.get("warning").is_none(),+ "a short result list is not a wrong scope: {out}"+ );+ }++ /// Truncation here produces a false POSITIVE, which is why conflicts is+ /// loud too. Cap 1: whichever validity row survives, one graph is left+ /// undescribed, an undescribed period is timeless, and a timeless period+ /// overlaps everything.+ #[test]+ fn a_cut_validity_scan_turns_a_correction_into_a_false_contradiction() {+ let sound = json(Temporal::new(store(CORRECTION)).conflicts().unwrap());+ assert_eq!(sound["contradiction_count"], 0);+ assert_eq!(sound["superseded_count"], 1);+ assert_eq!(sound["complete"], true, "{sound}");++ let cut = json(+ Temporal::with_limits(+ store(CORRECTION),+ Limits {+ validity_scan: 1,+ ..Limits::default()+ },+ )+ .conflicts()+ .unwrap(),+ );+ assert_eq!(+ cut["contradiction_count"], 1,+ "the same correction now reads as a live conflict: {cut}"+ );+ assert_eq!(cut["superseded_count"], 0);+ assert_eq!(cut["complete"], false, "and it is labelled: {cut}");+ assert!(+ cut["warning"]+ .as_str()+ .unwrap()+ .contains("UNSOUND CLASSIFICATION"),+ "{cut}"+ );+ assert_eq!(cut["truncated"][0]["scan"], "validities");+ assert_eq!(cut["truncated"][0]["limit"], 1);+ }++ /// The pair scan is the mild cap: fewer pairs examined, no claim turned+ /// upside down, so it is reported without the warning.+ #[test]+ fn a_cut_pair_scan_is_reported_quietly() {+ let t = Temporal::with_limits(+ store(CORRECTION),+ Limits {+ conflict_pairs: 0,+ ..Limits::default()+ },+ );+ let out = json(t.conflicts().unwrap());+ assert_eq!(out["contradiction_count"], 0);+ assert_eq!(out["superseded_count"], 0);+ assert_eq!(out["complete"], false, "{out}");+ assert_eq!(out["truncated"][0]["scan"], "conflict_pairs");+ assert!(+ out.get("warning").is_none(),+ "an unexamined pair is a gap, not a wrong classification: {out}"+ );+ }++ /// Both caps at once: every cut scan is listed, and the warning may not+ /// call the count an upper bound full stop — unexamined pairs can hold+ /// contradictions the count never saw, so the claim is bounded to the+ /// pairs that were actually compared.+ #[test]+ fn both_conflict_scans_cut_report_both_and_bound_the_claim() {+ let t = Temporal::with_limits(+ store(CORRECTION),+ Limits {+ validity_scan: 1,+ conflict_pairs: 0,+ ..Limits::default()+ },+ );+ let out = json(t.conflicts().unwrap());+ let scans: Vec<&str> = out["truncated"]+ .as_array()+ .unwrap()+ .iter()+ .map(|c| c["scan"].as_str().unwrap())+ .collect();+ assert_eq!(scans, ["validities", "conflict_pairs"], "{out}");+ assert_eq!(out["complete"], false);+ let warning = out["warning"].as_str().unwrap();+ assert!(+ warning.contains("over the pairs that were examined"),+ "with the candidate scan cut too, the count bounds nothing about the store: {out}"+ );}}