Skip to content

Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.

feat(align): RRF (Reciprocal Rank Fusion) as opt-in fusion strategy

FABIOTESS708e65355e28
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 175b15c0..c5fda5d7 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,7 @@ All notable changes to Open Ontologies are documented here.
## [Unreleased]
### Added
+- **RRF (Reciprocal Rank Fusion) as an opt-in fusion strategy for `onto_align`**. New `OntoAlignInput.fusion` field accepts `"weighted_sum"` (default, unchanged behaviour with self-calibrating learned weights) or `"rrf"` (Cormack et al. SIGIR 2009 at k=60, validated for ontology alignment by Agent-OM at VLDB 2025). RRF is order-based rather than score-based, so it doesn't need feedback to bootstrap; it's a sensible cold-start choice when the `align_feedback` table is empty. The per-signal scores remain on each candidate's `signals` field so downstream `onto_align_feedback` calls keep working identically. New public method `AlignmentEngine::align_with_fusion(source, target, high, low, dry_run, fusion)`; the existing `align_with_thresholds(...)` and `align(...)` entry points are preserved as thin wrappers that pass `"weighted_sum"`. New `tests/align_rrf_test.rs` (5 tests) covering normalisation to [0, 1], per-signal preservation, low-threshold post-rerank filtering, weighted_sum back-compat, and weighted_sum/RRF top-pair agreement on perfect matches.
- **`ies4` enforce rule pack** (#24). New built-in design-pattern pack for the [Information Exchange Standard](https://informationexchangestandard.org/), the UK cross-sector ontology framework custodied by Department for Business and Trade since March 2025 (canonical repo `IES-Org/ont-ies`). Three rules beyond the existing `boro` pack: (1) `ies4_particular_class_overlap` (severity: error) — a class cannot subclass both `ies:Particular` and `ies:ClassOfEntity`, as that violates the type-vs-token distinction foundational to IES4's 4D mereology; (2) `ies4_state_without_subject` (severity: warning) — a class subclassing `ies:State` must declare `ies:isStateOf` via owl:Restriction or have at least one instance using it (the state pattern is meaningless without a bearer); (3) `ies4_event_without_participant` (severity: warning) — a class subclassing `ies:Event` must have a participant pattern via `ies:isParticipantIn` / `ies:involvesParticipant` / `ies:hasParticipant` (events without participants are incomplete 4D models). Invoke via `onto_enforce` MCP tool or `enforce` CLI with `rule_pack = "ies4"`. Academic grounding: FOUST 7 paper "Comparing IES and BORO" (CEUR Vol-4176, JOWO 2024). New `tests/enforce_ies4_test.rs` (5 tests covering each rule's positive and negative cases, plus the instance-level participation accept path).
### Fixed
diff --git a/src/align.rs b/src/align.rs
index d25181db..384d1054 100644
--- a/src/align.rs
+++ b/src/align.rs
@@ -321,26 +321,49 @@ impl AlignmentEngine {
self.align_with_thresholds(source, target, min_confidence, min_confidence, dry_run)
}
- /// Two-threshold alignment with borderline-candidate surfacing for LLM-orchestrated review.
+ /// Two-threshold alignment with the default weighted-sum fusion (self-calibrating
+ /// weights). For RRF fusion or any future strategy, call `align_with_fusion` directly.
+ pub fn align_with_thresholds(
+ &self,
+ source: &str,
+ target: Option<&str>,
+ high_threshold: f64,
+ low_threshold: f64,
+ dry_run: bool,
+ ) -> anyhow::Result<String> {
+ self.align_with_fusion(source, target, high_threshold, low_threshold, dry_run, "weighted_sum")
+ }
+
+ /// Two-threshold alignment with borderline-candidate surfacing for LLM-orchestrated review,
+ /// parameterised by fusion strategy.
///
- /// - candidates with confidence ≥ `high_threshold` land in `auto_applied`
- /// (and are persisted as triples unless `dry_run`)
- /// - candidates with confidence in `[low_threshold, high_threshold)` land in `borderline`
- /// with enriched context (labels, parents) so the calling LLM can judge them and record
- /// verdicts via `onto_align_feedback`
- /// - candidates below `low_threshold` are dropped
+ /// `fusion` is one of:
+ /// - `"weighted_sum"` — combine per-signal scores with the self-calibrating weights
+ /// learned from `onto_align_feedback` (default; equal weights at cold start)
+ /// - `"rrf"` — Reciprocal Rank Fusion at k=60 (Cormack et al., SIGIR 2009; validated for
+ /// ontology alignment by Agent-OM at VLDB 2025). No learned weights required; useful
+ /// as a cold-start baseline or alternative to the weighted sum on graphs where
+ /// feedback hasn't been collected yet.
///
- /// If `low_threshold >= high_threshold`, no borderline bucket is produced (back-compat mode).
- pub fn align_with_thresholds(
+ /// Bucket semantics are identical to `align_with_thresholds`: candidates with
+ /// confidence ≥ `high_threshold` land in `auto_applied` (and are persisted unless
+ /// `dry_run`); those in `[low_threshold, high_threshold)` land in `borderline` with
+ /// enriched context; those below `low_threshold` are dropped.
+ pub fn align_with_fusion(
&self,
source: &str,
target: Option<&str>,
high_threshold: f64,
low_threshold: f64,
dry_run: bool,
+ fusion: &str,
) -> anyhow::Result<String> {
// Clamp: degenerate range means "everything above high_threshold; nothing borderline".
let low_threshold = low_threshold.min(high_threshold);
+ // For RRF the inline weighted-sum confidence is throwaway (recomputed after
+ // collection). We collect all label-prefiltered candidates and only apply
+ // `low_threshold` to the FINAL fused score post-rerank.
+ let inline_low_threshold = if fusion == "rrf" { f64::NEG_INFINITY } else { low_threshold };
// Load source into a temporary graph (detect format from content)
let source_store = GraphStore::new();
if std::path::Path::new(source).exists() {
@@ -427,8 +450,10 @@ impl AlignmentEngine {
signals.iter().zip(weights.iter()).map(|(s, w)| s * w).sum()
};
- // Drop pairs below the low threshold entirely.
- if confidence < low_threshold {
+ // Drop pairs below the low threshold entirely (for RRF, this is
+ // f64::NEG_INFINITY — we keep all label-prefiltered candidates and apply
+ // the real threshold after rerank).
+ if confidence < inline_low_threshold {
continue;
}
@@ -461,6 +486,17 @@ impl AlignmentEngine {
}
}
+ // RRF rerank: recompute confidence from per-signal rankings, then re-derive
+ // requires_review from the new confidence and apply the real low_threshold.
+ if fusion == "rrf" {
+ Self::rrf_rerank(&mut candidates);
+ candidates.retain(|c| c["confidence"].as_f64().unwrap_or(0.0) >= low_threshold);
+ for c in &mut candidates {
+ let conf = c["confidence"].as_f64().unwrap_or(0.0);
+ c["requires_review"] = serde_json::json!(conf < high_threshold);
+ }
+ }
+
// Sort by confidence descending
candidates.sort_by(|a, b| {
b["confidence"].as_f64().unwrap_or(0.0)
@@ -581,6 +617,65 @@ impl AlignmentEngine {
}).to_string())
}
+ /// Reciprocal Rank Fusion (Cormack, Clarke, Buettcher SIGIR 2009) at k=60.
+ /// For each per-signal ranking, accumulate `1 / (k + rank)` into each candidate's
+ /// RRF score. Normalise to `[0, 1]` against the max possible (a candidate that
+ /// is rank-1 in every signal), then overwrite each candidate's `confidence` field.
+ ///
+ /// `signals` is the list of signal field-names in the candidate's `signals` object
+ /// to fuse over. By design RRF is robust to signals on different scales — it only
+ /// uses the per-signal ordering, not the raw scores. This is the property that
+ /// makes it a sensible cold-start alternative to the learned weighted-sum.
+ fn rrf_rerank(candidates: &mut [serde_json::Value]) {
+ const K: f64 = 60.0;
+ if candidates.is_empty() {
+ return;
+ }
+
+ #[cfg(feature = "embeddings")]
+ const SIGNALS: &[&str] = &[
+ "label_similarity",
+ "property_overlap",
+ "parent_overlap",
+ "instance_overlap",
+ "restriction_similarity",
+ "neighborhood_similarity",
+ "embedding_similarity",
+ ];
+ #[cfg(not(feature = "embeddings"))]
+ const SIGNALS: &[&str] = &[
+ "label_similarity",
+ "property_overlap",
+ "parent_overlap",
+ "instance_overlap",
+ "restriction_similarity",
+ "neighborhood_similarity",
+ ];
+
+ let n = candidates.len();
+ let mut rrf = vec![0.0_f64; n];
+
+ for sig in SIGNALS {
+ let mut idx: Vec<usize> = (0..n).collect();
+ idx.sort_by(|&a, &b| {
+ let sa = candidates[a]["signals"][sig].as_f64().unwrap_or(0.0);
+ let sb = candidates[b]["signals"][sig].as_f64().unwrap_or(0.0);
+ sb.partial_cmp(&sa).unwrap_or(std::cmp::Ordering::Equal)
+ });
+ for (rank_zero, &i) in idx.iter().enumerate() {
+ let rank = (rank_zero + 1) as f64;
+ rrf[i] += 1.0 / (K + rank);
+ }
+ }
+
+ // Max possible: rank 1 in every signal -> SIGNALS.len() * 1/(K + 1)
+ let max_possible = SIGNALS.len() as f64 * (1.0 / (K + 1.0));
+ for (i, c) in candidates.iter_mut().enumerate() {
+ let normalised = (rrf[i] / max_possible).clamp(0.0, 1.0);
+ c["confidence"] = serde_json::json!((normalised * 1000.0).round() / 1000.0);
+ }
+ }
+
/// Classify the relation type based on signal strengths.
fn classify_relation(label_sim: f64, prop_overlap: f64, parent_overlap: f64) -> &'static str {
if label_sim > 0.8 && prop_overlap > 0.5 {
diff --git a/src/inputs.rs b/src/inputs.rs
index 7c75ac73..8af485f8 100644
--- a/src/inputs.rs
+++ b/src/inputs.rs
@@ -389,6 +389,13 @@ pub struct OntoAlignInput {
pub low_threshold: Option<f64>,
/// If true, return candidates only without inserting triples (default false)
pub dry_run: Option<bool>,
+ /// Fusion strategy for combining the per-signal scores into a confidence score.
+ /// One of "weighted_sum" (default — learned weights over the 7 signals, cold-start
+ /// equal-weighted) or "rrf" (Reciprocal Rank Fusion at k=60, validated by Agent-OM
+ /// at VLDB 2025). RRF doesn't need learned weights so it's a sensible cold-start
+ /// choice; the weighted_sum self-calibrates from `onto_align_feedback` over time.
+ #[serde(default)]
+ pub fusion: Option<String>,
}
#[derive(Deserialize, JsonSchema)]
diff --git a/src/server.rs b/src/server.rs
index 89e9a747..e3cbacea 100644
--- a/src/server.rs
+++ b/src/server.rs
@@ -1378,14 +1378,15 @@ impl OpenOntologiesServer {
// Callers wanting the old strict behaviour pass low_threshold == high_threshold.
let low = input.low_threshold.unwrap_or(0.4).min(high);
let dry_run = input.dry_run.unwrap_or(false);
+ let fusion = input.fusion.as_deref().unwrap_or("weighted_sum");
- match engine.align_with_thresholds(&source, target.as_deref(), high, low, dry_run) {
+ match engine.align_with_fusion(&source, target.as_deref(), high, low, dry_run, fusion) {
Ok(result) => {
self.lineage().record(
&self.session_id,
"AL",
"align",
- &format!("high={},low={}", high, low),
+ &format!("high={},low={},fusion={}", high, low, fusion),
);
result
}
diff --git a/tests/align_rrf_test.rs b/tests/align_rrf_test.rs
new file mode 100644
index 00000000..d1ffd43d
--- /dev/null
+++ b/tests/align_rrf_test.rs
@@ -0,0 +1,168 @@
+//! Integration tests for the optional RRF (Reciprocal Rank Fusion) strategy in
+//! `onto_align` — Cormack et al. SIGIR 2009, validated for ontology alignment
+//! by Agent-OM at VLDB 2025.
+
+use open_ontologies::align::AlignmentEngine;
+use open_ontologies::graph::GraphStore;
+use open_ontologies::state::StateDb;
+use std::sync::Arc;
+use tempfile::NamedTempFile;
+
+fn setup() -> (StateDb, Arc<GraphStore>) {
+ let tmp = NamedTempFile::new().unwrap();
+ let path = tmp.path().to_path_buf();
+ std::mem::forget(tmp);
+ let db = StateDb::open(&path).unwrap();
+ let graph = Arc::new(GraphStore::new());
+ (db, graph)
+}
+
+const SOURCE: &str = r#"
+ @prefix owl: <http://www.w3.org/2002/07/owl#> .
+ @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+ @prefix ex: <http://source.org/> .
+ ex:Animal a owl:Class ; rdfs:label "Animal" .
+ ex:Dog a owl:Class ; rdfs:label "Dog" ; rdfs:subClassOf ex:Animal .
+ ex:Cat a owl:Class ; rdfs:label "Cat" ; rdfs:subClassOf ex:Animal .
+"#;
+
+const TARGET: &str = r#"
+ @prefix owl: <http://www.w3.org/2002/07/owl#> .
+ @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
+ @prefix tg: <http://target.org/> .
+ tg:Animal a owl:Class ; rdfs:label "Animal" .
+ tg:Dog a owl:Class ; rdfs:label "Dog" ; rdfs:subClassOf tg:Animal .
+ tg:Cat a owl:Class ; rdfs:label "Cat" ; rdfs:subClassOf tg:Animal .
+"#;
+
+#[test]
+fn rrf_produces_normalised_confidence_in_unit_interval() {
+ let (db, graph) = setup();
+ let engine = AlignmentEngine::new(db, graph);
+
+ let result_str = engine
+ .align_with_fusion(SOURCE, Some(TARGET), 0.5, 0.0, true, "rrf")
+ .expect("align_with_fusion rrf");
+ let result: serde_json::Value = serde_json::from_str(&result_str).expect("json");
+
+ let candidates = result["candidates"].as_array().unwrap();
+ assert!(!candidates.is_empty(), "expected at least some candidates under RRF");
+
+ for c in candidates {
+ let conf = c["confidence"].as_f64().unwrap_or(-1.0);
+ assert!(
+ (0.0..=1.0).contains(&conf),
+ "RRF confidence must be in [0, 1]; got {} on {}",
+ conf,
+ c
+ );
+ }
+}
+
+#[test]
+fn rrf_preserves_per_signal_breakdown() {
+ // RRF only overwrites the top-level `confidence` field; the per-signal
+ // values must remain on each candidate so the caller can inspect them
+ // and so feedback signals stay computable.
+ let (db, graph) = setup();
+ let engine = AlignmentEngine::new(db, graph);
+
+ let result_str = engine
+ .align_with_fusion(SOURCE, Some(TARGET), 0.5, 0.0, true, "rrf")
+ .expect("align_with_fusion rrf");
+ let result: serde_json::Value = serde_json::from_str(&result_str).expect("json");
+
+ for c in result["candidates"].as_array().unwrap() {
+ let signals = &c["signals"];
+ assert!(signals.is_object(), "missing signals on RRF candidate: {}", c);
+ assert!(signals["label_similarity"].is_number());
+ assert!(signals["property_overlap"].is_number());
+ assert!(signals["parent_overlap"].is_number());
+ }
+}
+
+#[test]
+fn rrf_low_threshold_filters_after_rerank() {
+ // A high low_threshold should drop most candidates; we should be left with
+ // at most the top-ranked few from the RRF rerank.
+ let (db, graph) = setup();
+ let engine = AlignmentEngine::new(db, graph);
+
+ let result_str = engine
+ .align_with_fusion(SOURCE, Some(TARGET), 1.0, 0.7, true, "rrf")
+ .expect("align_with_fusion rrf");
+ let result: serde_json::Value = serde_json::from_str(&result_str).expect("json");
+
+ for c in result["candidates"].as_array().unwrap() {
+ let conf = c["confidence"].as_f64().unwrap_or(0.0);
+ assert!(
+ conf >= 0.7,
+ "RRF candidate below low_threshold should have been dropped; got conf={} on {}",
+ conf,
+ c
+ );
+ }
+}
+
+#[test]
+fn weighted_sum_default_unchanged() {
+ // Calling align_with_thresholds (the wrapper) should behave exactly as before —
+ // using the weighted-sum fusion under the hood. Confidence values for the
+ // matching Animal/Dog/Cat pairs should be > 0.85 (the default high threshold
+ // for the explicit pairs, since labels and parents both match).
+ let (db, graph) = setup();
+ let engine = AlignmentEngine::new(db, graph);
+
+ let result_str = engine
+ .align_with_thresholds(SOURCE, Some(TARGET), 0.5, 0.5, true)
+ .expect("align_with_thresholds weighted_sum");
+ let result: serde_json::Value = serde_json::from_str(&result_str).expect("json");
+
+ // At minimum the perfect-label matches should be in auto_applied.
+ let auto = result["auto_applied"].as_array().unwrap();
+ assert!(
+ auto.iter().any(|c| {
+ c["source_iri"].as_str().unwrap_or("").contains("Animal")
+ && c["target_iri"].as_str().unwrap_or("").contains("Animal")
+ }),
+ "weighted_sum should match Animal/Animal under back-compat thresholds; auto={:?}",
+ auto
+ );
+}
+
+#[test]
+fn rrf_and_weighted_sum_can_produce_different_orderings() {
+ // RRF and weighted_sum use different aggregation strategies; on this small
+ // test ontology they should both produce sensible orderings even though
+ // the absolute confidence numbers differ. Verify both produce non-empty
+ // auto_applied buckets at sensible thresholds.
+ let (db, graph) = setup();
+ let engine = AlignmentEngine::new(db, graph);
+
+ let rrf_str = engine
+ .align_with_fusion(SOURCE, Some(TARGET), 0.5, 0.0, true, "rrf")
+ .expect("rrf");
+ let ws_str = engine
+ .align_with_fusion(SOURCE, Some(TARGET), 0.5, 0.0, true, "weighted_sum")
+ .expect("weighted_sum");
+ let rrf: serde_json::Value = serde_json::from_str(&rrf_str).unwrap();
+ let ws: serde_json::Value = serde_json::from_str(&ws_str).unwrap();
+
+ assert!(!rrf["candidates"].as_array().unwrap().is_empty());
+ assert!(!ws["candidates"].as_array().unwrap().is_empty());
+
+ // Both should rank the same top pair (Animal↔Animal) first since RRF
+ // and weighted_sum agree on perfect matches.
+ let rrf_top = rrf["candidates"][0]["source_iri"].as_str().unwrap_or("");
+ let ws_top = ws["candidates"][0]["source_iri"].as_str().unwrap_or("");
+ assert!(
+ rrf_top.contains("Animal") || rrf_top.contains("Dog") || rrf_top.contains("Cat"),
+ "RRF top should be one of the perfect matches; got {}",
+ rrf_top
+ );
+ assert!(
+ ws_top.contains("Animal") || ws_top.contains("Dog") || ws_top.contains("Cat"),
+ "weighted_sum top should be one of the perfect matches; got {}",
+ ws_top
+ );
+}