Skip to content

Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)

[skip ci] Screen shared-clock associations with certified residual ceilings

Cinna Davis17bbd500164f
diff --git a/README.md b/README.md
index e5c8a45..8b58f9d 100644
--- a/README.md
+++ b/README.md
@@ -82,6 +82,8 @@ The target is a supported location estimate within **1,000 m**, preferably **100
Recording-time evidence matters: audio-relative seconds do not identify which day's flights to search. A defensible start time or bounded UTC window is needed; file modification and upload times do not establish the recording clock. The robust search considers all prepared independent flyover groups under one shared clock, with no default two- or three-event cap. Harmonics count once and unconfirmed proposals may remain unmatched. Explicit event, tile or candidate budgets can leave a search incomplete.
+The automatic search uses conservative score ceilings before completing every fit: incompatible residuals and an established Top-K cutoff can stop further group fitting at a shared clock. Deferred acoustic batches avoid computing groups those trials never need. [Search screening and real-data timing](docs/DATA_AND_SCALE.md#search) describe the bounds and their measured limits.
+
Recent search work makes omitted alternatives and failed model evaluations visible. A disk-backed ledger records scored candidates before ranking limits discard them, including the best discarded score. Robust clock trials distinguish evaluated fits and known poor fits from missing or unsupported work; one finite winning score cannot hide unknown competing trials. This accounts for the **evaluated finite grid**. It does not prove that a better location cannot exist between grid points. See [finite-search evidence](docs/FINITE_SEARCH_PRODUCER.md) and [clock-trial accounting](docs/SCORING_ASSESSMENT.md).
Missing events, incomplete flight coverage, competing locations or inadequate sensitivity evidence can require an **inconclusive** outcome. Ranking scores and sensitivity envelopes are not calibrated probabilities. The configured research CLI emits ranked hypotheses; it does not execute the complete upload acceptance workflow. See [Decision rules](docs/CONFIDENCE.md).
diff --git a/docs/DATA_AND_SCALE.md b/docs/DATA_AND_SCALE.md
index c13d207..495d732 100644
--- a/docs/DATA_AND_SCALE.md
+++ b/docs/DATA_AND_SCALE.md
@@ -91,6 +91,34 @@ coarse spacing is a computational budget, not a proof that an isolated minimum
could not be missed between cells. Keep this limitation visible in confidence
decisions. `top_k` limits retained candidates; raw scores are not probabilities.
+Robust subset scoring visits larger associations first **within each receiver
+point**, preserving fit-cache locality. Every accepted group contributes at most
+one nonnegative score unit. The current kth score from distinct, fully assessed
+modes therefore supplies a safe lower cutoff: an association with too few groups
+cannot enter Top-K. While fitting a shared-clock trial, validated residuals can
+tighten that ceiling or establish a residual rejection before the remaining
+groups are fitted. Comparisons are strict, so ties remain eligible. Acoustic
+batches for later groups are deferred until a surviving trial needs them.
+
+Unperformed trials remain unknown with explicit conservative bounds in the
+retention ledger. A competitive candidate receives full evaluation, preserving
+its original fitted diagnostics. The initial zero cutoff only rejects trials
+that cannot satisfy the existing residual limits. This does not narrow the
+automatic speed/range constraints or establish completeness between grid cells.
+
+Real-observation computation replays for PR #152 measured the following; neither
+is an independent localization-accuracy test:
+
+| Replay | Source fits, before → after | Acoustic calls, before → after | Retained results |
+| --- | --- | --- | --- |
+| 169-point refinement neighborhood, 1,859 associations | 14,196 → 14,196 | 3,380 → 1,963 | All 100 identical, including diagnostics and best discarded score |
+| 256 evenly spaced associations from 10,549 at one saved receiver point | 30,361 → 19,629 | 879 → 689 | All 92 identical, with conservative omitted-trial bounds |
+
+The first replay took 20.13 → 15.86 seconds in one paired run. The second saved
+about 8–28% wall time across paired runs under varying machine load; operation
+counts were stable. These measurements do not predict nationwide runtime. The
+group-count ceiling alone did not show a reliable speedup on the existing cache.
+
Manifest and tile receipts hash configuration, input index files, event
measurements, implementation and terrain files. Interrupted jobs reuse completed
tiles with the same identity. Changed inputs require a new output directory.
@@ -127,12 +155,21 @@ than converting scores into that accuracy claim.
## Checks
+Use actual recorded observations; do not run the historical generated-data suites.
+
```bash
-PYTHONPATH=src /path/to/python -m pytest -q tests/test_data_access.py tests/test_tiled_search.py
+DOPPLERGEIST_REAL_CPA_FIXTURE=/path/to/real-cpa-diagnostic.json \
+ PYTHONPATH=src python -m pytest -q \
+ tests/test_real_retention_floor.py tests/test_real_score_ceiling_validation.py
+
+PYTHONPATH=src OPENBLAS_NUM_THREADS=1 python scripts/benchmark_acoustic_sieve.py \
+ --fixture /path/to/real-cpa-diagnostic.json --baseline 72cdebb \
+ --output results/my-new-sieve-comparison
```
-Tests include split archive streaming, byte-range/checksum failures, explicit
-offline errors, jets/unknowns, gap/no-extrapolation behavior, whole-segment queries,
-datum conversion, tiled/untiled agreement at a boundary, deterministic repeated
-approach preservation, checkpoint reuse, and visible partial/budget failures.
-Scaling experiments and location-accuracy evidence must be labelled separately.
+The diagnostic contains measured event ridges, observed `cpa_rows`, one receiver
+`point` and the local `real_index` path. The benchmark can override that path with
+`--index`; no media or native index is bundled. It compares isolated baseline
+matching/scoring modules with the current fitter and requires identical shared
+geometry. Timing includes scoring with a bounded fit cache. Scaling experiments
+and location-accuracy evidence must be labelled separately.
diff --git a/scripts/benchmark_acoustic_sieve.py b/scripts/benchmark_acoustic_sieve.py
new file mode 100644
index 0000000..5d87f69
--- /dev/null
+++ b/scripts/benchmark_acoustic_sieve.py
@@ -0,0 +1,129 @@
+"""Compare acoustic screening on a saved real-observation CPA diagnostic.
+
+The fixture contains events with measured ridges, cpa_rows, one receiver point,
+and real_index (a local FlightIndex). This is a computation replay, not a location
+accuracy test. Neither audio nor aircraft observations are generated here.
+"""
+import argparse
+import json
+import math
+from pathlib import Path
+import subprocess
+import sys
+import time
+import types
+from types import SimpleNamespace
+
+import numpy as np
+import pandas as pd
+
+from dopplergeist import matching, search, tiled_search
+from dopplergeist.data_access import FlightIndex
+from dopplergeist.finite_search import bounded_assessment, classified_assessment
+from dopplergeist.geometry import Track
+
+
+parser = argparse.ArgumentParser(description=__doc__)
+parser.add_argument("--fixture", type=Path, required=True)
+parser.add_argument("--index", type=Path, help="Override the fixture FlightIndex path")
+parser.add_argument("--baseline", default="origin/main", help="Git revision before the sieve change")
+parser.add_argument("--associations", type=int, default=256, help="Deterministically spaced real associations")
+parser.add_argument("--output", type=Path, required=True, help="New result directory; never overwritten")
+args = parser.parse_args()
+if args.associations < 1:
+ parser.error("--associations must be positive")
+
+base_commit = subprocess.check_output(["git", "rev-parse", args.baseline], text=True).strip()
+baseline_geometry = subprocess.check_output(
+ ["git", "show", f"{base_commit}:src/dopplergeist/geometry.py"], text=True)
+geometry = Path(__file__).resolve().parents[1] / "src/dopplergeist/geometry.py"
+if baseline_geometry != geometry.read_text():
+ parser.error("The shared geometry must match the baseline for this focused comparison")
+
+# Isolate both baseline modules, so they cannot accidentally use the new fitter.
+for name in ("matching", "search"):
+ module = types.ModuleType("dopplergeist._point_baseline_" + name)
+ module.__package__ = "dopplergeist"
+ sys.modules[module.__name__] = module
+ source = subprocess.check_output(
+ ["git", "show", f"{base_commit}:src/dopplergeist/{name}.py"], text=True)
+ source = source.replace("from .matching import ", "from ._point_baseline_matching import ")
+ exec(compile(source, "baseline-" + name + ".py", "exec"), module.__dict__)
+base_matching = sys.modules["dopplergeist._point_baseline_matching"]
+base_search = sys.modules["dopplergeist._point_baseline_search"]
+
+fixture = json.loads(args.fixture.read_text())
+events, measurements = matching.prepare_events(fixture["events"])
+all_candidates, _ = matching.association_candidates(
+ pd.DataFrame(fixture["cpa_rows"]), events, max_candidates=20000)
+if not all_candidates or any(row["point_id"] != fixture["point"]["point_id"] for row in all_candidates):
+ parser.error("Need actual associations for exactly the fixture receiver point")
+# Fixed spacing through the associations; selection does not use their scores.
+indices = np.linspace(0, len(all_candidates) - 1, min(args.associations, len(all_candidates)), dtype=int)
+candidates = [all_candidates[index] for index in indices]
+identities = sorted({entry["flight_id"] for row in candidates for entry in row["passes"]})
+tracks = {track.id: track for track in FlightIndex(args.index or fixture["real_index"]).lookup(identities)}
+point = SimpleNamespace(**fixture["point"], x=0.0, y=0.0)
+args.output.mkdir(parents=True, exist_ok=False)
+runs, retained, all_rows = [], [], []
+
+for name, backend, scorer in [("baseline", base_matching, base_search), ("screened", matching, search)]:
+ stats = {"source_fits": 0, "acoustic_calls": 0, "acoustic_samples": 0}
+ rows = []
+ leader = tiled_search._RetainedScoreFloor(100, 0.0)
+ original_fit = backend._fit_ridge_uncached
+ original_acoustic = Track.acoustic
+
+ def counted_fit(*args, **kwargs):
+ stats["source_fits"] += 1
+ return original_fit(*args, **kwargs)
+
+ def counted_acoustic(track, receive, *args, **kwargs):
+ stats["acoustic_calls"] += 1
+ stats["acoustic_samples"] += np.asarray(receive).size
+ return original_acoustic(track, receive, *args, **kwargs)
+
+ @backend._cache_ridge_fits
+ def run():
+ for candidate in sorted(candidates, key=lambda row: -len(row["event_indices"])):
+ settings = {"score_floor": leader.value} if name == "screened" else {}
+ row = scorer.score_candidate(candidate, point, tracks, events, measurements,
+ mode="robust_ridge", **settings)
+ row["pass_emission_times_s"] = [float(entry["emission_time"]) for entry in candidate["passes"]]
+ row["core_assessment_complete"] = (classified_assessment(row) and row["score"] is not None
+ and math.isfinite(row["score"]))
+ rows.append(row)
+ if name == "screened":
+ leader.observe(row)
+
+ backend._fit_ridge_uncached = counted_fit
+ Track.acoustic = counted_acoustic
+ began, cpu = time.perf_counter(), time.process_time()
+ try:
+ run()
+ finally:
+ backend._fit_ridge_uncached = original_fit
+ Track.acoustic = original_acoustic
+ wall, used_cpu = time.perf_counter() - began, time.process_time() - cpu
+ finite = [row for row in rows if row["score"] is not None and math.isfinite(row["score"])]
+ kept = tiled_search.merge_candidates(finite, 100)
+ retained.append(kept)
+ all_rows.append(rows)
+ result = dict(name=name, seconds=wall, cpu_seconds=used_cpu, **stats,
+ associations=len(candidates), finite=len(finite), retained=len(kept),
+ bounded=sum(bounded_assessment(row) for row in rows),
+ unknown=sum(row["scoring_assessment"]["unknown_trials"] for row in rows))
+ runs.append(result)
+ print(json.dumps(result), flush=True)
+
+assert retained[0] == retained[1], "Retained diagnostics changed"
+for baseline, screened in zip(*all_rows):
+ if baseline["score"] is not None and math.isfinite(baseline["score"]) and bounded_assessment(screened):
+ upper = screened["scoring_assessment"]["unknown_score_upper_bound"]
+ actual = screened["score"] if screened["score"] is not None else -math.inf
+ assert baseline["score"] <= max(actual, -math.inf if upper is None else upper)
+summary = dict(baseline_commit=base_commit, total_associations=len(all_candidates),
+ sampled_associations=len(candidates), same_retained=True, runs=runs,
+ scope="Actual associations at one saved receiver point; computation only, no location accuracy claim")
+(args.output / "comparison.json").write_text(json.dumps(summary, indent=2) + "\n")
+print("Same retained results and conservative upper bounds", flush=True)
diff --git a/src/dopplergeist/matching.py b/src/dopplergeist/matching.py
index 77cd2b5..7c27a95 100644
--- a/src/dopplergeist/matching.py
+++ b/src/dopplergeist/matching.py
@@ -449,13 +449,18 @@ def score_ridges(tracks, events, measurements, observer, recording_start_s, c=34
def batch_acoustic_trials(tracks, measurements, observer, starts, c=343.0, max_samples=262144):
"""Evaluate repeated receive times together, falling back on support errors."""
+ return list(_iter_acoustic_trials(tracks, measurements, observer, starts, c, max_samples))
+
+
+def _iter_acoustic_trials(tracks, measurements, observer, starts, c=343.0, max_samples=262144):
+ """Defer each group's batch until a surviving trial actually needs it."""
from .geometry import AcousticSupportError
starts = np.asarray(starts, dtype=float)
if starts.ndim != 1 or not np.all(np.isfinite(starts)):
- return [None] * len(measurements)
+ yield from (None for _ in measurements)
+ return
remaining = max_samples
- batches = []
for track, measurement in zip(tracks, measurements):
spec, ridge, weight, _ = measurement
times, ridge, weight = (np.asarray(a, dtype=float) for a in (spec.time, ridge, weight))
@@ -464,26 +469,25 @@ def batch_acoustic_trials(tracks, measurements, observer, starts, c=343.0, max_s
# Negative or lower-precision epochs retain the per-trial path.
if (track.time[0] < 0 or
np.spacing(track.time[0]) <= 2 * _EMISSION_ITERATION_TOLERANCE_S):
- batches.append(None)
+ yield None
continue
if times.shape != ridge.shape or times.shape != weight.shape or times.ndim != 1:
- batches.append(None)
+ yield None
continue
measured = np.isfinite(times) & np.isfinite(ridge) & (weight > 0)
size = len(starts) * int(measured.sum())
if measured.sum() < 10 or not np.all(np.isfinite(weight)) or np.any(weight < 0) or size > remaining:
- batches.append(None)
+ yield None
continue
receive = starts[:, None] + times[measured][None, :]
unique, inverse = np.unique(receive, return_inverse=True)
try:
factor, emission, _ = track.acoustic(unique, observer, c)
except AcousticSupportError:
- batches.append(None)
+ yield None
continue
- batches.append((factor[inverse].reshape(receive.shape), emission[inverse].reshape(receive.shape)))
remaining -= size
- return batches
+ yield factor[inverse].reshape(receive.shape), emission[inverse].reshape(receive.shape)
def refine_candidate(candidate, tracks, events, measurements, terrain, radius_m=500.0,
diff --git a/src/dopplergeist/search.py b/src/dopplergeist/search.py
index 5aa43b4..7b871c6 100644
--- a/src/dopplergeist/search.py
+++ b/src/dopplergeist/search.py
@@ -379,10 +379,61 @@ def _association_trial_bound(result, events):
return min(maximum, upper)
+def _validate_score_ceiling_inputs(events, measurements):
+ """Do not hide malformed fit inputs behind an unperformed-fit certificate."""
+ from .matching import _frequency_scale, _ridge_fit_inputs
+ for event, measurement in zip(events, measurements):
+ value = _frequency_scale(event)
+ if not isinstance(value, (int, float, np.number)) or isinstance(value, (bool, np.bool_)):
+ raise ValueError("Frequency loss scales must be finite and positive")
+ try:
+ scale = float(value)
+ except (TypeError, ValueError, OverflowError) as error:
+ raise ValueError("Frequency loss scales must be finite and positive") from error
+ if not np.isfinite(scale) or scale <= 0 or not np.isfinite(3*scale):
+ raise ValueError("Frequency loss scales must be finite and positive")
+ _ridge_fit_inputs(measurement, scale, event.get("quality", {}).get("source_drift_fraction", 0.02))
+ bounds = event.get("f0_range_hz")
+ if (not isinstance(bounds, (list, tuple)) or len(bounds) != 2
+ or not all(isinstance(value, (int, float, np.number)) and not isinstance(value, (bool, np.bool_))
+ and np.isfinite(value) for value in bounds)
+ or not 0 < bounds[0] <= bounds[1]):
+ raise ValueError("Source frequency bounds must be finite, positive and ordered")
+
+
+def _screen_association_trial(tracks, events, measurements, observer, start, c, acoustic_at, floor):
+ """Fit groups until a conservative residual ceiling cannot reach the floor.
+
+ Unperformed groups each contribute at most one. The existing validated-fit
+ bound rounds upward and never uses missing evidence as a rejection.
+ """
+ from .matching import score_ridges
+ diagnostics = []
+ missing = {"score": -np.inf, "f0_hz": np.nan, "support_fraction": 0.0}
+ for index, (track, event, measurement) in enumerate(zip(tracks, events, measurements)):
+ acoustic = None if acoustic_at is None else [acoustic_at(index)]
+ single = score_ridges([track], [event], [measurement], observer, start, c, acoustic_results=acoustic)
+ diagnostics.extend(single["event_diagnostics"])
+ remaining = len(events) - len(diagnostics)
+ if remaining:
+ partial = diagnostics + [missing] * remaining
+ bound = _association_trial_bound({
+ "score_kind": "matched_group_support_minus_normalized_residual",
+ "event_scores": [d["score"] for d in partial],
+ "f0_hz": [d["f0_hz"] for d in partial], "event_diagnostics": partial}, events)
+ if bound is not None and bound < floor:
+ return None, bound
+ return {"score": float(sum(d["score"] for d in diagnostics)),
+ "event_scores": [d["score"] for d in diagnostics],
+ "f0_hz": [d["f0_hz"] for d in diagnostics], "event_diagnostics": diagnostics,
+ "score_kind": "negative_robust_ridge_loss_hz"}, None
+
+
def score_candidate(candidate, point, tracks, events, measurements, mode="brightness",
time_origin=None, time_sweep_s=5.0, time_step_s=0.5, c=343.0,
score_floor=None):
"""Score one hypothesis; a bounded score-floor result has no numeric score."""
+ all_events, all_measurements = events, measurements
association = "event_indices" in candidate
association_fields = {}
if association:
@@ -412,6 +463,7 @@ def score_candidate(candidate, point, tracks, events, measurements, mode="bright
raise ValueError("score_floor must be finite") from error
if not np.isfinite(score_floor):
raise ValueError("score_floor must be finite")
+ _validate_score_ceiling_inputs(events, measurements)
observer = ecef(point.lat,point.lon,point.elev_m)
# Derive one recording start from the first event; all other events retain
# their true absolute geometry/time. This implements the locked offset.
@@ -443,39 +495,49 @@ def score_candidate(candidate, point, tracks, events, measurements, mode="bright
"empty_clock_domain": len(shifts) == 0}
reasons = set()
unknown_bounds = []
+ partial_prunes = 0
# Every accepted group contributes at most one; a strict comparison keeps ties eligible.
floor_pruned = bool(score_floor is not None and len(shifts) > 0 and len(events) < score_floor)
if floor_pruned:
- # A ceiling must not turn malformed fit inputs into a successful assessment.
- from .matching import _frequency_scale, _ridge_fit_inputs
- for event, measurement in zip(events, measurements):
- value = _frequency_scale(event)
- if isinstance(value, (bool, np.bool_)):
- raise ValueError("Frequency loss scales must be finite and positive")
- try:
- scale = float(value)
- except (TypeError, ValueError, OverflowError) as error:
- raise ValueError("Frequency loss scales must be finite and positive") from error
- if not np.isfinite(scale) or scale <= 0 or not np.isfinite(3*scale):
- raise ValueError("Frequency loss scales must be finite and positive")
- _ridge_fit_inputs(measurement, scale, event.get("quality", {}).get("source_drift_fraction", 0.02))
best["score"] = None
assessment["unknown_trials"] = len(shifts)
reasons.add("matched_group_count_below_score_floor")
selected_tracks = ([tracks[p["flight_id"]] for p in candidate["passes"]]
if mode == "robust_ridge" and not floor_pruned else None)
batches = None
+ lazy_batches = None
if mode == "robust_ridge" and not floor_pruned and len(shifts) > 1:
- from .matching import batch_acoustic_trials
- batches = batch_acoustic_trials(selected_tracks, measurements, observer, anchor + shifts, c)
+ from .matching import _iter_acoustic_trials, batch_acoustic_trials
+ if association and score_floor is not None:
+ lazy_batches = iter(_iter_acoustic_trials(selected_tracks, measurements, observer, anchor + shifts, c))
+ batches = []
+ else:
+ batches = batch_acoustic_trials(selected_tracks, measurements, observer, anchor + shifts, c)
for trial, shift in enumerate(() if floor_pruned else shifts):
if mode == "robust_ridge":
from .matching import score_ridges
- precomputed = ([None if batch is None else (batch[0][trial], batch[1][trial]) for batch in batches]
- if batches is not None else None)
+ def acoustic_at(index):
+ if lazy_batches is not None:
+ while len(batches) <= index:
+ batches.append(next(lazy_batches))
+ batch = batches[index]
+ return None if batch is None else (batch[0][trial], batch[1][trial])
+ precomputed = ([acoustic_at(index) for index in range(len(events))]
+ if batches is not None and lazy_batches is None else None)
try:
- result = score_ridges(selected_tracks, events, measurements, observer, anchor+shift, c,
- acoustic_results=precomputed)
+ if association and score_floor is not None:
+ result, upper = _screen_association_trial(selected_tracks, events, measurements, observer,
+ anchor+shift, c, acoustic_at if batches is not None else None,
+ score_floor)
+ if result is None:
+ partial_prunes += 1
+ assessment["unknown_trials"] += 1
+ reasons.add("partial_group_score_below_retention_floor")
+ unknown_bounds.append(upper)
+ continue
+ else:
+ result = score_ridges(selected_tracks, events, measurements, observer, anchor+shift, c,
+ acoustic_results=precomputed)
except AcousticSupportError as error:
assessment["unknown_trials"] += 1
reasons.add(str(error))
@@ -509,6 +571,11 @@ def score_candidate(candidate, point, tracks, events, measurements, mode="bright
best = {"score":total,"event_scores":scores,"f0_hz":f0s,
"recording_start_time_s":float(anchor+shift),"time_adjustment_s":float(shift)}
if mode == "robust_ridge":
+ if partial_prunes and np.isfinite(best["score"]) and best["score"] >= score_floor:
+ # Competitive modes retain their complete original diagnostics. Cached
+ # fits are reused; skipped clocks are evaluated rather than relabelled.
+ return score_candidate(candidate, point, tracks, all_events, all_measurements, mode,
+ time_origin, time_sweep_s, time_step_s, c)
if not len(shifts):
reasons.add("no_clock_trials")
assessment["complete"] = bool(len(shifts) and assessment["unknown_trials"] == 0)
@@ -529,6 +596,9 @@ def score_candidate(candidate, point, tracks, events, measurements, mode="bright
unknown_score_upper_bound=max(finite_bounds) if finite_bounds else None,
unknown_score_bound_kind="matched_group_support_minus_normalized_residual",
unknown_score_bound_method="nonnegative_loss_and_validated_event_fits")
+ if partial_prunes:
+ assessment["retention_score_floor"] = score_floor
+ assessment["retention_pruned_trials"] = partial_prunes
best["scoring_assessment"] = assessment
return dict(best,**association_fields,point_id=int(point.point_id),lat=float(point.lat),lon=float(point.lon),elev_m=float(point.elev_m),
x=float(point.x),y=float(point.y),flight_ids=[p["flight_id"] for p in candidate["passes"]])
diff --git a/src/dopplergeist/tiled_search.py b/src/dopplergeist/tiled_search.py
index 866495b..05baead 100644
--- a/src/dopplergeist/tiled_search.py
+++ b/src/dopplergeist/tiled_search.py
@@ -270,7 +270,9 @@ def _score(points, tracks, events, measurements, budget, parameters, filters, st
raise DataUnavailable("Tile candidate memory budget exceeded; reduce tile_cells")
candidates, counts = pair_candidates(table, events, **kwargs)
screened = subset and parameters.get("mode") == "robust_ridge"
- cutoff = _RetainedScoreFloor(budget.top_k, score_floor) if screened else None
+ # Every accepted association has a nonnegative score, even before K modes
+ # exist. A strict zero floor can reject only residual-incompatible trials.
+ cutoff = _RetainedScoreFloor(budget.top_k, 0.0 if score_floor is None else max(0.0, score_floor)) if screened else None
if screened:
# Find competitive multi-group modes before spending fits on singletons.
# All remaining modes still receive a score or a persisted score bound.
@@ -349,7 +351,7 @@ def _score(points, tracks, events, measurements, budget, parameters, filters, st
bounds.append({"bound_id": "bound-" + content_hash(detail), "detail": detail,
"groups_count": len(detail["event_indices"]),
"upper_score": detail["scoring_assessment"]["unknown_score_upper_bound"]})
- retention_pruned += detail["scoring_assessment"]["unknown_score_bound_method"] == "group_count_ceiling"
+ retention_pruned += "retention_score_floor" in detail["scoring_assessment"]
if finite_score:
ranked.append(row)
if cutoff is not None:
diff --git a/tests/test_real_score_ceiling_validation.py b/tests/test_real_score_ceiling_validation.py
index baa471d..86c6cf8 100644
--- a/tests/test_real_score_ceiling_validation.py
+++ b/tests/test_real_score_ceiling_validation.py
@@ -11,6 +11,7 @@ import pytest
from dopplergeist.data_access import FlightIndex
from dopplergeist.finite_search import bounded_assessment
from dopplergeist.matching import association_candidates, prepare_events
+from dopplergeist import matching
from dopplergeist.search import score_candidate
@@ -60,3 +61,46 @@ def test_real_singleton_retains_source_drift_parameter_errors():
score_candidate(singleton, point, tracks, configured, measurements,
mode="robust_ridge", score_floor=floor)
+
+def test_real_partial_residual_ceiling_skips_fits_without_claiming_support(monkeypatch):
+ source = os.environ.get("DOPPLERGEIST_REAL_CPA_FIXTURE")
+ if not source:
+ pytest.skip("set DOPPLERGEIST_REAL_CPA_FIXTURE to the real reference diagnostic")
+ fixture = json.loads(Path(source).read_text())
+ events, measurements = prepare_events(fixture["events"])
+ candidates, _ = association_candidates(pd.DataFrame(fixture["cpa_rows"]), events, max_candidates=20000)
+ pair = next(candidate for candidate in candidates if len(candidate["event_indices"]) == 2)
+ tracks = {track.id: track for track in FlightIndex(fixture["real_index"]).lookup(
+ [entry["flight_id"] for entry in pair["passes"]])}
+ point = SimpleNamespace(**fixture["point"], x=0.0, y=0.0)
+ original = matching._fit_ridge_uncached
+ calls = []
+
+ def counted(*args, **kwargs):
+ calls.append(1)
+ return original(*args, **kwargs)
+
+ monkeypatch.setattr(matching, "_fit_ridge_uncached", counted)
+ baseline = score_candidate(pair, point, tracks, events, measurements, mode="robust_ridge", time_sweep_s=0)
+ full_fits = len(calls)
+ calls.clear()
+ screened = score_candidate(pair, point, tracks, events, measurements, mode="robust_ridge",
+ time_sweep_s=0, score_floor=2.0)
+ assessment = screened["scoring_assessment"]
+ assert len(calls) < full_fits
+ assert bounded_assessment(screened)
+ assert assessment["complete"] is False
+ assert assessment["retention_pruned_trials"] == assessment["unknown_trials"]
+ upper = assessment["unknown_score_upper_bound"]
+ if upper is None:
+ assert assessment["bounded_unknown_rejections"] == assessment["unknown_trials"]
+ upper = -float("inf")
+ assert baseline["score"] <= upper < 2.0
+
+ # A skipped group's invalid model parameter still fails, using the same
+ # recorded observations and candidate rather than generated field data.
+ configured = copy.deepcopy(events)
+ configured[pair["event_indices"][1]]["quality"]["source_drift_fraction"] = 0.2
+ with pytest.raises(ValueError, match="drift fraction"):
+ score_candidate(pair, point, tracks, configured, measurements, mode="robust_ridge",
+ time_sweep_s=0, score_floor=2.0)