Skip to content

Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)

Bound CPA motion-support loss to irrelevant native intervals

Cinna Davis165c9e031363
diff --git a/docs/MODEL_SUPPORT_COVERAGE.md b/docs/MODEL_SUPPORT_COVERAGE.md
index 51b4dd2..c98ab90 100644
--- a/docs/MODEL_SUPPORT_COVERAGE.md
+++ b/docs/MODEL_SUPPORT_COVERAGE.md
@@ -25,6 +25,24 @@ establish geographic coverage. The HTTP evidence verifier checks the recorded
inequality against the search clock and stage totals; it does not independently
reload the original aircraft index to verify the recorded emission endpoints.
+After these clock exclusions, a tiled CPA stage can prove every unsupported native
+interval spatially irrelevant to its actual receiver set. Both the native spline
+hulls and every original CPA sample edge that touches those intervals must pass
+strict distance bounds. The original spline, sampling lattice, global sample
+step and identity remain intact. See [motion support](MOTION_SUPPORT.md) for the
+proof and numerical limits. Failed proofs keep the original whole-track unknown
+envelope. Successful proofs appear as `interval_supported_cpa_tracks` in stage
+diagnostics, with their original invalid intervals, receiver box, sound speed,
+sampling step and distance bounds. The report counts these evaluations in
+`interval_supported_cpa_track_evaluations` without counting them as unsupported
+CPA work. Acoustic trials still check arrivals from those same invalid intervals
+and remain unknown when their receive envelopes overlap.
+
+The HTTP verifier checks each recorded interval-proof inequality, configuration
+scope, coarse checkpoint, and aggregate count. It does not reconstruct the source
+spline or receiver grid from the aircraft index. This proof narrows a CPA loss
+envelope; it does not establish source correctness or complete spatial coverage.
+
The additive `model_support` report contains these integer work counts:
- `unsupported_cpa_track_evaluations`: trajectory/stage CPA failures, including
diff --git a/docs/MOTION_SUPPORT.md b/docs/MOTION_SUPPORT.md
index 42593dd..9d7a6e4 100644
--- a/docs/MOTION_SUPPORT.md
+++ b/docs/MOTION_SUPPORT.md
@@ -26,6 +26,33 @@ scope as lost model coverage. It must keep the search partial and finite evidenc
incomplete. Calling the CPA function directly still propagates the typed failure;
it does not silently drop the source.
+The tiled search can make a narrower spatial proof after its existing whole-track
+clock exclusions. Every unsupported native interval's cubic Bezier position box
+must lie strictly outside the receiver box by more than the search radius. In
+addition, both endpoints of every original CPA sampling edge that touches an
+unsupported interval must lie strictly outside the original KD query radius,
+`search_radius + 2 * global_max_sample_step`. Such an edge cannot enter the
+sampled-minimum or endpoint-refinement path. This second condition rejects a
+native interval whose own curve is distant but whose crossing sample edge is
+not proved irrelevant.
+
+The proof uses the original Track, spline, timestamps, sampling lattice, global
+maximum sample step and flight ID. It does not split the trajectory or create
+new segment endpoints. The receiver box encloses the actual stage receivers;
+proof reuse is limited to that box, the same sound speed and sampling step, and
+no larger radius. A one-micrometre plus scale-dependent floating-point margin is
+subtracted from distance lower bounds, followed by outward rounding. Time-edge
+overlap includes the existing solver-scale timestamp tolerance. Equality,
+nonfinite bounds, modified motion methods or any failed proof retain the original
+whole-track unsupported result. These are conservative floating-point checks,
+not formally certified interval arithmetic.
+
+Only this bounded tiled CPA path opts into the proof. Direct public CPA calls
+retain whole-source validation. A successful spatial proof records the excluded
+native intervals and bounding inequalities as `interval_supported_cpa_tracks`;
+it supplies no acoustic-arrival exemption. The direct acoustic check below still
+uses every original unsupported interval.
+
Direct `Track.acoustic` scoring checks only possible arrivals from unsupported
native intervals. Cubic Bezier position boxes bound minimum/maximum range to the
observer; adding those travel-time bounds to the native interval encloses all
@@ -59,3 +86,45 @@ acoustic fit, a 300 m/s control, an unaffected separate event, actual sparse/den
CPA failures, sound-speed sensitivity, conservative arrival envelopes and a dense
continuous-speed oracle. W2's outer isolation must additionally verify an actual
bad-source plus clean-source search and all-invalid abstention.
+
+## Real interval-exclusion checks
+
+`tests/test_real_motion_interval_cpa.py` uses the unchanged released reference
+archive and the measured ridge workload in `tests/data/real-search-workloads.json`.
+Set `DOPPLERGEIST_REAL_REFERENCE_ROOT` to its restored checkout and
+`DOPPLERGEIST_REAL_FLIGHT_INDEX` to the matching existing regional index, then run
+that test file. These are development receiver hypotheses, not microphone ground
+truth or complete aircraft coverage.
+
+At the two recorded hypotheses, the proof admits 26 and 15 track/receiver pairs;
+36 and 11 remain unsupported. The resulting 27 and 20 CPA rows exactly match the
+unchanged sparse proposer with only the whole-track validator bypassed for this
+diagnostic comparison. That bypass supplies no physical support; the spatial
+inequalities supply the narrower CPA support. The measured-ridge offset workload
+keeps the same 28 ranked candidates, scores and 7,623 unknown clock trials while
+recovering two CPA rows from one previously excluded track. Separate checks retain
+two actual first-interval passages and a final-interval passage on unchanged
+tracks. Acoustic arrivals from invalid native intervals remain unknown, including
+the arrival-envelope endpoints. The finite ledgers
+retain the same 28 scored modes and 168 unknown-score bound envelopes, with
+identical validated stage checksums.
+
+The recovered track contributes no retained mode or unknown-score envelope within
+that original limited clock window. A separate component check calls
+`Track.acoustic` at all 47 recovered CPA receive times and obtains finite
+supported output. The same recovered track also reaches the ordinary free-clock
+scorer with the unchanged measured ridges: 35 associations produce 20 finite
+hypotheses, while 253 clock trials remain unknown. Rows and trial assessments
+exactly match the diagnostic original CPA path. This check uses the normal clock
+hypothesis generator outside the original start window; it does not establish
+the recording clock, aircraft association or receiver location.
+
+The independent two-second dense proposer returns 25 and 18 rows. Its minima
+rule uses the sequence of per-edge minimum distances, which can erase a high
+shared vertex between nearby passes. The four extra sparse proposals are
+interior passes, not new endpoints. A 0.125-second continuous-spline reference
+finds a nearby minimum for every sparse row, with maximum observed differences
+of 1.20 seconds and 16.03 metres. It also finds additional narrow minima. These
+comparisons do not claim equality of the existing dense and sparse proposers or
+continuous-minimum completeness. The increment preserves the existing finite
+proposal algorithm and its known sampling limits.
diff --git a/scripts/verify_upload_http.py b/scripts/verify_upload_http.py
index 98eb120..c17bd04 100644
--- a/scripts/verify_upload_http.py
+++ b/scripts/verify_upload_http.py
@@ -378,6 +378,53 @@ def valid_clock_exclusion(item, configuration):
return False
+def valid_interval_cpa_support(item, configuration):
+ """Check recorded spatial inequalities; source splines are not reloaded."""
+ from decimal import Decimal, localcontext
+
+ def number(value):
+ if type(value) not in (int, float) or not math.isfinite(value):
+ raise ValueError("invalid interval bound number")
+ return Decimal.from_float(float(value))
+
+ try:
+ if (not isinstance(item, dict) or item.get("method") != "outside_native_hulls_and_sample_edges"
+ or not isinstance(item["track_id"], str) or not item["track_id"]
+ or not all(type(item.get(key)) is int and item[key] > 0
+ for key in ("sample_count", "affected_sample_edges", "receiver_point_count"))
+ or not item["affected_sample_edges"] < item["sample_count"]
+ or item["sample_count"] > configuration["budget"]["max_track_samples_per_tile"]):
+ return False
+ with localcontext() as ctx:
+ ctx.prec = 1100
+ c = number(configuration["scoring"].get("c", 343.0))
+ radius = max(number(event["slant_range_m"][1]) for event in configuration["events"])
+ step = number(configuration["budget"]["cpa_step_s"])
+ maximum_step = number(item["max_sample_step_m"])
+ query_radius = number(item["query_radius_m"])
+ if (c <= 0 or radius <= 0 or step <= 0 or maximum_step < 0
+ or number(item["sound_speed_mps"]) != c or number(item["max_distance_m"]) != radius
+ or number(item["sample_step_s"]) != step or query_radius < radius + 2*maximum_step
+ or number(item["hull_distance_lower_m"]) <= radius
+ or number(item["edge_distance_lower_m"]) <= query_radius):
+ return False
+ first, last = map(number, item["emission_time_range_s"])
+ intervals = item["emission_intervals_s"]
+ if first >= last or not isinstance(intervals, list) or not intervals:
+ return False
+ previous = first
+ for pair in intervals:
+ start, end = map(number, pair)
+ if not previous <= start < end <= last:
+ return False
+ previous = end
+ lower, upper = item["observer_box_ecef_m"]
+ return (len(lower) == len(upper) == 3
+ and all(number(lo) <= number(hi) for lo, hi in zip(lower, upper)))
+ except (KeyError, IndexError, TypeError, ValueError, OverflowError):
+ return False
+
+
def inspect_finite_search(artifact, read, search, configuration, refinement):
"""Audit recorded retention read-only; physical trials are not recomputed."""
import sqlite3
@@ -418,7 +465,7 @@ def inspect_finite_search(artifact, read, search, configuration, refinement):
totals, names, originals = {}, {}, {}
evaluated = discarded = coarse_unknown = 0
coarse_trials = dict(evaluated_trials=0, rejected_trials=0, unknown_trials=0, unreported_trial_envelopes=0)
- bounded_count = bounded_rejections = clock_excluded = 0
+ bounded_count = bounded_rejections = clock_excluded = interval_supported = 0
bound = unknown_bound = None
bound_fields = ("bounded_unknown_envelopes", "bounded_unknown_rejections", "best_unknown_score_upper_bound")
bound_contract = any(name in finite for name in bound_fields) or "model_assessment_complete" in search
@@ -568,7 +615,7 @@ def inspect_finite_search(artifact, read, search, configuration, refinement):
coarse_unknown += unknown
if bound_contract and phase in {"coarse", "fine"}:
support_path = path.parent / "support" / (name.replace(":", "-") + ".json")
- details, unsupported, excluded = [], [], []
+ details, unsupported, excluded, local_support = [], [], [], []
if support_path.exists():
support_record = read("search/support/" + support_path.name)
details, unsupported = (support_record.get(key) for key in
@@ -579,6 +626,11 @@ def inspect_finite_search(artifact, read, search, configuration, refinement):
excluded = support_record.get("clock_excluded_unsupported_tracks", [])
require(isinstance(excluded, list) and all(valid_clock_exclusion(item, configuration) for item in excluded),
"Invalid unsupported-track clock exclusion evidence")
+ local_support = support_record.get("interval_supported_cpa_tracks", [])
+ require(isinstance(local_support, list)
+ and all(valid_interval_cpa_support(item, configuration) for item in local_support)
+ and len({item["track_id"] for item in local_support}) == len(local_support),
+ "Invalid interval CPA motion-support evidence")
certificates = {"bound-" + digest(detail): detail for detail in details
if bounded_finite_clock_assessment(detail, finite["max_event_groups"])}
require(stage_bounds == certificates, "Finite bound ledger omits or invents a recorded candidate assessment")
@@ -587,12 +639,14 @@ def inspect_finite_search(artifact, read, search, configuration, refinement):
require(counts.get("unclassified_candidate_assessments", []) == details
and counts.get("unsupported_cpa_tracks", []) == unsupported
and counts.get("clock_excluded_unsupported_tracks", []) == excluded
+ and counts.get("interval_supported_cpa_tracks", []) == local_support
and counts.get("unclassified_score_candidates", 0) == len(details)
and counts.get("bounded_score_candidates", 0) == len(stage_bounds)
and counts.get("unbounded_score_candidates", 0) == len(details)-len(stage_bounds),
"Coarse support counts differ from persisted candidate assessments")
support_totals["unsupported_cpa_track_evaluations"] += len(unsupported)
clock_excluded += len(excluded)
+ interval_supported += len(local_support)
support_totals["unclassified_score_candidates"] += len(details)
support_totals["bounded_score_candidates"] += len(stage_bounds)
support_totals["unbounded_score_candidates"] += len(details)-len(stage_bounds)
@@ -649,6 +703,8 @@ def inspect_finite_search(artifact, read, search, configuration, refinement):
"Finite model-support totals differ from persisted candidate assessments")
require(support.get("clock_excluded_unsupported_track_evaluations", 0) == clock_excluded,
"Clock-excluded unsupported-track totals differ from stage evidence")
+ require(support.get("interval_supported_cpa_track_evaluations", 0) == interval_supported,
+ "Interval-supported CPA totals differ from stage evidence")
require(all(type(support.get(key)) is int and support[key] >= value
for key, value in coarse_trials.items()),
"Search model-support totals omit recorded coarse clock trials")
diff --git a/src/dopplergeist/geometry.py b/src/dopplergeist/geometry.py
index 75b2c04..436c707 100644
--- a/src/dopplergeist/geometry.py
+++ b/src/dopplergeist/geometry.py
@@ -148,6 +148,86 @@ class Track:
raise AcousticSupportError(self.id, "non_subsonic_motion",
emission_intervals_s=[[float(self.time[i]), float(self.time[i+1])] for i in indices])
+ def _cpa_interval_exclusion(self, observers, radius, c, sample_time, sample_position, max_step):
+ """Prove bad native intervals cannot affect this sparse CPA query.
+
+ Keep the original spline, sample lattice and global maximum sample
+ step. Bad curves must be outside every receiver sphere. Both endpoints
+ of every lattice edge touching a bad interval must also be outside the
+ original KD-tree radius, so no such edge, sampled minimum or endpoint
+ refinement can be used. A failed proof leaves whole-track CPA unknown.
+ This says nothing about later acoustic arrivals from those intervals.
+ """
+ methods = ("position", "speed_vector", "_unsupported_motion_intervals", "validate_motion_support")
+ if (type(self) is not Track or any(name in self.__dict__ for name in methods)
+ or tuple(getattr(Track, name) for name in methods) != _CANONICAL_CPA_MOTION_FUNCTIONS
+ or type(radius) not in (int, float) or type(c) not in (int, float)
+ or not np.isfinite([radius, c, max_step]).all() or radius <= 0 or c <= 0 or max_step < 0):
+ return None
+ observers = np.asarray(observers)
+ if (observers.dtype != np.dtype(float) or observers.ndim != 2 or observers.shape[1] != 3
+ or not len(observers) or not np.isfinite(observers).all()
+ or sample_time.dtype != np.dtype(float) or sample_time.ndim != 1 or len(sample_time) < 2
+ or sample_position.shape != (len(sample_time), 3)
+ or not np.isfinite(sample_time).all() or not np.isfinite(sample_position).all()
+ or sample_time[0] != self.time[0] or sample_time[-1] != self.time[-1]
+ or not np.all(np.diff(sample_time) > 0)):
+ return None
+ indices = self._unsupported_motion_intervals(c)
+ if not len(indices):
+ return None
+ lower, upper = observers.min(axis=0), observers.max(axis=0)
+ query_radius = float(np.nextafter(radius + 2 * max_step, np.inf))
+ if not np.isfinite(query_radius):
+ return None
+ affected_edges = np.zeros(len(sample_time)-1, dtype=bool)
+ hull_lower = np.inf
+ # No interval-by-receiver matrix: the receiver box encloses the actual
+ # receiver set, and fixed-size interval blocks bound temporary storage.
+ for first in range(0, len(indices), _MOTION_SUPPORT_INTERVAL_BATCH):
+ selected = indices[first:first+_MOTION_SUPPORT_INTERVAL_BATCH]
+ times = self.time[np.column_stack((selected, selected+1))]
+ dt = (times[:, 1]-times[:, 0])[:, None]
+ velocity = self.speed_vector(times)
+ p0, p1 = self.xyz[selected], self.xyz[selected+1]
+ with np.errstate(over="ignore", invalid="ignore"):
+ controls = np.stack((p0, p0+dt*velocity[:, 0]/3,
+ p1-dt*velocity[:, 1]/3, p1), axis=1)
+ if not np.isfinite(controls).all():
+ return None
+ lo, hi = controls.min(axis=1), controls.max(axis=1)
+ separation = np.maximum(np.maximum(lo-upper, lower-hi), 0)
+ scale = max(1., float(np.abs(controls).max()), float(np.abs(observers).max()), radius)
+ margin = 1e-6 + 128 * np.finfo(float).eps * scale
+ distances = np.nextafter(np.linalg.norm(separation, axis=1)-margin, -np.inf)
+ if not np.isfinite(distances).all() or not np.all(distances > radius):
+ return None
+ hull_lower = min(hull_lower, float(distances.min()))
+ for start, end in times:
+ tolerance = max(1e-7, 4*abs(np.spacing(max(abs(start), abs(end)))))
+ left = max(0, int(np.searchsorted(sample_time, start-tolerance, side="left"))-1)
+ right = min(len(affected_edges), int(np.searchsorted(sample_time, end+tolerance, side="right")))
+ affected_edges[left:right] = True
+ edges = np.flatnonzero(affected_edges)
+ if not len(edges):
+ return None
+ vertices = np.unique(np.r_[edges, edges+1])
+ positions = sample_position[vertices]
+ separation = np.maximum(np.maximum(positions-upper, lower-positions), 0)
+ scale = max(1., float(np.abs(positions).max()), float(np.abs(observers).max()), query_radius)
+ margin = 1e-6 + 128 * np.finfo(float).eps * scale
+ distances = np.nextafter(np.linalg.norm(separation, axis=1)-margin, -np.inf)
+ if not np.isfinite(distances).all() or not np.all(distances > query_radius):
+ return None
+ return {"track_id": self.id, "method": "outside_native_hulls_and_sample_edges",
+ "emission_time_range_s": [float(self.time[0]), float(self.time[-1])],
+ "emission_intervals_s": [[float(self.time[i]), float(self.time[i+1])] for i in indices],
+ "sound_speed_mps": float(c), "max_distance_m": float(radius),
+ "max_sample_step_m": float(max_step), "query_radius_m": query_radius,
+ "hull_distance_lower_m": hull_lower, "edge_distance_lower_m": float(distances.min()),
+ "sample_count": len(sample_time), "affected_sample_edges": len(edges),
+ "receiver_point_count": len(observers), "observer_box_ecef_m": [lower.tolist(), upper.tolist()]}
+
def outside_observer_radius(self, observers, radius, c=343.0):
"""Prove all observers are beyond a validated track's range.
@@ -363,6 +443,8 @@ class Track:
_NATIVE_POSITION = Track.position
_NATIVE_POSITION_VELOCITY = Track._position_velocity
+_CANONICAL_CPA_MOTION_FUNCTIONS = (Track.position, Track.speed_vector,
+ Track._unsupported_motion_intervals, Track.validate_motion_support)
_CANONICAL_ACOUSTIC_FUNCTIONS = (Track.acoustic, Track.position, Track.speed_vector,
Track._position_velocity, Track._check_motion_support, Track._check_motion_support_scalar,
Track._unsupported_motion_intervals, _try_native_acoustic, _packed_kinematics)
diff --git a/src/dopplergeist/search.py b/src/dopplergeist/search.py
index e7a21cc..f68307a 100644
--- a/src/dopplergeist/search.py
+++ b/src/dopplergeist/search.py
@@ -9,7 +9,7 @@ from scipy.optimize import minimize_scalar
from scipy.signal import find_peaks
from scipy.spatial import cKDTree
-from .geometry import AcousticSupportError, _CANONICAL_ECEF, ecef
+from .geometry import AcousticSupportError, Track, _CANONICAL_ECEF, ecef
from .matching import _CANONICAL_NATIVE_EVENT_VALIDATOR, _native_validate_ridge_events
from .native_cpa import try_cpa_edges
@@ -115,8 +115,15 @@ def _cpa_rows(tracks, observers, point_ids, max_distance, c=343.0, step_s=2.0, c
class _PreparedCPATrack:
"""One validated track/step, reused only across one call's receiver chunks."""
- def __init__(self, track, c, step_s, *, _sample_cache=None):
- track.validate_motion_support(c)
+ def __init__(self, track, c, step_s, *, _sample_cache=None, motion_observers=None, max_distance=None):
+ self.motion_support = None
+ motion_error = None
+ try:
+ track.validate_motion_support(c)
+ except AcousticSupportError as error:
+ if motion_observers is None or error.reason != "non_subsonic_motion" or _sample_cache is not None:
+ raise
+ motion_error = error
self.track, self.c, self.step_s = track, c, step_s
key = (id(track), step_s)
sampled = None if _sample_cache is None else _sample_cache.get(key)
@@ -130,10 +137,24 @@ class _PreparedCPATrack:
_sample_cache[key] = (track, self.time, self.position, self.max_step)
else:
_, self.time, self.position, self.max_step = sampled
+ if motion_error is not None:
+ self.motion_support = Track._cpa_interval_exclusion(track, motion_observers, max_distance, c,
+ self.time, self.position, float(self.max_step))
+ if self.motion_support is None:
+ raise motion_error
+ self.motion_support["sample_step_s"] = float(step_s)
self.lower, self.upper = self.position.min(axis=0), self.position.max(axis=0)
self.tree = None
def raw_rows(self, observers, point_ids, max_distance, chunk_size=512):
+ if self.motion_support is not None:
+ support = self.motion_support
+ lower, upper = np.asarray(support["observer_box_ecef_m"])
+ if (not np.isfinite(max_distance) or max_distance > support["max_distance_m"]
+ or self.c != support["sound_speed_mps"] or self.step_s != support["sample_step_s"]
+ or not np.isfinite(observers).all() or not np.all((observers >= lower) & (observers <= upper))):
+ raise AcousticSupportError(self.track.id, "non_subsonic_motion",
+ emission_intervals_s=support["emission_intervals_s"])
rows = []
track, t, pos, c = self.track, self.time, self.position, self.c
near=np.flatnonzero(np.all((observers>=self.lower-max_distance)&
diff --git a/src/dopplergeist/tiled_search.py b/src/dopplergeist/tiled_search.py
index 5987223..a14e73c 100644
--- a/src/dopplergeist/tiled_search.py
+++ b/src/dopplergeist/tiled_search.py
@@ -522,13 +522,26 @@ def _clock_children(tile_id, cells, start_window, parameters, filters, events):
return children
+def _track_cpa_batches(prepared, observers, point_ids, radius, step_s, chunk_points, max_rows):
+ batches, count = [], 0
+ for first in range(0, len(observers), chunk_points):
+ rows = _ordered_cpa_rows(prepared.raw_rows(observers[first:first + chunk_points],
+ point_ids[first:first + chunk_points], radius), step_s)
+ count += len(rows)
+ if count > max_rows:
+ raise DataUnavailable("Tile CPA-row memory budget exceeded")
+ if rows:
+ batches.append(np.array(rows, dtype=_CPA_DTYPE))
+ return batches, count
+
+
@_cache_ridge_fits
def _score(points, tracks, events, measurements, budget, parameters, filters, start_window=None, *, ledger=None, stage=None, score_floor=None, anchor_window=None):
radius = max(e["slant_range_m"][1] for e in events)
observers = ecef(points.lat.to_numpy(), points.lon.to_numpy(), points.elev_m.to_numpy())
point_ids = points.point_id.to_numpy(dtype=int)
cpa_batches, count = [], 0
- unsupported, clock_excluded = [], []
+ unsupported, clock_excluded, interval_supported = [], [], []
receive_bounds = _event_receive_time_bounds(events, parameters, filters, start_window)
# Bound materialized CPA rows before pairing; one track at a time keeps the
# national traffic count out of the CPA tensor/list allocation.
@@ -545,15 +558,8 @@ def _score(points, tracks, events, measurements, budget, parameters, filters, st
if track.outside_observer_radius(observers, radius, parameters.get("c", 343.0)):
continue
prepared = _PreparedCPATrack(track, parameters.get("c", 343.0), budget.cpa_step_s)
- for first in range(0, len(points), chunk_points):
- rows = _ordered_cpa_rows(prepared.raw_rows(observers[first:first + chunk_points],
- point_ids[first:first + chunk_points], radius), budget.cpa_step_s)
- track_rows += len(rows)
- if count + track_rows > budget.max_cpa_rows:
- raise DataUnavailable("Tile CPA-row memory budget exceeded")
- if rows:
- track_batches.append(np.array(rows, dtype=_CPA_DTYPE))
- del rows
+ track_batches, track_rows = _track_cpa_batches(prepared, observers, point_ids, radius,
+ budget.cpa_step_s, chunk_points, budget.max_cpa_rows-count)
except AcousticSupportError as error:
if receive_bounds is not None and track.time[0] > receive_bounds[1]:
clock_excluded.append({"track_id": track.id, "reason": error.reason,
@@ -574,9 +580,24 @@ def _score(points, tracks, events, measurements, budget, parameters, filters, st
"latest_possible_receive_s": last_receive,
"earliest_event_receive_s": receive_bounds[0]})
continue
- # The typed failure does not identify an exact bad interval. Keep
- # the entire track/stage as explicitly unassessed, and retain rows
- # from other tracks. Never repair or clip the source payload here.
+ if error.reason == "non_subsonic_motion":
+ try:
+ prepared = _PreparedCPATrack(track, parameters.get("c", 343.0), budget.cpa_step_s,
+ motion_observers=observers, max_distance=radius)
+ if prepared.motion_support is None:
+ raise error
+ track_batches, track_rows = _track_cpa_batches(prepared, observers, point_ids, radius,
+ budget.cpa_step_s, chunk_points, budget.max_cpa_rows-count)
+ except AcousticSupportError as local_error:
+ error = local_error
+ else:
+ interval_supported.append(prepared.motion_support)
+ count += track_rows
+ cpa_batches.extend(track_batches)
+ continue
+ # A failed interval proof still loses the whole track/stage. Direct
+ # acoustic scoring separately checks arrivals from every original
+ # invalid interval, including intervals proved irrelevant to CPA.
unsupported.append({"track_id": track.id, "reported_track_id": error.track_id, "reason": error.reason,
"reported_intervals": _support_error(error),
"emission_time_range_s": [float(track.time[0]), float(track.time[-1])],
@@ -592,7 +613,8 @@ def _score(points, tracks, events, measurements, budget, parameters, filters, st
if ledger is not None:
ledger.record(stage, [], [], 0)
return [], {"cpa_rows": 0, "candidates": 0, "unsupported_cpa_tracks": unsupported,
- "clock_excluded_unsupported_tracks": clock_excluded}
+ "clock_excluded_unsupported_tracks": clock_excluded,
+ "interval_supported_cpa_tracks": interval_supported}
if parameters.get("mode") == "robust_ridge":
from .matching import _prepare_score_inputs
events, measurements = _prepare_score_inputs(events, measurements)
@@ -786,6 +808,7 @@ def _score(points, tracks, events, measurements, budget, parameters, filters, st
bounded_score_candidates=len(bounds), unbounded_score_candidates=unclassified-len(bounds),
evaluated_rejections=rejected, unsupported_cpa_tracks=unsupported,
clock_excluded_unsupported_tracks=clock_excluded,
+ interval_supported_cpa_tracks=interval_supported,
unclassified_candidate_assessments=unclassified_assessments, **trial_counts)
@@ -795,6 +818,10 @@ def _record_support(output, stage, counts, summary):
if clock_excluded:
summary["clock_excluded_unsupported_track_evaluations"] = (
summary.get("clock_excluded_unsupported_track_evaluations", 0) + clock_excluded)
+ interval_supported = len(counts.get("interval_supported_cpa_tracks", []))
+ if interval_supported:
+ summary["interval_supported_cpa_track_evaluations"] = (
+ summary.get("interval_supported_cpa_track_evaluations", 0) + interval_supported)
for key in ("unclassified_score_candidates", "bounded_score_candidates", "unbounded_score_candidates",
"evaluated_trials", "rejected_trials", "unknown_trials", "unreported_trial_envelopes"):
summary[key] += counts.get(key, 0)
@@ -810,11 +837,12 @@ def _record_support(output, stage, counts, summary):
write_json(output / "support" / (stage.replace(":", "-") + "-event-pair-screen.json"),
{"stage": stage, **pair_screen})
if (counts.get("unsupported_cpa_tracks") or counts.get("unclassified_candidate_assessments")
- or counts.get("clock_excluded_unsupported_tracks")):
+ or counts.get("clock_excluded_unsupported_tracks") or counts.get("interval_supported_cpa_tracks")):
(output / "support").mkdir(exist_ok=True)
write_json(output / "support" / (stage.replace(":", "-") + ".json"),
{"stage": stage, "unsupported_cpa_tracks": counts.get("unsupported_cpa_tracks", []),
"clock_excluded_unsupported_tracks": counts.get("clock_excluded_unsupported_tracks", []),
+ "interval_supported_cpa_tracks": counts.get("interval_supported_cpa_tracks", []),
"unclassified_candidate_assessments": counts.get("unclassified_candidate_assessments", [])},
compact=True)
diff --git a/tests/test_real_motion_interval_cpa.py b/tests/test_real_motion_interval_cpa.py
new file mode 100644
index 0000000..91f81b1
--- /dev/null
+++ b/tests/test_real_motion_interval_cpa.py
@@ -0,0 +1,368 @@
+"""Localized CPA checks on unchanged released ADS-B and measured audio inputs.
+
+Receiver hypotheses and clock bounds are the existing development workload,
+not microphone-location truth. No aircraft samples or audio are generated.
+"""
+from copy import deepcopy
+import importlib.util
+import json
+import os
+from pathlib import Path
+import time
+from unittest.mock import patch
+
+import numpy as np
+import pandas as pd
+import pytest
+from scipy.optimize import minimize_scalar
+
+from dopplergeist.geometry import AcousticSupportError, Track, ecef
+from dopplergeist.matching import prepare_events
+from dopplergeist.search import _PreparedCPATrack, _ordered_cpa_rows, cpa_table_dense
+from dopplergeist.tracks import load_tracks
+
+
+ROOT = Path(__file__).resolve().parents[1]
+
+
[email protected](scope="module")
+def real_intervals():
+ root = Path(os.environ.get("DOPPLERGEIST_REAL_REFERENCE_ROOT", ROOT))
+ source = root / "data/adsb/2026-09-08/reference-region.parquet"
+ if not source.is_file():
+ pytest.skip("restore the real reference archive or set DOPPLERGEIST_REAL_REFERENCE_ROOT")
+ workload = json.loads((ROOT / "tests/data/real-search-workloads.json").read_text())
+ tracks, _ = load_tracks(source)
+ cases = []
+ for name, case in workload["cases"].items():
+ points = pd.DataFrame([case["point"]])
+ observers = ecef(points.lat.to_numpy(), points.lon.to_numpy(), points.elev_m.to_numpy())
+ admitted, unknown = [], []
+ for track in tracks:
+ try:
+ if track.outside_observer_radius(observers, 20000., 343.):
+ continue
+ except AcousticSupportError:
+ pass
+ else:
+ continue
+ try:
+ prepared = _PreparedCPATrack(track, 343., 2., motion_observers=observers, max_distance=20000.)
+ except AcousticSupportError:
+ unknown.append(track)
+ else:
+ admitted.append(prepared)
+ assert admitted and unknown
+ cases.append((name, points, observers, admitted, unknown))
+ return workload, tracks, cases
+
+
+def rows(prepared, observers):
+ return _ordered_cpa_rows(prepared.raw_rows(observers, np.arange(len(observers)), 20000.), 2.)
+
+
+def test_real_local_cpa_preserves_original_lattice_and_every_proposal(real_intervals):
+ _, tracks, cases = real_intervals
+ summary = []
+ for name, _, observers, admitted, unknown in cases:
+ count = nonempty = multiple = 0
+ for prepared in admitted:
+ track = prepared.track
+ original_time = track.time.copy()
+ expected_lattice = np.unique(np.r_[np.arange(track.time[0], track.time[-1], 2.), track.time[-1]])
+ assert np.array_equal(prepared.time, expected_lattice)
+ actual = rows(prepared, observers)
+ # Diagnostic arithmetic reference only. The interval proof above,
+ # not bypassing this validator, supplies physical CPA support.
+ with patch.object(Track, "validate_motion_support", lambda *args: None):
+ previous = _PreparedCPATrack(track, 343., 2.)
+ expected = rows(previous, observers)
+ assert actual == expected
+ assert prepared.max_step == previous.max_step
+ assert np.array_equal(track.time, original_time)
+ count += len(actual)
+ nonempty += bool(actual)
+ multiple += len(prepared.motion_support["emission_intervals_s"]) > 1
+ assert count > 0 and nonempty > 0 and multiple > 0
+ summary.append({"case": name, "admitted": len(admitted), "unresolved": len(unknown),
+ "cpa_rows": count, "nonempty_tracks": nonempty})
+ # Clean sources keep the same path and do not acquire an exclusion receipt.
+ clean_checked = 0
+ for track in tracks:
+ try:
+ old = _PreparedCPATrack(track, 343., 2.)
+ except AcousticSupportError:
+ continue
+ expected = rows(old, cases[0][2])
+ if not expected:
+ continue
+ local = _PreparedCPATrack(track, 343., 2., motion_observers=cases[0][2], max_distance=20000.)
+ assert local.motion_support is None
+ assert rows(local, cases[0][2]) == expected
+ clean_checked += 1
+ if clean_checked == 5:
+ break
+ assert clean_checked == 5
+ print(json.dumps({"exact_sparse_cpa": summary, "clean_controls": clean_checked}))
+
+
+def test_real_dense_comparison_keeps_existing_sampling_limits_visible(real_intervals):
+ _, _, cases = real_intervals
+ summary = []
+ for name, points, observers, admitted, _ in cases:
+ count = dense_count = fine_count = 0
+ largest_time = largest_range = 0.
+ for prepared in admitted:
+ actual = rows(prepared, observers)
+ with patch.object(Track, "validate_motion_support", lambda *args: None):
+ dense = cpa_table_dense([prepared.track], points, 20000., c=343., step_s=2.)
+ fine = prepared.track.closest_approaches(observers[0], max_distance=20000., c=343., step=.125)
+ count += len(actual)
+ dense_count += len(dense)
+ fine_count += len(fine)
+ for row in actual:
+ assert fine
+ closest = min(fine, key=lambda value: abs(value["emission_time"]-row[2]))
+ delta_time = abs(closest["emission_time"]-row[2])
+ delta_range = abs(closest["distance_m"]-row[4])
+ # Sparse CPA deliberately projects its two-second lattice.
+ # Compare with a finer continuous-spline solve without claiming
+ # that the existing finite proposer finds every narrow minimum.
+ assert delta_time < 2. * 1.1
+ assert delta_range < 343. * 2.
+ largest_time, largest_range = max(largest_time, delta_time), max(largest_range, delta_range)
+ assert count and dense_count and fine_count
+ summary.append({"case": name, "sparse_rows": count, "dense_2s_rows": dense_count,
+ "continuous_0_125s_minima": fine_count, "max_time_difference_s": largest_time,
+ "max_range_difference_m": largest_range})
+ print(json.dumps({"independent_dense_comparison": summary}))
+
+
+def test_real_first_and_final_sample_intervals_remain_intact(real_intervals):
+ _, tracks, cases = real_intervals
+ # These complete observed tracks have endpoint-adjacent minima at the
+ # workload's existing receiver hypotheses. No trajectory is cut or shifted.
+ controls = (("previous-fit", "a46d03:38", False),
+ ("previous-fit", "a740a3:3", True), ("offset", "ad6cf9:2", False))
+ for name, identity, final in controls:
+ observers = next(case[2] for case in cases if case[0] == name)
+ track = next(track for track in tracks if track.id == identity)
+ prepared = _PreparedCPATrack(track, 343., 2., motion_observers=observers, max_distance=20000.)
+ assert prepared.motion_support is None
+ lo, hi = prepared.time[-2:] if final else prepared.time[:2]
+ reference = minimize_scalar(lambda dt: np.sum((track.position(lo+dt)-observers[0])**2),
+ bounds=(0., hi-lo), method="bounded", options={"xatol": .0001})
+ assert reference.success and 0 < reference.x < hi-lo and reference.fun < 20000.**2
+ matches = [row for row in rows(prepared, observers) if lo < row[2] < hi]
+ assert len(matches) == 1
+ assert abs(matches[0][2]-(lo+reference.x)) < .001
+ assert abs(matches[0][4]-np.sqrt(reference.fun)) < .01
+ print(json.dumps({"observed_endpoint_controls": {"first": 2, "final": 1}}))
+
+
+def test_real_unproven_boundaries_and_query_changes_fail_closed(real_intervals):
+ _, _, cases = real_intervals
+ for _, _, observers, admitted, unknown in cases:
+ for track in unknown:
+ with pytest.raises(AcousticSupportError) as original:
+ track.validate_motion_support(343.)
+ with pytest.raises(AcousticSupportError) as scoped:
+ _PreparedCPATrack(track, 343., 2., motion_observers=observers, max_distance=20000.)
+ assert vars(scoped.value) == vars(original.value)
+ prepared = admitted[0]
+ proof = prepared.motion_support
+ # Exercise equality at the outward-rounded hull bound using the same
+ # observed spline and receiver, not altered aircraft observations.
+ with pytest.raises(AcousticSupportError):
+ _PreparedCPATrack(prepared.track, 343., 2., motion_observers=observers,
+ max_distance=proof["hull_distance_lower_m"])
+ with pytest.raises(AcousticSupportError):
+ prepared.raw_rows(observers, np.array([0]), 20001.)
+ other = cases[1 if cases[0][2] is observers else 0][2]
+ with pytest.raises(AcousticSupportError):
+ prepared.raw_rows(other, np.array([0]), 20000.)
+ # Alternate sound speeds must recheck the native intervals.
+ changed = _PreparedCPATrack(prepared.track, 333.3, 2., motion_observers=observers, max_distance=20000.)
+ assert changed.motion_support["sound_speed_mps"] == 333.3
+
+
+def test_real_acoustic_unknown_arrivals_survive_local_cpa_support(real_intervals):
+ _, _, cases = real_intervals
+ for _, _, observers, admitted, _ in cases:
+ prepared = admitted[0]
+ first, last = prepared.motion_support["emission_intervals_s"][0]
+ emission = first + (last-first)/2
+ receive = emission + np.linalg.norm(prepared.track.position(emission)-observers[0])/343.
+ with pytest.raises(AcousticSupportError) as supported:
+ prepared.track._check_motion_support(np.array([receive]), observers[0], 343.)
+ assert [first, last] in supported.value.emission_intervals_s
+ with pytest.raises(AcousticSupportError):
+ prepared.track.acoustic(np.array([receive]), observers[0], 343.)
+ with pytest.raises(AcousticSupportError):
+ prepared.track.validate_motion_support(343.)
+
+
+def test_real_recovered_cpa_supports_acoustic_and_measured_hypotheses(real_intervals):
+ from dopplergeist.tiled_search import SearchBudget, _score
+
+ workload, _, cases = real_intervals
+ acoustic_counts = []
+ for name, _, observers, admitted, _ in cases:
+ count = 0
+ for prepared in admitted:
+ proposals = rows(prepared, observers)
+ if not proposals:
+ continue
+ receive = np.asarray([row[3] for row in proposals])
+ factor, emission, distance = prepared.track.acoustic(receive, observers[0], 343.)
+ assert np.isfinite([factor, emission, distance]).all()
+ residual = emission + np.linalg.norm(prepared.track.position(emission)-observers[0], axis=1)/343. - receive
+ assert np.all(np.abs(residual) <= np.maximum(1e-7, 4*np.abs(np.spacing(receive))))
+ count += len(proposals)
+ assert count
+ acoustic_counts.append({"case": name, "supported_cpa_arrivals": count})
+
+ # Use the very track recovered in the limited-clock regression below. The
+ # ordinary free-clock generator aligns unchanged measured ridges to its two
+ # actual CPA proposals. This component check supplies no recording-clock or
+ # location truth; the original limited-clock result remains separate.
+ _, points, _, admitted, _ = next(case for case in cases if case[0] == "offset")
+ track = next(prepared.track for prepared in admitted if prepared.track.id == "ad340b:8")
+ events, measurements = prepare_events(workload["events"])
+ args = (points, [track], events, measurements, SearchBudget(**workload["budget"]), workload["scoring"], {})
+ actual, counts = _score(*args)
+ assert actual and all(np.isfinite(row["score"]) and row["flight_ids"] == [track.id] for row in actual)
+ assert any(row["core_assessment_complete"] for row in actual)
+ assert counts["evaluated_trials"] > 0 and counts["unknown_trials"] > 0
+ assert counts["unclassified_candidate_assessments"]
+ assert [item["track_id"] for item in counts["interval_supported_cpa_tracks"]] == [track.id]
+ # Diagnostic comparison changes only the global CPA gate. Original acoustic
+ # motion/arrival guards still classify every clock trial on both paths.
+ with patch.object(Track, "validate_motion_support", lambda *args: None):
+ expected, expected_counts = _score(*args)
+ expected_counts["interval_supported_cpa_tracks"] = counts["interval_supported_cpa_tracks"]
+ assert actual == expected and counts == expected_counts
+ print(json.dumps({"recovered_acoustic_components": {"cpa_arrivals": acoustic_counts,
+ "free_clock_associations": counts["candidates"], "finite_hypotheses": len(actual),
+ "evaluated_trials": counts["evaluated_trials"], "rejected_trials": counts["rejected_trials"],
+ "unknown_trials": counts["unknown_trials"], "exact_diagnostic_parity": True}}))
+
+
+def verifier():
+ spec = importlib.util.spec_from_file_location("verify_interval_cpa", ROOT / "scripts/verify_upload_http.py")
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
+ return module
+
+
+def test_real_interval_receipts_reject_corrupted_inequalities(real_intervals):
+ workload, _, cases = real_intervals
+ check = verifier().valid_interval_cpa_support
+ events, _ = prepare_events(workload["events"])
+ config = {"events": events, "scoring": workload["scoring"], "budget": workload["budget"]}
+ for _, _, _, admitted, _ in cases:
+ for prepared in admitted:
+ proof = prepared.motion_support
+ assert check(proof, config)
+ for key, value in (("hull_distance_lower_m", proof["max_distance_m"]),
+ ("edge_distance_lower_m", proof["query_radius_m"]),
+ ("query_radius_m", proof["max_distance_m"]),
+ ("sound_speed_mps", 333.3), ("sample_step_s", 3.),
+ ("receiver_point_count", 0), ("emission_intervals_s", [])):
+ assert not check({**proof, key: value}, config)
+ damaged = deepcopy(proof)
+ damaged["observer_box_ecef_m"][0][0] = damaged["observer_box_ecef_m"][1][0] + 1
+ assert not check(damaged, config)
+
+
+def test_real_interval_proof_overhead(real_intervals):
+ _, _, cases = real_intervals
+ comparisons = []
+ for name, _, observers, admitted, _ in cases:
+ original_times, local_times = [], []
+ for repetition in range(3):
+ for local in ((False, True) if repetition % 2 == 0 else (True, False)):
+ began = time.process_time()
+ for previous in admitted:
+ track = previous.track
+ if local:
+ prepared = _PreparedCPATrack(track, 343., 2., motion_observers=observers, max_distance=20000.)
+ else:
+ with patch.object(Track, "validate_motion_support", lambda *args: None):
+ prepared = _PreparedCPATrack(track, 343., 2.)
+ rows(prepared, observers)
+ (local_times if local else original_times).append(time.process_time()-began)
+ comparisons.append({"case": name, "track_receiver_pairs": len(admitted),
+ "original_diagnostic_cpu_s": float(np.median(original_times)),
+ "local_proof_cpu_s": float(np.median(local_times))})
+ print(json.dumps({"warm_cpa_overhead": comparisons}))
+
+
+def test_real_measurements_preserve_nonempty_ranking_and_unknown_work(real_intervals, tmp_path):
+ from dopplergeist.data_access import FlightIndex
+ from dopplergeist.finite_search import RetentionLedger
+ from dopplergeist.tiled_search import SearchBudget, _record_support, _score
+
+ path = os.environ.get("DOPPLERGEIST_REAL_FLIGHT_INDEX")
+ if not path or not Path(path).is_file():
+ pytest.skip("set DOPPLERGEIST_REAL_FLIGHT_INDEX to the restored reference-region index")
+ workload, _, _ = real_intervals
+ case = workload["cases"]["offset"]
+ index = FlightIndex(path)
+ assert [{key: value[key] for key in ("sha256", "bytes")} for value in index.report["sources"]] == workload["index_identity"]["sources"]
+ tracks = list(index.query(case["query_bounds"], workload["query_time_s"]))
+ events, measurements = prepare_events(workload["events"])
+ points = pd.DataFrame([case["point"]])
+ args = (points, tracks, events, measurements, SearchBudget(**workload["budget"]),
+ workload["scoring"], {}, workload["recording_start_window_s"])
+ ledgers = {name: RetentionLedger(tmp_path / f"{name}.sqlite", "real-motion-interval-cpa",
+ "matched_group_support_minus_normalized_residual")
+ for name in ("actual", "baseline", "diagnostic")}
+ stage = "coarse:observed"
+ actual, counts = _score(*args, ledger=ledgers["actual"], stage=stage)
+ selected = {item["track_id"] for item in counts["interval_supported_cpa_tracks"]}
+ assert actual and selected and counts["unknown_trials"] > 0
+ with patch.object(Track, "_cpa_interval_exclusion", return_value=None):
+ old, old_counts = _score(*args, ledger=ledgers["baseline"], stage=stage)
+ assert actual == old
+ assert counts["cpa_rows"] > old_counts["cpa_rows"]
+ assert {item["track_id"] for item in old_counts["unsupported_cpa_tracks"]} == (
+ {item["track_id"] for item in counts["unsupported_cpa_tracks"]} | selected)
+ assert counts["unknown_trials"] == old_counts["unknown_trials"]
+ assert counts["unclassified_candidate_assessments"] == old_counts["unclassified_candidate_assessments"]
+
+ original = Track.validate_motion_support
+ def diagnostic_validate(track, c=343.):
+ if track.id not in selected:
+ original(track, c)
+ with patch.object(Track, "validate_motion_support", diagnostic_validate):
+ expected, expected_counts = _score(*args, ledger=ledgers["diagnostic"], stage=stage)
+ assert actual == expected
+ expected_counts["interval_supported_cpa_tracks"] = counts["interval_supported_cpa_tracks"]
+ assert counts == expected_counts
+
+ persisted = []
+ for ledger in ledgers.values():
+ checksum = ledger.checksum(stage)
+ assert checksum and ledger.stage_valid(stage, checksum)
+ with ledger.connect() as db:
+ persisted.append((db.execute("SELECT * FROM records ORDER BY stage,mode_id").fetchall(),
+ db.execute("SELECT * FROM bounds ORDER BY stage,bound_id").fetchall(),
+ db.execute("SELECT * FROM stages ORDER BY name").fetchall()))
+ assert persisted[0] == persisted[1] == persisted[2]
+ assert persisted[0][0] and persisted[0][1]
+
+ summary = dict.fromkeys(("unsupported_cpa_track_evaluations", "unclassified_score_candidates",
+ "bounded_score_candidates", "unbounded_score_candidates", "evaluated_trials", "rejected_trials",
+ "unknown_trials", "unreported_trial_envelopes"), 0)
+ _record_support(tmp_path, stage, counts, summary)
+ saved = json.loads((tmp_path / "support/coarse-observed.json").read_text())
+ assert saved["interval_supported_cpa_tracks"] == counts["interval_supported_cpa_tracks"]
+ assert summary["interval_supported_cpa_track_evaluations"] == len(selected)
+ config = {"events": events, "scoring": workload["scoring"], "budget": workload["budget"]}
+ assert all(verifier().valid_interval_cpa_support(item, config) for item in saved["interval_supported_cpa_tracks"])
+ print(json.dumps({"real_measured_scoring": {"tracks": len(tracks), "retained": len(actual),
+ "previous_cpa_rows": old_counts["cpa_rows"], "current_cpa_rows": counts["cpa_rows"],
+ "candidates": counts["candidates"], "remaining_unknown_trials": counts["unknown_trials"],
+ "interval_supported": len(selected), "retained_ledger_modes": len(persisted[0][0]),
+ "retained_unknown_bounds": len(persisted[0][1]), "exact_ranking_and_ledger": True}}))