Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)
Python acoustic-geolocationads-baircraft-trackingairplaneaudio-forensicsdoppler-effectdoppler-shiftgeolocationosintsignal-processing
Integrate latest main and reject empty receiver selections
Cinna Davised8b26612996
diff --cc docs/COUNTRIES.mdindex 56a0510,0000000..ca94a1dmode 100644,000000..100644--- a/docs/COUNTRIES.md+++ b/docs/COUNTRIES.md@@@ -1,103 -1,0 +1,111 @@@+# Country profiles++The automatic request, CLI and upload interface support two receiver regions:++| Country | Coverage | Envelope: west, south, east, north | Search CRS | Receiver region |+| --- | --- | --- | --- | --- |+| `US` | `conus` | `-125, 24, -66, 50` | `EPSG:5070` | Existing CONUS rectangle |+| `DE` | `germany` | `5.5, 47, 15.5, 55.2` | `EPSG:32632` | BKG VG250 Germany land polygon inside the envelope |++The envelope limits optional rectangular subsets. Germany's envelope includes+foreign territory; the land polygon removes those receiver locations before+terrain sampling or scoring. Islands, exclaves and polygon holes are preserved.+The boundary is generalized geographic data, not a survey of a microphone site.+Country support does not establish German localization accuracy.++## Requests and commands++```bash+dopplergeist analyze recording.mp3 --country DE --jobs jobs --online \+ --start-utc "$RECORDING_START_UTC_MIN" --end-utc "$RECORDING_START_UTC_MAX" \+ --time-source 'Evidence establishing the recording-start interval'+```++The upload API accepts `"search":{"country":"DE"}`. It also accepts+`"search":{"coverage":"germany"}`. A normalized request records the coverage;+an explicitly supplied country is normalized to uppercase. If both are supplied,+they must agree. `bounds_wsen` and `--bounds` narrow the selected profile. They+must be an independently justified prior, not the evaluation location.++Omitting country preserves the previous CONUS behavior. Old requests/results+without a country field remain readable. An older service advertises no country+catalog, so the new browser offers only CONUS and sends the legacy request shape.++The configured research command also accepts a top-level `country` or a `search`+object. A supplied `grid.crs` must match the country profile, and `grid.bounds`+must agree with any `search.bounds_wsen`. That command retains its existing+configured-event and uncertified-coverage limitations.++## Germany boundary acquisition++The first online German search obtains the country layer `vg250:vg250_sta`,+filtered to `GF=4`, from the [official BKG WFS](https://sgx.geodatenzentrum.de/wfs_vg250).+It requests `EPSG:25832` so the original ETRS89 / UTM 32N geometry is retained.+Receiver coordinates are transformed into that CRS for point-in-polygon tests.+The search grid and reported projected coordinates use `EPSG:32632` consistently.++The WFS product advertises the 2025-01-01 dataset. The pinned land feature has+object ID `DEBKGVG200000CKM` and `BEGINN=2025-04-11Z`; it is not described as the+separate 2026 download product. Its canonical geometry, properties and CRS have+SHA-256 `b5e9276fe37ca1c0d65b4542a075cc246e97194ed70d3259b0d7900ad944a1d3`.+The response's request timestamp is excluded from that content identity.++The validated feature is cached at+`CACHE_ROOT/geography/de-vg250-b5e9276fe37ca1c0.json`. The download is limited to+8 MiB and written atomically after validation. Offline searches require this+cache entry. Copying this entry to another machine preserves the same boundary.+Missing, damaged or changed source data produce an explicit data-unavailable+outcome. A future WFS update does not silently replace the pinned geometry;+review its data and update the pin deliberately.++No source simplification is applied. The source's 110 polygon components and+their holes remain intact. Prepared vector predicates avoid creating a separate+geometry object for every receiver. The mask is serialized to each search worker.++## Search and checkpoint behavior++The mask applies to serial and parallel coarse tiles, fine tiles, research grids,+continuous refinement and every sensitivity scenario. A masked-out continuous+proposal never reaches terrain sampling or acoustic scoring. A valid seed remains+available when optimization proposes an invalid location. A refinement whose+explored radius meets the country boundary is marked boundary-limited.+++Before automatic flight acquisition, a bounded geographic check establishes at++least one eligible coarse-grid receiver. Foreign-only selections and small land++selections containing no coarse-grid point return `needs_input` with a specific++area or grid-resolution explanation. An inconclusive bounded check stops with an++explicit limitation; it does not report an empty area or a completed flight search.++The full-country check uses land-component probes rather than enumerating the++national grid.+++Aircraft are not country-masked. Automatic acquisition retains full historical+trajectories, and tile queries include the existing audible-range border margin.+Archive absence or incomplete trajectory/model support remains visible.++German search identities include the boundary content, source, selected CRS and+Shapely version. Changing those inputs cannot reuse incompatible search results.+Legacy CONUS requests retain their meaning, but a new code revision does not+automatically make an old source-pinned checkpoint resumable. Preserve the old+producer or use a separately verified migration; no identity bypass is added.++## Geographic checks++```bash+python scripts/check_country_profiles.py --online --cache-root /tmp/dopplergeist-geography+python scripts/check_country_profiles.py --cache-root /tmp/dopplergeist-geography+```++The check uses the actual pinned BKG geometry, named geographic points, country- boundary grids and worker serialization. It generates no audio or flight tracks.++boundary grids, empty-area checks and worker serialization. It generates no audio or flight tracks.+These are software/geography checks, not an acoustic localization benchmark.++## Boundary license and source++© [BKG](https://www.bkg.bund.de) 2026,+[dl-de/by-2-0](https://www.govdata.de/dl-de/by-2-0).+[Data sources](https://sgx.geodatenzentrum.de/web_public/gdz/datenquellen/datenquellen_vg_nuts.pdf).+[Official product metadata](https://gdz.bkg.bund.de/index.php/default/wfs-verwaltungsgebiete-1-250-000-stand-01-01-wfs-vg250.html).++Processing: selection of the land feature and coordinate transformation for+receiver tests; no polygon simplification. Boundary data retains its source+license. The application code retains the repository's PolyForm license.diff --cc scripts/check_country_profiles.pyindex 84e2a26,0000000..563e4e3mode 100644,000000..100644--- a/scripts/check_country_profiles.py+++ b/scripts/check_country_profiles.py@@@ -1,229 -1,0 +1,283 @@@+#!/usr/bin/env python3+"""Check country contracts and real BKG receiver geometry without audio or tracks.++Run against an installed package, or set PYTHONPATH=src in a checkout:++ python scripts/check_country_profiles.py --cache-root /path/to/cache++The default is offline. --online permits the production loader to acquire its+pinned public BKG feature when absent. No historical-flight or terrain download,+localization run, generated recording, or generated aircraft track is involved.+"""+from copy import deepcopy+import argparse+import json+import math+from pathlib import Path+import pickle+import sys+++class CheckFailure(RuntimeError):+ pass+++def require(condition, message):+ if not condition:+ raise CheckFailure(message)+++def check_contracts():+ from dopplergeist.contracts import validate_request+ from dopplergeist.geography import profile_for_search, public_profiles++ # This is only request validation. The existing script path is not decoded+ # or represented as a real recording.+ request_base = {"schema_version": 1, "audio_path": str(Path(__file__).resolve())}+ conus_subset = [-90.0, 40.0, -87.0, 43.0]+ buesingen_subset = [8.66, 47.68, 8.72, 47.72]+ cases = (+ ("omitted search", None, {"coverage": "conus"}, "US"),+ ("legacy CONUS", {"coverage": "conus"}, {"coverage": "conus"}, "US"),+ ("legacy CONUS subset", {"coverage": "conus", "bounds_wsen": conus_subset},+ {"coverage": "conus", "bounds_wsen": conus_subset}, "US"),+ ("US country", {"country": "us"}, {"country": "US", "coverage": "conus"}, "US"),+ ("DE country", {"country": "de"}, {"country": "DE", "coverage": "germany"}, "DE"),+ ("Germany coverage", {"coverage": "germany"}, {"coverage": "germany"}, "DE"),+ ("explicit Germany", {"country": "DE", "coverage": "germany"},+ {"country": "DE", "coverage": "germany"}, "DE"),+ ("Germany subset", {"country": "DE", "bounds_wsen": buesingen_subset},+ {"country": "DE", "coverage": "germany", "bounds_wsen": buesingen_subset}, "DE"),+ )+ validated = {}+ for name, search, expected, country in cases:+ request = deepcopy(request_base)+ if search is not None:+ request["search"] = deepcopy(search)+ original = deepcopy(request)+ normalized = validate_request(request)+ require(request == original, f"{name}: validation changed the input request")+ require(normalized["search"] == expected, f"{name}: unexpected normalized search")+ require(profile_for_search(normalized["search"]).country == country,+ f"{name}: wrong resolved country")+ serialized = json.loads(json.dumps(normalized, allow_nan=False))+ require(validate_request(serialized) == normalized,+ f"{name}: normalization is not stable through a JSON round trip")+ validated[name] = normalized++ rejected = (+ {"country": "DE", "coverage": "conus"},+ {"country": "US", "coverage": "germany"},+ {"country": "FR"},+ {"coverage": "custom"},+ {"country": None},+ {"coverage": None},+ {},+ {"country": "DE", "bounds_wsen": conus_subset},+ {"country": "US", "bounds_wsen": buesingen_subset},+ {"country": "DE", "bounds_wsen": [8.72, 47.68, 8.66, 47.72]},+ {"country": "DE", "bounds_wsen": [8.66, 47.68, math.nan, 47.72]},+ {"country": "DE", "unsupported_field": True},+ )+ for search in rejected:+ try:+ validate_request(dict(request_base, search=search))+ except ValueError:+ pass+ else:+ raise CheckFailure(f"Invalid search was accepted: {search!r}")++ catalog = public_profiles()+ require(len(catalog) == 2 and {p["country"] for p in catalog} == {"US", "DE"},+ "Capabilities must describe exactly the supported US and DE profiles")+ for item in catalog:+ profile = profile_for_search({"country": item["country"]})+ require(item["coverage"] == profile.coverage and item["crs"] == profile.crs+ and item["bounds_wsen"] == list(profile.bounds_wsen),+ f"Capabilities disagree with the {profile.country} runtime profile")+ require(profile_for_search().crs == "EPSG:5070", "Legacy CONUS CRS changed")+ require(profile_for_search({"country": "DE"}).crs == "EPSG:32632",+ "Germany must use its declared UTM 32N search CRS")+ print(f"PASS requests: {len(cases)} normalization/JSON round-trip cases; "+ f"{len(rejected)} invalid declarations rejected; profile catalog agrees")+ return validated+++# Fixed real locations, not simulated receivers or localization truth.+# The small-component controls use coordinates published by their local bodies:+# Helgoland: https://www.helgoland.de/content/uploads/2024/02/Faltplan-HELGOLAND-GB_web.pdf+# Sylt: https://www.sylt.de/eventlocation/tui-blue-sylt+# Buesingen: routing link at https://www.buesingen.de/de/Rathaus/Gemeindeverwaltung+# Mainland city coordinates are rounded landmarks; they are far from the border.+LOCATIONS = (+ ("Berlin centre", 13.4050, 52.5200, True),+ ("Munich Marienplatz", 11 + 34 / 60 + 31 / 3600, 48 + 8 / 60 + 14 / 3600, True),+ ("Helgoland", 7 + 53 / 60 + 1 / 3600, 54 + 10 / 60 + 57 / 3600, True),+ ("Sylt, Rantum", 8.2949, 54.85991, True),+ ("Buesingen town hall", 8.687434, 47.696354, True),+ ("Zurich centre", 8.5417, 47.3769, False),+ ("Prague centre", 14.4378, 50.0755, False),+ ("Schaffhausen centre", 8.6349, 47.6965, False),+)+++def check_mask(cache_root, online, validated):+ import numpy as np+ from dopplergeist.geography import (+ BKG_BOUNDARY_SHA256, BKG_BOUNDARY_URL, profile_for_search, receiver_mask_for_search,+ )++ require(receiver_mask_for_search(validated["legacy CONUS"]["search"],+ cache_root=cache_root, offline=True) is None,+ "The legacy CONUS profile unexpectedly acquired a country polygon")+ search = validated["DE country"]["search"]+ mask = receiver_mask_for_search(search, cache_root=cache_root, offline=not online)+ require(mask is not None, "Germany must not fall back to a rectangular mask")+ identity = mask.identity+ require(identity["boundary_sha256"] == BKG_BOUNDARY_SHA256+ and identity["source_url"] == BKG_BOUNDARY_URL,+ "Loaded boundary does not identify the pinned BKG source")+ require(identity["source_crs"] == "EPSG:25832"+ and identity["search_crs"] == profile_for_search(search).crs,+ "Boundary/source CRS identity disagrees with the selected search profile")++ lon = np.array([item[1] for item in LOCATIONS])+ lat = np.array([item[2] for item in LOCATIONS])+ expected = np.array([item[3] for item in LOCATIONS], dtype=bool)+ west, south, east, north = profile_for_search(search).bounds_wsen+ require(bool(np.all((lon >= west) & (lon <= east) & (lat >= south) & (lat <= north))),+ "Named controls must all be inside the Germany rectangular envelope")+ actual = np.asarray(mask.contains(lon, lat), dtype=bool)+ require(actual.shape == expected.shape, "Vector mask changed the coordinate shape")+ mismatches = [item[0] for item, keep, want in zip(LOCATIONS, actual, expected) if keep != want]+ require(not mismatches, f"Incorrect land membership: {', '.join(mismatches)}")+ scalar = np.array([bool(mask.contains(float(x), float(y))) for x, y in zip(lon, lat)])+ require(np.array_equal(actual, scalar), "Scalar and vector country masks disagree")++ # Workers receive the real polygon through serialization. Also reload the+ # actual cache offline after acquisition, so a successful online fetch alone+ # cannot hide an unusable persisted feature.+ transferred = pickle.loads(pickle.dumps(mask, protocol=pickle.HIGHEST_PROTOCOL))+ cached = receiver_mask_for_search(search, cache_root=cache_root, offline=True)+ for name, copy in (("pickled worker mask", transferred), ("offline cache", cached)):+ require(copy.identity == identity, f"{name}: source identity changed")+ require(np.array_equal(copy.contains(lon, lat), expected),+ f"{name}: real-location membership changed")+ require(not mask.boundary_limited(13.4050, 52.5200, 1000.0),+ "Central Berlin was incorrectly flagged within 1 km of the country boundary")+ require(mask.boundary_limited(8.687434, 47.696354, 10000.0),+ "A 10 km neighbourhood of Buesingen must be boundary-limited")+ require(mask.boundary_limited(8.5417, 47.3769, 0.0),+ "An outside-country receiver must be boundary-limited")+ print(f"PASS BKG mask: {int(expected.sum())} German locations retained "+ f"(including two islands and Buesingen); {int((~expected).sum())} foreign locations excluded")+ print("PASS scalar/vector, worker pickle, offline reload and boundary-proximity controls")+ return mask+++def check_border_grid(mask, validated):+ import numpy as np+ from pyproj import Transformer+ from dopplergeist.geography import profile_for_search+ from dopplergeist.tiled_search import _tile_coordinates++ profile = profile_for_search(validated["DE country"]["search"])+ forward = Transformer.from_crs("EPSG:4326", profile.crs, always_xy=True)+ # The actual Buesingen exclave and surrounding Swiss territory exercise both+ # sides of a land border, unlike an inland rectangle-only grid check.+ centre_x, centre_y = forward.transform(8.687434, 47.696354)+ subset = validated["Germany subset"]["search"]["bounds_wsen"]+ summaries = []+ for spacing in (1000.0, 250.0):+ cells = (math.floor((centre_x - 6000) / spacing), math.floor((centre_y - 6000) / spacing),+ math.ceil((centre_x + 6000) / spacing), math.ceil((centre_y + 6000) / spacing))+ for label, bounds in (("national envelope", profile.bounds_wsen), ("requested subset", subset)):+ rectangle = _tile_coordinates(cells, spacing, bounds, profile.crs)+ clipped = _tile_coordinates(cells, spacing, bounds, profile.crs, receiver_mask=mask)+ require(0 < len(clipped) < len(rectangle),+ f"Buesingen {spacing:g} m/{label}: clipping did not preserve both border sides")+ expected = np.asarray(mask.contains(rectangle.lon.to_numpy(), rectangle.lat.to_numpy()), dtype=bool)+ columns = ["x", "y", "lon", "lat"]+ require(np.array_equal(clipped[columns].to_numpy(), rectangle.loc[expected, columns].to_numpy()),+ f"Buesingen {spacing:g} m/{label}: the grid lost or added country cells")+ projected_x, projected_y = forward.transform(clipped.lon.to_numpy(), clipped.lat.to_numpy())+ require(np.allclose(projected_x, clipped.x.to_numpy(), rtol=0, atol=1e-5)+ and np.allclose(projected_y, clipped.y.to_numpy(), rtol=0, atol=1e-5),+ f"Buesingen {spacing:g} m/{label}: grid x/y disagree with the country CRS")+ summaries.append(f"{spacing:g} m/{label}: {len(rectangle)} -> {len(clipped)} cells")+ print("PASS real border-grid clipping and UTM 32N coordinate agreement: " + "; ".join(summaries))++++def check_empty_areas(mask, validated):++ from dopplergeist.geography import EmptySearchArea, profile_for_search++ from dopplergeist.tiled_search import _grid_extent, _tile_coordinates++++ profile = profile_for_search(validated["DE country"]["search"])++ foreign = (++ ("Zurich", [8.53, 47.37, 8.55, 47.39]),++ ("Prague", [14.43, 50.07, 14.45, 50.08]),++ ("Schaffhausen", [8.63, 47.69, 8.64, 47.70]),++ )++ for name, bounds in foreign:++ try:++ mask.preflight_grid(bounds, 2000.0, profile.crs)++ except EmptySearchArea as error:++ require(error.reason_code == "empty_search_area",++ f"{name}: foreign rectangle did not identify an empty country area")++ else:++ raise CheckFailure(f"{name}: foreign rectangle passed the receiver preflight")++++ # This real Berlin location is on German land, but the sub-metre rectangle++ # contains no point of the actual 2 km search grid. Check the grid directly++ # before requiring the distinct coarse-resolution explanation.++ small_berlin = [13.404999, 52.519999, 13.405001, 52.520001]++ require(bool(mask.contains(13.4050, 52.5200)), "Berlin land control changed")++ xmin, ymin, xmax, ymax = _grid_extent(small_berlin, 2000.0, profile.crs)++ require(len(_tile_coordinates((xmin, ymin, xmax + 1, ymax + 1), 2000.0,++ small_berlin, profile.crs, mask)) == 0,++ "The Berlin small-area control unexpectedly contains a coarse grid point")++ try:++ mask.preflight_grid(small_berlin, 2000.0, profile.crs)++ except EmptySearchArea as error:++ require(error.reason_code == "empty_search_grid",++ "A land selection without coarse receivers must identify the grid limitation")++ else:++ raise CheckFailure("A land selection without coarse receivers passed preflight")++++ available = (++ ("Berlin", [13.3, 52.45, 13.5, 52.6], 2000.0),++ ("Helgoland", [7.87, 54.17, 7.90, 54.19], 250.0),++ ("Sylt", [8.28, 54.85, 8.31, 54.87], 250.0),++ ("Buesingen", validated["Germany subset"]["search"]["bounds_wsen"], 1000.0),++ )++ for name, bounds, spacing in available:++ require(mask.preflight_grid(bounds, spacing, profile.crs) is True,++ f"{name}: real eligible receiver grid was not established")++ require(mask.preflight_grid(profile.bounds_wsen, 2000.0, profile.crs, max_points=9) is True,++ "The national preflight must find a receiver within nine grid probes")++ require(mask.preflight_grid(profile.bounds_wsen, 2000.0, profile.crs, max_points=1) is None,++ "An exhausted preflight limit must remain undecided, not claim an empty area")++ print("PASS empty-area preflight: three foreign rectangles rejected; Berlin land/grid "++ "distinguished; mainland, islands and exclave retained; national check bounded to nine points")+++++def main(argv=None):+ parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)+ parser.add_argument("--cache-root", required=True, type=Path, help="Existing DOPPLERGEIST data cache")+ parser.add_argument("--online", action="store_true", help="Allow acquisition of the pinned public BKG feature")+ args = parser.parse_args(argv)+ try:+ validated = check_contracts()+ mask = check_mask(args.cache_root, args.online, validated)+ check_border_grid(mask, validated)++ check_empty_areas(mask, validated)+ except (CheckFailure, ImportError, OSError, ValueError, RuntimeError) as error:+ print(f"FAIL country verification: {error}", file=sys.stderr)+ return 1+ print("PASS geographic software checks. Not exercised: flight acquisition/association, terrain sampling, "+ "job/checkpoint resume, full refinement, or recording localization.")+ return 0+++if __name__ == "__main__":+ raise SystemExit(main())diff --cc scripts/verify_real_upload_browser.pyindex f02ea8b,690ee15..857f6bf--- a/scripts/verify_real_upload_browser.py+++ b/scripts/verify_real_upload_browser.py@@@ -513,3 -465,3 +513,4 @@@ def main()if __name__ == "__main__":raise SystemExit(main())++diff --cc src/dopplergeist/automatic.pyindex 1c3b4b7,8e3018e..50beb22--- a/src/dopplergeist/automatic.py+++ b/src/dopplergeist/automatic.py@@@ -9,6 -9,6 +9,7 @@@ from pathlib import Patimport timefrom .contracts import utc_timestamp, validate_candidate, validate_event, validate_request++from .geography import EmptySearchAreafrom .jobs import write_json@@@ -153,6 -153,6 +154,9 @@@ def analyze(request, output_dir, configrequest, recording, events, output, config, report_search_progress)except ImportError as exc:return finish("failed", "component_unavailable", f"Automatic matching/data components are not installed: {exc}")++ except EmptySearchArea as exc:++ state["coverage"]["limitations"].append(str(exc))++ return finish("needs_input", exc.reason_code, str(exc), required_fields=["search.bounds_wsen"])except MissingData as exc:state["coverage"]["limitations"].append(str(exc))return finish("inconclusive", "data_unavailable", str(exc))@@@ -255,9 -254,6 +259,18 @@@ def _search(request, recording, eventsraise MissingData("Recording time window exceeds the configured historical-data acquisition budget. Supply a narrower defensible window.")ground = config.get("terrain", {})hashes = {"terrain_configuration": ground}+ profile = profile_for_search(request["search"])+ receiver_mask = receiver_mask_for_search(request["search"],+ cache_root=config.get("cache_root", output / "cache"), offline=config.get("offline", True))++ budget = None++ if receiver_mask is not None:++ budget = SearchBudget(**config.get("search_budget", {}))++ if receiver_mask.preflight_grid(request["search"].get("bounds_wsen", profile.bounds_wsen),++ budget.spacing_m, profile.crs) is not True:++ raise MissingData("The bounded receiver-grid check could not establish a German receiver "++ "point in this selection. No flight data were acquired. Choose a smaller area, "++ "or ask the operator to inspect the search area and coarse-grid spacing; "++ "this does not establish that the receiver area is empty.")if ground.get("rasters") or "constant_m" in ground:# Explicit offline/test terrain override; normal provider acquisition stays# in the data owner's adapter. Never infer flight coverage from this model.@@@ -284,18 -280,13 +297,18 @@@"recording_start_window_s": [lo, hi]}if config.get("resume_legacy_first_batch"):scoring["legacy_first_batch"] = True+ geographic_options = {"receiver_mask": receiver_mask} if receiver_mask is not None else {}report = search_national(indexes, terrain, prepared, measurements, time_range_s=time_range, output_dir=output / "search",-- bounds=bounds, budget=SearchBudget(**config.get("search_budget", {})),- scoring=scoring, filters=filters, input_hashes=hashes,++ bounds=bounds, budget=budget or SearchBudget(**config.get("search_budget", {})),+ scoring=scoring, filters=filters, input_hashes=hashes, crs=profile.crs,workers=config.get("search_workers"),- search_progress=search_progress)- write_json(output / "data-provenance.json", {"flight_indexes": [index.report for index in indexes],- "terrain": terrain.provenance() if hasattr(terrain, "provenance") else hashes})+ search_progress=search_progress, **geographic_options)+ provenance = {"flight_indexes": [index.report for index in indexes],+ "terrain": terrain.provenance() if hasattr(terrain, "provenance") else hashes}+ if receiver_mask is not None:+ provenance["search_geography"] = {"country": profile.country, "coverage": profile.coverage,+ "bounds_wsen": list(bounds), "crs": profile.crs, "receiver_mask": receiver_mask.identity}+ write_json(output / "data-provenance.json", provenance)return report, prepared, measurements, indexes, terraindiff --cc src/dopplergeist/geography.pyindex c9f1a5d,0000000..cb86b9fmode 100644,000000..100644--- a/src/dopplergeist/geography.py+++ b/src/dopplergeist/geography.py@@@ -1,208 -1,0 +1,283 @@@+"""Supported receiver regions and the pinned BKG Germany land boundary.++Country masks restrict receiver locations, never aircraft trajectories. The US+profile deliberately retains the historical CONUS rectangle and CRS. Germany+uses BKG VG250 land geometry in its native ETRS89 / UTM 32N coordinates.+"""+from copy import deepcopy+from dataclasses import dataclass+import hashlib+import json+import math+import os+from pathlib import Path+import tempfile+from urllib.parse import urlencode+from urllib.request import Request, urlopen+++@dataclass(frozen=True)+class SearchProfile:+ country: str+ coverage: str+ name: str+ bounds_wsen: tuple[float, float, float, float]+ crs: str+++_PROFILES = {+ "US": SearchProfile("US", "conus", "United States (CONUS)",+ (-125.0, 24.0, -66.0, 50.0), "EPSG:5070"),+ "DE": SearchProfile("DE", "germany", "Germany",+ (5.5, 47.0, 15.5, 55.2), "EPSG:32632"),+}+_COVERAGE_COUNTRIES = {p.coverage: code for code, p in _PROFILES.items()}++# WFS metadata advertises the 2025-01-01 product; the returned land object's+# BEGINN is 2025-04-11. Pin the actual feature, not an assumed download vintage.+# The digest covers canonical {properties, geometry, crs}, excluding the WFS+# request timestamp. A server update requires deliberate review of a new pin.+BKG_BOUNDARY_SHA256 = "b5e9276fe37ca1c0d65b4542a075cc246e97194ed70d3259b0d7900ad944a1d3"+BKG_BOUNDARY_URL = "https://sgx.geodatenzentrum.de/wfs_vg250?" + urlencode({+ "service": "WFS", "version": "2.0.0", "request": "GetFeature",+ "typeNames": "vg250:vg250_sta", "outputFormat": "application/json",+ "srsName": "EPSG:25832", "CQL_FILTER": "gf=4",+})+BKG_LICENSE_URL = "https://www.govdata.de/dl-de/by-2-0"+BKG_SOURCES_URL = "https://sgx.geodatenzentrum.de/web_public/gdz/datenquellen/datenquellen_vg_nuts.pdf"+_MAX_BOUNDARY_BYTES = 8 * 1024 * 1024++++class EmptySearchArea(ValueError):++ """A selected country area has no land or no eligible coarse receivers."""++++ def __init__(self, reason_code, message):++ super().__init__(message)++ self.reason_code = reason_code+++++def profile_for_search(search=None):+ """Resolve one supported profile, rejecting contradictory declarations."""+ if search is None:+ search = {}+ if not isinstance(search, dict):+ raise ValueError("search must be an object")+ country = search.get("country")+ coverage = search.get("coverage")+ if "country" in search:+ if not isinstance(country, str) or country.upper() not in _PROFILES:+ raise ValueError("Supported countries are US (CONUS) and DE (Germany)")+ country = country.upper()+ if "coverage" in search and (not isinstance(coverage, str) or coverage not in _COVERAGE_COUNTRIES):+ raise ValueError("Supported coverage values are conus (US/CONUS) and germany (DE)")+ if country is None:+ country = _COVERAGE_COUNTRIES.get(coverage, "US")+ profile = _PROFILES[country]+ if coverage is not None and coverage != profile.coverage:+ raise ValueError(f"Country {country} requires coverage {profile.coverage}")+ return profile+++def public_profiles():+ """Return JSON-safe profiles for the CLI/API/browser, with no local paths."""+ return [{"country": p.country, "coverage": p.coverage, "name": p.name,+ "bounds_wsen": list(p.bounds_wsen), "crs": p.crs}+ for p in _PROFILES.values()]+++def _canonical_bytes(record):+ return json.dumps(record, sort_keys=True, separators=(",", ":"), allow_nan=False).encode("utf-8")+++def _checked_record(record):+ """Validate the exact recorded BKG feature before constructing a mask."""+ if not isinstance(record, dict) or set(record) != {"properties", "geometry", "crs"}:+ raise ValueError("Invalid cached Germany boundary record")+ digest = hashlib.sha256(_canonical_bytes(record)).hexdigest()+ if digest != BKG_BOUNDARY_SHA256:+ raise ValueError("Germany boundary differs from the pinned BKG feature; review the source before updating the cache")+ return record+++class ReceiverMask:+ """A pickle-safe prepared land polygon, in its original projected CRS."""++ def __init__(self, record):+ record = _checked_record(record)+ from shapely.geometry import shape+ geometry = shape(record["geometry"])+ if geometry.geom_type != "MultiPolygon" or geometry.is_empty or not geometry.is_valid:+ raise ValueError("BKG Germany boundary is not a valid nonempty MultiPolygon")+ properties = record["properties"]+ if (properties.get("gf") != 4 or properties.get("lkz") != "DE"+ or record["crs"].get("properties", {}).get("name") != "urn:ogc:def:crs:EPSG::25832"):+ raise ValueError("BKG boundary has an unsupported country, component or CRS")+ self._identity = {+ "country": "DE", "coverage": "germany", "search_crs": "EPSG:32632",+ "source_crs": "EPSG:25832", "boundary_sha256": BKG_BOUNDARY_SHA256,+ "source_url": BKG_BOUNDARY_URL, "source_layer": "vg250:vg250_sta",+ "selection": "GF=4 (land)", "source_object_id": properties["objid"],+ "source_begin": properties["beginn"], "advertised_product_date": "2025-01-01",+ "attribution": "© BKG 2026, dl-de/by-2-0",+ "license_url": BKG_LICENSE_URL, "data_sources_url": BKG_SOURCES_URL,+ "processing": "Unmodified land polygon; receiver coordinates transformed from WGS84 to ETRS89 / UTM 32N",+ }+ self._initialize_geometry(geometry)++ def _initialize_geometry(self, geometry):+ import shapely+ from pyproj import Transformer+ self._geometry = geometry+ shapely.prepare(self._geometry)+ self._boundary = self._geometry.boundary+ self._to_native = Transformer.from_crs("EPSG:4326", "EPSG:25832", always_xy=True)++ @property+ def identity(self):+ return deepcopy(self._identity)++ def contains(self, lon, lat):+ """Include the polygon boundary, retaining islands and polygon holes."""+ import shapely+ x, y = self._to_native.transform(lon, lat)+ return shapely.intersects_xy(self._geometry, x, y)+++ def preflight_grid(self, bounds_wsen, spacing_m, crs, *, max_points=65536):++ """Reject empty selections before flight acquisition, with bounded work.++++ True means an actual coarse-grid receiver was found. None means the++ bounded check was inconclusive; it must not be called an empty area.++ Small grids are examined completely. Large grids probe near interior++ points of the actual clipped land components, without enumerating the++ national grid. Every probe uses the production grid/mask predicate.++ """++ from pyproj import Transformer++ from shapely.geometry import box++ from .tiled_search import _grid_extent, _tile_coordinates++++ if type(max_points) is not int or max_points < 1:++ raise ValueError("Receiver preflight point limit must be a positive integer")++ if not math.isfinite(spacing_m) or spacing_m <= 0:++ raise ValueError("Receiver grid spacing must be finite and positive")++ west, south, east, north = bounds_wsen++ left, bottom, right, top = self._to_native.transform_bounds(*bounds_wsen, densify_pts=101)++ # Within the DE profile (5.5--15.5 E, 47--55.2 N), UTM 32N northing++ # along a latitude edge is smallest nearest the 9 E central meridian.++ # Include that interior extremum explicitly; other edge extrema are++ # at corners. Do not rely on densification happening to sample 9 E.++ meridian = max(west, min(east, 9.0))++ _, northings = self._to_native.transform([meridian, meridian], [south, north])++ bottom, top = min(bottom, *northings), max(top, *northings)++ # Ten metres is a numerical allowance for this fixed projection and++ # region, not a general reprojection error bound. Only the exclusion++ # shortcut uses this expansion; actual grid membership remains exact.++ envelope = box(left - 10.0, bottom - 10.0, right + 10.0, top + 10.0)++ if not self._geometry.intersects(envelope):++ raise EmptySearchArea("empty_search_area",++ "The selected rectangle contains no German land in the BKG receiver boundary. "++ "Choose an area that includes a supported receiver location.")++++ xmin, ymin, xmax, ymax = _grid_extent(bounds_wsen, spacing_m, crs)++ total = (xmax - xmin + 1) * (ymax - ymin + 1)++ if total <= max_points:++ points = _tile_coordinates((xmin, ymin, xmax + 1, ymax + 1), spacing_m,++ bounds_wsen, crs, self)++ if len(points):++ return True++ raise EmptySearchArea("empty_search_grid",++ "The selected rectangle contains no German receiver points on the configured "++ f"{spacing_m:g} m coarse grid. Choose a wider area containing German land, "++ "or ask the operator to use a smaller coarse-grid spacing.")++++ overlap = self._geometry.intersection(envelope)++ native_to_grid = Transformer.from_crs("EPSG:25832", crs, always_xy=True)++ parts = (overlap, *getattr(overlap, "geoms", ()))++ checked = 0++ for part in parts:++ if part.is_empty:++ continue++ interior = part.representative_point()++ x, y = native_to_grid.transform(interior.x, interior.y)++ ix, iy = math.floor(x / spacing_m), math.floor(y / spacing_m)++ cells = (max(xmin, ix - 1), max(ymin, iy - 1),++ min(xmax + 1, ix + 2), min(ymax + 1, iy + 2))++ count = max(0, cells[2] - cells[0]) * max(0, cells[3] - cells[1])++ if checked + count > max_points:++ break++ checked += count++ if count and len(_tile_coordinates(cells, spacing_m, bounds_wsen, crs, self)):++ return True++ return None+++ def boundary_limited(self, lon, lat, radius_m):+ """Conservatively flag a local sensitivity search that meets the mask.++ One percent covers UTM scale variation across this Germany envelope.+ This is a truncation flag, not a calibrated geographic error bound.+ """+ from shapely.geometry import Point+ if not math.isfinite(radius_m) or radius_m < 0:+ raise ValueError("Boundary proximity radius must be finite and nonnegative")+ x, y = self._to_native.transform(lon, lat)+ if not math.isfinite(x) or not math.isfinite(y):+ return True+ point = Point(x, y)+ return not self._geometry.covers(point) or self._boundary.distance(point) <= 1.01 * radius_m++ def __getstate__(self):+ from shapely import to_wkb+ return {"geometry_wkb": to_wkb(self._geometry), "identity": self._identity}++ def __setstate__(self, state):+ from shapely import from_wkb+ self._identity = state["identity"]+ self._initialize_geometry(from_wkb(state["geometry_wkb"]))+++def receiver_mask_for_search(search, *, cache_root, offline=True):+ """Load the pinned Germany polygon, acquiring it only when online is allowed.++ No boundary is inferred from a rectangular envelope or a failed download.+ Existing cache bytes are checked each time; missing or changed data produce+ an explicit data-unavailable outcome through the acquisition adapter.+ """+ if profile_for_search(search).country == "US":+ return None+ from .data_access import DataUnavailable+ path = Path(cache_root) / "geography" / f"de-vg250-{BKG_BOUNDARY_SHA256[:16]}.json"+ try:+ if path.exists():+ with path.open("rb") as handle:+ data = handle.read(_MAX_BOUNDARY_BYTES + 1)+ if len(data) > _MAX_BOUNDARY_BYTES:+ raise ValueError("Cached Germany boundary exceeds the size limit")+ return ReceiverMask(json.loads(data))+ if offline:+ raise DataUnavailable(f"Germany receiver boundary is not cached at {path}; enable online acquisition once to obtain the pinned BKG land mask")+ request = Request(BKG_BOUNDARY_URL, headers={"User-Agent": "DOPPLERGEIST/0.1 country-boundary"})+ with urlopen(request, timeout=60) as response:+ raw = response.read(_MAX_BOUNDARY_BYTES + 1)+ if len(raw) > _MAX_BOUNDARY_BYTES:+ raise ValueError("BKG Germany boundary exceeds the download limit")+ collection = json.loads(raw)+ features = collection.get("features", [])+ if collection.get("type") != "FeatureCollection" or len(features) != 1:+ raise ValueError("Expected exactly one BKG Germany land feature")+ feature = features[0]+ record = {"properties": feature["properties"], "geometry": feature["geometry"], "crs": collection["crs"]}+ mask = ReceiverMask(record)+ path.parent.mkdir(parents=True, exist_ok=True)+ temporary = None+ try:+ with tempfile.NamedTemporaryFile(dir=path.parent, prefix=path.name + ".", suffix=".tmp", delete=False) as handle:+ temporary = Path(handle.name)+ handle.write(_canonical_bytes(record))+ os.replace(temporary, path)+ finally:+ if temporary is not None:+ temporary.unlink(missing_ok=True)+ return mask+ except DataUnavailable:+ raise+ except (OSError, ValueError, KeyError, TypeError) as error:+ raise DataUnavailable(f"Germany receiver boundary unavailable: {error}") from errordiff --cc src/dopplergeist/matching.pyindex c33d6c5,d6e9dec..d2d35a0--- a/src/dopplergeist/matching.py+++ b/src/dopplergeist/matching.py@@@ -833,8 -816,6 +833,12 @@@ def _refine_candidate(candidate, tracksdt = lower if fixed_clock else parameters[2]spatial_boundary = any(min(value-lo, hi-value) <= radius_m * 0.02for value, (lo, hi) in zip(parameters[:2], bounds[:2]))+ if receiver_mask is not None:- spatial_boundary |= bool(receiver_mask.boundary_limited(lon, lat, radius_m))++ # Powell explores a square about the seed. Its diagonal reaches++ # sqrt(2) times the per-axis radius; checking only the winning point++ # can miss a country edge that truncated another part of that square.++ spatial_boundary |= bool(receiver_mask.boundary_limited(++ candidate["lon"], candidate["lat"], np.sqrt(2.0) * radius_m))clock_boundary = bool(not fixed_clock and min(dt-lower, upper-dt) <= (upper-lower)*0.02)profiles.append(dict(score, lat=lat, lon=lon, elev_m=elevation,recording_start_time_s=float(start + dt), altitude_offset_m=float(offset),