Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)
Python acoustic-geolocationads-baircraft-trackingairplaneaudio-forensicsdoppler-effectdoppler-shiftgeolocationosintsignal-processing
Add Italy search profile including Sicily and Sardinia (#400)
andrea veneroni66fe55c47e0c
diff --git a/README.md b/README.mdindex 2f554f5..c5bc2b4 100644--- a/README.md+++ b/README.md@@ -41,7 +41,7 @@ Colsto did not release the original tools or source code from his [aircraft Dopp| **Automatic acoustic discovery** | Scans an entire recording for moving frequency tracks and groups related harmonics, without manually selecting event windows or frequency bands. || **Historical aircraft data** | Acquires and indexes ADSB.lol flight archives; caches reusable trajectory, terrain, and geoid inputs. || **Location search** | Compares predicted Doppler and sound-arrival patterns across candidate receiver locations, including evidence from multiple independent flyovers. |-| **Geographic profiles** | Search profiles for the contiguous United States, Germany, Spain, and metropolitan France (including Corsica), with optional user-provided regional bounds. |+| **Geographic profiles** | Search profiles for the contiguous United States, Germany, Spain, metropolitan France (including Corsica), and Italy (including Sicily and Sardinia), with optional user-provided regional bounds. || **Local browser application** | Recording uploads, persistent analysis jobs, inspectable results, progress monitoring, and supported resume/cancellation workflows. || **Research and reproducibility** | Configured measurement/replay commands, documented experiments, diagnostic artifacts, and decision rules for unresolved searches. |@@ -107,7 +107,7 @@ To check progress after the job ID is printed:The progress percentage is stage-based and can remain unchanged during long tile subdivisions. Recent search-file activity is a liveness clue, not proof that the queue is shrinking or that a location was found.-The default country profile is `US` (contiguous United States); use `--country DE` for Germany, `--country ES` for Spain, or `--country FR` for metropolitan France including Corsica. Spain and France use broad geographic envelopes without country polygons, so add independently justified `--bounds WEST SOUTH EAST NORTH` for a regional search. The France profile excludes overseas territories. See [country profiles](docs/COUNTRIES.md) and the [local upload workflow](docs/PRODUCT.md) for request options, cancellation, resuming jobs, and upload limits.+The default country profile is `US` (contiguous United States); use `--country DE` for Germany, `--country ES` for Spain, or `--country FR` for metropolitan France including Corsica, or `--country IT` for Italy including Sicily and Sardinia. Spain, France and Italy use broad geographic envelopes without country polygons, so add independently justified `--bounds WEST SOUTH EAST NORTH` for a regional search. The France profile excludes overseas territories. See [country profiles](docs/COUNTRIES.md) and the [local upload workflow](docs/PRODUCT.md) for request options, cancellation, resuming jobs, and upload limits.**Data requirements:** Initial online acquisition may take hours and tens of gigabytes of disk space, depending on the recording time and search region. Downloads and caches are reusable; the application does not enforce an automatic disk-space cap.diff --git a/docs/COUNTRIES.md b/docs/COUNTRIES.mdindex 712deb4..0c1d915 100644--- a/docs/COUNTRIES.md+++ b/docs/COUNTRIES.md@@ -1,6 +1,6 @@# Country profiles-The automatic request and upload interface support four receiver regions:+The automatic request and upload interface support five receiver regions:| Country | Coverage | Envelope: west, south, east, north | Search CRS | Receiver region || --- | --- | --- | --- | --- |@@ -8,6 +8,7 @@ The automatic request and upload interface support four receiver regions:| `DE` | `germany` | `5.5, 47, 15.5, 55.2` | `EPSG:32632` | BKG VG250 Germany land polygon inside the envelope || `ES` | `spain` | `-18.2, 27.5, 4.5, 43.9` | `EPSG:6933` | Spain and territories bounding envelope; no country polygon || `FR` | `france` | `-5.5, 41.2, 10, 51.2` | `EPSG:2154` | Metropolitan France including Corsica bounding envelope; no country polygon |+| `IT` | `italy` | `6.6, 35.4, 18.6, 47.1` | `EPSG:7794` | Italy including Sicily, Sardinia and Lampedusa bounding envelope; no country polygon |The envelope limits optional rectangular subsets. Germany's envelope includesforeign territory; the land polygon removes those receiver locations before@@ -15,7 +16,10 @@ terrain sampling or scoring. Islands, exclaves and polygon holes are preserved.The boundary is generalized geographic data, not a survey of a microphone site.Spain uses an outer envelope covering mainland Spain and its island territories.France covers mainland France and Corsica, excluding overseas departments and-territories, with a Lambert-93 metric search grid. These two profiles do not+territories, with a Lambert-93 metric search grid. Italy covers the mainland,+Sicily, Sardinia and the Pelagie islands with the national RDN2008 / Italy zone+(E-N) transverse Mercator grid; San Marino, Vatican City and parts of+neighboring countries and the sea fall inside its envelope. These three profiles do notapply country land masks: their envelopes include sea and neighboring countries.Use explicit bounds that describe the intended receiver area. Aircraft outsidethe receiver bounds remain eligible when they are within the acoustic search margin. Country support@@ -29,11 +33,11 @@ dopplergeist analyze recording.mp3 --country DE --jobs jobs --online \--time-source 'Evidence establishing the recording-start interval'```-Use `--country FR` for metropolitan France, including Corsica, or select France-in the browser's country selector. The upload API accepts+Use `--country FR` for metropolitan France, including Corsica, or `--country IT`+for Italy, or select the country in the browser's country selector. The upload API accepts`"search":{"country":"DE"}`, `"search":{"country":"ES"}` or-`"search":{"country":"FR"}`. It also accepts the matching coverage names-`germany`, `spain` and `france`. A normalized request records the coverage;+`"search":{"country":"FR"}` or `"search":{"country":"IT"}`. It also accepts+the matching coverage names `germany`, `spain`, `france` and `italy`. A normalized request records the coverage;an explicitly supplied country is normalized to uppercase. If both are supplied,they must agree. `bounds_wsen` and `--bounds` narrow the selected profile. Theymust be an independently justified prior, not the evaluation location.diff --git a/docs/INTERFACES.md b/docs/INTERFACES.mdindex 74e6631..53eb448 100644--- a/docs/INTERFACES.md+++ b/docs/INTERFACES.md@@ -33,16 +33,17 @@ returns `needs_input` with `required_fields: ["time_window"]` if a defensible wiis still unavailable. Do not fabricate a timestamp. A derived best recording startis a result field and must not be mislabeled independent timestamp evidence.-`search.country` accepts `US` (CONUS), `DE` (Germany), `ES` (Spain) or `FR`-(metropolitan France including Corsica); lowercase ISO codes are normalized.+`search.country` accepts `US` (CONUS), `DE` (Germany), `ES` (Spain), `FR`+(metropolitan France including Corsica) or `IT` (Italy including Sicily and+Sardinia); lowercase ISO codes are normalized.The corresponding `search.coverage` values are-`conus`, `germany`, `spain` and `france`.+`conus`, `germany`, `spain`, `france` and `italy`.Either country or coverage is sufficient, and contradictory values are rejected.Legacy `{"coverage":"conus"}` objects keep their serialized shape; the validatordoes not insert a country into old requests. Optional `bounds_wsen` further restrictthe selected profile to `[west, south, east, north]` in longitude/latitude degrees.Bounds must lie inside its declared envelope. Germany's land mask further limits-receiver points. Spain and metropolitan France use bounding envelopes without+receiver points. Spain, metropolitan France and Italy use bounding envelopes withoutcountry polygons; provide explicit bounds for a suitably regional search. TheFrance profile excludes overseas territories. Neighboring aircraftremain eligible outside the receiver area when they fall within acoustic range. The capabilitydiff --git a/docs/PRODUCT.md b/docs/PRODUCT.mdindex cba22a5..28bb1a6 100644--- a/docs/PRODUCT.md+++ b/docs/PRODUCT.md@@ -10,7 +10,7 @@ scope and limitations.The automatic product path is separate from the historical research `run` and`measure` commands. Uploads supply audio, an optional defensible UTC window for**sample zero**, and a declared country profile (US/CONUS by default, Germany,-Spain or metropolitan France including Corsica).+Spain, metropolitan France including Corsica, or Italy).An optional rectangle narrows that profile. Event windows, frequencybands, flight IDs and ground-truth coordinates are not upload inputs.@@ -93,6 +93,13 @@ browser option, or `"search":{"country":"FR"}` in the API. Requests normalize toenvelope without a country polygon; use independently justified `--bounds` fora regional search. Overseas territories are outside this profile.+For Italy, use `analyze --country IT`, the Italy browser option, or+`"search":{"country":"IT"}`. Requests normalize to `"coverage":"italy"` and use+`EPSG:7794` (RDN2008 / Italy zone, E-N). The envelope covers the mainland,+Sicily, Sardinia and Lampedusa without a country polygon, so it also contains+sea and border areas of neighboring countries; use independently justified+`--bounds` for a regional search.+`analyze` prints the job ID and a monitor command to standard error as soon asthe upload is queued. Search progress is based on completed grid work and ispersisted after a 2.5 percentage-point increase or at least every 15 secondsdiff --git a/scripts/check_country_profiles.py b/scripts/check_country_profiles.pyindex 8f85f71..4bd9757 100644--- a/scripts/check_country_profiles.py+++ b/scripts/check_country_profiles.py@@ -39,6 +39,8 @@ def check_contracts():madrid_subset = [-4.16, 40.05, -2.97, 40.95]paris_subset = [2.2, 48.8, 2.5, 48.95]corsica_subset = [9.0, 41.3, 9.3, 41.5]+ milan_subset = [9.05, 45.4, 9.3, 45.55]+ lampedusa_subset = [12.5, 35.48, 12.65, 35.55]cases = (("omitted search", None, {"coverage": "conus"}, "US"),("legacy CONUS", {"coverage": "conus"}, {"coverage": "conus"}, "US"),@@ -63,6 +65,12 @@ def check_contracts():{"country": "FR", "coverage": "france", "bounds_wsen": paris_subset}, "FR"),("Corsica region", {"coverage": "france", "bounds_wsen": corsica_subset},{"coverage": "france", "bounds_wsen": corsica_subset}, "FR"),+ ("Italy country", {"country": "it"}, {"country": "IT", "coverage": "italy"}, "IT"),+ ("Italy coverage", {"coverage": "italy"}, {"coverage": "italy"}, "IT"),+ ("Italy region", {"country": "IT", "bounds_wsen": milan_subset},+ {"country": "IT", "coverage": "italy", "bounds_wsen": milan_subset}, "IT"),+ ("Lampedusa region", {"coverage": "italy", "bounds_wsen": lampedusa_subset},+ {"coverage": "italy", "bounds_wsen": lampedusa_subset}, "IT"),)validated = {}for name, search, expected, country in cases:@@ -84,7 +92,10 @@ def check_contracts():{"country": "DE", "coverage": "conus"},{"country": "US", "coverage": "germany"},{"country": "DE", "coverage": "spain"},- {"country": "IT"},+ {"country": "PT"},+ {"country": "IT", "coverage": "france"},+ {"country": "FR", "coverage": "italy"},+ {"country": "IT", "bounds_wsen": madrid_subset},{"country": "FR", "coverage": "conus"},{"country": "FR", "coverage": "spain"},{"country": "DE", "coverage": "france"},@@ -111,8 +122,8 @@ def check_contracts():raise CheckFailure(f"Invalid search was accepted: {search!r}")catalog = public_profiles()- require(len(catalog) == 4 and {p["country"] for p in catalog} == {"US", "DE", "ES", "FR"},- "Capabilities must describe exactly the supported US, DE, ES and FR profiles")+ require(len(catalog) == 5 and {p["country"] for p in catalog} == {"US", "DE", "ES", "FR", "IT"},+ "Capabilities must describe exactly the supported US, DE, ES, FR and IT profiles")for item in catalog:profile = profile_for_search({"country": item["country"]})require(item["coverage"] == profile.coverage and item["crs"] == profile.crs@@ -125,6 +136,8 @@ def check_contracts():"Spain must use its declared global equal-area search CRS")require(profile_for_search({"country": "FR"}).crs == "EPSG:2154","Metropolitan France must use its declared Lambert-93 search CRS")+ require(profile_for_search({"country": "IT"}).crs == "EPSG:7794",+ "Italy must use its declared RDN2008 / Italy zone (E-N) search CRS")print(f"PASS requests: {len(cases)} normalization/JSON round-trip cases; "f"{len(rejected)} invalid declarations rejected; profile catalog agrees")return validated@@ -230,24 +243,21 @@ def check_spain_grid(validated):"no country polygon applied")-def check_france_grids(cache_root, validated):+def check_envelope_grids(cache_root, validated, profile_case, places, regions, label):import numpy as npfrom pyproj import CRS, Transformerfrom dopplergeist.geography import profile_for_search, receiver_mask_for_searchfrom dopplergeist.tiled_search import _grid_extent, _tile_coordinates- profile = profile_for_search(validated["France country"]["search"])- # Rounded real landmarks check the envelope, not recording-location truth.- places = (("Brest", -4.49, 48.39), ("Dunkirk", 2.38, 51.04),- ("Paris", 2.35, 48.86), ("Bonifacio, Corsica", 9.16, 41.39))+ profile = profile_for_search(validated[profile_case]["search"])west, south, east, north = profile.bounds_wsen- require(all(west < lon < east and south < lat < north for _, lon, lat in places),- "France envelope must include western/northern mainland France and southern Corsica")+ for place, lon, lat in places:+ require(west < lon < east and south < lat < north, f"{label} envelope must include {place}")area = CRS(profile.crs).area_of_userequire(area.west <= west < east <= area.east and area.south <= south < north <= area.north,- "France envelope must stay within the search CRS area of use")+ f"{label} envelope must stay within the search CRS area of use")forward = Transformer.from_crs("EPSG:4326", profile.crs, always_xy=True)- for name in ("France region", "Corsica region"):+ for name in regions:search = validated[name]["search"]require(receiver_mask_for_search(search, cache_root=cache_root, offline=True) is None,f"{name}: the bounding envelope must not acquire a German polygon")@@ -261,12 +271,29 @@ def check_france_grids(cache_root, validated):require(np.isfinite(x).all() and np.isfinite(y).all()and np.allclose(x, receivers.x.to_numpy(), rtol=0, atol=1e-5)and np.allclose(y, receivers.y.to_numpy(), rtol=0, atol=1e-5),- f"{name}: grid x/y disagree with Lambert-93")+ f"{name}: grid x/y disagree with {profile.crs}")w, s, e, n = boundsrequire(bool(np.all((receivers.lon >= w) & (receivers.lon <= e)& (receivers.lat >= s) & (receivers.lat <= n))),f"{name}: receiver grid escaped the selected bounds")- print(f"PASS {name}: {len(receivers)} receivers at {spacing:g} m in Lambert-93; no country polygon")+ print(f"PASS {name}: {len(receivers)} receivers at {spacing:g} m in {profile.crs}; no country polygon")+++def check_france_grids(cache_root, validated):+ # Rounded real landmarks check the envelope, not recording-location truth.+ check_envelope_grids(cache_root, validated, "France country",+ (("Brest", -4.49, 48.39), ("Dunkirk", 2.38, 51.04),+ ("Paris", 2.35, 48.86), ("Bonifacio, Corsica", 9.16, 41.39)),+ ("France region", "Corsica region"), "France")+++def check_italy_grids(cache_root, validated):+ # Rounded real landmarks at the national extremes check the envelope only.+ check_envelope_grids(cache_root, validated, "Italy country",+ (("Vetta d'Italia", 12.18, 47.09), ("Aosta valley west", 6.63, 45.83),+ ("Otranto", 18.52, 40.14), ("Lampedusa", 12.6, 35.5),+ ("Cagliari, Sardinia", 9.11, 39.22), ("Palermo, Sicily", 13.36, 38.12)),+ ("Italy region", "Lampedusa region"), "Italy")def check_border_grid(mask, validated):@@ -363,6 +390,7 @@ def main(argv=None):try:validated = check_contracts()check_france_grids(args.cache_root, validated)+ check_italy_grids(args.cache_root, validated)mask = check_mask(args.cache_root, args.online, validated)check_spain_grid(validated)check_border_grid(mask, validated)diff --git a/src/dopplergeist/cli.py b/src/dopplergeist/cli.pyindex 4b66c9d..7f8d13e 100644--- a/src/dopplergeist/cli.py+++ b/src/dopplergeist/cli.py@@ -120,8 +120,8 @@ def main():analyze.add_argument("--start-utc",help="Earliest UTC recording start, including explicit timezone")analyze.add_argument("--end-utc",help="Latest UTC recording start; defaults to earliest")analyze.add_argument("--time-source",help="How the recording start window was established")- analyze.add_argument("--country",type=str.upper,choices=("US", "DE", "ES", "FR"),- help="Receiver country: US (CONUS, default), DE (Germany), ES (Spain), or FR (metropolitan France, including Corsica)")+ analyze.add_argument("--country",type=str.upper,choices=("US", "DE", "ES", "FR", "IT"),+ help="Receiver country: US (CONUS, default), DE (Germany), ES (Spain), FR (metropolitan France, including Corsica), or IT (Italy, including Sicily and Sardinia)")analyze.add_argument("--bounds",type=float,nargs=4,metavar=("WEST", "SOUTH", "EAST", "NORTH"),help="Limit the selected country profile to a WGS84 rectangle in decimal degrees")analyze.add_argument("--jobs",default="jobs",help="Persistent job directory")diff --git a/src/dopplergeist/contracts.py b/src/dopplergeist/contracts.pyindex 9f429b4..e2b5ead 100644--- a/src/dopplergeist/contracts.py+++ b/src/dopplergeist/contracts.py@@ -21,8 +21,8 @@ class RecordingTimeWindow(TypedDict):class SearchArea(TypedDict):- country: NotRequired[Literal["US", "DE", "ES", "FR"]]- coverage: NotRequired[Literal["conus", "germany", "spain", "france"]]+ country: NotRequired[Literal["US", "DE", "ES", "FR", "IT"]]+ coverage: NotRequired[Literal["conus", "germany", "spain", "france", "italy"]]bounds_wsen: NotRequired[list[float]]diff --git a/src/dopplergeist/geography.py b/src/dopplergeist/geography.pyindex 31a947e..49ef251 100644--- a/src/dopplergeist/geography.py+++ b/src/dopplergeist/geography.py@@ -2,9 +2,9 @@Country masks restrict receiver locations, never aircraft trajectories. The USprofile deliberately retains the historical CONUS rectangle and CRS. Germany-uses BKG VG250 land geometry in its native ETRS89 / UTM 32N coordinates. Spain-and metropolitan France use outer geographic envelopes; the caller's declared-bounds select the actual receiver search region.+uses BKG VG250 land geometry in its native ETRS89 / UTM 32N coordinates. Spain,+metropolitan France and Italy use outer geographic envelopes; the caller's+declared bounds select the actual receiver search region."""from copy import deepcopyfrom dataclasses import dataclass@@ -36,6 +36,8 @@ _PROFILES = {(-18.2, 27.5, 4.5, 43.9), "EPSG:6933"),"FR": SearchProfile("FR", "france", "France (metropolitan; bounding envelope)",(-5.5, 41.2, 10.0, 51.2), "EPSG:2154"),+ "IT": SearchProfile("IT", "italy", "Italy (including Sicily and Sardinia; bounding envelope)",+ (6.6, 35.4, 18.6, 47.1), "EPSG:7794"),}_COVERAGE_COUNTRIES = {p.coverage: code for code, p in _PROFILES.items()}@@ -72,10 +74,10 @@ def profile_for_search(search=None):coverage = search.get("coverage")if "country" in search:if not isinstance(country, str) or country.upper() not in _PROFILES:- raise ValueError("Supported countries are US (CONUS), DE (Germany), ES (Spain) and FR (metropolitan France)")+ raise ValueError("Supported countries are US (CONUS), DE (Germany), ES (Spain), FR (metropolitan France) and IT (Italy)")country = country.upper()if "coverage" in search and (not isinstance(coverage, str) or coverage not in _COVERAGE_COUNTRIES):- raise ValueError("Supported coverage values are conus (US/CONUS), germany (DE), spain (ES) and france (FR)")+ raise ValueError("Supported coverage values are conus (US/CONUS), germany (DE), spain (ES), france (FR) and italy (IT)")if country is None:country = _COVERAGE_COUNTRIES.get(coverage, "US")profile = _PROFILES[country]