Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)
Python acoustic-geolocationads-baircraft-trackingairplaneaudio-forensicsdoppler-effectdoppler-shiftgeolocationosintsignal-processing
perf(discovery): make event detail plots opt-in (#215)
Cinna Davis79282549045a
diff --git a/docs/PERFORMANCE_AUDIO_DISCOVERY.md b/docs/PERFORMANCE_AUDIO_DISCOVERY.mdnew file mode 100644index 0000000..ae70df7--- /dev/null+++ b/docs/PERFORMANCE_AUDIO_DISCOVERY.md@@ -0,0 +1,102 @@+# Real audio discovery performance notes++Date: 2026-11-03. Measurements below use real recordings only. No audio or+flight data was synthesized, and no location search was run on these clips.++## Measured cost in DOPPLERGEIST++The 180-second YouTube recording `yt-yjJrC9UvIrk` produced 32 ridge records.+At the same code revision and on the same input, the original CLI rendered a+summary plot and 32 per-event spectrograms: **142.86 s** wall time and+**306,232 KiB** peak RSS. With all plots disabled it took **17.51 s** and+**148,608 KiB** and produced the same `discovery.json` bytes and ridge CSVs.+The per-event plot renders alone accounted for 85.90 s under `cProfile`.++A longer real recording provides a scale check: the 3,726.45-second Bourne+Garden recording on Wikimedia Commons ([source and CC BY-SA 4.0 attribution](https://commons.wikimedia.org/wiki/File:Bourne_Garden_2020-08-28_0843.mp3))+produced 222 ridge records in 62 chunks. At `80ca06c`, the plot-heavy scan+took 462.66 s and 398,036 KiB peak RSS. At `7a8dd27`, with CLI plots disabled,+it took 285.94 s and 158,208 KiB. The output discovery JSON is byte-identical+across all three saved runs (SHA-256+`750a54c1dda9f85f06ef556ca3241e0f5ff04ced2ed7fec7f8da153afcbaf480`). The+two revisions have identical `discovery.py` and CLI implementation before the+plot-option change. The 38% wall-time reduction is an observed sequential run,+not a hardware-isolated benchmark.++An instrumented no-plot run of the hour-long recording is useful for locating+the remaining detector work, but its timings include profiler overhead and a+low-priority process competing with the national search. In that profile,+`discover_events` took 147 s cumulative: `_paths` took 97.6 s and `_spectrum`+40.6 s. Repeated short-window slope medians were a substantial `_paths` cost.+Intake took 218 s, including 204 s inside reads from FFmpeg's output pipe;+this run does not yet separate decoder CPU from wait time under host load.++The immediate change makes per-event PNGs opt-in while retaining all ridge+measurements in JSON and CSV. The CLI also makes its summary plot opt-in. The+library keeps its existing summary-plot default for callers that use the image+as an inspectable artifact. These changes do not reduce event recall.++DOPPLERGEIST's application and detector control flow is Python. NumPy/SciPy+provide compiled array, FFT and peak-finding kernels; the repository has no+Rust or C extension for this path. The one-hour profile shows that Python path+linking and repeated small-array work remain candidates for a focused native+kernel, while the array transforms already call compiled libraries.++## Event count and downstream cost++In the active real national search, 15 harmonic groups led to 1,527,829 timing+trials across 680 completed tiles. That confirms that additional independent+event groups can increase association and scoring work. It does not justify+dropping detections indiscriminately: multiple correct events constrain the+shared recording clock and reject wrong hypotheses. Harmonics and fragments+must be consolidated before counting independent events; uncertain groups need+to be screened with measured evidence and recall checks.++## Relevant open implementations++- [doppler-locate](https://github.com/digitalis-p/doppler-locate) keeps its+ workflow in Python and uses NumPy/SciPy, Numba, and optional OpenMP/CUDA+ kernels. Its automatic detector uses a bank of 108 constant-speed flyby+ shapes, FFT cross-correlation on a whitened log-frequency spectrogram,+ steady-tone suppression, harmonic grouping, then ridge tracing. Its README+ reports about 70 seconds for one hour on 32 CPU cores. The method is a+ candidate for a coarse acoustic sieve before the current tracker; adopting+ it needs real-recording recall comparisons before it can reject paths. Its+ separate geometric search uses compiled track-local CPA/candidate loops and+ reports 126 s / 20 GB on CPU versus 80 s / 12 GB on an RTX 4090 for its+ CONUS case. The kernel's float32/parabolic CPA is a proposal technique;+ DOPPLERGEIST's continuous-track scorer must still decide final candidates.+- [dopplerprint](https://github.com/Hexay/dopplerprint) uses a Rust core with+ Python orchestration. On its real one-hour recording, caching repeated+ fragment-pair curve fits cut discovery CPU from 249 s to 11.5 s with+ byte-identical JSON. Its real search scorer fell from 443 to 117 CPU-seconds+ after hoisting log-frequency terms and using binary search for monotone f0+ sweeps. DOPPLERGEIST already uses local-slope fragment stitching rather than+ those repeated curve fits, and its primary ranking uses physical ridge+ residuals rather than the same brightness score, so these exact changes do+ not transfer directly.+- [Hermes aircraft Doppler geolocation](https://github.com/mojomast/hermes-aircraft-doppler-geolocation)+ uses Python/SciPy Viterbi ridge tracing with a manually selected band and+ time interval. It is a compact reference for ridge fitting and an explicit+ false-fit gate, but it does not provide a whole-recording automatic scan or+ a measured optimization.+- [EchoTrace](https://github.com/Goprogabriel/echotrace) is a browser-local+ TypeScript implementation that runs analysis in Web Workers. It bounds+ spectrogram storage at four million cells by increasing the frame hop for+ long clips, and its ridge dynamic program uses a linear-time L1 distance+ transform instead of a pairwise transition search. Those are useful+ algorithmic ideas, though DOPPLERGEIST already streams 120-second chunks and+ its tracker has different association semantics. EchoTrace caps proposals at+ eight and has users select up to three; copying that cap would trade away+ recall. Its own documentation says there is no independent real-world+ validation. The synthetic demo was not run or used as evidence.++## Next measurements worth doing++1. Time FFmpeg intake separately from scanning, on an otherwise idle host, so+ wall time is not mistaken for Python detector cost.+2. Compare a real-recording template-bank pre-screen against current discovery,+ measuring recall and false proposals before feeding it into association.+3. If `_paths` remains the dominant detector cost after that comparison, compile+ only its path-linking loop and keep orchestration in Python. The present+ evidence does not support a blanket language rewrite.diff --git a/scripts/discover_audio.py b/scripts/discover_audio.pyindex dcf6543..e8e6d85 100644--- a/scripts/discover_audio.py+++ b/scripts/discover_audio.py@@ -20,12 +20,18 @@ def main():parser = argparse.ArgumentParser(description=__doc__)parser.add_argument("audio")parser.add_argument("--output", required=True)+ parser.add_argument("--plot-summary", action="store_true",+ help="render the recording-level ridge overview plot")+ parser.add_argument("--plot-event-details", action="store_true",+ help="render one spectrogram PNG per discovered event (slower)")args = parser.parse_args()output = Path(args.output)output.mkdir(parents=True, exist_ok=False)began = time.monotonic()recording = intake_recording(args.audio, output / "intake")- events = discover_events(recording, output_dir=output / "discovery")+ events = discover_events(recording, output_dir=output / "discovery",+ plot_summary=args.plot_summary,+ plot_event_details=args.plot_event_details)root = Path(__file__).resolve().parents[1]sources = sorted((root / "src" / "dopplergeist").glob("*.py"))code_hash = hashlib.sha256()diff --git a/src/dopplergeist/discovery.py b/src/dopplergeist/discovery.pyindex 70aca19..ee75eda 100644--- a/src/dopplergeist/discovery.py+++ b/src/dopplergeist/discovery.py@@ -247,13 +247,14 @@ def _stitch_fragments(paths, settings, diagnostics=None):return result-def discover_events(recording, *, output_dir=None, settings=None):+def discover_events(recording, *, output_dir=None, settings=None, plot_summary=True, plot_event_details=False):"""Return shared DetectedEvent dictionaries from a disk-backed recording.Fixed defaults scan 40–1600 Hz in overlapping chunks; no event hints, flightdata or truth paths are accepted. Each measured ridge gets a record; ashared harmonic_group_id makes those records one independent proposal.Optional diagnostics retain all measured harmonic ridges and chunk counts.+ Per-event spectrograms are expensive and are rendered only when requested."""settings = settings or DiscoverySettings()sr = recording.metadata["sample_rate_hz"]@@ -315,12 +316,14 @@ def discover_events(recording, *, output_dir=None, settings=None):"harmonic_ridges": [[points.tolist() for points in group] for group in groups],"scope": "automatic acoustic moving-tone proposals; aircraft identity unconfirmed"}(output / "discovery.json").write_text(json.dumps(diagnostics, indent=2, allow_nan=False) + "\n")- _plot_groups(groups, output / "discovery.png", recording.metadata["duration_s"], settings.max_gap_s)+ if plot_summary:+ _plot_groups(groups, output / "discovery.png", recording.metadata["duration_s"], settings.max_gap_s)for event, group in zip(events, event_groups):points = np.array([[p["time_s"], p["frequency_hz"], p["weight"]] for p in event["ridge"]])np.savetxt(output / f"{event['event_id']}.csv", points, delimiter=",",header="time_s,frequency_hz,weight", comments="")- _plot_event(recording, group, event, settings, output / f"{event['event_id']}.png")+ if plot_event_details:+ _plot_event(recording, group, event, settings, output / f"{event['event_id']}.png")return events