Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)
Python acoustic-geolocationads-baircraft-trackingairplaneaudio-forensicsdoppler-effectdoppler-shiftgeolocationosintsignal-processing
feat: prototype optional ENF and document single-ridge candidates (#223)
Cinna Davis7979ba2ae0d8
diff --git a/docs/AUDIO_DISCOVERY.md b/docs/AUDIO_DISCOVERY.mdindex 9839789..62f090c 100644--- a/docs/AUDIO_DISCOVERY.md+++ b/docs/AUDIO_DISCOVERY.md@@ -17,6 +17,9 @@ integer ratios and rational ratios (e.g. measured 2f and 3f with an absentfundamental) count as one independent event. The lowest measured harmonic is therepresentative ridge; no fundamental is invented. Harmonic count denotes distinctmeasured ratios, including fragmented traces of the same harmonic only once.+Harmonic grouping is optional consolidation: a ridge without a compatible+partner remains a single-ridge event candidate. The detector does not require a+second harmonic.The output preserves recording-relative ridge samples and weights, measuredwindow/band, harmonic-group identity, contrast/support, spectral resolution and@@ -26,6 +29,13 @@ source-frequency estimate constrains downstream search. These are **acousticmoving-tone proposals**, with aircraft identity unconfirmed. Association mustreject unsupported proposals and count harmonic groups once.+In the current real one-hour upload, automatic discovery produced 15 independent+groups: 14 have one observed ridge and one has three. This is a useful reminder+that harmonic grouping can consolidate evidence when multiple bands are+recorded, but a single-ridge candidate must remain eligible for downstream+assessment. The audio-only discovery result does not confirm aircraft identity+or recording location.+With output_dir, discovery saves `discovery.json` (all events, settings, chunkcounts, original harmonic ridges and input hash), `discovery.png` (overview), andper-event CSV plus spectrogram PNG. Each spectrogram image is limited to a chunk'sdiff --git a/docs/ENF_TIMESTAMPING.md b/docs/ENF_TIMESTAMPING.mdnew file mode 100644index 0000000..75c971a--- /dev/null+++ b/docs/ENF_TIMESTAMPING.md@@ -0,0 +1,112 @@+# ENF timestamp recovery: research status++Electrical Network Frequency (ENF) may provide recording-time evidence when a+real mains-related signal is captured in the audio and a matching, timestamped+grid reference exists. A stationary 50/60 Hz tone by itself is not enough: fans,+power supplies, motors and other equipment can produce similar tones. ENF+cannot be reported as a timestamp until the extracted frequency variation has+been matched to an independent reference and ambiguity and clock drift have+been assessed.++## Current prototype++[`scripts/scan_enf_signal.py`](../scripts/scan_enf_signal.py) screens real audio+for peaks near 50/60 Hz and their first three harmonics. It reports per-window+frequency estimates, local spectral prominence and coverage in a new output+directory. It does **not** identify grid hum, compare a historical reference,+estimate UTC, infer a location, or alter the aircraft search request. A strong+screen result is only a reason to investigate the trace further.++Example for a new output directory:++```bash+python scripts/scan_enf_signal.py recording.mp3 \+ --output-dir results/enf-screening-v1+```++The default uses the same 4 kHz mono decoding path already used by DOPPLERGEIST.+The 4-second window and 1-second hop are screening settings, not a calibrated+ENF extractor. A future timestamp estimator should test each usable band against+real references. One sufficiently strong line can be evaluated at its known or+hypothesized harmonic order; additional harmonics are optional corroboration,+not an acceptance requirement. Longer windows and held-out real recordings are+still needed to validate frequency precision and false-match risk.++## First real-audio screen (2026-11-03)++The prototype screened 50 real recordings at 4 kHz: 24 files from the+publisher-site-labelled corpus, 5 videos without an exact microphone point, 20+AeroSonic recordings, and the active one-hour upload. The site labels are not+independent recording-time evidence. The scanner did not read truth-coordinate+files or pass any result into aircraft search.++For this exploratory summary, a *persistent tone candidate* means at least+one of the six 50/60 Hz harmonic bands had local prominence of at least 10 dB+in at least 80% of its complete 4-second windows (1-second hop). This is only+a spectral peak threshold; it does not distinguish grid frequency from a+motor, power supply, interference or other sound.++| Real-audio group | Recordings | Persistent tone candidate | Two harmonics at one nominal base (descriptive) |+| --- | ---: | ---: | ---: |+| Publisher-site-labelled corpus | 24 | 2 | 0 |+| Unknown exact microphone point | 5 | 0 | 0 |+| AeroSonic recordings | 20 | 0 | 0 |+| Active one-hour upload | 1 | 0 | 0 |++Two recordings had at least one persistent tone candidate: `site-914.mp3` (a+50 Hz fundamental and a separate 120 Hz band) and `site-919.mp3` (a 50 Hz+fundamental). No recording had two persistent bands at one nominal base. That+last column is descriptive only: one coherent line remains eligible for a+future reference match, and the lack of a second harmonic is not grounds to+reject it. The one-hour upload had no band meeting this screen's persistence+threshold; that does not prove its audio lacks a weaker or longer-window ENF+trace. No reference matching or UTC estimate was attempted, so this screen+establishes neither ENF availability nor timestamp recovery rates.++Full per-window traces, summaries and SHA-256 receipts are preserved locally+at `/home/cinna/.local/state/dopplergeist-enf-screening-20261103/corpus-v1`.+The recordings and their local paths are not included in the repository.++## Reference-data audit++- The official [FNET/GridEye real-time display](https://fnetpublic.utk.edu/frequencygauge.html)+ describes its live plot as real-time frequency measurements.+- The FNET [table display](https://fnetpublic.utk.edu/tabledisplay.html) warns+ that displayed frequency values are sample data, not current grid+ measurements, and that actual values have been hidden from that display+ since 2018.+- [OpenENF](https://github.com/openenf/openenf) documents public references for+ parts of Europe and says public North American reference data is unavailable.+- [nate-griff/enf](https://github.com/nate-griff/enf) is a useful implementation+ reference. Its README describes QIFFT harmonic extraction and FFT-accelerated+ sliding correlation, but labels the tool an investigative aid rather than a+ proof system. Its documented collector captures the current FNET chart, and+ its checked-in daily CSVs cover 2026-03-20 through 2026-05-12. That archive+ cannot match the active 2026-09-08 upload.+- A 2025 ENF extraction evaluation reports that target duration, reference+ duration and signal-to-noise ratio affect timestamp accuracy: [Maiberger,+ Gusakov and Routtenberg, CISS 2025](https://doi.org/10.1109/CISS64860.2025.10944719).++FNET can support building a prospective archive by collecting its real-time+charts, but this does not create historical references for past recordings.+The repository's image-derived reference series also needs independent+validation of sampling cadence, frequency calibration, missing-data handling+and redistribution terms before it can be used for an accepted timestamp.++## Integration gate++Only consider passing an ENF-derived time window into the existing aircraft+pipeline after real recordings with independently established sample-zero UTC+have been matched to a genuine historical grid reference. Keep extraction+quality, reference provenance, clock-rate drift, alternative timestamp peaks+and match separation in the result. Do not require multiple harmonics: one+well-tracked line can support a timestamp when its independent reference match+is unique and strong enough under the measured uncertainty. Other harmonics+can strengthen that evidence when present. If no usable line exists, the+reference is missing, or the match remains ambiguous, return the existing+`needs_input`/inconclusive path. ENF region identification is at most coarse+interconnection evidence; it is not an observer coordinate.++The initial screen must run on DOPPLERGEIST's real audio only. Do not synthesize+hum, fill missing reference intervals, tune the matcher on the recording under+evaluation, or feed recording truth into extraction or reference search.diff --git a/scripts/scan_enf_signal.py b/scripts/scan_enf_signal.pynew file mode 100644index 0000000..dc96901--- /dev/null+++ b/scripts/scan_enf_signal.py@@ -0,0 +1,95 @@+"""Measure 50/60 Hz harmonic-tone availability in real recordings.++This scanner does not identify grid hum, match a reference, or estimate UTC.+Use a new output directory for each run; do not use its results as search time+evidence.+"""+import argparse+import csv+import hashlib+import json+from pathlib import Path++from dopplergeist.audio import decode+from dopplergeist.enf import scan_enf_tones, summarize_enf_tones+++def sha256(path):+ digest = hashlib.sha256()+ with Path(path).open("rb") as stream:+ for block in iter(lambda: stream.read(1024 * 1024), b""):+ digest.update(block)+ return digest.hexdigest()+++def main():+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("inputs", nargs="+", type=Path, help="real audio files to screen")+ parser.add_argument("--output-dir", required=True, type=Path,+ help="new directory for scan receipts and CSV evidence")+ parser.add_argument("--sample-rate", type=int, default=4000)+ parser.add_argument("--frame-s", type=float, default=4.0)+ parser.add_argument("--hop-s", type=float, default=1.0)+ parser.add_argument("--nominals", type=float, nargs="+", default=(50.0, 60.0))+ parser.add_argument("--harmonics", type=int, nargs="+", default=(1, 2, 3))+ parser.add_argument("--prominence-db", type=float, default=10.0,+ help="screening threshold for local peak prominence, not an ENF confidence")+ args = parser.parse_args()+ if args.output_dir.exists():+ raise FileExistsError("Use a new output directory to preserve prior scan evidence")+ if len(set(args.inputs)) != len(args.inputs):+ raise ValueError("Input paths must be unique")+ for path in args.inputs:+ if not path.is_file():+ raise FileNotFoundError(path)++ args.output_dir.mkdir(parents=True)+ trace_dir = args.output_dir / "traces"+ trace_dir.mkdir()+ summary_rows = []+ receipts = []+ for source in args.inputs:+ source = source.resolve()+ source_digest = sha256(source)+ samples, sr = decode(source, sample_rate=args.sample_rate)+ traces = scan_enf_tones(samples, sr, nominals=args.nominals, harmonics=args.harmonics,+ frame_s=args.frame_s, hop_s=args.hop_s)+ estimates = summarize_enf_tones(traces, prominence_threshold_db=args.prominence_db)+ stem = f"{source.stem}-{source_digest[:12]}"+ trace_path = trace_dir / f"{stem}.csv"+ with trace_path.open("w", newline="") as stream:+ writer = csv.DictWriter(stream, fieldnames=("offset_s", "nominal_hz", "harmonic",+ "target_hz", "frequency_hz",+ "local_prominence_db"))+ writer.writeheader()+ writer.writerows(traces)+ for estimate in estimates:+ summary_rows.append({"source": str(source), "source_sha256": source_digest,+ "duration_s": len(samples) / sr, **estimate})+ receipts.append({"source": str(source), "source_sha256": source_digest,+ "sample_rate_hz": sr, "duration_s": len(samples) / sr,+ "trace_csv": str(trace_path.relative_to(args.output_dir)),+ "harmonic_summaries": estimates})+ print(f"scanned {source.name}: {len(samples) / sr:.1f} s, {len(traces)} tone observations")++ with (args.output_dir / "summary.csv").open("w", newline="") as stream:+ columns = ("source", "source_sha256", "duration_s", "nominal_hz", "harmonic", "target_hz",+ "frames", "prominence_threshold_db", "prominent_frame_fraction",+ "median_local_prominence_db", "median_peak_frequency_hz",+ "peak_frequency_std_hz", "peak_frequency_p05_hz", "peak_frequency_p95_hz",+ "interpretation")+ writer = csv.DictWriter(stream, fieldnames=columns)+ writer.writeheader()+ writer.writerows(summary_rows)+ (args.output_dir / "receipt.json").write_text(json.dumps({+ "schema_version": 1, "method": "windowed FFT peak screen; not ENF identification",+ "sample_rate_hz": args.sample_rate, "frame_s": args.frame_s, "hop_s": args.hop_s,+ "nominals_hz": args.nominals, "harmonics": args.harmonics,+ "prominence_threshold_db": args.prominence_db,+ "reference_match_performed": False, "utc_estimate_emitted": False,+ "inputs": receipts,+ }, indent=2, allow_nan=False) + "\n")+++if __name__ == "__main__":+ main()diff --git a/src/dopplergeist/enf.py b/src/dopplergeist/enf.pynew file mode 100644index 0000000..b33e12c--- /dev/null+++ b/src/dopplergeist/enf.py@@ -0,0 +1,105 @@+"""Screen audio for sustained tones near electrical-network harmonics.++These measurements are not an ENF identification or timestamp. A tone can come+from equipment, interference, or other sound; only a defensible reference match+can establish an ENF clock candidate.+"""+import math++import numpy as np+from scipy.signal.windows import hann+++def scan_enf_tones(samples, sample_rate, *, nominals=(50.0, 60.0), harmonics=(1, 2, 3),+ frame_s=4.0, hop_s=1.0):+ """Return per-frame peak frequency and local prominence near ENF harmonics.++ The output is deliberately a tone screen. It does not claim that a peak is+ caused by the grid or that it can recover a timestamp.+ """+ samples = np.asarray(samples, dtype=float)+ if samples.ndim != 1 or not len(samples) or not np.isfinite(samples).all():+ raise ValueError("ENF screening requires a finite, nonempty mono signal")+ if type(sample_rate) is not int or sample_rate <= 0:+ raise ValueError("sample_rate must be a positive integer")+ if not math.isfinite(frame_s) or not math.isfinite(hop_s) or frame_s <= 0 or hop_s <= 0:+ raise ValueError("frame_s and hop_s must be positive and finite")+ if any(not math.isfinite(float(value)) or float(value) <= 0 for value in nominals):+ raise ValueError("nominal frequencies must be positive and finite")+ if any(type(value) is not int or value < 1 for value in harmonics):+ raise ValueError("harmonics must be positive integers")++ frame = int(round(frame_s * sample_rate))+ hop = int(round(hop_s * sample_rate))+ if frame < 8 or hop < 1:+ raise ValueError("Frame and hop durations are too short for this sample rate")+ if len(samples) < frame:+ return []+ nfft = 4 * frame+ frequency = np.fft.rfftfreq(nfft, 1.0 / sample_rate)+ window = hann(frame, sym=False)+ specs = []+ for nominal in nominals:+ for harmonic in harmonics:+ center = float(nominal) * harmonic+ if center + 0.75 >= sample_rate / 2:+ continue+ search = np.flatnonzero(np.abs(frequency - center) <= 0.75)+ side = np.flatnonzero((np.abs(frequency - center) <= 0.75) &+ (np.abs(frequency - center) >= 0.18))+ if len(search) < 3 or len(side) < 3:+ continue+ specs.append((float(nominal), harmonic, center, search, side))++ if not specs:+ return []++ rows = []+ tiny = np.finfo(float).tiny+ for first in range(0, len(samples) - frame + 1, hop):+ segment = samples[first:first + frame]+ magnitude = np.abs(np.fft.rfft(segment * window, n=nfft))+ offset = (first + frame / 2) / sample_rate+ for nominal, harmonic, center, search, side in specs:+ peak = int(search[np.argmax(magnitude[search])])+ left, middle, right = np.log(np.maximum(magnitude[peak - 1:peak + 2], tiny))+ denominator = left - 2 * middle + right+ delta = (0.5 * (left - right) / denominator+ if len((left, middle, right)) == 3 and abs(denominator) > 1e-12 else 0.0)+ delta = float(np.clip(delta, -0.5, 0.5))+ local_floor = float(np.median(magnitude[side]))+ prominence = 20 * math.log10(max(float(magnitude[peak]), tiny) /+ max(local_floor, tiny))+ rows.append({"offset_s": float(offset), "nominal_hz": nominal,+ "harmonic": harmonic, "target_hz": center,+ "frequency_hz": float((peak + delta) * (sample_rate / nfft)),+ "local_prominence_db": float(prominence)})+ return rows+++def summarize_enf_tones(rows, *, prominence_threshold_db=10.0):+ """Summarize tone candidates without labeling them as ENF."""+ if not math.isfinite(prominence_threshold_db):+ raise ValueError("prominence threshold must be finite")+ groups = {}+ for row in rows:+ groups.setdefault((row["nominal_hz"], row["harmonic"]), []).append(row)+ result = []+ for (nominal, harmonic), values in sorted(groups.items()):+ prominence = np.asarray([row["local_prominence_db"] for row in values])+ frequency = np.asarray([row["frequency_hz"] for row in values])+ present = prominence >= prominence_threshold_db+ estimates = frequency[present]+ result.append({+ "nominal_hz": float(nominal), "harmonic": int(harmonic),+ "target_hz": float(nominal * harmonic), "frames": int(len(values)),+ "prominence_threshold_db": float(prominence_threshold_db),+ "prominent_frame_fraction": float(np.mean(present)),+ "median_local_prominence_db": float(np.median(prominence)),+ "median_peak_frequency_hz": float(np.median(estimates)) if len(estimates) else None,+ "peak_frequency_std_hz": float(np.std(estimates)) if len(estimates) else None,+ "peak_frequency_p05_hz": float(np.percentile(estimates, 5)) if len(estimates) else None,+ "peak_frequency_p95_hz": float(np.percentile(estimates, 95)) if len(estimates) else None,+ "interpretation": "mains-band tone candidate; ENF/reference match unverified",+ })+ return result