Skip to content

Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)

feat: prototype optional ENF and document single-ridge candidates (#223)

Cinna Davis7979ba2ae0d8
diff --git a/docs/AUDIO_DISCOVERY.md b/docs/AUDIO_DISCOVERY.md
index 9839789..62f090c 100644
--- a/docs/AUDIO_DISCOVERY.md
+++ b/docs/AUDIO_DISCOVERY.md
@@ -17,6 +17,9 @@ integer ratios and rational ratios (e.g. measured 2f and 3f with an absent
fundamental) count as one independent event. The lowest measured harmonic is the
representative ridge; no fundamental is invented. Harmonic count denotes distinct
measured ratios, including fragmented traces of the same harmonic only once.
+Harmonic grouping is optional consolidation: a ridge without a compatible
+partner remains a single-ridge event candidate. The detector does not require a
+second harmonic.
The output preserves recording-relative ridge samples and weights, measured
window/band, harmonic-group identity, contrast/support, spectral resolution and
@@ -26,6 +29,13 @@ source-frequency estimate constrains downstream search. These are **acoustic
moving-tone proposals**, with aircraft identity unconfirmed. Association must
reject unsupported proposals and count harmonic groups once.
+In the current real one-hour upload, automatic discovery produced 15 independent
+groups: 14 have one observed ridge and one has three. This is a useful reminder
+that harmonic grouping can consolidate evidence when multiple bands are
+recorded, but a single-ridge candidate must remain eligible for downstream
+assessment. The audio-only discovery result does not confirm aircraft identity
+or recording location.
+
With output_dir, discovery saves `discovery.json` (all events, settings, chunk
counts, original harmonic ridges and input hash), `discovery.png` (overview), and
per-event CSV plus spectrogram PNG. Each spectrogram image is limited to a chunk's
diff --git a/docs/ENF_TIMESTAMPING.md b/docs/ENF_TIMESTAMPING.md
new file mode 100644
index 0000000..75c971a
--- /dev/null
+++ b/docs/ENF_TIMESTAMPING.md
@@ -0,0 +1,112 @@
+# ENF timestamp recovery: research status
+
+Electrical Network Frequency (ENF) may provide recording-time evidence when a
+real mains-related signal is captured in the audio and a matching, timestamped
+grid reference exists. A stationary 50/60 Hz tone by itself is not enough: fans,
+power supplies, motors and other equipment can produce similar tones. ENF
+cannot be reported as a timestamp until the extracted frequency variation has
+been matched to an independent reference and ambiguity and clock drift have
+been assessed.
+
+## Current prototype
+
+[`scripts/scan_enf_signal.py`](../scripts/scan_enf_signal.py) screens real audio
+for peaks near 50/60 Hz and their first three harmonics. It reports per-window
+frequency estimates, local spectral prominence and coverage in a new output
+directory. It does **not** identify grid hum, compare a historical reference,
+estimate UTC, infer a location, or alter the aircraft search request. A strong
+screen result is only a reason to investigate the trace further.
+
+Example for a new output directory:
+
+```bash
+python scripts/scan_enf_signal.py recording.mp3 \
+ --output-dir results/enf-screening-v1
+```
+
+The default uses the same 4 kHz mono decoding path already used by DOPPLERGEIST.
+The 4-second window and 1-second hop are screening settings, not a calibrated
+ENF extractor. A future timestamp estimator should test each usable band against
+real references. One sufficiently strong line can be evaluated at its known or
+hypothesized harmonic order; additional harmonics are optional corroboration,
+not an acceptance requirement. Longer windows and held-out real recordings are
+still needed to validate frequency precision and false-match risk.
+
+## First real-audio screen (2026-11-03)
+
+The prototype screened 50 real recordings at 4 kHz: 24 files from the
+publisher-site-labelled corpus, 5 videos without an exact microphone point, 20
+AeroSonic recordings, and the active one-hour upload. The site labels are not
+independent recording-time evidence. The scanner did not read truth-coordinate
+files or pass any result into aircraft search.
+
+For this exploratory summary, a *persistent tone candidate* means at least
+one of the six 50/60 Hz harmonic bands had local prominence of at least 10 dB
+in at least 80% of its complete 4-second windows (1-second hop). This is only
+a spectral peak threshold; it does not distinguish grid frequency from a
+motor, power supply, interference or other sound.
+
+| Real-audio group | Recordings | Persistent tone candidate | Two harmonics at one nominal base (descriptive) |
+| --- | ---: | ---: | ---: |
+| Publisher-site-labelled corpus | 24 | 2 | 0 |
+| Unknown exact microphone point | 5 | 0 | 0 |
+| AeroSonic recordings | 20 | 0 | 0 |
+| Active one-hour upload | 1 | 0 | 0 |
+
+Two recordings had at least one persistent tone candidate: `site-914.mp3` (a
+50 Hz fundamental and a separate 120 Hz band) and `site-919.mp3` (a 50 Hz
+fundamental). No recording had two persistent bands at one nominal base. That
+last column is descriptive only: one coherent line remains eligible for a
+future reference match, and the lack of a second harmonic is not grounds to
+reject it. The one-hour upload had no band meeting this screen's persistence
+threshold; that does not prove its audio lacks a weaker or longer-window ENF
+trace. No reference matching or UTC estimate was attempted, so this screen
+establishes neither ENF availability nor timestamp recovery rates.
+
+Full per-window traces, summaries and SHA-256 receipts are preserved locally
+at `/home/cinna/.local/state/dopplergeist-enf-screening-20261103/corpus-v1`.
+The recordings and their local paths are not included in the repository.
+
+## Reference-data audit
+
+- The official [FNET/GridEye real-time display](https://fnetpublic.utk.edu/frequencygauge.html)
+ describes its live plot as real-time frequency measurements.
+- The FNET [table display](https://fnetpublic.utk.edu/tabledisplay.html) warns
+ that displayed frequency values are sample data, not current grid
+ measurements, and that actual values have been hidden from that display
+ since 2018.
+- [OpenENF](https://github.com/openenf/openenf) documents public references for
+ parts of Europe and says public North American reference data is unavailable.
+- [nate-griff/enf](https://github.com/nate-griff/enf) is a useful implementation
+ reference. Its README describes QIFFT harmonic extraction and FFT-accelerated
+ sliding correlation, but labels the tool an investigative aid rather than a
+ proof system. Its documented collector captures the current FNET chart, and
+ its checked-in daily CSVs cover 2026-03-20 through 2026-05-12. That archive
+ cannot match the active 2026-09-08 upload.
+- A 2025 ENF extraction evaluation reports that target duration, reference
+ duration and signal-to-noise ratio affect timestamp accuracy: [Maiberger,
+ Gusakov and Routtenberg, CISS 2025](https://doi.org/10.1109/CISS64860.2025.10944719).
+
+FNET can support building a prospective archive by collecting its real-time
+charts, but this does not create historical references for past recordings.
+The repository's image-derived reference series also needs independent
+validation of sampling cadence, frequency calibration, missing-data handling
+and redistribution terms before it can be used for an accepted timestamp.
+
+## Integration gate
+
+Only consider passing an ENF-derived time window into the existing aircraft
+pipeline after real recordings with independently established sample-zero UTC
+have been matched to a genuine historical grid reference. Keep extraction
+quality, reference provenance, clock-rate drift, alternative timestamp peaks
+and match separation in the result. Do not require multiple harmonics: one
+well-tracked line can support a timestamp when its independent reference match
+is unique and strong enough under the measured uncertainty. Other harmonics
+can strengthen that evidence when present. If no usable line exists, the
+reference is missing, or the match remains ambiguous, return the existing
+`needs_input`/inconclusive path. ENF region identification is at most coarse
+interconnection evidence; it is not an observer coordinate.
+
+The initial screen must run on DOPPLERGEIST's real audio only. Do not synthesize
+hum, fill missing reference intervals, tune the matcher on the recording under
+evaluation, or feed recording truth into extraction or reference search.
diff --git a/scripts/scan_enf_signal.py b/scripts/scan_enf_signal.py
new file mode 100644
index 0000000..dc96901
--- /dev/null
+++ b/scripts/scan_enf_signal.py
@@ -0,0 +1,95 @@
+"""Measure 50/60 Hz harmonic-tone availability in real recordings.
+
+This scanner does not identify grid hum, match a reference, or estimate UTC.
+Use a new output directory for each run; do not use its results as search time
+evidence.
+"""
+import argparse
+import csv
+import hashlib
+import json
+from pathlib import Path
+
+from dopplergeist.audio import decode
+from dopplergeist.enf import scan_enf_tones, summarize_enf_tones
+
+
+def sha256(path):
+ digest = hashlib.sha256()
+ with Path(path).open("rb") as stream:
+ for block in iter(lambda: stream.read(1024 * 1024), b""):
+ digest.update(block)
+ return digest.hexdigest()
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("inputs", nargs="+", type=Path, help="real audio files to screen")
+ parser.add_argument("--output-dir", required=True, type=Path,
+ help="new directory for scan receipts and CSV evidence")
+ parser.add_argument("--sample-rate", type=int, default=4000)
+ parser.add_argument("--frame-s", type=float, default=4.0)
+ parser.add_argument("--hop-s", type=float, default=1.0)
+ parser.add_argument("--nominals", type=float, nargs="+", default=(50.0, 60.0))
+ parser.add_argument("--harmonics", type=int, nargs="+", default=(1, 2, 3))
+ parser.add_argument("--prominence-db", type=float, default=10.0,
+ help="screening threshold for local peak prominence, not an ENF confidence")
+ args = parser.parse_args()
+ if args.output_dir.exists():
+ raise FileExistsError("Use a new output directory to preserve prior scan evidence")
+ if len(set(args.inputs)) != len(args.inputs):
+ raise ValueError("Input paths must be unique")
+ for path in args.inputs:
+ if not path.is_file():
+ raise FileNotFoundError(path)
+
+ args.output_dir.mkdir(parents=True)
+ trace_dir = args.output_dir / "traces"
+ trace_dir.mkdir()
+ summary_rows = []
+ receipts = []
+ for source in args.inputs:
+ source = source.resolve()
+ source_digest = sha256(source)
+ samples, sr = decode(source, sample_rate=args.sample_rate)
+ traces = scan_enf_tones(samples, sr, nominals=args.nominals, harmonics=args.harmonics,
+ frame_s=args.frame_s, hop_s=args.hop_s)
+ estimates = summarize_enf_tones(traces, prominence_threshold_db=args.prominence_db)
+ stem = f"{source.stem}-{source_digest[:12]}"
+ trace_path = trace_dir / f"{stem}.csv"
+ with trace_path.open("w", newline="") as stream:
+ writer = csv.DictWriter(stream, fieldnames=("offset_s", "nominal_hz", "harmonic",
+ "target_hz", "frequency_hz",
+ "local_prominence_db"))
+ writer.writeheader()
+ writer.writerows(traces)
+ for estimate in estimates:
+ summary_rows.append({"source": str(source), "source_sha256": source_digest,
+ "duration_s": len(samples) / sr, **estimate})
+ receipts.append({"source": str(source), "source_sha256": source_digest,
+ "sample_rate_hz": sr, "duration_s": len(samples) / sr,
+ "trace_csv": str(trace_path.relative_to(args.output_dir)),
+ "harmonic_summaries": estimates})
+ print(f"scanned {source.name}: {len(samples) / sr:.1f} s, {len(traces)} tone observations")
+
+ with (args.output_dir / "summary.csv").open("w", newline="") as stream:
+ columns = ("source", "source_sha256", "duration_s", "nominal_hz", "harmonic", "target_hz",
+ "frames", "prominence_threshold_db", "prominent_frame_fraction",
+ "median_local_prominence_db", "median_peak_frequency_hz",
+ "peak_frequency_std_hz", "peak_frequency_p05_hz", "peak_frequency_p95_hz",
+ "interpretation")
+ writer = csv.DictWriter(stream, fieldnames=columns)
+ writer.writeheader()
+ writer.writerows(summary_rows)
+ (args.output_dir / "receipt.json").write_text(json.dumps({
+ "schema_version": 1, "method": "windowed FFT peak screen; not ENF identification",
+ "sample_rate_hz": args.sample_rate, "frame_s": args.frame_s, "hop_s": args.hop_s,
+ "nominals_hz": args.nominals, "harmonics": args.harmonics,
+ "prominence_threshold_db": args.prominence_db,
+ "reference_match_performed": False, "utc_estimate_emitted": False,
+ "inputs": receipts,
+ }, indent=2, allow_nan=False) + "\n")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/dopplergeist/enf.py b/src/dopplergeist/enf.py
new file mode 100644
index 0000000..b33e12c
--- /dev/null
+++ b/src/dopplergeist/enf.py
@@ -0,0 +1,105 @@
+"""Screen audio for sustained tones near electrical-network harmonics.
+
+These measurements are not an ENF identification or timestamp. A tone can come
+from equipment, interference, or other sound; only a defensible reference match
+can establish an ENF clock candidate.
+"""
+import math
+
+import numpy as np
+from scipy.signal.windows import hann
+
+
+def scan_enf_tones(samples, sample_rate, *, nominals=(50.0, 60.0), harmonics=(1, 2, 3),
+ frame_s=4.0, hop_s=1.0):
+ """Return per-frame peak frequency and local prominence near ENF harmonics.
+
+ The output is deliberately a tone screen. It does not claim that a peak is
+ caused by the grid or that it can recover a timestamp.
+ """
+ samples = np.asarray(samples, dtype=float)
+ if samples.ndim != 1 or not len(samples) or not np.isfinite(samples).all():
+ raise ValueError("ENF screening requires a finite, nonempty mono signal")
+ if type(sample_rate) is not int or sample_rate <= 0:
+ raise ValueError("sample_rate must be a positive integer")
+ if not math.isfinite(frame_s) or not math.isfinite(hop_s) or frame_s <= 0 or hop_s <= 0:
+ raise ValueError("frame_s and hop_s must be positive and finite")
+ if any(not math.isfinite(float(value)) or float(value) <= 0 for value in nominals):
+ raise ValueError("nominal frequencies must be positive and finite")
+ if any(type(value) is not int or value < 1 for value in harmonics):
+ raise ValueError("harmonics must be positive integers")
+
+ frame = int(round(frame_s * sample_rate))
+ hop = int(round(hop_s * sample_rate))
+ if frame < 8 or hop < 1:
+ raise ValueError("Frame and hop durations are too short for this sample rate")
+ if len(samples) < frame:
+ return []
+ nfft = 4 * frame
+ frequency = np.fft.rfftfreq(nfft, 1.0 / sample_rate)
+ window = hann(frame, sym=False)
+ specs = []
+ for nominal in nominals:
+ for harmonic in harmonics:
+ center = float(nominal) * harmonic
+ if center + 0.75 >= sample_rate / 2:
+ continue
+ search = np.flatnonzero(np.abs(frequency - center) <= 0.75)
+ side = np.flatnonzero((np.abs(frequency - center) <= 0.75) &
+ (np.abs(frequency - center) >= 0.18))
+ if len(search) < 3 or len(side) < 3:
+ continue
+ specs.append((float(nominal), harmonic, center, search, side))
+
+ if not specs:
+ return []
+
+ rows = []
+ tiny = np.finfo(float).tiny
+ for first in range(0, len(samples) - frame + 1, hop):
+ segment = samples[first:first + frame]
+ magnitude = np.abs(np.fft.rfft(segment * window, n=nfft))
+ offset = (first + frame / 2) / sample_rate
+ for nominal, harmonic, center, search, side in specs:
+ peak = int(search[np.argmax(magnitude[search])])
+ left, middle, right = np.log(np.maximum(magnitude[peak - 1:peak + 2], tiny))
+ denominator = left - 2 * middle + right
+ delta = (0.5 * (left - right) / denominator
+ if len((left, middle, right)) == 3 and abs(denominator) > 1e-12 else 0.0)
+ delta = float(np.clip(delta, -0.5, 0.5))
+ local_floor = float(np.median(magnitude[side]))
+ prominence = 20 * math.log10(max(float(magnitude[peak]), tiny) /
+ max(local_floor, tiny))
+ rows.append({"offset_s": float(offset), "nominal_hz": nominal,
+ "harmonic": harmonic, "target_hz": center,
+ "frequency_hz": float((peak + delta) * (sample_rate / nfft)),
+ "local_prominence_db": float(prominence)})
+ return rows
+
+
+def summarize_enf_tones(rows, *, prominence_threshold_db=10.0):
+ """Summarize tone candidates without labeling them as ENF."""
+ if not math.isfinite(prominence_threshold_db):
+ raise ValueError("prominence threshold must be finite")
+ groups = {}
+ for row in rows:
+ groups.setdefault((row["nominal_hz"], row["harmonic"]), []).append(row)
+ result = []
+ for (nominal, harmonic), values in sorted(groups.items()):
+ prominence = np.asarray([row["local_prominence_db"] for row in values])
+ frequency = np.asarray([row["frequency_hz"] for row in values])
+ present = prominence >= prominence_threshold_db
+ estimates = frequency[present]
+ result.append({
+ "nominal_hz": float(nominal), "harmonic": int(harmonic),
+ "target_hz": float(nominal * harmonic), "frames": int(len(values)),
+ "prominence_threshold_db": float(prominence_threshold_db),
+ "prominent_frame_fraction": float(np.mean(present)),
+ "median_local_prominence_db": float(np.median(prominence)),
+ "median_peak_frequency_hz": float(np.median(estimates)) if len(estimates) else None,
+ "peak_frequency_std_hz": float(np.std(estimates)) if len(estimates) else None,
+ "peak_frequency_p05_hz": float(np.percentile(estimates, 5)) if len(estimates) else None,
+ "peak_frequency_p95_hz": float(np.percentile(estimates, 95)) if len(estimates) else None,
+ "interpretation": "mains-band tone candidate; ENF/reference match unverified",
+ })
+ return result