Skip to content

Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)

test(#341): record fixed real-clip controls and held-out limit

Cinna Davisff7216ca02be
diff --git a/benchmarks/coherent-proposals-341-20261104.csv b/benchmarks/coherent-proposals-341-20261104.csv
new file mode 100644
index 0000000..d584141
--- /dev/null
+++ b/benchmarks/coherent-proposals-341-20261104.csv
@@ -0,0 +1,25 @@
+case_id,split,publisher_aircraft_label,production_groups,coherent_groups,union_has_proposal,coherent_elapsed_s,baseline_trace_parity
+case-0018,development,0,0,0,False,8.029138,True
+case-0027,development,0,0,0,False,10.213210,True
+case-0037,holdout,0,0,0,False,9.586693,not_replayed
+case-0040,holdout,0,0,0,False,10.169844,not_replayed
+case-0001,development,1,1,2,True,29.874282,True
+case-0002,development,1,0,0,False,17.428045,True
+case-0003,holdout,1,2,1,True,16.225479,not_replayed
+case-0004,holdout,1,4,1,True,16.580701,not_replayed
+case-0028,development,0,0,0,False,11.463499,True
+case-0041,development,0,0,0,False,7.688888,True
+case-0046,holdout,0,0,0,False,10.925905,not_replayed
+case-0089,holdout,0,0,0,False,9.781349,not_replayed
+case-0033,development,1,1,3,True,18.132085,True
+case-0039,development,1,0,0,False,15.677529,True
+case-0074,holdout,1,1,1,True,14.456808,not_replayed
+case-0085,holdout,1,0,0,False,13.535634,not_replayed
+case-0009,development,0,0,0,False,9.186861,True
+case-0038,development,0,0,0,False,10.445173,True
+case-0043,holdout,0,0,0,False,7.846644,not_replayed
+case-0060,holdout,0,0,0,False,9.597013,not_replayed
+case-0008,development,1,0,0,False,12.766453,True
+case-0013,development,1,0,1,True,21.337992,True
+case-0024,holdout,1,0,0,False,22.924753,not_replayed
+case-0025,holdout,1,0,0,False,24.733032,not_replayed
diff --git a/docs/COHERENT_DISCOVERY.md b/docs/COHERENT_DISCOVERY.md
index 1ea3867..438dc02 100644
--- a/docs/COHERENT_DISCOVERY.md
+++ b/docs/COHERENT_DISCOVERY.md
@@ -36,3 +36,33 @@ The regression fixture contains only measured frequencies/times derived from
for attribution. It contains no audio, coordinates or flight IDs. The original
recording is [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/);
see the source page for the recordist attribution.
+
+## Fixed real-clip development and held-out check
+
+From the existing seeded AeroSonicDB sample, take the first two aircraft-labeled
+and first two background-labeled clips per publisher location for development,
+then the next two per class/location as a held-out set. No labels, flight IDs or
+locations enter the detector. The complete case outcomes are in the
+[24-row table](../benchmarks/coherent-proposals-341-20261104.csv).
+
+| Partition and publisher label | Clips | Ordinary discovery | Coherent adapter | Union |
+| --- | ---: | ---: | ---: | ---: |
+| Development aircraft | 6 | 2 | 3 | 3 |
+| Held-out aircraft | 6 | 3 | 3 | 3 |
+| Development background | 6 | 0 | 0 | 0 |
+| Held-out background | 6 | 0 | 0 | 0 |
+
+Counts mean clips with proposals, not confirmed aircraft or independent events.
+The initial unrestricted reference-trace experiment proposed on 5/6 development
+aircraft clips and 1/6 backgrounds; the existing quality rules discard short or
+noisy traces. The final adapter adds one development clip and no held-out clip.
+These small, source-defined strata do not establish improved general recall or
+a false-alarm rate, and do not justify changing normal upload defaults. See the
+[AeroSonicDB source and sampling report](../benchmarks/aerosonic-scanner-survey-20261103.md).
+
+All twelve development replays preserved the reference detector's measured
+traces exactly while omitting profile ranges. Summed wall time was 185.72 s
+with reference profiles and 172.24 s for the adapter, but the median paired
+ratio was 1.031. These single passes had different system load; there is no
+consistent measured speedup claim. Neither this clip test nor the Michigan
+regression establishes a location.