Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)
Local upload workflow
The current location goal accepts up to 1,000 m error, with 100–500 m or better preferred. True benchmark error and uncalibrated model sensitivity remain separate. Historical reports below retain their original 100 m thresholds.
The real-world data policy stops new synthetic data generation, tests, benchmarks, and replays. New validation uses real recordings and measured trajectories. Historical results retain their original scope and limitations.
The automatic product path is separate from the historical research run and
measure commands. Uploads supply audio, an optional defensible UTC window for
sample zero, and a declared country profile (US/CONUS by default, Germany,
Spain, metropolitan France including Corsica, Italy, or any other country in
COUNTRIES.md).
An optional rectangle narrows that profile. Event windows, frequency
bands, flight IDs and ground-truth coordinates are not upload inputs.
Launch the local interface after installing the project:
dopplergeist serve --jobs jobs --online --max-runtime 14400 --port 8765
Open http://127.0.0.1:8765. The same persistent directory must be supplied after
restarting. Each upload gets a new job ID and its own original file, normalized
request, provenance, intermediate diagnostics, result and worker log. Completed
jobs are read back without rerunning. Interrupted jobs are explicitly failed;
they are never silently replayed. Stopping preserves partial artifacts. After
whole-recording discovery finishes, the job writes an atomic resume.json.
Use Resume job to continue a stopped job with that checkpoint. Completed
audio discovery is reused, as are valid completed search tiles; a partially
processed stage or tile is repeated. Input, detector and checkpoint integrity
are checked before reuse; incompatible search checkpoints fail explicitly.
Stopping before discovery completes cannot resume from that stage.
Completed scored fine-grid pieces are also saved and reused when their scoring and
retention evidence match. Fine-stage checkpoint reuse currently requires the
same effective search worker count; use a new output directory if that count
must change after fine checkpoints have been written.
To back up or move a complete job, stop it, wait for its terminal status and
worker exit, then copy its entire job directory. If a service owns the store and
you cannot confirm the worker has exited, stop the service cleanly after its other
jobs finish before copying. Do not copy a live retention SQLite file with cp
or rsync: the database and its journal can change during the copy. SQLite's
online backup can produce a consistent database snapshot, but it does not also
freeze the job's independently changing JSON and tile files, so it is not by
itself a consistent full-job backup.
Saved searches from the November 3 225bf6b scorer have a compatible upgrade
path to the faster staged scorer. Resume retains their completed scores and
tile files, archives the previous search and identities, and processes only
unfinished work. Input, event, runtime and model checks still reject unsupported
transitions; the upgrade does not make an inconclusive result conclusive.
The server binds to loopback by default. For Docker bridge networking, use
--host 0.0.0.0 and publish the same port on the host's loopback address; see
Docker setup. The local Host/Origin checks remain enabled.
It runs one analysis process at a time, accepts up
to four pending jobs, limits uploads to 100 MiB by default, and limits analysis
to one hour by default. Set --max-upload-mb and --max-runtime to change these
operator limits. Cancellation terminates the analysis process group, including
decoders. No API credentials or paid service are required by the local service.
--online enables public flight and terrain downloads into ~/.cache/dopplergeist
when a job has a defensible recording time. Archives and indexes can take tens of gigabytes or more and hours; no automatic
disk-space limit is currently enforced. The example gives each job four hours; adjust --max-runtime
for your system. Omit --online and supply --config operator.json to use
preconfigured offline data. An unconfigured backend reports missing data.
CLI equivalents:
dopplergeist analyze recording.mp3 --jobs jobs --online --max-runtime 14400 \
--start-utc "$RECORDING_START_UTC_MIN" --end-utc "$RECORDING_START_UTC_MAX" \
--time-source 'Your evidence for this recording time'
dopplergeist job JOB_ID --jobs jobs
dopplergeist job JOB_ID --jobs jobs --watch --interval 10
dopplergeist cancel JOB_ID
dopplergeist cancel JOB_ID --jobs jobs
dopplergeist resume JOB_ID --jobs jobs
# If the upload service still owns the job store:
dopplergeist resume JOB_ID --server-url http://127.0.0.1:8765
dopplergeist job JOB_ID --jobs jobs --watch
Use analyze --country DE or the upload form's country selector for Germany.
The API equivalent is "search":{"country":"DE"}; normalized requests record
"coverage":"germany". The same country, CRS and receiver mask apply to coarse
search, parallel workers, fine grids and continuous refinement. See
country profiles for boundary acquisition, offline use and limits.
For metropolitan France including Corsica, use analyze --country FR, the France
browser option, or "search":{"country":"FR"} in the API. Requests normalize to
"coverage":"france" and use EPSG:2154 (Lambert-93). The profile is a bounding
envelope without a country polygon; use independently justified --bounds for
a regional search. Overseas territories are outside this profile.
For Italy, use analyze --country IT, the Italy browser option, or
"search":{"country":"IT"}. Requests normalize to "coverage":"italy" and use
EPSG:7794 (RDN2008 / Italy zone, E-N). The envelope covers the mainland,
Sicily, Sardinia and Lampedusa without a country polygon, so it also contains
sea and border areas of neighboring countries; use independently justified
--bounds for a regional search.
analyze prints the job ID and a monitor command to standard error as soon as
the upload is queued. Search progress is based on completed grid work and is
persisted after a 2.5 percentage-point increase or at least every 15 seconds
when another tile completes. A single long-running tile cannot report partial
work it has not completed. The CLI and upload UI hide the percentage when an
active job's status has not refreshed for 60 seconds; the job may still be
processing.
job --watch reports the status-file progress field, the
count/rate of matching coarse and fine search filenames, the newest matching
file modification age, and the job status-file age on standard error. These are
unverified filesystem indicators: the watcher does not validate payload hashes
or retention evidence, and the file-count rate is not a completed-work rate or
ETA. Coarse split-marker counts and rates show recorded subdivisions, not the number of
pending leaves or evidence that the search is converging. Remotely staged work
appears after it is imported. Pressing Ctrl-C stops the watcher without
cancelling the analysis.
Direct analyze and resume --jobs print the terminal result as JSON and exit
with status 1 for a failed job or 130 for cancellation. They exit 0 for
succeeded, needs_input, or inconclusive: a zero exit status means the
command completed, not that a location was established. Automation must inspect
the JSON status. A server-side resume only requests that the job be queued;
check the job result separately.
Direct analyze accepts up to 100 MiB by default. Set --max-upload-mb 250,
for example, to allow a larger local recording. The serve command has its own
--max-upload-mb setting for uploads through the local web interface.
cancel sends the cancellation request to the local service at
http://127.0.0.1:8765. If serve uses a different port, pass its base URL with
--server-url. The service must be running and managing the job directory.
For a job started by direct analyze or resume --jobs, use
cancel JOB_ID --jobs PATH. It writes the cooperative cancellation marker
without opening a second manager; the active worker stops at its next
cancellation check, so it may finish its current tile first. This path requires
an active manager to own the job directory. The service API path instead stops
the worker process group promptly and returns the terminal job result.
When serve owns the job-store lock, use the same --server-url option with
resume; it prints the queued result after the service accepts the request.
Monitor it with job --watch and the service's --jobs directory. Without
--server-url, resume --jobs runs directly against a job store and requires
that no service currently owns its lock.
The UTC window bounds the recording start, not the event occurrence or upload. Missing time is a required-input result. Scores are ranking statistics, not probabilities. Sensitivity radii must not be described as calibrated confidence. An inconclusive result may retain candidate hypotheses for inspection.
Curve evidence and local review
Automatic jobs write one media record per discovered ridge in discovery/:
{event_id}.png: a spectrogram crop with the measured ridge overlaid;{event_id}-audio.wav: a bounded excerpt of the preserved recording, with 1.5 s of padding and at most 60 s of audio;{event_id}-band.wav: an illustrative band-limited excerpt for listening;{event_id}.csv: the measured ridge points;event-media.json: an index of the generated media and any per-event render errors.
The raw excerpt is the primary listening evidence. The band-limited excerpt is
not used for detection, scoring or acceptance. A media-rendering failure is
recorded in event-media.json and does not fail the analysis. Jobs analyzed
before this feature show the measured ridge chart and data, with a note that
media generation requires a new analysis.
The browser groups ridges by harmonic group and lets a local reviewer label each
ridge, or all ridges in a group, as confirmed or false_positive, with an
optional note. Labels are stored in event-reviews.json in the job directory.
They are annotations: they never change that job's scores, candidates, status,
coverage or displayed location. The labels are downloadable with the other job
artifacts.
Re-analyze without N false positives starts a new job from the preserved
original upload. The new request records excluded_event_ids and
review_source_job_id; after discovery, only the explicitly marked event IDs are
removed before search. The new job reports review.user_assisted: true and its
provenance records the exclusion. The original job and its result remain
unchanged. A reviewed exclusion is a user-assisted analysis, not an automatic
result, and must not be presented as independent automatic validation.
HTTP surface
GET /api/capabilities: declared coverage, upload/work limits, and whether a flight provider and online acquisition are configured.POST /api/jobs: raw audio bytes withContent-Type: application/octet-stream,Content-Length, optionalX-Filename, andX-Dopplergeist-Requestcontaining the shared request JSON withoutaudio_path. The server saves the upload and sets that path. Returns HTTP 202 and a shared queuedJobResult.GET /api/jobs/{id}: the current sharedJobResult.POST /api/jobs/{id}/cancel: cancel a queued/running job; terminal jobs remain.POST /api/jobs/{id}/resume: explicitly continue a stopped job with a valid completed audio checkpoint.GET /api/jobs/{id}/artifacts: relative file paths and byte counts.GET /api/jobs/{id}/artifacts/{path}: download one artifact. Private operator configuration is excluded. Paths cannot escape the job directory. PNG, SVG, WAV, MP3, FLAC, M4A and OGG artifacts are served inline; other files download.GET /api/jobs/{id}/reviews: savedconfirmed/false_positivelabels and optional notes for this job.POST /api/jobs/{id}/reviews: JSON body{"label":"confirmed|false_positive|unreviewed","event_ids":["..."],"note":"optional"}. The label applies to every listed event. Unknown event IDs or labels are rejected. This changes annotations only, never the result file.POST /api/jobs/{id}/reanalyze: start a new user-assisted job that excludes every curve currently markedfalse_positive. Returns the new queued job. Requires at least one false-positive label.
Upload example (header metadata uses actual JSON, not a string-encoded file):
curl -H 'Content-Type: application/octet-stream' \
-H 'X-Dopplergeist-Request: {"schema_version":1,"search":{"coverage":"conus"}}' \
--data-binary @recording.mp3 http://127.0.0.1:8765/api/jobs
The historical checks in tests/test_service.py cover HTTP transport, original
preservation, state readback, cancellation, interrupted jobs, path restrictions,
and input validation. They do not establish scientific localization accuracy.
Automatic CI checks installation, source compilation, CLI help, and packaged
assets without generated data.
Verify the complete HTTP pipeline
scripts/verify_upload_http.py submits a real recording to a local service and
checks the saved inputs, stage outputs, terminal decision and persisted result.
Use --require-full-pipeline to require acquisition, search, refinement and
decision evidence as well as intake/discovery, with at least one returned
candidate. A correctly handled missing clock or empty search cannot pass this
stronger operational check.
For a regional run, pass --bounds WEST SOUTH EAST NORTH using independently
justified WGS84 longitude/latitude bounds. The verifier records these bounds in
the actual upload request; omitting the option searches the full CONUS area.
python scripts/verify_upload_http.py \
--audio path/to/real-recording.mp3 \
--operator-config operator.json \
--start-utc "$RECORDING_START_UTC_MIN" \
--end-utc "$RECORDING_START_UTC_MAX" \
--time-source 'Evidence establishing the recording-start interval' \
--expect-status inconclusive --require-full-pipeline \
--output results/new-real-http-run
Declare the expected outcome before running. An expected inconclusive result
can verify that every stage ran; it does not establish a supported location.
It can pass with incomplete search coverage; the result retains those limitations.
Use --expect-status succeeded when testing that acceptance requirement, then
evaluate location error against an independent reference after the run. The
verifier reports operational verification and supported_decision separately.
It counts independent harmonic groups separately from their ridge records.
Verify runtime limits, cancellation and restart
--timeout-s bounds the entire verification. --job-max-runtime-s separately
bounds the actual worker. It defaults to --timeout-s for existing callers.
Allow time for service startup, upload, terminal readback and shutdown when testing
a worker limit. A verifier timeout does not count as an observed work_limit.
python scripts/verify_upload_http.py \
--audio path/to/real-recording.mp3 --operator-config operator.json \
--start-utc "$RECORDING_START_UTC_MIN" --end-utc "$RECORDING_START_UTC_MAX" \
--expect-status inconclusive --expect-reason work_limit \
--job-max-runtime-s 30 --timeout-s 180 --verify-restart \
--output results/new-real-runtime-control
Use a recording and budget that actually reach the intended limit. A different
terminal outcome fails the declared expectation and remains in the output.
For an API cancellation control, use --cancel-at-stage discovery with
--expect-status cancelled. The verifier waits for that actual running stage
before it sends the cancellation request. It fails if the job finishes first.
The cancellation response and terminal result must both confirm cancelled,
even when a Python caller uses expect_status=None.
--verify-restart stops the verifier's service, starts a new service over its
completed job, and reads the result and original audio through HTTP again.
It checks unchanged saved result bytes and original audio. A cancellation control
also repeats cancellation after restart and requires the terminal result to stay
unchanged. These operations affect only the new verification directory.
To exercise unavailable data, supply a new empty cache with offline: true and
a recording-start window. Preserve the actual data_unavailable outcome;
do not replace it with a successful result. These terminal controls do not satisfy
--require-full-pipeline or establish a supported location.
Python callers can pass expect_status=None to observe any terminal outcome.
All input, artifact and stage checks still apply, including
require_full_pipeline=True when requested. The CLI requires --expect-status.
The optional Python arguments are job_max_runtime_s=None,
cancel_at_stage=None and verify_restart=False.
Verify guidance in the browser
The browser verifier checks actual guidance at desktop and mobile widths. It compares each observation, suggestion and evidence link with the API response. Every evidence pointer must resolve in the saved result or decision artifact. Downloaded evidence must match the saved bytes, which must remain unchanged.
python scripts/verify_real_upload_browser.py \
--saved-job path/to/completed-real-job --source src \
--expect-status inconclusive --expect-reason no_supported_events \
--expect-guidance-code no_events \
--output results/new-real-guidance-readback
Repeat --expect-guidance-code to require several observations. Use work_limit
for a completed runtime-limit case and data_access_limit for an unavailable-data
case with a recorded acquisition limit. The saved job is copied before readback.
No job is resumed and no result is manufactured. The check preserves the saved
status, candidates and scores. Issue #63 still requires real positive and negative
counterfactual controls before it can claim measured guidance benefit.
Provider and search configuration
--online requires no JSON configuration: it uses the per-user cache and public
providers automatically. For a different cache location, existing indexes, or
custom budgets, provide an operator config with --config; --online enables
network acquisition while retaining the configured cache path. Users upload
audio without choosing trajectory files. The acquisition path delegates to
data_access.acquire_inputs:
{
"cache_root": "/absolute/path/to/dopplergeist-cache",
"offline": false,
"terrain": {},
"search_workers": 12,
"search_budget": {
"spacing_m": 2000,
"tile_cells": 32,
"top_k": 100,
"refine_spacing_m": 100
}
}
This acquires public ADSB.lol all-type daily archives and Copernicus terrain/geoid
inputs, then reuses their indexed cache. It can require substantial disk space and
time. offline defaults to true; online retrieval is an explicit operator choice.
search_workers is an optional positive integer for coarse-search processes;
omit it to use one fewer than the available logical CPUs. More workers increase
memory use. Custom terrain providers currently search serially.
No hidden account credentials or paid fallback are used. The default acquisition
budget is three UTC archive dates, including propagation guards and the full recording (max_acquisition_days); wider recording windows need
a narrower defensible input or an explicit operator budget change. Do not put
credentials in this configuration.
For existing offline data, supply flight_indexes (paths to Worker 2 FlightIndex
SQLite files). Terrain may use the cached provider options (geoid_path,
adopt_existing) or a declared local override: {"rasters":["/dem.tif"], "geoid_raster":"/geoid.tif"}. The historical {"constant_m":0} terrain override
does not establish real terrain or altitude-datum accuracy. Its result retains
that limitation for real recordings.
Local overrides require prebuilt indexes and retain their actual coverage.
search_budget.max_tiles limits the initial root-tile prefix. Recursive
subdivision can create more work than that root limit, so use
search_budget.max_tile_evaluations to cap newly scored coarse tiles in one
search invocation. Verified tile checkpoints do not consume that per-invocation
allowance, allowing a later resume to continue with the same configuration.
Exhausting the work cap leaves the result partial. Reaching a tile, memory, data
or wall-time limit leaves incomplete coverage and cannot produce a supported
national location. Full search completion does not prove the archive captured
every real aircraft. Grid finalists undergo jointly constrained position/clock
refinement; refine_finalists (default 5) limits physical sensitivity
refinement. Unassessed retained approaches prevent an affirmative confidence
decision.
Discovery uses fixed detector settings. The settings and measured proposals are
saved in discovery/discovery.json; custom discovery settings are rejected.
An operator can add "spectral_comparison": {} to save a separate
spectral-comparison.json artifact after refinement. This optional stage reads
the actual decoded recording and named native flight segments. It compares
constant source frequencies with one shared clock, using the reference band of
each prepared flyover group. The existing ridge decision and its uncertainty
remain unchanged. Spectral scores only compare supplied modes with identical
event sets; mixed sets have separate rankings_by_event_set lists and an empty
top-level ranked list. Each event-set ranking also contains sample_sum_ranked,
a separate ranking summing contrast over the sampled time bins and optimizing the
same shared clock. Unlike the event-mean statistic, it weights longer spectral
windows more heavily. It depends on the window, hop and contrast normalization;
the source video does not specify its exact brightness scaling. The comparator
chooses a clock that maximizes the combined score; the video describes choosing
the first event's alignment before locking later events to it. Neither ranking
establishes a national optimum or location acceptance, and ridge retention
bounds do not apply to these scores.
The spectral configuration accepts work limits and grid settings, including
max_modes, max_spectrogram_bins, max_event_clock_trials,
max_curve_samples, time_sweep_s, time_step_s, and frequency_step_hz.
Track loading uses search_budget.max_tracks_per_tile and
search_budget.max_track_samples_per_tile as total limits for this comparison.
Missing trajectories, ambiguous segment IDs and exhausted budgets remain
explicit in the artifact. These controls are private operator settings.
Search reports distinguish raw model support from finite assessment. If any
trajectory or scoring envelope lacks support, model_support_complete stays false.
model_support retains the raw work counts, unknown trials, and reasons. Missing
or malformed counters cannot become zero work. These counts describe work
evaluations, not distinct aircraft.
Unreported trial envelopes are explicit: zero reported unknown trials cannot
establish completion when any trial accounting is absent, including the legacy
fully supported path.
If every unknown finite scoring envelope has a validated ceiling or rejection,
model_assessment_complete can be true. An unavailable event fit contributes at
most one unit to the normalized support score. A fully validated event that
exceeds its residual limit rejects that association at that clock. The producer
retains the greatest ceiling across all scheduled clock trials in the retention
ledger. This accounting does not create a measured fit or restore missing support.
The product requires agreement between raw counters, persisted bounds, completion flags, and finite discard accounting. The confidence decision compares the combined discard bound with the leading score under the existing margin. Unsupported CPA trajectories, unreported trials, and unbounded alternatives keep assessment incomplete. Missing tiles, data gaps, event caps, and incomplete sensitivity evidence still prevent a supported location.
Source availability remains a separate data_status. An available archive can
still have unassessed trajectory models. Per-stage artifacts retain affected IDs,
reasons, and conservative coverage envelopes. See
model-support coverage for the complete contract.
The automatic path always uses one shared clock inside the supplied start window and robust ridge loss. Its default association path considers every proposed independent group; harmonics count once and unsupported acoustic proposals remain explicitly unmatched. A smaller operator event cap is incomplete work and is shown in coverage limitations, rather than silently keeping an initial subset. Every discovered event remains in the diagnostics. A location requires the confidence producer's evidence checks, including independent events, alternative hypotheses, complete coverage, residuals and measurement/model sensitivity. A small altitude-only profile is insufficient. The final decision and full candidate/refinement diagnostics are saved as downloadable JSON.
Current limits
The application is available on main and remains experimental. Automatic
synthetic runs have produced accurate hypotheses and inconclusive decisions;
those are different outcomes. See the README
for the reported results.
Curve review labels are local human annotations, not calibrated probabilities or aircraft identities. A user-assisted re-analysis is explicitly marked and is not automatic validation. Media excerpts are illustrative diagnostics; listening to the band-limited excerpt does not establish that a proposal is an aircraft.
Independent real recordings with established recording time and location, reliable global hypothesis discrimination, and calibrated false-localization rates remain open. The acceptance goal is within 1,000 m, preferably 100–500 m; landing the application does not establish that goal.
Accounted finite-search evidence
The current integration consumes the optional artifact-only producer/decision
contract in FINITE_SEARCH_DECISION.md. The search
producer assigns stable seed mode_id values including position, clock, event
indices and flight/pass association, and records every lossy retention bound.
The product relays finite_search_evidence unchanged, preserves seed IDs through
refinement, and supplies candidate_mode_ids parallel to the wire candidates.
For the explicitly declared evaluated-finite-grid scope, assessed_mode_ids
requires the producer's explicit core_assessed_mode_ids mapping plus an actually
completed finite core score with full measured-trajectory support at the original
seed/model and a valid trusted clock. A returned fit alone does not establish
classification of every bounded clock/optimizer trial; missing/malformed mappings
or producer-omitted IDs remain unassessed. It does not require
jointly optimizing every retained point. Failed, missing or unsupported core
records are unassessed. Raw and refined representations retain their original
mode ID and original-or-improved score; no association-only deduplication is used.
The accepted leading estimate still needs successful full sensitivity, datum,
optimizer, clock, actual-radius and competing-mode checks.
Unperformed refinements and continuous locations are outside that conditional
finite claim. Their limitations are displayed in coverage and saved in the
artifacts. The old alternatives_evaluated flag keeps its older meaning and is
not flipped to force success. An absent finite object preserves old outcomes;
missing mappings, incomplete work or an unaccounted near-tied discard remain
inconclusive. Validated bounds account for unknown discarded trials without adding
them to assessed_mode_ids. The adapter and decision code consume this finite
contract. An accepted real-recording location through this path remains unverified.