Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)
Python acoustic-geolocationads-baircraft-trackingairplaneaudio-forensicsdoppler-effectdoppler-shiftgeolocationosintsignal-processing
Integrate cb996dc continuation with verified consumer restrictions
Cinna Davis32e2ad9cff42
diff --cc docs/verification/CB996DC_CONTINUATION.mdindex 0000000,0000000..ccda075new file mode 100644--- /dev/null+++ b/docs/verification/CB996DC_CONTINUATION.md@@@ -1,0 -1,0 +1,112 @@@++# Continuation of cb996dc checkpoints++++This profile continues stopped searches from `cb996dcb49b9d31a24c10b9b5418c35423f7bbcd` under a specific Python consumer.++It ports the unique behavior from [PR #260](https://github.com/1337isnot1337/DOPPLERGEIST/pull/260) after the #263 compatibility changes.++The profile preserves completed tile bytes, their producer identities, and the retained scores and bounds.++++The consumer must match the 30 source hashes in [the verification receipt](cb996dc-continuation-20261104.json).++Those files are identical at `350c9bfa1a56ce2b46d0418b39779b12cdfa04de` and `be56c2fae81fe65159103ddb2bf3ccd59ca3bb67`.++The consumer also needs the `checkpoint_resume.py` profile and tests from this change.++The source list covers search orchestration, input interpretation, scoring, retention, and their Python dependencies.++It excludes this adapter and files that only provide CLI, API, or UI behavior.++++The geometry change proposed in #293 changes which trajectory segments can produce scores.++The shared-observation change for #275 changes which event groups can receive independent scores.++Those consumers need a separate compatibility proof before they can reuse these scores.++This profile rejects changes to `geometry.py`, `search.py`, `tiled_search.py`, or any other pinned file.++It does not authorize continuation on arbitrary future versions of main.++++## Preserved behavior and restrictions++++The old scorer used two matched groups by default for a broad recording clock with multiple groups.++The profile writes that effective minimum into both the search manifest and the operator configuration.++An explicit saved minimum remains unchanged.++The profile uses ordinary batched arithmetic and does not enable `legacy_first_batch`.++++The profile requires the Python ridge fitter, acoustic solver, CPA calculation, and flight decoder.++The current native flight decoder has no environment switch, so the source checkout must have no compiled `_native_index` extension.++The producer and consumer must have the same Python version, machine architecture, and recorded package versions.++Audio, request, runtime, event preparation, and saved-file identity checks remain active.++The job must have a terminal status of `cancelled`, `failed`, or `inconclusive`.++The profile rejects fine checkpoints and the legacy first-batch mode because this producer did not create them.++++The adapter copies the retention database independently before it changes the run identity.++It keeps the previous search directory and identity files for rollback.++An interrupted metadata commit restores the original search, operator configuration, and audio identity.++The compatibility reader binds the old producer to the new run hash and the pinned consumer fingerprint.++++## Real-data evidence++++Both replay pairs used the same recorded input and numerical runtime within each pair.++All recordings, trajectories, terrain, and ridge measurements came from recorded or released data.++The receiver location and regional bounds were source-informed.++The historical regional index excluded known jets during its original ingestion and does not establish complete coverage.++These comparisons establish software equivalence for these workloads, not independent location accuracy or nationwide completion time.++++| Replay | Candidate associations | Ranked payloads | Retained records | Retained bounds | Result |++| --- | ---: | ---: | ---: | ---: | --- |++| Regional tile, two points, 317 tracks, 16 groups | 1,699 | 20 | 112 | 1,617 | Exact scores, order, retention, and remaining payload fields |++| Full-day query, one point, 1,354 tracks, 16 groups | 16,015 | 100 | 499 | 15,691 | Exact scores, order, retention, and remaining payload fields |++++The full-day replay used an unknown recording clock and one documented association-anchor partition.++Its query covered `1788825600.17` through `1788911999.6` seconds UTC.++Its anchor partition covered `1788864888.7151732` through `1788865236.7214916` seconds UTC.++Physical fit intervals remained unchanged.++This bounded partition satisfies the dense replay check without a complete nationwide search.++++The full-day retention checksum was `ca75b434b7776ca1507c9efed4e3e0f7df891a623c0b28b30fe19d0d1b5048e7` on both revisions.++The producer exposed a `minimum_matched_events: 2` diagnostic that the consumer no longer emits.++The consumer added an empty `clock_excluded_unsupported_tracks` diagnostic.++All other compared payload fields were equal.++++The full-day runs finished within independent 180-second limits.++An earlier two-point full-day attempt exceeded the same candidate budget on both revisions before scoring.++Its failure receipts and partial artifacts remain available in the local evidence directory.++The successful point partition used the existing documented benchmark configuration instead of an increased candidate budget.++The shared machine ran concurrent work, so the elapsed times do not support a performance claim.++++A separate run used the unmodified cb996dc JobManager and the whole real MP3.++It produced an actual inconclusive job with two nonempty coarse tiles.++The continuation tests reused both tiles and reproduced the ordered ranked result exactly.++The tests also rejected changed producer and geometry identities, invalid manifests, changed runtimes, native processing, and active jobs.++The interrupted-commit control restored the original run identity and metadata.++No original checkpoint or live job changed during these checks.++++## Continue a compatible stopped job++++1. Stop the job and wait for its worker to exit.++2. Copy the complete stopped job directory, including its retention database and referenced inputs.++3. Use the consumer files from `be56c2fae81fe65159103ddb2bf3ccd59ca3bb67` with this continuation change applied.++4. Use the Python version and package versions recorded in the saved manifest.++5. Verify that the checkout has no compiled `_native_index` extension.++6. Run the continuation from that checkout with all three numerical backend variables set to `python`.++++```bash++DOPPLERGEIST_RIDGE_BACKEND=python \++DOPPLERGEIST_ACOUSTIC_BACKEND=python \++DOPPLERGEIST_CPA_BACKEND=python \++PYTHONPATH=src \++python -m dopplergeist.cli resume JOB_ID --jobs /path/to/copied-jobs++```++++The normal resume command performs the compatibility checks before it queues work.++An unsupported consumer leaves the old checkpoint incompatible.++An inconclusive result remains inconclusive unless the resumed search independently meets the normal decision requirements.++++## Repeat the continuation controls++++1. Use an actual stopped cb996dc job with at least one nonempty coarse tile.++2. Run the tests in the pinned consumer checkout.++++```bash++DOPPLERGEIST_RIDGE_BACKEND=python \++DOPPLERGEIST_ACOUSTIC_BACKEND=python \++DOPPLERGEIST_CPA_BACKEND=python \++DOPPLERGEIST_REAL_CB996DC_JOB=/path/to/stopped-job \++PYTHONPATH=src \++python -m pytest -q tests/test_real_cb996dc_checkpoint.py++```++++The tests copy the real checkpoint before fault injection or continuation.++They skip when the environment does not provide a real job.++They do not generate replacement recordings, trajectories, or scores.diff --cc docs/verification/cb996dc-continuation-20261104.jsonindex 0000000,0000000..838ded7new file mode 100644--- /dev/null+++ b/docs/verification/cb996dc-continuation-20261104.json@@@ -1,0 -1,0 +1,586 @@@++{++ "scope": "Recorded-input software equivalence and checkpoint continuation. No independent location accuracy or complete-coverage claim.",++ "original_pull_request": "https://github.com/1337isnot1337/DOPPLERGEIST/pull/260",++ "producer_revision": "cb996dcb49b9d31a24c10b9b5418c35423f7bbcd",++ "producer_code_fingerprint": "c66d402b750381303bd0c7d9c828d2294942227176ccb3cb157f0ef6299cbe40",++ "validated_numerical_consumer_revision": "350c9bfa1a56ce2b46d0418b39779b12cdfa04de",++ "integration_base_revision": "be56c2fae81fe65159103ddb2bf3ccd59ca3bb67",++ "target_code_fingerprint": "2e6a9601eaf42be7ca2b440eef55eeafa894ab21f9f90adcbb216c9f08a16d2d",++ "target_file_hashes": {++ "__init__.py": "01428b7938817c1cfe7eb3cf7e186f35b3fa7bc2d85f14204d07e0aa251d2c88",++ "acoustic_screen.py": "a0a479a46d6c48ab973ec539d1457868a7eb6f4b9c361a0426dc7993f84970ec",++ "audio.py": "7ae1cbd3e682057096f103d34fa5664b11cf4e7a74e25af29736ed6618cc9c6d",++ "automatic.py": "8de96b1450eb26ab33acac5d36680c743c3d2c537b4b0f7182aae9b6faf245bd",++ "batch_ridge.py": "d8187c64c3fcfb2d084fbdaa302b78791c68fa6aed33e2d61f95537058ab7f3a",++ "confidence.py": "6a35edb42bc5321999a179f09f33727122f6de2549924d47f1da99673dcae00d",++ "contracts.py": "602735eef7de44e6410d79336e33c47d35e2c3d00d531729a4c137995ccd1a70",++ "data_access.py": "77da0d39e1e95476fdf02e29e9a345914545af7e62227a93b4cb6786edb37a71",++ "discovery.py": "0a973a1238b7d0c344efa23383ac42a7a8d5b37a2e975b4fad9ac878cee5b2a0",++ "finite_search.py": "f499ba1f298b5869ac5ae375a2eddb46aabfabcfb9f3fd0d848e362c636c0367",++ "flight_payload.py": "d74d218529d91db6397c5450a4b57f1f5efd2b3bc942120858dfabe6e69fd4e5",++ "geometry.py": "85336bd0e3739f890386e9f93965cefb78c4477ae2e8a4fe489353501c5e26a2",++ "index_compatibility.py": "a9aa315a3cdaf9b01bcac027bb4f7b5074864db8623402bc1afcbee24e6ecc83",++ "intake.py": "c88f51f8c7edd6abb74fca830a839d0bbce758beb9a582a7791a8a174b603781",++ "jobs.py": "40517440e0dc380898e0685b8c4bec3320e9cf121626983ee207e42729ad6ebc",++ "matching.py": "777115ddc2752e27a4a4e69cc25eab727ff819d54398a688bccbf8b10ebdff5a",++ "native_acoustic.py": "608475fc2254e9a80549ebd0fc337ba2893d938ee8d4834f1573d0022b1b6f7f",++ "native_cpa.py": "32c02cd43f6a1a67bdaa17ee3cdaef385fd763a4781610aeada9e1d8d714566d",++ "parallel_retention.py": "2861608faa3490601aec9870647ec2a5f9268ff051548565b6d657cb1e95155f",++ "parallel_tiles.py": "3605621c3c5fd9b35008fe1680e1bea99f968ec841f69b4a5bc9e3ced3c52108",++ "provenance.py": "b3dfda5b31f1aec195eef8ef763b08b2734265da06ce41c129daf70acce8c041",++ "resource_usage.py": "2704ff96bc9ae5a39fa29b3f4cee590e38bc5e59f3ec369e6aa28aab307f5d04",++ "scoring_inputs.py": "9cbc11678ba5a555acf234945e037662b494591df6a33d21b460e726c62c3588",++ "search.py": "41b75f9f2d4a951cd2e1cac42b81ca5a7e15620fca4cd44e75767de5a025f9fb",++ "spectral_grid.py": "67fe613a0ee9505f1d9cfd8db468ee03ccdf8664fe1608d6df3d484ebcaad27c",++ "spectral_matching.py": "8de9273ea8090ce9e5a639c6d9fb4b88c2455afc02413643935983149fe9d2e6",++ "terrain.py": "749d6e1d526324fc3015511e9f83fde3663d31a1fc863f069bd18f56601869f8",++ "tiled_search.py": "592ebec1d077cc9b42c55ea07bccc4097c082e2c455a7a1580711deae2956eed",++ "tracks.py": "543f483cee756fdb9efdd69c8e8f05b32e290f19187af3523160f19d2eba90e0",++ "trial_prefix.py": "cd428883c036d52e9eb6f06cb453fa4cfa63614695ff229db21f3ca5ebfba285"++ },++ "backend_environment": {++ "DOPPLERGEIST_RIDGE_BACKEND": "python",++ "DOPPLERGEIST_ACOUSTIC_BACKEND": "python",++ "DOPPLERGEIST_CPA_BACKEND": "python"++ },++ "runtime": {++ "python": "3.12.14",++ "machine": "x86_64",++ "packages": {++ "numpy": "2.3.5",++ "scipy": "1.17.0",++ "pandas": "2.2.3",++ "pyproj": "3.8.0",++ "rasterio": "1.5.2",++ "pyarrow": "26.0.1"++ }++ },++ "same_full_day_runtime": true,++ "numerical_thread_environment": {++ "OPENBLAS_NUM_THREADS": null,++ "OMP_NUM_THREADS": null,++ "MKL_NUM_THREADS": null,++ "NUMEXPR_NUM_THREADS": null,++ "VECLIB_MAXIMUM_THREADS": null++ },++ "regional_replay": {++ "producer_revision": "cb996dcb49b9d31a24c10b9b5418c35423f7bbcd",++ "consumer_revision": "06693a457f4f1dde0d194588232d19a459f946e1",++ "scope": "Complete nonempty regional tile from real MP3 discovery and recorded ADS-B/DEM; 20-second source-informed clock; not the canonical full-day checkpoint/runtime.",++ "points": 2,++ "tracks": 317,++ "groups": 16,++ "candidates": 1699,++ "ranked_count": 20,++ "same_ordered_ranked_rows": true,++ "same_points": true,++ "same_tracks": true,++ "same_events": true,++ "same_runtime": true,++ "same_retention": true,++ "retention": {++ "stages": {++ "rows": 1,++ "sha256": "93c084095e2e9ec2f6a36c6840d6879b0361fbe70b4705a382c289bb6f856b3d"++ },++ "records": {++ "rows": 112,++ "sha256": "dfe03cfaafdec30c9cec19a5e809bf59a232fa0acd278590008d57d0ca88910e"++ },++ "bounds": {++ "rows": 1617,++ "sha256": "660201d145a54d34970a129d9dff7cc0a983029452eb6779cc7dbc699075a3a4"++ }++ },++ "retention_checksum": "62a465ba1ef79dab292143b8d7af4c41e0c9ed4e3672bbfd2eaa2d6b2b16e31f",++ "counts_differences": {++ "clock_excluded_unsupported_tracks": {++ "cb996dc": null,++ "current": []++ },++ "minimum_matched_events": {++ "cb996dc": 2,++ "current": null++ }++ },++ "all_other_payload_fields_equal": true,++ "source_checkpoint_preserved": {++ "search/manifest.json": true,++ "search/retention.sqlite": true,++ "search/tile-58_68.json": true,++ "discovery/discovery.json": true++ },++ "options": {++ "bounds": [++ -73.545,++ 40.865,++ -73.505,++ 40.895++ ],++ "time_range_s": [++ 1788898746.3333333,++ 1788902290.1845++ ],++ "budget": {++ "spacing_m": 2000,++ "tile_cells": 16,++ "max_tracks_per_tile": 2000,++ "max_track_samples_per_tile": 500000,++ "max_cpa_rows": 100000,++ "max_candidates_per_tile": 20000,++ "top_k": 20,++ "refine_spacing_m": 500,++ "refine_finalists": 3,++ "cpa_step_s": 2.0,++ "max_tiles": null++ },++ "scoring": {++ "mode": "robust_ridge",++ "c": 343.0,++ "time_origin": null,++ "time_sweep_s": 5,++ "time_step_s": 0.5,++ "recording_start_window_s": [++ 1788898640.0,++ 1788898660.0++ ]++ },++ "filters": {++ "association": "subsets",++ "max_events": 16++ },++ "crs": "EPSG:5070",++ "coarse_floor_policy": "global_and_per_tile",++ "execution": "Direct production _tile_coordinates -> _tracks_for_points -> _sample_tile_terrain -> _score for all recorded cells in tile58_68; no candidate/event truncation and no initial score-floor override."++ },++ "inputs": [++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/search/manifest.json",++ "bytes": 139863,++ "sha256": "cb4510a238c5de5014ff0225a64380aa8ac26e32be6e252bb55364a79a530605"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/search/tile-58_68.json",++ "bytes": 2018801,++ "sha256": "50294472363ac7e70d8a1005dfedd7b3110a9506c1889d15915fb6ecac10be4c"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/discovery.json",++ "bytes": 213364,++ "sha256": "c640045ef8aedf29a0cf50f6cdef07d53aa3f024d403ee243978c7250314d80c"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/input.mp3",++ "bytes": 42085293,++ "sha256": "e1c13dafd8d1634bbf84df843c31771318e15a31ebb86dd165af7ee312ab80a0"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/real-ledger-inputs/reference-region.sqlite",++ "bytes": 85929984,++ "sha256": "17ab5d9b860513c86fa850b6c7e7318cf5d507339dfa1921e86611439b214c6a"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/replay_tile.py",++ "bytes": 4978,++ "sha256": "cb431588a3502d0b75d8e716ea655b7a33c0b168b53cd3e8cfa4176ac834d2d0"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/DOPPLERGEIST/data/terrain/Copernicus_DSM_COG_10_N40_00_W074_00_DEM.tif",++ "bytes": 15806993,++ "sha256": "6a0f4273de0bbc0709cc9db4793a89ac40c811388f59f23f644c0304b66ab94d"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/DOPPLERGEIST/data/terrain/us_nga_egm96_15.tif",++ "bytes": 2710815,++ "sha256": "db493027562c9b004d7220fa881f5603adada4e1c5029b933fa7de4547b0e78d"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0001.csv",++ "bytes": 1602,++ "sha256": "db32c8e940b5512873ec18fe405eb06cf363d4567815db3ee2531f9190bf5ab4"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0002.csv",++ "bytes": 1827,++ "sha256": "afe993bfe19457324ee47ff8f2cf938214a90aa00fd4a518e1ec54050837e643"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0003.csv",++ "bytes": 2727,++ "sha256": "6ca82441bd4346d21827d99ca0910ea51ead05101d5261a44e5f1ad604d35a90"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0004.csv",++ "bytes": 1677,++ "sha256": "277d1c95b08dee58b17fd8ada90a38b9a2d7b0f0cc45ea1a6f4217de9dde81ce"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0005.csv",++ "bytes": 2502,++ "sha256": "9f74137474bf5daf8ea0a0386e4d8111ec20495082c798a42ba109f8c540049c"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0006.csv",++ "bytes": 2277,++ "sha256": "ae5abab11a3c5d546c2efb75a1d045b786a101e238ee4a9609284d6a5d2c7aa5"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0007.csv",++ "bytes": 3627,++ "sha256": "3fe3027c1985975537f70927e8f5490d0ab86465441a74888f2dc4ff418cb701"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0008.csv",++ "bytes": 3327,++ "sha256": "fb53735e349b179538fa65e86b0db40716159e84c1de43f51dc9ecd380186c51"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0009.csv",++ "bytes": 1752,++ "sha256": "a691392e4348390f0f681677699a536d4ff05e775abfcfee6827d5f961935301"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0010.csv",++ "bytes": 2952,++ "sha256": "cfb556b2203a4fe8949aad384db33d310d2158cba754ed422aaf054bded88be3"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0011-harmonic-2.csv",++ "bytes": 4527,++ "sha256": "9ef47f55d3b7972c1ec4e32cc123e01e91e4317c4154b9b72bbd392130efa89f"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0011-harmonic-3.csv",++ "bytes": 1752,++ "sha256": "bad92f16bbb3da88d85c6d23e4b9bfc7c6d917c6be78b034198a5c1140eb17d8"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0011.csv",++ "bytes": 15027,++ "sha256": "c2381e2d365fa15f02ed6389f19e3a3c8bda75147181ffed3f4e253f807cd1d4"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0012.csv",++ "bytes": 4377,++ "sha256": "cfec0566d225300f0d9a0bf6dc9b92551ebad62610c964afa00753ae57f9c474"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0013.csv",++ "bytes": 6327,++ "sha256": "b660a01a291000ad451e30ed44f0fb0cea327571c9a81a7437238d8c1df0c81d"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0014.csv",++ "bytes": 1827,++ "sha256": "ce724f76e6169ecc3b43526d300c653d92cc8222e786abaf7a10cfcd9d34d1be"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0015.csv",++ "bytes": 4752,++ "sha256": "0ecebac047804687a696c4153e2157baeffc8bfcb18afe04b2a1362788810e47"++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-main-ledger/jobs/6f9890441e4b4add8a7eb35890c185a5/discovery/event-0016.csv",++ "bytes": 2127,++ "sha256": "ecf83cc2872b207a9005a216a2ef58af453899c334aba4383104afbe64568d28"++ }++ ],++ "outputs": {++ "cb996dc": {++ "report.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/cb996dc/report.json",++ "bytes": 7342,++ "sha256": "d87a38b664773546b7bada3280a5ad93a3a2917bca56986a556ea58774480f1b"++ },++ "payload.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/cb996dc/payload.json",++ "bytes": 3108129,++ "sha256": "14025e0d691cdba59b7d52b7f9d5b201f8106ad21ae39b5fe6f888bbdebdaf1a"++ },++ "retention.sqlite": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/cb996dc/retention.sqlite",++ "bytes": 2654208,++ "sha256": "fc677b7a95621bef5797be1dc6b3220a7f7c07e8f222129a352c8898cbd553e7"++ }++ },++ "current": {++ "report.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/current/report.json",++ "bytes": 7328,++ "sha256": "1df2c1c3f827a461eee32c6480376f0cbb4b4e02ccc601bd9483ae73a28ebc2f"++ },++ "payload.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/current/payload.json",++ "bytes": 3108141,++ "sha256": "797900746ffd868ef6686189d7466d8031d655b3c235fa1e700c065503d5cb0a"++ },++ "retention.sqlite": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/current/retention.sqlite",++ "bytes": 2654208,++ "sha256": "fc677b7a95621bef5797be1dc6b3220a7f7c07e8f222129a352c8898cbd553e7"++ }++ }++ }++ },++ "full_day_replay": {++ "producer_revision": "cb996dcb49b9d31a24c10b9b5418c35423f7bbcd",++ "consumer_revision": "350c9bfa1a56ce2b46d0418b39779b12cdfa04de",++ "scope": "Documented real full-day dense point/association-anchor partition; unknown recording clock, source-informed receiver, incomplete regional coverage; no accuracy claim.",++ "fixture_path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-fixture.json",++ "fixture_sha256": "d7ad8de1657d524b933454fb22c842c8d14bf4c35bb595f4e481269b6afefdcd",++ "runtime": {++ "python": "3.12.14",++ "machine": "x86_64",++ "packages": {++ "numpy": "2.3.5",++ "scipy": "1.17.0",++ "pandas": "2.2.3",++ "pyproj": "3.8.0",++ "rasterio": "1.5.2",++ "pyarrow": "26.0.1"++ }++ },++ "points": 1,++ "tracks": 1354,++ "groups": 16,++ "candidates": 16015,++ "ranked_count": 100,++ "retention": {++ "stages": {++ "rows": 1,++ "sha256": "973d246388dd4523e770a52ba98edf77a9dcc684e6c463cfebb253a1a651ba02"++ },++ "records": {++ "rows": 499,++ "sha256": "316ed354465d0548133f84ab118b9d96f00dce8bf6f0df4d44c247df52a2e488"++ },++ "bounds": {++ "rows": 15691,++ "sha256": "3d72914f96e31f5b3c1954e292a1f295be40c1dedf6ac188bd75048e6469bfc0"++ }++ },++ "retention_checksum": "ca75b434b7776ca1507c9efed4e3e0f7df891a623c0b28b30fe19d0d1b5048e7",++ "same_ordered_ranked_rows": true,++ "same_retention": true,++ "same_points_tracks_events": true,++ "all_other_payload_fields_equal": true,++ "counts_differences": {++ "minimum_matched_events": {++ "producer": 2,++ "consumer": null++ },++ "clock_excluded_unsupported_tracks": {++ "producer": null,++ "consumer": []++ }++ },++ "recording_start_window_s": null,++ "query_time_s": [++ 1788825600.17,++ 1788911999.6++ ],++ "anchor_window_s": [++ 1788864888.7151732,++ 1788865236.7214916++ ],++ "producer_outputs": {++ "report.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-cb996dc/report.json",++ "sha256": "30e1af59679cddd5bdf5515561d8aca779002c7b20479e871a9d0df1cac14eee",++ "bytes": 24090++ },++ "payload.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-cb996dc/payload.json",++ "sha256": "6f16fa437e1b9b67348b6f4979f907a19dcc73dfa414fff8913f52b10f4d4bae",++ "bytes": 29506029++ },++ "retention.sqlite": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-cb996dc/retention.sqlite",++ "sha256": "e893fc635ea60b15207f208d91c4a34f352b06472b93cbcd1039f40351f8942a",++ "bytes": 25264128++ }++ },++ "consumer_outputs": {++ "report.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-current318/report.json",++ "sha256": "2accec444f07f7ea1f951fedf16514927615be415f618ed12563198bfd27d913",++ "bytes": 24075++ },++ "payload.json": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-current318/payload.json",++ "sha256": "defc23048b7afd29670d512c70e8b1a23337dd9d99193e2a50a2f4476ae44b60",++ "bytes": 29506041++ },++ "retention.sqlite": {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-current318/retention.sqlite",++ "sha256": "e893fc635ea60b15207f208d91c4a34f352b06472b93cbcd1039f40351f8942a",++ "bytes": 25264128++ }++ }++ },++ "full_day_input_files": [++ {++ "path": "/workspace/scratch/e20de6666de2/DOPPLERGEIST-pr260-review/evidence/audio/w1-sensitivity-20261102/reference/discovery.json",++ "sha256": "89281081ba16310e622a90001373bb1d21ad59eed5667842e98957753feab26c",++ "bytes": 188675++ },++ {++ "path": "/workspace/scratch/e20de6666de2/DOPPLERGEIST-pr260-review/evidence/results/reference-local/report.json",++ "sha256": "29b01918f2bb20e9f9a3770bfcc229f1bd1b8e8862fe2f66483b988a73cbf090",++ "bytes": 4311++ },++ {++ "path": "/workspace/scratch/e20de6666de2/real-ledger-inputs/reference-region.sqlite",++ "sha256": "17ab5d9b860513c86fa850b6c7e7318cf5d507339dfa1921e86611439b214c6a",++ "bytes": 85929984++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-fixture.json",++ "sha256": "d7ad8de1657d524b933454fb22c842c8d14bf4c35bb595f4e481269b6afefdcd",++ "bytes": 358130++ },++ {++ "path": "/workspace/scratch/e20de6666de2/verification-pr260-parity/replay_documented_full_day.py",++ "sha256": "d3d5b45e71098e8c21eb3fe035ac21ef697595841f7238c1886887e8d8cb5d57",++ "bytes": 4404++ }++ ],++ "full_day_configuration": {++ "index_input_hash": "229d57dc810fc3f2985bae5c7fc68eb1132912cf8b95fdea2376c739d66e22a2",++ "query_bounds": [++ -74.299988,++ 40.100005,++ -72.700011,++ 41.599964++ ],++ "query_time_s": [++ 1788825600.17,++ 1788911999.6++ ],++ "budget": {++ "spacing_m": 2000.0,++ "tile_cells": 32,++ "max_tracks_per_tile": 2000,++ "max_track_samples_per_tile": 500000,++ "max_cpa_rows": 100000,++ "max_candidates_per_tile": 20000,++ "top_k": 100,++ "refine_spacing_m": 100.0,++ "refine_finalists": 10,++ "cpa_step_s": 2.0,++ "max_tiles": null++ },++ "scoring": {++ "mode": "robust_ridge"++ },++ "recording_start_window_s": null,++ "anchor_window_s": [++ 1788864888.7151732,++ 1788865236.7214916++ ],++ "track_query_preparation": {++ "queried_trajectories": 4694,++ "queried_trajectory_samples": 713100,++ "operations": {++ "query_calls": 1,++ "decoded_track_yields": 4694,++ "decoded_sample_yields": 713100++ }++ },++ "counts": {++ "trajectories": 1354,++ "trajectory_samples": 357324,++ "ridge_records": 16,++ "independent_groups": 16,++ "ridge_samples": 724,++ "cpa_rows": 944,++ "unsupported_cpa_tracks": 62++ }++ },++ "full_day_receipts": [++ {++ "label": "documented-full-day-cb996dc",++ "source": "/workspace/scratch/e20de6666de2/DOPPLERGEIST-cb996dc-review",++ "started_utc": "2026-11-04T03:10:52.091754+00:00",++ "wall_cap_s": 180.0,++ "passed": true,++ "command": [++ "/workspace/scratch/e20de6666de2/.venv/bin/python",++ "/workspace/scratch/e20de6666de2/verification-pr260-parity/replay_documented_full_day.py",++ "/workspace/scratch/e20de6666de2/DOPPLERGEIST-cb996dc-review",++ "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-cb996dc"++ ],++ "backend_environment": {++ "DOPPLERGEIST_RIDGE_BACKEND": "python",++ "DOPPLERGEIST_ACOUSTIC_BACKEND": "python",++ "DOPPLERGEIST_CPA_BACKEND": "python"++ },++ "source_revision": "cb996dcb49b9d31a24c10b9b5418c35423f7bbcd",++ "returncode": 0,++ "timed_out": false,++ "wall_seconds": 69.19827941400035,++ "partial_artifacts_preserved": true++ },++ {++ "label": "documented-full-day-current318",++ "source": "/workspace/scratch/e20de6666de2/DOPPLERGEIST-pr260-review",++ "started_utc": "2026-11-04T03:10:52.086807+00:00",++ "wall_cap_s": 180.0,++ "passed": true,++ "command": [++ "/workspace/scratch/e20de6666de2/.venv/bin/python",++ "/workspace/scratch/e20de6666de2/verification-pr260-parity/replay_documented_full_day.py",++ "/workspace/scratch/e20de6666de2/DOPPLERGEIST-pr260-review",++ "/workspace/scratch/e20de6666de2/verification-pr260-parity/documented-full-day-current318",++ "--normalize-min-events"++ ],++ "backend_environment": {++ "DOPPLERGEIST_RIDGE_BACKEND": "python",++ "DOPPLERGEIST_ACOUSTIC_BACKEND": "python",++ "DOPPLERGEIST_CPA_BACKEND": "python"++ },++ "source_revision": "350c9bfa1a56ce2b46d0418b39779b12cdfa04de",++ "returncode": 0,++ "timed_out": false,++ "wall_seconds": 42.10764088199994,++ "partial_artifacts_preserved": true++ }++ ],++ "real_legacy_job": {++ "job": "/workspace/scratch/e20de6666de2/verification-pr260-parity/genuine-cb996dc-job/jobs/c00d7c754bc845ccaca8f8c0a3d60c92",++ "status": "inconclusive",++ "reason_code": "search_incomplete",++ "run_hash": "1305ac416229590a1133166f5ebab5af67432942636a4bf74c3c15da6bc2d91c",++ "producer_code_fingerprint": "c66d402b750381303bd0c7d9c828d2294942227176ccb3cb157f0ef6299cbe40",++ "coarse_floor_policy": "per_tile",++ "nonempty_tiles": [++ {++ "name": "tile-58_68.json",++ "candidates": 1699,++ "ranked": 20,++ "retention_checksum": "7a26d507ad4da1ec11e2e5361585eedd35c5ac69853bef3a100171fa158f0150"++ },++ {++ "name": "tile-58_69.json",++ "candidates": 1681,++ "ranked": 20,++ "retention_checksum": "27555384b433d759a2ff8525c043b5c23c1fa6511d5eb4423ebca8579f5491f7"++ }++ ]++ },++ "continuation_controls": {++ "command": "DOPPLERGEIST_RIDGE_BACKEND=python DOPPLERGEIST_ACOUSTIC_BACKEND=python DOPPLERGEIST_CPA_BACKEND=python DOPPLERGEIST_REAL_CB996DC_JOB=/path/to/real-stopped-job PYTHONPATH=src python -m pytest -q tests/test_real_cb996dc_checkpoint.py",++ "result": "10 passed",++ "source_checkpoints_unchanged": true++ },++ "limitations": [++ "Only this pinned Python consumer family is admitted.",++ "The geometry changes proposed in #293 require a new compatibility proof and invalidate this target fingerprint.",++ "The shared-observation changes for #275 invalidate the matching.py/search.py consumer pin and require a new compatibility proof.",++ "The full-day comparison covers one documented association-anchor partition at a source-informed receiver, not a complete search.",++ "The historical regional data omit known jets and do not establish complete coverage.",++ "The replay used Python 3.12.14 and the recorded package versions. Continuation also requires exact producer/consumer runtime identity.",++ "An earlier full-day two-point attempt exceeded the candidate budget on both revisions; its partial artifacts are preserved.",++ "Elapsed times on a shared machine are not a performance result."++ ]++}diff --cc src/dopplergeist/checkpoint_resume.pyindex 69bb4ec,4219d4b..511d6ae--- a/src/dopplergeist/checkpoint_resume.py+++ b/src/dopplergeist/checkpoint_resume.py@@@ -24,35 -25,43 +25,69 @@@ LEGACY_SOURCES ="finite_search.py": "78ed9fca6cb13c9754ea0f4438f1bb54bbcdc488ead2a4d77a15f149b6d77b1c",}-# Complete code-map identity for the real combined scorer/scheduler used by-# the 7ae saved continuation. Unlike the older legacy scorer above, this code-# already used the batched scoring arithmetic now on main.++# The already upgraded reference search has been replayed exactly against this+# Python scoring family. Keep this transition bound to its complete input hash.+BATCHED_REFERENCE_RUN = "9890647b03bc1f966f3b039727e7c0f00ccef0b4250b7312d5b423c191ca3d3f"+ORIGINAL_REFERENCE_RUN = "d1ef045417a64a792b81c7355e508bb204046a70b614fb5815aba14b74a75456"+THROUGHPUT_SOURCES = {+ "matching.py": "8c2ba9b32ce035f07a8357bf0da05f63a5ac693baad5f6dc52930a82f94439c9",+ "search.py": "be84feaef43fa356e3e77e0132e1e427b677216f4c3e84166e566a1d9787c99e",+ "batch_ridge.py": "314d4837fadd80f94410d21c8d5115fa65aff0515e3de2cf4bb08fefd427680a",+ "geometry.py": "f30f6aa755387f7c28f4bb49d0750e7afc42551f3e5af34f9d8be87c5f0cdc26",+ "tiled_search.py": "8b54a363611863e896cb86b388daa0fd5722161eac7b19ea8dbe1937b10c45ea",+ "acoustic_screen.py": "ce2b4e86fed8dfba92ae4742f8cd273f80711976df6ca0a399f1f233d6a6ff32",+ "finite_search.py": "f1bae3b5a947a3d3234df3b3b1bc8edecca5cf7682ed561d45ad38fe34681fe6",+ "parallel_tiles.py": "6b5ff3def69b2433c00f9c4afbd1c52ccc0a79e0f623bdf0dff3f61c0d7ea16a",+ "data_access.py": "1b599aadb8cafa6be0c6aeec959e0c450c1b5c515c24fad6e33cb97719bce6c1",+ "tracks.py": "543f483cee756fdb9efdd69c8e8f05b32e290f19187af3523160f19d2eba90e0",+ "native_acoustic.py": "a048cb3bd2d308c91995c97e587f181fbf6d74d6dd9598ab3ab3fd1cc4a50796",+}+++# cb996dc's combined scheduler/scorer uses ordinary batched arithmetic. Its++# real broad-clock tile and a regional tile were replayed against this Python++# consumer, including the complete ranked payload and retention records/bounds.+ CB996DC_CODE_FINGERPRINT = "c66d402b750381303bd0c7d9c828d2294942227176ccb3cb157f0ef6299cbe40"++CB996DC_TARGET_FINGERPRINT = "2e6a9601eaf42be7ca2b440eef55eeafa894ab21f9f90adcbb216c9f08a16d2d"++CB996DC_TARGET_FILES = (++ "__init__.py", "acoustic_screen.py", "audio.py", "automatic.py", "batch_ridge.py",++ "confidence.py", "contracts.py", "data_access.py", "discovery.py", "finite_search.py",++ "flight_payload.py", "geometry.py", "index_compatibility.py", "intake.py", "jobs.py",++ "matching.py", "native_acoustic.py", "native_cpa.py", "parallel_retention.py",++ "parallel_tiles.py", "provenance.py", "resource_usage.py", "scoring_inputs.py",++ "search.py", "spectral_grid.py", "spectral_matching.py", "terrain.py", "tiled_search.py",++ "tracks.py", "trial_prefix.py",++)++-def _legacy_source_profile(configuration):- code = configuration.get("code", {})- if all(code.get(name) == digest for name, digest in LEGACY_SOURCES.items()):- return "legacy-first-batch"- if content_hash(code) == CB996DC_CODE_FINGERPRINT:- return "combined-cb996dc"- return None++def _consumer_fingerprint(code):++ # Pin orchestration, input interpretation, numerical scoring, retention and++ # their Python dependencies. CLI/API/UI-only changes do not change this++ # proof. The adapter cannot hash itself; native execution is gated below.++ return content_hash({name: code.get(name) for name in CB996DC_TARGET_FILES})+++ def _combined_min_events(configuration):- """Preserve cb996dc's broad-clock default when current main resumes it."""++ """Preserve the producer's default instead of adopting main's singleton default."""+ filters = configuration.get("filters", {})+ if "min_events" in filters:+ return filters["min_events"]+ events = configuration["events"]- max_events = filters.get("max_events") or len(events)- start_window = configuration["scoring"].get("recording_start_window_s")- clock_window_width = math.inf if start_window is None else start_window[1] - start_window[0]- default = min(2, max_events) if len(events) > 1 and clock_window_width > 10.0 else 1- return default++ maximum = filters.get("max_events") or len(events)++ window = configuration["scoring"].get("recording_start_window_s")++ width = math.inf if window is None else window[1] - window[0]++ return min(2, maximum) if len(events) > 1 and width > 10.0 else 1+def upgrade_legacy_job(directory):- """Prepare a compatible replacement search before swapping a stopped job.+ """Prepare a replacement search before swapping it into a stopped job.Completed payloads and retention records stay unchanged. The old directory- and audio/operator identities remain available for rollback. New work uses- staged batching with the saved first-group arithmetic and fallback checks.- and audio/operator identities remain available for rollback. The old- first-batch arithmetic is retained only for the matching legacy source.++ and audio/operator identities remain available for rollback. Each profile++ preserves its scoring arithmetic and requires its validated consumer."""from .automatic import _audio_identity, _digest- from . import geometry, matching, native_cpa- from . import geometry, matching++ from . import data_access, geometry, matching, native_cpadirectory = Path(directory)search = directory / "search"@@@ -60,36 -69,18 +95,44 @@@return Nonemanifest = json.loads((search / "manifest.json").read_text())old = manifest["configuration"]- source_profile = _legacy_source_profile(old)- if source_profile is None:+ if content_hash(old) != manifest.get("run_hash"):return None- legacy_first_batch = source_profile == "legacy-first-batch"++ combined = content_hash(old.get("code", {})) == CB996DC_CODE_FINGERPRINT+ previous_upgrade = None- if any(old["code"].get(name) != digest for name, digest in LEGACY_SOURCES.items()):++ if not combined and any(old["code"].get(name) != digest for name, digest in LEGACY_SOURCES.items()):+ report_path = directory / "checkpoint-upgrade.json"+ if manifest["run_hash"] != BATCHED_REFERENCE_RUN or not report_path.exists():+ return None+ previous_upgrade = json.loads(report_path.read_text())+ if (previous_upgrade.get("version") != 1+ or previous_upgrade.get("old_run_hash") != ORIGINAL_REFERENCE_RUN+ or previous_upgrade.get("new_run_hash") != BATCHED_REFERENCE_RUN+ or previous_upgrade.get("legacy_first_batch") is not True+ or old["scoring"].get("legacy_first_batch") is not True):+ return Noneif getattr(matching, "_native_fit_irls", None) is not None:- raise ValueError("Checkpoint compatibility requires the Python ridge fitter")+ raise ValueError("Legacy checkpoint continuation requires the Python ridge fitter")if geometry.ACOUSTIC_BACKEND != "python-acoustic-v1":- raise ValueError("Checkpoint compatibility requires the Python acoustic solver")+ raise ValueError("Legacy checkpoint continuation requires the Python acoustic solver")+ if native_cpa.CPA_BACKEND != "python-cpa-v1":+ raise ValueError("Legacy checkpoint continuation requires Python closest-approach processing")++ if combined and getattr(data_access, "_native_payload_columns", None) is not None:++ raise ValueError("The cb996dc continuation requires the verified Python flight decoder")+ from .provenance import implementation_files+ package_root = Path(__file__).parent+ code = {p.relative_to(package_root).as_posix(): _digest(p)+ for p in implementation_files(package_root, recursive=False)}++ if combined and _consumer_fingerprint(code) != CB996DC_TARGET_FINGERPRINT:++ return None+ if previous_upgrade and any(code.get(name) != digest for name, digest in THROUGHPUT_SOURCES.items()):+ return Noneif (old["scoring"].get("mode") != "robust_ridge" or old.get("coarse_floor_policy") != "per_tile"or old["scoring"].get("time_origin") is not Noneor old.get("filters", {}).get("association") != "subsets"):raise ValueError("This legacy checkpoint mode has no compatible upgrade")++ if combined and (old["scoring"].get("legacy_first_batch")++ or any(search.glob("fine-tile-*.json"))):++ raise ValueError("The cb996dc producer has no compatible legacy-batch or fine-checkpoint mode")runtime = {"python": platform.python_version(), "machine": platform.machine(),"packages": {name: version(name) for name in old["runtime"]["packages"]}}if runtime != old["runtime"]:@@@ -111,8 -102,14 +154,11 @@@if events != old["events"]:raise ValueError("Prepared events no longer match the saved search")current = deepcopy(old)- if source_profile == "combined-cb996dc":+ current["code"] = code- current["scoring"]["legacy_first_batch"] = True++ if combined:+ current.setdefault("filters", {})["min_events"] = _combined_min_events(old)- from .provenance import implementation_files- package_root = Path(__file__).parent- current["code"] = {p.relative_to(package_root).as_posix(): _digest(p)- for p in implementation_files(package_root, recursive=False)}- if legacy_first_batch:++ else:+ current["scoring"]["legacy_first_batch"] = Truenew_hash = content_hash(current)suffix = uuid.uuid4().hex[:12]staged = directory / f".search-upgrade-{suffix}"@@@ -141,21 -138,22 +187,29 @@@# each payload and retention stage before reuse.tiles = len(list(staged.glob("tile-*.json")))write_json(staged / "manifest.json", {"run_hash": new_hash, "configuration": current})- report = {"version": 2 if previous_upgrade else 1,- report = {"version": 1, "created_utc": datetime.now(timezone.utc).isoformat(),++ report = {"version": 3 if combined else (2 if previous_upgrade else 1),+ "created_utc": datetime.now(timezone.utc).isoformat(),"old_run_hash": manifest["run_hash"], "new_run_hash": new_hash,- "tiles_preserved": tiles, "legacy_first_batch": True,- "tiles_preserved": tiles, "legacy_first_batch": legacy_first_batch,- "compatible_source": source_profile,++ "tiles_preserved": tiles, "legacy_first_batch": not combined,"previous_search": str(previous), "identity_backup": str(backup)}+ if previous_upgrade:+ report["compatible_run_hashes"] = sorted([ORIGINAL_REFERENCE_RUN, BATCHED_REFERENCE_RUN])+ report["upgrade_kind"] = "exact_python_throughput_v1"+ report["previous_upgrade_report"] = str(backup / "checkpoint-upgrade.json")++ if combined:++ report.update(upgrade_kind="exact_python_cb996dc_v1",++ source_code_fingerprint=CB996DC_CODE_FINGERPRINT,++ target_code_fingerprint=CB996DC_TARGET_FINGERPRINT)write_json(backup / "upgrade.json", report)search.rename(previous)try:staged.rename(search)audio["identity"] = identity- operator["resume_legacy_first_batch"] = True- if legacy_first_batch:- operator["resume_legacy_first_batch"] = True- else:++ if combined:+ operator.pop("resume_legacy_first_batch", None)- if source_profile == "combined-cb996dc":- operator["filters"] = deepcopy(current["filters"])++ operator["filters"] = deepcopy(current["filters"])++ else:++ operator["resume_legacy_first_batch"] = Truewrite_json(directory / "resume.json", audio)write_json(directory / "operator.json", operator)write_json(directory / "checkpoint-upgrade.json", report)@@@ -181,11 -175,8 +235,21 @@@ def compatible_checkpoint_runs(output,path = Path(output).parent / "checkpoint-upgrade.json"if path.exists():report = json.loads(path.read_text())- if (report.get("version") == 1 and report.get("new_run_hash") == run_hash- and (report.get("legacy_first_batch") is True- or report.get("compatible_source") == "combined-cb996dc")):- accepted.add(report["old_run_hash"])++ if (report.get("version") == 3 and report.get("new_run_hash") == run_hash++ and report.get("legacy_first_batch") is False++ and report.get("upgrade_kind") == "exact_python_cb996dc_v1"++ and report.get("source_code_fingerprint") == CB996DC_CODE_FINGERPRINT++ and report.get("target_code_fingerprint") == CB996DC_TARGET_FINGERPRINT):++ manifest = json.loads((Path(output) / "manifest.json").read_text())++ configuration = manifest.get("configuration", {})++ if (manifest.get("run_hash") == run_hash and content_hash(configuration) == run_hash++ and _consumer_fingerprint(configuration.get("code", {})) == CB996DC_TARGET_FINGERPRINT):++ accepted.add(report["old_run_hash"])+ if report.get("new_run_hash") == run_hash and report.get("legacy_first_batch") is True:+ if report.get("version") == 1:+ accepted.add(report["old_run_hash"])+ elif (report.get("version") == 2 and report.get("upgrade_kind") == "exact_python_throughput_v1"+ and report.get("old_run_hash") == BATCHED_REFERENCE_RUN+ and report.get("compatible_run_hashes") == sorted([ORIGINAL_REFERENCE_RUN, BATCHED_REFERENCE_RUN])):+ accepted.update([ORIGINAL_REFERENCE_RUN, BATCHED_REFERENCE_RUN])return accepteddiff --cc tests/test_real_cb996dc_checkpoint.pyindex 0000000,0000000..3f37441new file mode 100644--- /dev/null+++ b/tests/test_real_cb996dc_checkpoint.py@@@ -1,0 -1,0 +1,157 @@@++"""Continue a real cb996dc job; never generate audio, tracks or scored modes."""++import hashlib++import json++import os++from pathlib import Path++import shutil++import sqlite3++++import pytest++++from dopplergeist import checkpoint_resume, data_access, native_cpa++from dopplergeist.automatic import _load_audio_checkpoint, _search++from dopplergeist.data_access import content_hash++from dopplergeist.finite_search import RetentionLedger++++[email protected](scope="module")++def real_job():++ location = os.environ.get("DOPPLERGEIST_REAL_CB996DC_JOB")++ if not location:++ pytest.skip("Supply an actual stopped cb996dc job with nonempty coarse checkpoints")++ source = Path(location)++ manifest = json.loads((source / "search/manifest.json").read_text())++ assert content_hash(manifest["configuration"]) == manifest["run_hash"]++ assert content_hash(manifest["configuration"]["code"]) == checkpoint_resume.CB996DC_CODE_FINGERPRINT++ assert json.loads((source / "result.json").read_text())["status"] in {"cancelled", "failed", "inconclusive"}++ tiles = [path for path in sorted((source / "search").glob("tile-*.json"))++ if json.loads(path.read_text())["payload"].get("ranked")]++ assert tiles, "The recorded job must include actual scored candidates"++ return source, manifest, tiles++++++def _copy_job(source, destination):++ # request.json continues to identify the unchanged original input.mp3.++ # Copy metadata/ledgers independently so fault injection cannot alter source.++ shutil.copytree(source, destination, ignore=shutil.ignore_patterns("input.mp3"))++ return destination++++++def _digest(path):++ with path.open("rb") as stream:++ return hashlib.file_digest(stream, "sha256").hexdigest()++++++def test_real_nonempty_tiles_resume_after_pinned_upgrade(real_job, tmp_path):++ source, original, tiles = real_job++ original_files = {path: _digest(path) for path in [source / "search/manifest.json",++ source / "search/retention.sqlite", source / "resume.json", source / "operator.json", *tiles]}++ job = _copy_job(source, tmp_path / "job")++ saved = {path.name: path.read_bytes() for path in tiles}++ expected_ranked = json.loads((source / "search/search-report.json").read_text())["ranked"]++ report = checkpoint_resume.upgrade_legacy_job(job)++ assert report["version"] == 3++ assert report["upgrade_kind"] == "exact_python_cb996dc_v1"++ assert report["legacy_first_batch"] is False++ assert report["old_run_hash"] == original["run_hash"]++ assert report["target_code_fingerprint"] == checkpoint_resume.CB996DC_TARGET_FINGERPRINT++ manifest = json.loads((job / "search/manifest.json").read_text())++ operator = json.loads((job / "operator.json").read_text())++ assert operator["filters"] == manifest["configuration"]["filters"]++ assert operator["filters"]["min_events"] == checkpoint_resume._combined_min_events(original["configuration"])++ assert not operator.get("resume_legacy_first_batch")++ assert checkpoint_resume.compatible_checkpoint_runs(job / "search", report["new_run_hash"]) == {++ report["new_run_hash"], report["old_run_hash"]}++ assert checkpoint_resume.compatible_checkpoint_runs(job / "search", report["old_run_hash"]) == {report["old_run_hash"]}++ for name, data in saved.items():++ assert (job / "search" / name).read_bytes() == data++ assert (Path(report["previous_search"]) / name).read_bytes() == data++ assert checkpoint_resume.upgrade_legacy_job(job) is None++ request = json.loads((job / "request.json").read_text())++ recording, events = _load_audio_checkpoint(job, request, operator)++ result, *_ = _search(request, recording, events, job, operator, lambda *args, **kwargs: None)++ assert result["run_hash"] == report["new_run_hash"]++ assert result["tiles_resumed"] == result["tiles_completed"] > 0++ assert result["tile_failures"] == []++ assert result["ranked"] == expected_ranked++ ledger = RetentionLedger(job / "search/retention.sqlite", report["new_run_hash"],++ "matched_group_support_minus_normalized_residual")++ for name, data in saved.items():++ assert (job / "search" / name).read_bytes() == data++ payload = json.loads(data)["payload"]++ stage = "coarse:" + Path(name).stem.removeprefix("tile-")++ assert ledger.stage_valid(stage, payload["retention_checksum"])++ assert {path: _digest(path) for path in original_files} == original_files++++[email protected]("change", ["producer", "consumer", "manifest", "runtime", "native_cpa", "native_index", "active"])++def test_unverified_continuations_leave_real_checkpoint_unchanged(real_job, tmp_path, monkeypatch, change):++ source, original, _ = real_job++ job = tmp_path / "job"++ (job / "search").mkdir(parents=True)++ shutil.copy2(source / "search/manifest.json", job / "search/manifest.json")++ shutil.copy2(source / "result.json", job / "result.json")++ manifest = json.loads((job / "search/manifest.json").read_text())++ if change == "producer":++ manifest["configuration"]["code"]["matching.py"] = "0" * 64++ elif change == "consumer":++ # A later geometry change must invalidate the measured consumer even++ # when its checkpoint format and all other source files stay unchanged.++ from dopplergeist import automatic++ digest = automatic._digest++ monkeypatch.setattr(automatic, "_digest",++ lambda path: "0" * 64 if Path(path).name == "geometry.py" else digest(path))++ elif change == "manifest":++ manifest["run_hash"] = "0" * 64++ elif change == "runtime":++ manifest["configuration"]["runtime"]["python"] += ".changed"++ elif change == "native_cpa":++ monkeypatch.setattr(native_cpa, "CPA_BACKEND", "native-c-cpa-v1")++ elif change == "native_index":++ monkeypatch.setattr(data_access, "_native_payload_columns", object())++ elif change == "active":++ state = json.loads((job / "result.json").read_text())++ state["status"] = "running"++ (job / "result.json").write_text(json.dumps(state))++ if change in {"producer", "runtime"}:++ manifest["run_hash"] = content_hash(manifest["configuration"])++ (job / "search/manifest.json").write_text(json.dumps(manifest))++ before = (job / "search/manifest.json").read_bytes()++ if change in {"producer", "consumer", "manifest"}:++ assert checkpoint_resume.upgrade_legacy_job(job) is None++ else:++ with pytest.raises(ValueError, match={"runtime": "runtime changed", "native_cpa": "Python closest-approach", "native_index": "Python flight decoder", "active": "Stop the previous job"}[change]):++ checkpoint_resume.upgrade_legacy_job(job)++ assert (job / "search/manifest.json").read_bytes() == before++ assert not list(job.glob("checkpoint-upgrade*"))++++++def test_real_upgrade_rolls_back_operator_and_producer_binding(real_job, tmp_path, monkeypatch):++ source, manifest, tiles = real_job++ job = _copy_job(source, tmp_path / "job")++ names = ("search/manifest.json", "resume.json", "operator.json")++ before = {name: (job / name).read_bytes() for name in names}++ write_json = checkpoint_resume.write_json++++ def fail_operator(path, value):++ if Path(path) == job / "operator.json":++ raise OSError("interrupted real checkpoint upgrade")++ return write_json(path, value)++++ monkeypatch.setattr(checkpoint_resume, "write_json", fail_operator)++ with pytest.raises(OSError, match="interrupted real checkpoint"):++ checkpoint_resume.upgrade_legacy_job(job)++ assert {name: (job / name).read_bytes() for name in names} == before++ assert not (job / "checkpoint-upgrade.json").exists()++ with sqlite3.connect((job / "search/retention.sqlite").as_uri() + "?mode=ro", uri=True) as db:++ assert db.execute("SELECT identity FROM run").fetchone() == (manifest["run_hash"],)++ assert (job / "search" / tiles[0].name).read_bytes() == tiles[0].read_bytes()++++++def test_cb996dc_report_cannot_authorize_an_unverified_target(real_job, tmp_path):++ source, _, _ = real_job++ job = _copy_job(source, tmp_path / "job")++ report = checkpoint_resume.upgrade_legacy_job(job)++ report["target_code_fingerprint"] = "0" * 64++ (job / "checkpoint-upgrade.json").write_text(json.dumps(report))++ assert checkpoint_resume.compatible_checkpoint_runs(job / "search", report["new_run_hash"]) == {report["new_run_hash"]}