Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)
Python acoustic-geolocationads-baircraft-trackingairplaneaudio-forensicsdoppler-effectdoppler-shiftgeolocationosintsignal-processing
Preserve the frozen nine-arm real recording experiment runner
Cinna Davis309429278978
diff --git a/docs/REAL_METAMORPHIC.md b/docs/REAL_METAMORPHIC.mdnew file mode 100644index 0000000..5f46a3d--- /dev/null+++ b/docs/REAL_METAMORPHIC.md@@ -0,0 +1,143 @@+# Real recording representation checks++Issue [#61](https://github.com/1337isnot1337/DOPPLERGEIST/issues/61) requires fresh+end-to-end runs for equivalent representations of actual recordings.+The runner in `scripts/run_real_metamorphic.py` executes these runs through the+local HTTP upload service. It accepts a recorded source, a recording-start UTC+window, and an offline operator configuration.++Each arm starts a new service and job. Intake, discovery, selection, matching,+search, refinement, decision and serialization run from the uploaded bytes.+The runner shares only frozen historical indexes and terrain. It does not+resume audio or search checkpoints. The HTTP verifier checks the actual stage+artifacts and the persisted terminal result.++## Frozen protocol++The versioned protocol is+[`verification/metamorphic_protocol_v1.json`](verification/metamorphic_protocol_v1.json).+It was committed before the development matrix ran. It has nine sequential arms:++| Arm | Change from the baseline |+| --- | --- |+| Baseline | MP3, 128 kb/s, 48 kHz, mono |+| Low bitrate | 64 kb/s |+| High bitrate | 192 kb/s |+| Sample rate | 44.1 kHz |+| Gain | −6 dB |+| Channels | Two identical channels |+| Leading silence | 17.25 seconds |+| Short discovery chunks | 60-second chunks, 30-second overlap |+| Long discovery chunks | 180-second chunks, 60-second overlap |++All encodings start from the complete source recording's arithmetic channel+mean at 48 kHz. No arm is encoded from another MP3 arm. The baseline keeps the+operator's discovery settings. The current defaults are 120/60 seconds.++Only `operator.discovery.chunk_s` and `overlap_s` are exposed chunk controls.+The protocol explicitly lists unsupported upload, decoder and detector controls.+It does not insert hooks into the source algorithms.++Each job has a 240-second runtime limit and a 300-second client deadline.+The encoding/decode command limit is 300 seconds. Pipeline time is at most+45 minutes across nine client deadlines, plus preparation and cleanup.+Different budgets can be declared at preparation, before any outcomes exist.++Tolerances are fixed in the protocol: 2 ms for duration and residual alignment,+1 second for mapped event edges and fitted clocks, 2 Hz for median ridge+frequency, and 0.05 per matched group for comparable normalized scores.+Usability also requires nonclipping decoded peaks and gain-corrected PCM+NRMSE at most 0.15. Group counts, flight associations and score units must agree.+The runner reports displacement thresholds at 100, 500 and 1,000 metres.+Accepted locations that move over 1,000 metres receive a separate serious flag.++The PCM calculation reads bounded blocks. Lag checks use twelve fixed short+windows, not full-hour correlations. A measured lag is reported; it does not+shift samples to improve the result. Native-rate decoded peaks and identical+stereo channels are checked separately from the 4 kHz analysis PCM.++## Clock mapping++If leading silence is Δ seconds, subtract Δ from **both** sample-zero UTC+window endpoints. Subtract Δ from detected audio-relative times for comparison.+Add Δ to the fitted sample-zero UTC to recover the original recording clock.++For example, 20:17:20–20:17:40 UTC becomes 20:17:02.750–20:17:22.750 UTC+after 17.25 seconds of padding. The physical time of each recorded sound stays+the same. The runner retains codec skip/discard metadata, decoded sample counts,+and residual lag checks. Container start time is not a recording clock.++## Run++Use an installed checkout with FFmpeg and the repository's dependencies.+The request must have a defensible time window. Use absolute local paths for+historical indexes and terrain, or run preparation where relative paths resolve.+Use a stopped, immutable data cache. Active job directories are not data inputs.++```bash+PYTHONPATH=src python scripts/run_real_metamorphic.py prepare \+ --audio /absolute/path/actual-recording.wav \+ --request /absolute/path/request.json \+ --operator /absolute/path/operator.json \+ --source-note 'Identify the actual recorded source and its permission/provenance' \+ --scope-note 'Describe the search scope, clock evidence and evaluation limits' \+ --output /absolute/path/new-frozen-plan++PYTHONPATH=src python scripts/run_real_metamorphic.py run \+ --plan /absolute/path/new-frozen-plan/plan.json \+ --output /absolute/path/new-matrix-run+```++Preparation writes an immutable plan before encoding. It pins source bytes,+configuration, historical inputs, terrain, source code, native binaries,+dependencies, backend environment and FFmpeg version. Runs refuse changed+inputs. Use `--frozen-input PATH` for an additional immutable local dependency.+Every output directory must be new.++The runner preserves encoded inputs, exact transformation commands, actual+job artifacts and observations for every arm. It continues after an ordinary+arm failure. It preserves pending arms on interruption or changed inputs.+An unusable encoded arm still reaches the HTTP pipeline. Encoding failures+remain explicit failures.++`report.json` contains all pairwise comparisons. `report.md` gives a short+case table. Event matching uses a one-to-one assignment based on mapped time+overlap and ridge frequency. It reports unmatched ridges/groups, group splits,+group merges, and ambiguous alternatives. IDs and row positions do not define+correspondence. Retained distant modes are compared separately.++Each case includes a one-arm reproducer command. The `--case` option creates+a fresh job and records the other arms as not selected. It cannot pass the+full-matrix requirement. The `report` command compares preserved observations;+it never counts as another end-to-end run.++Exit zero requires all nine usable arms to reach the full pipeline and remain+within the declared comparison tolerances. Exit one preserves incomplete runs+or observed differences. An inconclusive scientific decision is not itself a+runner failure when the complete pipeline ran. It remains an inconclusive+location result.++The full-path count uses the verifier's stronger requirement: every stage must+be verified and at least one diagnostic candidate must survive. Empty outcomes+remain in the report. A separate count records all observed stage sequences.+Neither count requires a supported location decision.++## Scope of the development run++The affordable development control uses the whole released reference recording,+a source-informed regional box, a narrow source-informed UTC window, and+incomplete historical flight coverage. It cannot satisfy national acceptance.+It does not establish independent location accuracy.++No eligible independent truth is supplied in this protocol. Truth errors at+100/500/1,000 metres remain explicitly unevaluated. Pairwise displacement is+reported separately. A future eligible national evaluation needs its own+predeclared request, data, budgets and evaluation-only truth protocol.++Focused contract checks use archived real ridge observations and an optional+short actual recording. They do not replace any full-recording arm:++```bash+DOPPLERGEIST_REAL_METAMORPHIC_AUDIO=/absolute/path/actual-short-recording.mp3 \+ PYTHONPATH=src pytest -q tests/test_real_metamorphic_runner.py+```diff --git a/docs/verification/metamorphic_protocol_v1.json b/docs/verification/metamorphic_protocol_v1.jsonnew file mode 100644index 0000000..ed9ba50--- /dev/null+++ b/docs/verification/metamorphic_protocol_v1.json@@ -0,0 +1,71 @@+{+ "schema_version": 1,+ "protocol_id": "real-recording-metamorphic-v1",+ "issue": 61,+ "source_policy": "One complete actual recording; arithmetic channel mean; no generated events or tracks; no truth supplied to search.",+ "canonical_pcm": {"sample_rate_hz": 48000, "channels": 1, "sample_format": "f32le"},+ "baseline": {"bitrate_kbps": 128, "sample_rate_hz": 48000, "channels": 1, "gain_db": 0.0, "leading_silence_s": 0.0},+ "cases": [+ {"id": "baseline", "changes": {}},+ {"id": "bitrate_64", "changes": {"bitrate_kbps": 64}},+ {"id": "bitrate_192", "changes": {"bitrate_kbps": 192}},+ {"id": "rate_44100", "changes": {"sample_rate_hz": 44100}},+ {"id": "gain_minus_6db", "changes": {"gain_db": -6.0}},+ {"id": "identical_stereo", "changes": {"channels": 2}},+ {"id": "leading_17_25s", "changes": {"leading_silence_s": 17.25}},+ {"id": "chunks_60_30", "changes": {"discovery": {"chunk_s": 60.0, "overlap_s": 30.0}}},+ {"id": "chunks_180_60", "changes": {"discovery": {"chunk_s": 180.0, "overlap_s": 60.0}}}+ ],+ "budgets": {"job_max_runtime_s": 240.0, "http_timeout_s": 300.0, "encoding_timeout_s": 300.0},+ "tolerances": {+ "decoded_duration_error_s": 0.002,+ "residual_alignment_lag_s": 0.002,+ "decoded_absolute_peak_exclusive": 1.0,+ "gain_corrected_pcm_nrmse": 0.15,+ "mapped_event_edge_s": 1.0,+ "mapped_clock_s": 1.0,+ "median_ridge_frequency_hz": 2.0,+ "normalized_score_per_matched_group": 0.05,+ "displacement_thresholds_m": [100.0, 500.0, 1000.0],+ "accepted_jump_m": 1000.0,+ "distant_mode_m": 1000.0,+ "event_group_counts": "equal",+ "associated_flights": "equal after one-to-one event mapping",+ "score_units": "equal"+ },+ "pcm_comparison": {+ "sample_rate_hz": 4000,+ "block_samples": 65536,+ "lag_windows": 12,+ "lag_window_s": 8.0,+ "lag_search_radius_s": 0.1,+ "lag_rule": "Evenly spaced fixed windows; report each measured lag; never shift comparisons by fitted lag.",+ "gain_rule": "Remove only the declared gain for comparison; no fitted normalization.",+ "reference": "Decoded canonical mono PCM before MP3 encoding"+ },+ "event_correspondence": {+ "minimum_overlap_fraction_of_shorter_event": 0.5,+ "maximum_matching_median_frequency_hz": 25.0,+ "assignment": "Minimum-cost one-to-one assignment, maximizing eligible pairs first; cost is one minus overlap fraction plus median frequency difference divided by 25 Hz.",+ "ridge_comparison": "Interpolate the second mapped ridge at first-ridge times within their overlap; do not extrapolate.",+ "groups": "Report unmatched ridges and groups, group splits and group merges; IDs and row order are not correspondence."+ },+ "time_mapping": {+ "padding_request": "Subtract leading_silence_s from BOTH sample-zero UTC window endpoints.",+ "event_mapping": "Subtract leading_silence_s from audio-relative event and ridge times.",+ "clock_mapping": "Add leading_silence_s to fitted sample-zero UTC to recover the original recording clock.",+ "codec_delay": "Use first decoded sample after ffmpeg skip/discard; retain first-packet side data, decoded sample counts and residual lag checks."+ },+ "execution": {+ "order": "Protocol order, sequentially; baseline first.",+ "fresh_work": "Each arm starts a fresh local HTTP service and job with original upload bytes; no saved intake, discovery, search or decision is reused.",+ "shared_data": "Only pinned immutable historical indexes and terrain may be shared; require an offline operator.",+ "failure_policy": "Retain encoding, usability, verification, terminal and incomplete outcomes. Continue other cases after ordinary failures; stop and preserve pending cases on interruption or changed pinned inputs.",+ "truth_policy": "No eligible independent truth is supplied in this protocol. Truth error is explicitly not evaluated. Pairwise displacement is not accuracy."+ },+ "chunk_controls": {+ "supported": ["operator.discovery.chunk_s", "operator.discovery.overlap_s"],+ "unsupported": ["HTTP upload chunk size", "decoder block size", "detector spectral window/hop via the upload operator", "search scheduling chunks as an equivalent audio representation"]+ },+ "outcome_policy": "Compare all status/reason, coverage, work-completion and uncertainty-provenance transitions. Compare diagnostic candidates separately from accepted locations. Accepted jumps above 1000 m are serious findings. Regional source-informed runs do not satisfy national acceptance."+}diff --git a/scripts/run_real_metamorphic.py b/scripts/run_real_metamorphic.pynew file mode 100644index 0000000..598d0f8--- /dev/null+++ b/scripts/run_real_metamorphic.py@@ -0,0 +1,831 @@+#!/usr/bin/env python3+"""Run fresh HTTP jobs for a predeclared real-recording representation matrix.++Prepare freezes configuration, inputs and implementation before any arm runs.+Run preserves failed, inconclusive and unusable arms. Report only compares them.+"""+from __future__ import annotations++import argparse+from copy import deepcopy+from datetime import datetime, timedelta, timezone+from importlib import metadata+from importlib.machinery import EXTENSION_SUFFIXES+import inspect+import itertools+import json+import math+import os+from pathlib import Path+import shlex+import signal+import subprocess+import sys+import tempfile+import threading++import numpy as np+from pyproj import Geod+from scipy import signal as scipy_signal+from scipy.optimize import linear_sum_assignment++import dopplergeist+from dopplergeist.contracts import validate_request+from dopplergeist.discovery import DiscoverySettings+from dopplergeist.intake import IntakeRecording, intake_recording, probe_media+import verify_upload_http as http_verifier+++ROOT = Path(__file__).resolve().parents[1]+PROTOCOL = ROOT / "docs/verification/metamorphic_protocol_v1.json"+GEOD = Geod(ellps="WGS84")+++def read_json(path, default=None):+ path = Path(path)+ return json.loads(path.read_text()) if path.is_file() else default+++def write_json(path, value):+ """Atomic progress reports; immutable plans use exclusive creation."""+ path = Path(path)+ temporary = path.with_suffix(path.suffix + ".tmp")+ temporary.write_text(json.dumps(value, indent=2, allow_nan=False) + "\n")+ temporary.replace(path)+++def identity(path):+ path = Path(path).absolute()+ resolved = path.resolve(strict=True)+ return {"path": str(path), "resolved_path": str(resolved), "bytes": path.stat().st_size,+ "sha256": http_verifier.sha256(path)}+++def parse_utc(value):+ parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))+ if parsed.tzinfo is None:+ raise ValueError("UTC endpoints must include a timezone")+ return parsed.astimezone(timezone.utc)+++def format_utc(value):+ return value.isoformat().replace("+00:00", "Z")+++def shifted_request(request, silence_s):+ result = deepcopy(request)+ window = result.get("time_window")+ if not isinstance(window, dict):+ raise ValueError("The matrix needs a defensible sample-zero UTC window")+ for key in ("start_utc", "end_utc"):+ window[key] = format_utc(parse_utc(window[key]) - timedelta(seconds=silence_s))+ return result+++def implementation():+ from dopplergeist import data_access, matching, native_acoustic, native_cpa++ package = Path(dopplergeist.__file__).resolve().parent+ result = {+ "package_files": {p.relative_to(package).as_posix(): http_verifier.sha256(p)+ for p in sorted(package.rglob("*")) if p.is_file() and (+ p.suffix in {".py", ".c"} or any(p.name.endswith(suffix) for suffix in EXTENSION_SUFFIXES))},+ "runner_sha256": http_verifier.sha256(__file__),+ "verifier_sha256": http_verifier.sha256(http_verifier.__file__),+ "python": sys.version, "executable": sys.executable,+ "packages": {name: metadata.version(name) for name in+ ("numpy", "scipy", "pandas", "pyproj", "rasterio", "pyarrow", "matplotlib")},+ "backends": {"ridge": matching.RIDGE_FIT_BACKEND, "acoustic": native_acoustic.ACOUSTIC_BACKEND,+ "cpa": native_cpa.CPA_BACKEND,+ "index_payload": "native" if data_access._native_payload_columns is not None else "python"},+ "environment": {name: value for name, value in sorted(os.environ.items())+ if name in {"DOPPLERGEIST_CPA_BACKEND", "DOPPLERGEIST_ACOUSTIC_BACKEND",+ "DOPPLERGEIST_RIDGE_BACKEND",+ "OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS",+ "NUMEXPR_NUM_THREADS", "VECLIB_MAXIMUM_THREADS", "PYTHONHASHSEED"}},+ }+ for name in ("ffmpeg", "ffprobe"):+ result[name] = subprocess.run([name, "-version"], check=True,+ capture_output=True, text=True, timeout=10).stdout.splitlines()[0]+ return result+++def operator_inputs(operator, additional):+ """Pin local data named by the operator, without following active job state."""+ if operator.get("offline", True) is not True:+ raise ValueError("Use an offline operator with frozen historical data and terrain")+ paths = set()+ for value in operator.get("flight_indexes", []):+ paths.add(Path(value).resolve(strict=True))+ ground = operator.get("terrain", {})+ for value in ground.get("rasters", []):+ paths.add(Path(value).resolve(strict=True))+ if ground.get("geoid_raster"):+ paths.add(Path(ground["geoid_raster"]).resolve(strict=True))+ if operator.get("cache_root"):+ cache = Path(operator["cache_root"]).resolve(strict=True)+ paths.update(p.absolute() for p in cache.rglob("*") if p.is_file())+ paths.update(Path(p).resolve(strict=True) for p in additional)+ if not paths:+ raise ValueError("No immutable historical/terrain inputs were supplied")+ return [identity(path) for path in sorted(paths)]+++def prepare(audio, request, operator, output, *, source_note, scope_note,+ additional_inputs=(), job_max_runtime_s=None, http_timeout_s=None):+ audio = Path(audio).resolve(strict=True)+ operator = deepcopy(operator)+ for name in ("flight_indexes",):+ if name in operator:+ operator[name] = [str(Path(p).resolve(strict=True)) for p in operator[name]]+ ground = operator.get("terrain", {})+ if "rasters" in ground:+ ground["rasters"] = [str(Path(p).resolve(strict=True)) for p in ground["rasters"]]+ if ground.get("geoid_raster"):+ ground["geoid_raster"] = str(Path(ground["geoid_raster"]).resolve(strict=True))+ if operator.get("cache_root"):+ operator["cache_root"] = str(Path(operator["cache_root"]).resolve(strict=True))+ if set(operator.get("discovery", {})) - {"chunk_s", "overlap_s"}:+ raise ValueError("The upload operator exposes only discovery chunk_s and overlap_s")+ protocol = read_json(PROTOCOL)+ for key, override in (("job_max_runtime_s", job_max_runtime_s),+ ("http_timeout_s", http_timeout_s)):+ if override is not None:+ if not math.isfinite(override) or override <= 0:+ raise ValueError(f"{key} must be finite and positive")+ protocol["budgets"][key] = float(override)+ if protocol["budgets"]["job_max_runtime_s"] >= protocol["budgets"]["http_timeout_s"]:+ raise ValueError("The client deadline must exceed the job runtime limit")+ if not source_note.strip() or not scope_note.strip():+ raise ValueError("Declare the real source and the evaluation scope")+ request = dict(validate_request(dict(request, audio_path=str(audio))))+ request.pop("audio_path")+ shifted_request(request, 0)+ DiscoverySettings(**operator.get("discovery", {}))+ inputs = operator_inputs(operator, additional_inputs)+ cases = []+ for case in protocol["cases"]:+ settings = dict(protocol["baseline"], **case["changes"])+ arm_operator = deepcopy(operator)+ if "discovery" in settings:+ arm_operator["discovery"] = settings.pop("discovery")+ DiscoverySettings(**arm_operator.get("discovery", {}))+ cases.append({"id": case["id"], "encoding": settings,+ "request": shifted_request(request, settings["leading_silence_s"]),+ "operator": arm_operator})+ plan = {"schema_version": 1, "prepared_utc": http_verifier.now(),+ "protocol": protocol, "protocol_sha256": http_verifier.sha256(PROTOCOL),+ "source": identity(audio), "source_note": source_note, "scope_note": scope_note,+ "request": request, "operator": operator, "inputs": inputs, "cases": cases,+ "additional_inputs": [str(Path(p).resolve(strict=True)) for p in additional_inputs],+ "implementation": implementation(),+ "source_commit": subprocess.run(["git", "rev-parse", "HEAD"], cwd=ROOT,+ check=True, capture_output=True, text=True).stdout.strip(),+ "truth_evaluation": {"eligible": False, "evaluated": False,+ "reason": "No independent truth supplied to this protocol"},+ "national_gate_satisfied": False}+ output = Path(output).resolve()+ output.mkdir(parents=True, exist_ok=False)+ with (output / "plan.json").open("x") as stream:+ json.dump(plan, stream, indent=2, allow_nan=False)+ stream.write("\n")+ (output / "plan.sha256").write_text(http_verifier.sha256(output / "plan.json") + "\n")+ return output / "plan.json"+++def check_frozen(plan_path, plan):+ expected = plan_path.with_name("plan.sha256").read_text().strip()+ if http_verifier.sha256(plan_path) != expected:+ raise ValueError("The frozen plan changed; prepare a new experiment")+ if implementation() != plan["implementation"]:+ raise ValueError("Implementation, runtime or backend configuration changed after preparation")+ if identity(plan["source"]["path"]) != plan["source"]:+ raise ValueError("The recorded source changed")+ if operator_inputs(plan["operator"], plan["additional_inputs"]) != plan["inputs"]:+ raise ValueError("Pinned data bytes, paths or cache file set changed")+++def run_command(command, directory, name, timeout_s, *, env=None):+ write_json(directory / f"{name}-command.json", command)+ with (directory / f"{name}.log").open("wb") as log:+ process = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT,+ env=env, start_new_session=True)+ try:+ returncode = process.wait(timeout=timeout_s)+ if returncode:+ raise subprocess.CalledProcessError(returncode, command)+ except BaseException:+ # Intake starts FFmpeg. Stop this command's whole process group.+ try:+ os.killpg(process.pid, signal.SIGTERM)+ except ProcessLookupError:+ pass+ try:+ process.wait(timeout=2)+ except subprocess.TimeoutExpired:+ os.killpg(process.pid, signal.SIGKILL)+ process.wait()+ raise+++def preflight_decode(audio, directory, protocol):+ """Use actual intake in a bounded child; HTTP intake is still fresh."""+ command = [sys.executable, "-c",+ "import sys; from dopplergeist.intake import intake_recording; "+ "intake_recording(sys.argv[1], sys.argv[2], sample_rate_hz=int(sys.argv[3]))",+ str(audio), str(directory), str(protocol["pcm_comparison"]["sample_rate_hz"])]+ env = dict(os.environ, PYTHONPATH=str(Path(dopplergeist.__file__).resolve().parent.parent)+ + os.pathsep + os.environ.get("PYTHONPATH", ""))+ run_command(command, directory.parent, directory.name + "-decode",+ protocol["budgets"]["encoding_timeout_s"], env=env)+ return IntakeRecording(directory / "recording.f32", read_json(directory / "recording.json"),+ read_json(directory / "intake.json"))+++def make_canonical(source, directory, protocol):+ probe = probe_media(source)+ audio = next(s for s in probe["streams"] if s["codec_type"] == "audio")+ channels = int(audio["channels"])+ downmix = "pan=mono|c0=" + "+".join(f"{1/channels:.17g}*c{i}" for i in range(channels))+ path = directory / "canonical.wav"+ command = ["ffmpeg", "-nostdin", "-v", "error", "-xerror", "-i", str(source),+ "-map", "0:a:0", "-vn", "-sn", "-dn", "-af", downmix, "-ac", "1",+ "-ar", str(protocol["canonical_pcm"]["sample_rate_hz"]),+ "-c:a", "pcm_f32le", "-map_metadata", "-1", "-rf64", "auto", str(path)]+ run_command(command, directory, "canonical", protocol["budgets"]["encoding_timeout_s"])+ reference = preflight_decode(path, directory / "reference-pcm", protocol)+ return path, reference+++def encode(canonical, directory, settings, protocol):+ filters = []+ if settings["gain_db"]:+ filters.append(f"volume={10**(settings['gain_db']/20):.17g}:precision=double")+ if settings["leading_silence_s"]:+ samples = round(settings["leading_silence_s"] * protocol["canonical_pcm"]["sample_rate_hz"])+ filters.append(f"adelay={samples}S:all=1")+ if settings["channels"] == 2:+ filters.append("pan=stereo|c0=c0|c1=c0")+ path = directory / "input.mp3"+ command = ["ffmpeg", "-nostdin", "-v", "error", "-xerror", "-i", str(canonical),+ "-map", "0:a:0", "-vn", "-sn", "-dn"]+ if filters:+ command += ["-af", ",".join(filters)]+ command += ["-ac", str(settings["channels"]), "-ar", str(settings["sample_rate_hz"]),+ "-c:a", "libmp3lame", "-b:a", f"{settings['bitrate_kbps']}k",+ "-write_xing", "1", "-map_metadata", "-1", str(path)]+ run_command(command, directory, "encode", protocol["budgets"]["encoding_timeout_s"])+ return path+++def native_decode_metrics(audio, directory, protocol):+ """Scan blocks of a temporary native-rate decode; keep no second large PCM."""+ probe = probe_media(audio)+ stream = next(s for s in probe["streams"] if s["codec_type"] == "audio")+ channels = int(stream["channels"])+ command = ["ffmpeg", "-nostdin", "-v", "error", "-xerror", "-i", str(audio),+ "-map", "0:a:0", "-f", "f32le", "-c:a", "pcm_f32le", "pipe:1"]+ peak, frames, channel_difference = 0.0, 0, 0.0+ with tempfile.TemporaryFile(dir=directory) as raw, (directory / "native-decode.log").open("wb") as log:+ subprocess.run(command, stdout=raw, stderr=log, check=True,+ timeout=protocol["budgets"]["encoding_timeout_s"])+ if raw.tell() % (4 * channels):+ raise ValueError("Native decode has an incomplete audio frame")+ raw.seek(0)+ while len(block := np.fromfile(raw, dtype="<f4", count=65536 * channels)):+ if not np.isfinite(block).all():+ raise ValueError("Native decode contains nonfinite samples")+ values = block.reshape(-1, channels)+ peak = max(peak, float(np.max(np.abs(values))))+ if channels == 2:+ channel_difference = max(channel_difference, float(np.max(np.abs(values[:, 0] - values[:, 1]))))+ frames += len(values)+ packets = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "a:0",+ "-read_intervals", "%+#1", "-show_packets", "-of", "json", str(audio)],+ check=True, capture_output=True, text=True, timeout=10)+ return {"sample_rate_hz": int(stream["sample_rate"]), "channels": channels,+ "sample_count": frames, "duration_s": frames / int(stream["sample_rate"]),+ "absolute_peak": peak, "identical_channel_max_difference": channel_difference if channels == 2 else None,+ "first_packet": json.loads(packets.stdout).get("packets", [])[:1],+ "container_start_time": probe["format"].get("start_time"),+ "container_duration": probe["format"].get("duration"), "decode_command": command}+++def compare_pcm(reference_path, variant_path, settings, protocol):+ config, tol = protocol["pcm_comparison"], protocol["tolerances"]+ sr = config["sample_rate_hz"]+ left = np.memmap(reference_path, dtype="<f4", mode="r")+ right = np.memmap(variant_path, dtype="<f4", mode="r")+ padding = round(settings["leading_silence_s"] * sr)+ gain = 10 ** (settings["gain_db"] / 20)+ count = min(len(left), max(0, len(right) - padding))+ if count == 0:+ raise ValueError("No recorded samples remain after the declared padding")+ squared_error, energy = 0.0, 0.0+ for first in range(0, count, config["block_samples"]):+ last = min(count, first + config["block_samples"])+ a = np.asarray(left[first:last], dtype=np.float64)+ b = np.asarray(right[first+padding:last+padding], dtype=np.float64) / gain+ squared_error += float(np.dot(a-b, a-b))+ energy += float(np.dot(a, a))+ duration_error = (len(right) - padding - len(left)) / sr+ nrmse = math.sqrt(squared_error / energy) if energy > 0 else None+ radius = round(config["lag_search_radius_s"] * sr)+ width = min(round(config["lag_window_s"] * sr), count - 2*radius)+ lags = []+ if width > 0:+ for first in np.linspace(radius, count-radius-width, config["lag_windows"]).astype(int):+ a = np.asarray(left[first:first+width], dtype=np.float64)+ a = a - a.mean()+ if float(np.dot(a, a)) <= 1e-14:+ lags.append({"start_s": int(first)/sr, "lag_s": None, "reason": "no_alignment_energy"})+ continue+ b = np.asarray(right[first+padding-radius:first+padding+width+radius], dtype=np.float64) / gain+ values = scipy_signal.correlate(b, a, mode="valid", method="fft")+ sums, squares = np.r_[0.0, np.cumsum(b)], np.r_[0.0, np.cumsum(b*b)]+ variance = squares[width:] - squares[:-width] - (sums[width:] - sums[:-width])**2/width+ normalized = values / np.sqrt(np.maximum(variance, np.finfo(float).tiny))+ lag = (int(np.argmax(normalized)) - radius) / sr+ lags.append({"start_s": int(first)/sr, "lag_s": lag})+ measured = [abs(row["lag_s"]) for row in lags if row["lag_s"] is not None]+ max_lag = max(measured) if measured else None+ return {"reference_samples": len(left), "variant_samples": len(right),+ "compared_samples": count, "padding_samples": padding, "declared_gain": gain,+ "duration_error_s": duration_error, "gain_corrected_nrmse": nrmse,+ "fixed_window_lags": lags, "maximum_absolute_lag_s": max_lag,+ "passed": (abs(duration_error) <= tol["decoded_duration_error_s"]+ and nrmse is not None and nrmse <= tol["gain_corrected_pcm_nrmse"]+ and max_lag is not None and max_lag <= tol["residual_alignment_lag_s"])}+++class MemorySampler:+ """Sample only this runner's descendants, never another worker's jobs."""+ def __init__(self):+ self.stop = threading.Event()+ self.peak = 0+ self.samples = 0+ self.thread = threading.Thread(target=self.sample, daemon=True)++ def sample(self):+ while not self.stop.is_set():+ pending, visited, total = [os.getpid()], set(), 0+ while pending:+ pid = pending.pop()+ if pid in visited:+ continue+ visited.add(pid)+ try:+ # The HTTP service can spawn a worker from a non-main thread.+ for children in Path(f"/proc/{pid}/task").glob("*/children"):+ pending += [int(x) for x in children.read_text().split()]+ if pid != os.getpid():+ for line in Path(f"/proc/{pid}/status").read_text().splitlines():+ if line.startswith("VmRSS:"):+ total += int(line.split()[1]) * 1024+ except (OSError, ValueError):+ pass+ self.peak = max(self.peak, total)+ self.samples += 1+ self.stop.wait(0.1)++ def __enter__(self):+ self.thread.start()+ return self++ def __exit__(self, *_):+ self.stop.set()+ self.thread.join()++ def result(self):+ return {"sampled_descendant_peak_rss_mib": self.peak/(1024**2) if self.peak else None,+ "interval_s": 0.1, "samples": self.samples,+ "scope": "Sum of local service and worker descendant RSS; sampled lower bound, not unique physical memory"}+++def mapped_events(events, padding):+ result = deepcopy(events)+ for event in result:+ event["start_s"] -= padding+ event["end_s"] -= padding+ for sample in event["ridge"]:+ sample["time_s"] -= padding+ return result+++def observe_case(directory, case, receipt):+ job_id = receipt.get("job_id")+ job = directory / "http" / "jobs" / job_id if job_id else None+ final = read_json(directory / "http/result.json", {})+ if not final and job is not None:+ final = read_json(job / "result.json", {})+ discovery = read_json(job / "discovery/discovery.json", {}) if job else {}+ events = final.get("events", discovery.get("events", []))+ search = read_json(job / "search/search-report.json", {}) if job else {}+ refined = read_json(job / "refined-candidates.json", {}) if job else {}+ decision = read_json(job / "decision.json", {}) if job else {}+ evidence = read_json(directory / "http/pipeline-evidence.json", {})+ padding = case["encoding"]["leading_silence_s"]+ candidates = []+ groups = {event["event_id"]: event.get("harmonic_group_id", event["event_id"]) for event in events}+ for i, candidate in enumerate(final.get("candidates", [])):+ row = refined.get("ranked", [])[i] if i < len(refined.get("ranked", [])) else {}+ copied = deepcopy(candidate)+ copied["mapped_recording_start_utc"] = format_utc(+ parse_utc(candidate["recording_start_utc"]) + timedelta(seconds=padding))+ copied["matched_group_count"] = len({groups[e] for e in candidate["event_ids"] if e in groups})+ copied["normalized_score"] = candidate["score"] / max(1, copied["matched_group_count"])+ sensitivity = row.get("sensitivity", {})+ copied["uncertainty_provenance"] = {key: sensitivity.get(key) for key in+ ("kind", "method", "sources", "complete", "boundary_limited")}+ copied["uncertainty_radius_m"] = sensitivity.get("radius_m")+ copied["mode_id"] = row.get("mode_id")+ candidates.append(copied)+ keys = ("status", "tiles_total", "tiles_completed", "tiles_resumed", "unsearched_tiles",+ "grid_points_searched", "refinement_points_searched", "refinement_points_skipped",+ "refinement_stages_resumed", "model_assessment_complete", "model_support_complete",+ "data_scope_complete", "all_event_groups_considered", "event_groups_proposed",+ "alternative_modes_exhaustively_assessed", "elapsed_s", "process_peak_rss_mib",+ "bounded_concurrency")+ return {"job_id": job_id, "job_directory": str(job) if job else None,+ "status": final.get("status"), "reason_code": final.get("reason_code"),+ "stage": final.get("stage"), "accepted_location": final.get("status") == "succeeded",+ "verification_passed": receipt.get("verification_passed", False),+ "full_pipeline_observed": receipt.get("full_pipeline_observed", False),+ "full_pipeline_requirements_met": receipt.get("full_pipeline_requirements_met", False),+ "stages_verified": evidence.get("stages_verified", []),+ "http_elapsed_s": receipt.get("elapsed_s"), "interrupted": receipt.get("interrupted", False),+ "verification_errors": {key: receipt[key] for key in+ ("error", "finalization_error", "requirement_error", "pipeline_evidence_errors") if key in receipt},+ "mapped_events": mapped_events(events, padding),+ "ridge_count": len(events), "group_count": len(set(groups.values())),+ "candidates": candidates, "coverage": final.get("coverage"),+ "work": {key: search.get(key) for key in keys},+ "finite_search_evidence": search.get("finite_search_evidence"),+ "decision_provenance": decision.get("diagnostics"),+ "acquisition": read_json(job / "data-provenance.json", {}) if job else {}}+++def assign_pairs(left_count, right_count, eligible):+ """Maximum-cardinality, then minimum-cost one-to-one correspondence."""+ if not left_count or not right_count:+ return []+ penalty = 10 * (left_count + right_count + 1)+ costs = np.full((left_count, right_count+left_count), float(penalty))+ costs[:, :right_count] = 3 * penalty+ for (i, j), row in eligible.items():+ costs[i, j] = row["cost"]+ rows, columns = linear_sum_assignment(costs)+ return [(int(i), int(j)) for i, j in zip(rows, columns) if (i, j) in eligible]+++def event_correspondence(left, right, protocol):+ rules = protocol["event_correspondence"]+ eligible = {}+ for i, a in enumerate(left):+ for j, b in enumerate(right):+ first, last = max(a["start_s"], b["start_s"]), min(a["end_s"], b["end_s"])+ overlap = max(0.0, last-first) / min(a["end_s"]-a["start_s"], b["end_s"]-b["start_s"])+ if overlap < rules["minimum_overlap_fraction_of_shorter_event"]:+ continue+ ta, fa = np.array([(r["time_s"], r["frequency_hz"]) for r in a["ridge"]]).T+ tb, fb = np.array([(r["time_s"], r["frequency_hz"]) for r in b["ridge"]]).T+ mask = (ta >= tb[0]) & (ta <= tb[-1])+ if not mask.any():+ continue+ frequency = float(np.median(np.abs(fa[mask] - np.interp(ta[mask], tb, fb))))+ if frequency > rules["maximum_matching_median_frequency_hz"]:+ continue+ eligible[i, j] = {"left_event_id": a["event_id"], "right_event_id": b["event_id"],+ "left_group_id": a.get("harmonic_group_id", a["event_id"]),+ "right_group_id": b.get("harmonic_group_id", b["event_id"]),+ "start_delta_s": b["start_s"]-a["start_s"],+ "end_delta_s": b["end_s"]-a["end_s"],+ "overlap_fraction": overlap, "median_frequency_delta_hz": frequency,+ "compared_ridge_samples": int(mask.sum()),+ "cost": 1-overlap + frequency/rules["maximum_matching_median_frequency_hz"]}+ matches = [eligible[pair] for pair in assign_pairs(len(left), len(right), eligible)]+ left_ids = {row["left_event_id"] for row in matches}+ right_ids = {row["right_event_id"] for row in matches}+ forward, reverse, possible_forward, possible_reverse = {}, {}, {}, {}+ for rows, fwd, rev in ((matches, forward, reverse),+ (eligible.values(), possible_forward, possible_reverse)):+ for row in rows:+ fwd.setdefault(row["left_group_id"], set()).add(row["right_group_id"])+ rev.setdefault(row["right_group_id"], set()).add(row["left_group_id"])+ return {"matches": matches,+ "unmatched_left": [e["event_id"] for e in left if e["event_id"] not in left_ids],+ "unmatched_right": [e["event_id"] for e in right if e["event_id"] not in right_ids],+ "unmatched_left_groups": sorted({e.get("harmonic_group_id", e["event_id"]) for e in left} - forward.keys()),+ "unmatched_right_groups": sorted({e.get("harmonic_group_id", e["event_id"]) for e in right} - reverse.keys()),+ "split_groups": {key: sorted(value) for key, value in forward.items() if len(value) > 1},+ "merged_groups": {key: sorted(value) for key, value in reverse.items() if len(value) > 1},+ "possible_split_or_ambiguous_groups": {key: sorted(value) for key, value in possible_forward.items() if len(value) > 1},+ "possible_merge_or_ambiguous_groups": {key: sorted(value) for key, value in possible_reverse.items() if len(value) > 1}}+++def distance(a, b):+ return float(GEOD.inv(a["lon"], a["lat"], b["lon"], b["lat"])[2])+++def associations_equal(a, b, mapping):+ first = list(zip(a["event_ids"], a["flight_ids"]))+ if any(event not in mapping for event, _ in first):+ return False+ return sorted((mapping[event], flight) for event, flight in first) == sorted(zip(b["event_ids"], b["flight_ids"]))+++def compare_cases(left, right, protocol):+ a, b = left["observation"], right["observation"]+ tol = protocol["tolerances"]+ correspondence = event_correspondence(a["mapped_events"], b["mapped_events"], protocol)+ mapping = {m["left_event_id"]: m["right_event_id"] for m in correspondence["matches"]}+ result = {"left": left["id"], "right": right["id"], "event_correspondence": correspondence,+ "representation_usable": bool(left.get("usability", {}).get("passed") and right.get("usability", {}).get("passed")),+ "complete_pipelines": bool(a["full_pipeline_requirements_met"] and b["full_pipeline_requirements_met"]),+ "status_transition": [a["status"], b["status"]],+ "reason_transition": [a["reason_code"], b["reason_code"]],+ "group_count_delta": b["group_count"] - a["group_count"],+ "ridge_count_delta": b["ridge_count"] - a["ridge_count"],+ "coverage_equal": a["coverage"] == b["coverage"],+ "coverage_transition": [a["coverage"], b["coverage"]],+ "work_transition": [a["work"], b["work"]],+ "decision_provenance_equal": a["decision_provenance"] == b["decision_provenance"],+ "diagnostic_candidate_comparison": None, "accepted_location_displacement_m": None,+ "serious_accepted_jump": False, "violations": []}+ checks = {+ "status_changed": a["status"] != b["status"],+ "reason_changed": a["reason_code"] != b["reason_code"],+ "group_count_changed": a["group_count"] != b["group_count"],+ "ridge_correspondence_incomplete": bool(correspondence["unmatched_left"] or correspondence["unmatched_right"]),+ "group_split_or_merge": bool(correspondence["split_groups"] or correspondence["merged_groups"]),+ "mapped_event_edges": any(max(abs(m["start_delta_s"]), abs(m["end_delta_s"])) > tol["mapped_event_edge_s"]+ for m in correspondence["matches"]),+ "mapped_ridge_frequency": any(m["median_frequency_delta_hz"] > tol["median_ridge_frequency_hz"]+ for m in correspondence["matches"]),+ }+ if a["candidates"] and b["candidates"]:+ best_a, best_b = a["candidates"][0], b["candidates"][0]+ displacement = distance(best_a, best_b)+ clock_delta = abs((parse_utc(best_b["mapped_recording_start_utc"])+ - parse_utc(best_a["mapped_recording_start_utc"])).total_seconds())+ same_associations = associations_equal(best_a, best_b, mapping)+ same_units = best_a["score_kind"] == best_b["score_kind"]+ comparable = same_associations and same_units and best_a["matched_group_count"] == best_b["matched_group_count"]+ score_delta = abs(best_a["normalized_score"] - best_b["normalized_score"]) if comparable else None+ result["diagnostic_candidate_comparison"] = {+ "displacement_m": displacement, "within_threshold_m": {+ str(int(limit)): displacement <= limit for limit in tol["displacement_thresholds_m"]},+ "mapped_clock_delta_s": clock_delta, "associated_flights_equal": same_associations,+ "score_units_equal": same_units, "normalized_score_comparable": comparable,+ "normalized_score_delta": score_delta,+ "uncertainty_provenance_equal": best_a["uncertainty_provenance"] == best_b["uncertainty_provenance"],+ "uncertainty_radius_m": [best_a["uncertainty_radius_m"], best_b["uncertainty_radius_m"]]}+ checks.update(associated_flights_changed=not same_associations, score_units_changed=not same_units,+ normalized_score_changed=score_delta is not None and score_delta > tol["normalized_score_per_matched_group"],+ fitted_clock_changed=clock_delta > tol["mapped_clock_s"],+ diagnostic_candidate_jump=displacement > tol["accepted_jump_m"],+ uncertainty_provenance_changed=best_a["uncertainty_provenance"] != best_b["uncertainty_provenance"])+ if a["accepted_location"] and b["accepted_location"]:+ result["accepted_location_displacement_m"] = displacement+ result["serious_accepted_jump"] = displacement > tol["accepted_jump_m"]+ eligible = {}+ for i, first in enumerate(a["candidates"]):+ for j, second in enumerate(b["candidates"]):+ if associations_equal(first, second, mapping):+ separation = distance(first, second)+ clock = abs((parse_utc(first["mapped_recording_start_utc"])+ - parse_utc(second["mapped_recording_start_utc"])).total_seconds())+ same_units = first["score_kind"] == second["score_kind"]+ if separation <= tol["distant_mode_m"] and clock <= tol["mapped_clock_s"] and same_units:+ eligible[i, j] = {"cost": separation/tol["distant_mode_m"] + clock/tol["mapped_clock_s"],+ "distance_m": separation, "mapped_clock_delta_s": clock,+ "score_kind": first["score_kind"],+ "normalized_score_delta": abs(first["normalized_score"]-second["normalized_score"])+ if first["matched_group_count"] == second["matched_group_count"] else None,+ "uncertainty_provenance_equal": first["uncertainty_provenance"] == second["uncertainty_provenance"]}+ pairs = assign_pairs(len(a["candidates"]), len(b["candidates"]), eligible)+ result["retained_mode_correspondence"] = [+ dict(left_index=i, right_index=j, **{key: value for key, value in eligible[i, j].items() if key != "cost"})+ for i, j in pairs]+ result["unmatched_retained_left_indices"] = sorted(set(range(len(a["candidates"]))) - {i for i, _ in pairs})+ result["unmatched_retained_right_indices"] = sorted(set(range(len(b["candidates"]))) - {j for _, j in pairs})+ checks["retained_mode_score_changed"] = any(row["normalized_score_delta"] is not None+ and row["normalized_score_delta"] > tol["normalized_score_per_matched_group"]+ for row in result["retained_mode_correspondence"])+ checks["retained_mode_uncertainty_provenance_changed"] = any(+ not row["uncertainty_provenance_equal"] for row in result["retained_mode_correspondence"])+ distant = []+ for label, source, matched in (("left", a, {i for i, _ in pairs}),+ ("right", b, {j for _, j in pairs})):+ for index, candidate in enumerate(source["candidates"]):+ offset = distance(source["candidates"][0], candidate)+ if offset >= tol["distant_mode_m"]:+ distant.append({"side": label, "candidate_index": index,+ "distance_from_winner_m": offset, "retained_within_1000m": index in matched})+ result["distant_modes"] = distant+ checks["distant_mode_lost"] = any(not row["retained_within_1000m"] for row in distant)+ elif bool(a["candidates"]) != bool(b["candidates"]):+ checks["candidate_presence_changed"] = True+ result["violations"] = [name for name, failed in checks.items() if failed]+ return result+++def build_report(plan_path, plan, state):+ cases = state["cases"]+ observed = [row for row in cases if row.get("observation")]+ pairs = []+ for a, b in itertools.combinations(observed, 2):+ try:+ pairs.append(compare_cases(a, b, plan["protocol"]))+ except Exception as exc:+ pairs.append({"left": a["id"], "right": b["id"],+ "error": f"{type(exc).__name__}: {exc}",+ "violations": ["comparison_error"], "serious_accepted_jump": False})+ completed = [row for row in cases if row["state"] == "finished"]+ passed = [row for row in completed if row.get("observation", {}).get("verification_passed")]+ stages = [row for row in passed if row["observation"]["full_pipeline_observed"]]+ full = [row for row in passed if row["observation"]["full_pipeline_requirements_met"]]+ violations = [pair for pair in pairs if pair["violations"]]+ return {"schema_version": 1, "plan": str(plan_path), "plan_sha256": http_verifier.sha256(plan_path),+ "updated_utc": http_verifier.now(), "source_note": plan["source_note"], "scope_note": plan["scope_note"],+ "declared_case_count": len(plan["cases"]), "finished_cases": len(completed),+ "verified_cases": len(passed), "complete_stage_cases": len(stages), "full_pipeline_cases": len(full),+ "run_state": state["state"], "error": state.get("error"), "cases": cases, "pairwise": pairs,+ "pairs_with_observed_differences": len(violations),+ "serious_accepted_jumps": [p for p in pairs if p["serious_accepted_jump"]],+ "all_complete_usable_and_within_tolerances": (state["state"] == "finished"+ and len(full) == len(plan["cases"])+ and all(row.get("usability", {}).get("passed") for row in cases) and not violations),+ "truth_evaluation": plan["truth_evaluation"], "national_gate_satisfied": False,+ "interpretation": "Inconclusive decisions remain inconclusive. Pairwise agreement is not location accuracy. Early exits do not complete the pipeline requirement."}+++def save_report(plan_path, plan, state, output):+ report = build_report(plan_path, plan, state)+ write_json(output / "report.json", report)+ lines = ["# Real recording representation matrix", "", plan["scope_note"], "",+ f"Full pipeline observed in {report['full_pipeline_cases']}/{report['declared_case_count']} declared cases.",+ "No independent truth error was evaluated. National acceptance remains open.", "",+ "| Case | State | Status / reason | Groups / ridges | Usable | Full pipeline | HTTP seconds |",+ "| --- | --- | --- | --- | --- | --- | --- |"]+ for row in state["cases"]:+ obs = row.get("observation", {})+ seconds = obs.get("http_elapsed_s")+ lines.append(f"| {row['id']} | {row['state']} | {obs.get('status')} / {obs.get('reason_code')} | "+ f"{obs.get('group_count')} / {obs.get('ridge_count')} | {row.get('usability', {}).get('passed')} | "+ f"{obs.get('full_pipeline_requirements_met')} | {round(seconds, 2) if seconds is not None else 'unavailable'} |")+ lines += ["", f"Pairs with observed differences: {report['pairs_with_observed_differences']}.",+ f"Serious accepted location jumps: {len(report['serious_accepted_jumps'])}.", "",+ "The JSON report retains mapped ridges, correspondence, diagnostic candidates,",+ "coverage/work transitions, uncertainty provenance, and per-arm reproducers."]+ if state.get("error"):+ lines += ["", "Run error: " + state["error"]]+ (output / "report.md").write_text("\n".join(lines) + "\n")+ return report+++def run(plan_path, output, *, selected_case=None):+ plan_path, output = Path(plan_path).resolve(strict=True), Path(output).resolve()+ plan = read_json(plan_path)+ check_frozen(plan_path, plan)+ if "job_max_runtime_s" not in inspect.signature(http_verifier.verify_http).parameters:+ raise ValueError("This runner needs the HTTP verifier's job_max_runtime_s API")+ if selected_case and selected_case not in {case["id"] for case in plan["cases"]}:+ raise ValueError("Unknown case identifier")+ output.mkdir(parents=True, exist_ok=False)+ state = {"state": "running", "runner_pid": os.getpid(), "started_utc": http_verifier.now(),+ "cases": [{"id": case["id"], "state": "pending"} for case in plan["cases"]]}+ write_json(output / "state.json", state)+ save_report(plan_path, plan, state, output)++ def interrupted(signum, _frame):+ raise KeyboardInterrupt(f"Received {signal.Signals(signum).name}")++ previous = signal.signal(signal.SIGTERM, interrupted)+ active = None+ try:+ canonical, reference = make_canonical(plan["source"]["path"], output, plan["protocol"])+ write_json(output / "canonical.json", identity(canonical))+ for case, row in zip(plan["cases"], state["cases"]):+ if selected_case and case["id"] != selected_case:+ row["state"] = "not_selected"+ continue+ check_frozen(plan_path, plan)+ active = row+ directory = output / case["id"]+ directory.mkdir()+ row.update(state="running", directory=str(directory),+ reproducer=shlex.join([sys.executable, str(Path(__file__).resolve()), "run",+ "--plan", str(plan_path), "--output", str(directory)+"-reproducer", "--case", case["id"]]))+ write_json(output / "state.json", state)+ try:+ audio = encode(canonical, directory, case["encoding"], plan["protocol"])+ row["encoded_input"] = identity(audio)+ try:+ decoded = preflight_decode(audio, directory / "preflight", plan["protocol"])+ native = native_decode_metrics(audio, directory, plan["protocol"])+ pcm = compare_pcm(reference.samples_path, decoded.samples_path, case["encoding"], plan["protocol"])+ row["usability"] = {"pcm": pcm, "native_decode": native,+ "passed": bool(pcm["passed"] and native["absolute_peak"] < plan["protocol"]["tolerances"]["decoded_absolute_peak_exclusive"]+ and (native["channels"] != 2 or native["identical_channel_max_difference"] == 0.0))}+ except Exception as exc:+ row["usability"] = {"passed": False, "error": f"{type(exc).__name__}: {exc}"}+ baseline_input = state["cases"][0].get("encoded_input")+ if case["id"].startswith("chunks_") and baseline_input:+ same_input = row["encoded_input"]["sha256"] == baseline_input["sha256"]+ row["usability"]["chunk_input_matches_baseline"] = same_input+ row["usability"]["passed"] = bool(row["usability"]["passed"] and same_input)+ write_json(directory / "representation.json", {"encoding": case["encoding"],+ "encoded_input": row["encoded_input"], "usability": row["usability"],+ "request": case["request"], "operator": case["operator"]})+ # Failed usability checks do not remove encoded arms or cause+ # outcome-dependent settings changes. The upload remains fresh.+ with MemorySampler() as memory:+ receipt = http_verifier.verify_http(audio, directory / "http",+ request=case["request"], operator=case["operator"], expect_status=None,+ job_max_runtime_s=plan["protocol"]["budgets"]["job_max_runtime_s"],+ timeout_s=plan["protocol"]["budgets"]["http_timeout_s"],+ require_full_pipeline=False)+ row["memory"] = memory.result()+ row["observation"] = observe_case(directory, case, receipt)+ row["state"] = "finished"+ if receipt.get("interrupted"):+ raise KeyboardInterrupt("HTTP verification was interrupted")+ except Exception as exc:+ row.update(state="error", error=f"{type(exc).__name__}: {exc}")+ if (directory / "http/http-verification.json").is_file() and "observation" not in row:+ try:+ row["observation"] = observe_case(directory, case,+ read_json(directory / "http/http-verification.json"))+ except Exception as parse_error:+ row["observation_error"] = f"{type(parse_error).__name__}: {parse_error}"+ finally:+ write_json(output / "state.json", state)+ save_report(plan_path, plan, state, output)+ print(json.dumps({"case": row["id"], "state": row["state"],+ "status": row.get("observation", {}).get("status"),+ "reason": row.get("observation", {}).get("reason_code"),+ "representation_usable": row.get("usability", {}).get("passed")}), flush=True)+ active = None+ check_frozen(plan_path, plan)+ state["state"] = "finished"+ except BaseException as exc:+ state.update(state="interrupted" if isinstance(exc, KeyboardInterrupt) else "error",+ error=f"{type(exc).__name__}: {exc}")+ if active is not None and active["state"] == "running":+ active.update(state="interrupted", error=state["error"])+ finally:+ signal.signal(signal.SIGTERM, previous)+ write_json(output / "state.json", state)+ report = save_report(plan_path, plan, state, output)+ return report+++def main(argv=None):+ parser = argparse.ArgumentParser(description=__doc__)+ commands = parser.add_subparsers(dest="command", required=True)+ pre = commands.add_parser("prepare", help="Freeze a protocol before executing its cases")+ pre.add_argument("--audio", type=Path, required=True)+ pre.add_argument("--request", type=Path, required=True)+ pre.add_argument("--operator", type=Path, required=True)+ pre.add_argument("--output", type=Path, required=True)+ pre.add_argument("--source-note", required=True)+ pre.add_argument("--scope-note", required=True)+ pre.add_argument("--frozen-input", type=Path, action="append", default=[])+ pre.add_argument("--job-max-runtime-s", type=float)+ pre.add_argument("--http-timeout-s", type=float)+ execute = commands.add_parser("run", help="Execute fresh jobs; never resume old scientific work")+ execute.add_argument("--plan", type=Path, required=True)+ execute.add_argument("--output", type=Path, required=True)+ execute.add_argument("--case", help="One-arm reproducer; never counts as the full matrix")+ report = commands.add_parser("report", help="Compare preserved observations without executing the pipeline")+ report.add_argument("--plan", type=Path, required=True)+ report.add_argument("--output", type=Path, required=True)+ args = parser.parse_args(argv)+ try:+ if args.command == "prepare":+ path = prepare(args.audio, read_json(args.request), read_json(args.operator), args.output,+ source_note=args.source_note, scope_note=args.scope_note, additional_inputs=args.frozen_input,+ job_max_runtime_s=args.job_max_runtime_s, http_timeout_s=args.http_timeout_s)+ print(path)+ return 0+ if args.command == "run":+ result = run(args.plan, args.output, selected_case=args.case)+ else:+ plan_path = args.plan.resolve(strict=True)+ plan = read_json(plan_path)+ if http_verifier.sha256(plan_path) != plan_path.with_name("plan.sha256").read_text().strip():+ raise ValueError("The frozen plan changed")+ if http_verifier.sha256(__file__) != plan["implementation"]["runner_sha256"]:+ raise ValueError("Use the frozen runner to reproduce these comparisons")+ result = save_report(plan_path, plan, read_json(args.output / "state.json"), args.output)+ print(json.dumps({key: result[key] for key in+ ("run_state", "finished_cases", "full_pipeline_cases", "pairs_with_observed_differences",+ "all_complete_usable_and_within_tolerances")}))+ return 0 if result["all_complete_usable_and_within_tolerances"] else 1+ except (OSError, ValueError, TypeError, subprocess.SubprocessError) as exc:+ parser.error(str(exc))+++if __name__ == "__main__":+ raise SystemExit(main())diff --git a/scripts/verify_upload_http.py b/scripts/verify_upload_http.pyindex c17bd04..1c62af3 100644--- a/scripts/verify_upload_http.py+++ b/scripts/verify_upload_http.py@@ -6,7 +6,8 @@ The receipt separately preserves the producer's scientific status. An expectedinconclusive response is never counted as supported localization or accuracy.With --require-full-pipeline, verification also requires every pipeline stageand at least one returned candidate; an early exit cannot pass that requirement.-The existing browser verifier is independent and remains unchanged.+The browser verifier checks rendering separately. Optional runtime, cancellation,+and restart controls exercise actual terminal behavior without replacing workers."""from __future__ import annotations@@ -32,6 +33,9 @@ from dopplergeist.contracts import utc_timestamp, validate_recording, validate_rfrom dopplergeist.jobs import TERMINAL, write_json+CANCELLABLE_STAGES = ("intake", "discovery", "acquisition", "search", "refinement", "decision")++def _implementation_files(package):"""Independently cover Python sources, native C, and loaded-format binaries."""return sorted(path for path in package.iterdir()@@ -178,7 +182,8 @@ def wait_for_service(process, log_path, deadline):time.sleep(min(0.05, remaining(deadline)))-def poll_job(client, job_id, history_path, *, interval_s):+def poll_job(client, job_id, history_path, *, interval_s, cancel_at_stage=None, cancellation=None):+ cancellation = {} if cancellation is None else cancellationwith history_path.open("x") as history:previous = Nonewhile True:@@ -191,10 +196,75 @@ def poll_job(client, job_id, history_path, *, interval_s):history.flush()previous = valueif value["status"] in TERMINAL:+ if cancel_at_stage is not None and not cancellation:+ raise ValueError(f"Job ended before cancellation stage {cancel_at_stage!r} was observed")return value+ if (cancel_at_stage is not None and not cancellation+ and value["status"] == "running" and value.get("stage") == cancel_at_stage):+ cancellation.update(observed_status=value["status"], observed_stage=value["stage"],+ requested_utc=now())+ code, reply = client.request("POST", f"/api/jobs/{job_id}/cancel")+ cancellation["http_status"] = code+ if code != 200:+ raise ValueError(f"Cancellation returned HTTP {code}")+ validate_result(reply)+ if reply["job_id"] != job_id:+ raise ValueError("Cancellation returned a different job")+ cancellation["response_status"] = reply["status"]time.sleep(min(interval_s, remaining(client.deadline)))+def inspect_restart(output, command, environment, final, source_sha256, deadline,+ termination, report, *, repeat_cancel=False):+ """Restart only this verifier's service and read its completed job again."""+ directory = output / "jobs" / final["job_id"]+ result_digest = sha256(directory / "result.json")+ process = None+ report.update(passed=False, service={"shutdown_signal": None, "forced_kill": False, "returncode": None})+ try:+ with (output / "restart-service.log").open("xb") as log:+ try:+ termination.interruptible = False+ process = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT,+ env=environment, start_new_session=True)+ report["service"]["pid"] = process.pid+ termination.interruptible = True+ termination.raise_if_interrupted()+ client = wait_for_service(process, output / "restart-service.log", deadline)+ report["service"]["port"] = client.port+ job_path = f"/api/jobs/{final['job_id']}"+ if client.get(job_path) != final:+ raise ValueError("Restart changed the completed API result")+ if repeat_cancel:+ code, cancelled = client.request("POST", job_path + "/cancel")+ if code != 200 or cancelled != final:+ raise ValueError("Repeated cancellation changed the completed result")+ report["terminal_cancel_idempotent"] = True+ code, _ = client.request("GET", job_path + "/artifacts/input.mp3",+ destination=output / "downloaded-original-after-restart.mp3")+ if code != 200 or sha256(output / "downloaded-original-after-restart.mp3") != source_sha256:+ raise ValueError("Restart changed the original MP3 download")+ code, _ = client.request("GET", job_path + "/artifacts/result.json",+ destination=output / "downloaded-result-after-restart.json")+ if code != 200 or sha256(output / "downloaded-result-after-restart.json") != result_digest:+ raise ValueError("Restart changed the saved result bytes")+ if client.get(job_path) != final:+ raise ValueError("Terminal API result changed during restart readback")+ report.update(terminal_readback_verified=True, original_download_verified=True,+ saved_result_bytes_verified=True)+ finally:+ termination.interruptible = False+ stop_service(process, report, termination)+ if report["service"]["returncode"] != 0 or report["service"]["forced_kill"]:+ raise ValueError("Restarted service did not stop cleanly")+ if sha256(directory / "result.json") != result_digest or sha256(directory / "input.mp3") != source_sha256:+ raise ValueError("Restart shutdown changed completed artifacts")+ report["passed"] = True+ finally:+ termination.interruptible = True+ termination.raise_if_interrupted()++def inspect_job(directory, final, wire, operator, original, implementation):"""Check actual persisted evidence without changing it or opening truth."""read = lambda path: json.loads((directory / path).read_text())@@ -1029,13 +1099,25 @@ def inspect_pipeline_evidence(directory, final, wire, operator):def verify_http(audio, output, *, request=None, operator=None, expect_status,expect_reason=None, timeout_s=600.0, poll_interval_s=0.25,- require_full_pipeline=False):- if expect_status not in TERMINAL:- raise ValueError("Declare an expected terminal status before execution")+ require_full_pipeline=False, job_max_runtime_s=None,+ cancel_at_stage=None, verify_restart=False):+ """Verify an actual upload; None observes any terminal status for callers.++ A missing job runtime preserves the original timeout_s worker allowance.+ Give the client a larger timeout to observe an intentional worker time limit.+ """+ if expect_status is not None and expect_status not in TERMINAL:+ raise ValueError("Expected status must be terminal or None for observation")if type(require_full_pipeline) is not bool:raise ValueError("require_full_pipeline must be a boolean")- for name, value in (("timeout_s", timeout_s), ("poll_interval_s", poll_interval_s)):- if not math.isfinite(value) or value <= 0:+ if type(verify_restart) is not bool:+ raise ValueError("verify_restart must be a boolean")+ if cancel_at_stage is not None and cancel_at_stage not in CANCELLABLE_STAGES:+ raise ValueError("Cancellation stage must be an actual running pipeline stage")+ job_runtime = timeout_s if job_max_runtime_s is None else job_max_runtime_s+ for name, value in (("timeout_s", timeout_s), ("poll_interval_s", poll_interval_s),+ ("job_max_runtime_s", job_runtime)):+ if isinstance(value, bool) or not math.isfinite(value) or value <= 0:raise ValueError(f"{name} must be finite and positive")audio = Path(audio).resolve(strict=True)if audio.suffix.lower() != ".mp3" or not audio.is_file():@@ -1055,7 +1137,9 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,deadline = began + timeout_sreceipt = {"schema_version": 1, "scope": "actual_local_http_upload_verification","started_utc": now(), "expected_status": expect_status, "expected_reason": expect_reason,- "timeout_s": timeout_s, "source_sha256": sha256(audio), "source_bytes": audio.stat().st_size,+ "timeout_s": timeout_s, "job_max_runtime_s": job_runtime,+ "cancel_at_stage": cancel_at_stage, "verify_restart": verify_restart,+ "source_sha256": sha256(audio), "source_bytes": audio.stat().st_size,"request": wire, "implementation_files": implementation, "verifier_sha256": sha256(__file__),"python_executable": sys.executable, "python_version": sys.version,"verification_passed": False, "http_checks_passed": False, "pipeline_evidence_passed": False,@@ -1073,7 +1157,7 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,termination.raise_if_interrupted()command = [sys.executable, "-m", "dopplergeist.cli", "serve", "--port", "0","--jobs", str(output / "jobs"), "--config", str(output / "operator.json"),- "--max-runtime", str(timeout_s)]+ "--max-runtime", str(job_runtime)]receipt["service"]["command"] = commandenv = dict(os.environ, PYTHONPATH=str(package.parent) + os.pathsep + os.environ.get("PYTHONPATH", ""),MPLBACKEND="Agg")@@ -1085,7 +1169,10 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,termination.raise_if_interrupted()client = wait_for_service(process, output / "service.log", deadline)receipt["service"]["port"] = client.port- write_json(output / "capabilities.json", client.get("/api/capabilities"))+ capabilities = client.get("/api/capabilities")+ write_json(output / "capabilities.json", capabilities)+ if capabilities.get("max_runtime_s") != job_runtime:+ raise ValueError("Service did not retain the requested job runtime")with audio.open("rb") as stream:code, queued = client.request("POST", "/api/jobs", body=stream, headers={"Content-Type": "application/octet-stream", "Content-Length": str(audio.stat().st_size),@@ -1098,11 +1185,20 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,if uuid.UUID(job_id).hex != job_id:raise ValueError("Service returned an invalid job identifier")receipt["job_id"] = job_id- final = poll_job(client, job_id, output / "history.jsonl", interval_s=poll_interval_s)+ cancellation = {}+ if cancel_at_stage is not None:+ receipt["cancellation"] = cancellation+ final = poll_job(client, job_id, output / "history.jsonl", interval_s=poll_interval_s,+ cancel_at_stage=cancel_at_stage, cancellation=cancellation)write_json(output / "result.json", final)receipt.update(status=final["status"], reason_code=final.get("reason_code"),producer_reported_success=final["status"] == "succeeded",candidate_count=len(final.get("candidates", [])), event_count=len(final.get("events", [])))+ if cancel_at_stage is not None:+ cancellation["confirmed"] = (cancellation.get("response_status") == "cancelled"+ and final["status"] == "cancelled")+ if not cancellation["confirmed"]:+ raise ValueError("Cancellation was requested, but the job did not finish cancelled")listing = client.get(f"/api/jobs/{job_id}/artifacts")write_json(output / "artifacts.json", listing)if sum(item["path"] == "input.mp3" for item in listing["artifacts"]) != 1:@@ -1116,8 +1212,18 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,receipt["checks"] = inspect_job(output / "jobs" / job_id, final, wire, operator, audio, implementation)receipt["checks"].update(downloaded_original_verified=True, downloaded_result_verified=True)receipt["http_checks_passed"] = True- receipt["expectation_matched"] = (final["status"] == expect_status+ receipt["expectation_matched"] = ((expect_status is None or final["status"] == expect_status)and (expect_reason is None or final.get("reason_code") == expect_reason))+ if verify_restart:+ termination.interruptible = False+ stop_service(process, receipt, termination)+ termination.interruptible = True+ termination.raise_if_interrupted()+ if receipt["service"]["returncode"] != 0 or receipt["service"]["forced_kill"]:+ raise ValueError("Initial service did not stop cleanly before readback")+ receipt["restart_readback"] = {}+ inspect_restart(output, command, env, final, receipt["source_sha256"], deadline,+ termination, receipt["restart_readback"], repeat_cancel=bool(cancellation))except Exception as exc:receipt["error"] = f"{type(exc).__name__}: {exc}"receipt["timed_out"] = isinstance(exc, TimeoutError)@@ -1156,7 +1262,8 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,and receipt.get("pipeline_evidence_preserved_after_inspection") and receipt["implementation_unchanged"]and receipt.get("terminal_result_preserved_after_shutdown")and receipt["service"]["returncode"] == 0 and not receipt["service"]["forced_kill"]- and not receipt.get("shutdown_error") and not receipt["interrupted"])+ and not receipt.get("shutdown_error") and not receipt["interrupted"]+ and (not verify_restart or receipt.get("restart_readback", {}).get("passed")))if require_full_pipeline and not receipt["full_pipeline_requirements_met"]:receipt["requirement_error"] = ("Full pipeline verification requires verified acquisition, search, refinement, "@@ -1193,6 +1300,12 @@ def main(argv=None):parser.add_argument("--require-full-pipeline", action="store_true",help="Fail unless every pipeline stage is verified and at least one candidate is returned")parser.add_argument("--timeout-s", type=float, default=600)+ parser.add_argument("--job-max-runtime-s", type=float,+ help="Worker time budget; defaults to --timeout-s. Use a smaller value to verify work_limit")+ parser.add_argument("--cancel-at-stage", choices=CANCELLABLE_STAGES,+ help="Request actual API cancellation after this running stage is observed")+ parser.add_argument("--verify-restart", action="store_true",+ help="Restart this verifier's service and verify unchanged completed artifacts through HTTP")parser.add_argument("--poll-interval-s", type=float, default=0.25)args = parser.parse_args(argv)if args.end_utc and not args.start_utc:@@ -1206,7 +1319,8 @@ def main(argv=None):receipt = verify_http(args.audio, args.output, request=request, operator=operator,expect_status=args.expect_status, expect_reason=args.expect_reason,timeout_s=args.timeout_s, poll_interval_s=args.poll_interval_s,- require_full_pipeline=args.require_full_pipeline)+ require_full_pipeline=args.require_full_pipeline, job_max_runtime_s=args.job_max_runtime_s,+ cancel_at_stage=args.cancel_at_stage, verify_restart=args.verify_restart)except (OSError, ValueError, TypeError) as exc:parser.error(str(exc))print(json.dumps({"report": str(args.output.resolve() / "http-verification.json"),diff --git a/tests/test_real_metamorphic_runner.py b/tests/test_real_metamorphic_runner.pynew file mode 100644index 0000000..d87d7c2--- /dev/null+++ b/tests/test_real_metamorphic_runner.py@@ -0,0 +1,95 @@+"""Representation/mapping contracts using archived real ridges and recorded audio."""+from copy import deepcopy+import importlib.util+import json+import os+from pathlib import Path+import sys++import pytest+++ROOT = Path(__file__).resolve().parents[1]+sys.path.insert(0, str(ROOT / "scripts"))+spec = importlib.util.spec_from_file_location("real_metamorphic", ROOT / "scripts/run_real_metamorphic.py")+runner = importlib.util.module_from_spec(spec)+spec.loader.exec_module(runner)+PROTOCOL = json.loads(runner.PROTOCOL.read_text())+++def recorded_events():+ return json.loads((ROOT / "tests/data/shared-ridges-real.json").read_text())["shared"]["events"]+++def test_padding_preserves_both_absolute_utc_endpoints_and_real_ridge_times():+ request = {"schema_version": 1, "time_window": {+ "start_utc": "2026-09-08T20:17:20Z", "end_utc": "2026-09-08T20:17:40Z",+ "source": "user", "source_detail": "Source-informed reference development control"}}+ shifted = runner.shifted_request(request, 17.25)+ assert shifted["time_window"]["start_utc"] == "2026-09-08T20:17:02.750000Z"+ assert shifted["time_window"]["end_utc"] == "2026-09-08T20:17:22.750000Z"+ for key in ("start_utc", "end_utc"):+ assert (runner.parse_utc(request["time_window"][key])+ - runner.parse_utc(shifted["time_window"][key])).total_seconds() == 17.25+ events = recorded_events()+ padded = runner.mapped_events(events, -17.25)+ assert runner.mapped_events(padded, 17.25) == events+ assert request["time_window"]["start_utc"] == "2026-09-08T20:17:20Z"+++def test_recorded_ridges_use_one_to_one_correspondence_not_identifiers_or_order():+ left = recorded_events()+ right = deepcopy(list(reversed(left)))+ originals = {}+ for i, event in enumerate(right):+ originals[event["event_id"]] = f"renamed-{i}"+ event["event_id"] = f"renamed-{i}"+ event["harmonic_group_id"] = f"renamed-group-{i}"+ result = runner.event_correspondence(left, right, PROTOCOL)+ assert {row["left_event_id"]: row["right_event_id"] for row in result["matches"]} == originals+ assert len({row["right_event_id"] for row in result["matches"]}) == len(left)+ assert not result["unmatched_left"] and not result["unmatched_right"]+ assert all(row["median_frequency_delta_hz"] == 0 for row in result["matches"])+ missing = runner.event_correspondence(left, right[:1], PROTOCOL)+ assert len(missing["matches"]) == 1+ assert len(missing["unmatched_left"]) == 1+ assert not missing["unmatched_right"]+++def test_cache_freeze_detects_an_added_alias_to_existing_real_input(tmp_path):+ source = ROOT / "tests/data/shared-ridges-real.json"+ cache = tmp_path / "immutable-cache"+ cache.mkdir()+ (cache / "first.json").symlink_to(source)+ operator = {"cache_root": str(cache), "offline": True}+ before = runner.operator_inputs(operator, [])+ (cache / "another-name.json").symlink_to(source)+ after = runner.operator_inputs(operator, [])+ assert before != after+ assert len(before) == 1 and len(after) == 2+ assert len({row["resolved_path"] for row in after}) == 1+++def test_actual_recorded_clip_encoding_and_padding_contract(tmp_path):+ source = os.environ.get("DOPPLERGEIST_REAL_METAMORPHIC_AUDIO")+ if not source:+ pytest.skip("Set DOPPLERGEIST_REAL_METAMORPHIC_AUDIO to a short actual recording")+ canonical, reference = runner.make_canonical(source, tmp_path, PROTOCOL)+ for name in ("baseline", "identical_stereo", "leading_17_25s", "rate_44100", "gain_minus_6db"):+ case = next(row for row in PROTOCOL["cases"] if row["id"] == name)+ settings = dict(PROTOCOL["baseline"], **case["changes"])+ directory = tmp_path / name+ directory.mkdir()+ encoded = runner.encode(canonical, directory, settings, PROTOCOL)+ decoded = runner.intake_recording(encoded, directory / "preflight", sample_rate_hz=4000)+ metrics = runner.native_decode_metrics(encoded, directory, PROTOCOL)+ comparison = runner.compare_pcm(reference.samples_path, decoded.samples_path, settings, PROTOCOL)+ assert metrics["channels"] == settings["channels"]+ assert metrics["sample_rate_hz"] == settings["sample_rate_hz"]+ assert abs(comparison["duration_error_s"]) <= PROTOCOL["tolerances"]["decoded_duration_error_s"]+ assert comparison["maximum_absolute_lag_s"] <= PROTOCOL["tolerances"]["residual_alignment_lag_s"]+ assert comparison["padding_samples"] == round(settings["leading_silence_s"]*4000)+ if settings["channels"] == 2:+ assert metrics["identical_channel_max_difference"] == 0+ assert metrics["first_packet"]+ # Transform contracts are not full-pipeline or one-hour matrix evidence.diff --git a/tests/test_real_upload_terminal_controls.py b/tests/test_real_upload_terminal_controls.pynew file mode 100644index 0000000..d2fc631--- /dev/null+++ b/tests/test_real_upload_terminal_controls.py@@ -0,0 +1,66 @@+"""Actual upload cancellation controls; require an existing real-recording job.++The race control delays only the outgoing cancel request until the real worker+finishes. All audio, worker execution and HTTP responses remain actual inputs.+"""+import importlib.util+import json+import os+from pathlib import Path+import time++import pytest+++ROOT = Path(__file__).parents[1]+SPEC = importlib.util.spec_from_file_location("verify_real_terminal_controls",+ ROOT / "scripts/verify_upload_http.py")+HTTP = importlib.util.module_from_spec(SPEC)+SPEC.loader.exec_module(HTTP)++[email protected]("delay_cancel", [False, True], ids=["cancel-running", "completion-wins-race"])+def test_actual_cancellation_must_be_confirmed_even_without_expected_status(tmp_path, monkeypatch, delay_cancel):+ saved = os.environ.get("DOPPLERGEIST_REAL_UPLOAD_JOB")+ if not saved:+ pytest.skip("Set DOPPLERGEIST_REAL_UPLOAD_JOB to a real job with MP3 and recording-start window")+ source = Path(saved)+ request = json.loads((source / "request.json").read_text())+ request.pop("audio_path")+ assert request.get("time_window"), "The actual recording needs its existing recording-start window"+ audio = source / "input.mp3"+ assert HTTP.sha256(audio) == json.loads((source / "input.json").read_text())["sha256"]+ real_request = HTTP.LocalClient.request++ def request_after_real_completion(client, method, path, **kwargs):+ if delay_cancel and method == "POST" and path.endswith("/cancel"):+ while True:+ status, current = real_request(client, "GET", path.rsplit("/", 1)[0])+ assert status == 200+ if current["status"] in HTTP.TERMINAL:+ break+ time.sleep(min(0.02, HTTP.remaining(client.deadline)))+ return real_request(client, method, path, **kwargs)++ monkeypatch.setattr(HTTP.LocalClient, "request", request_after_real_completion)+ cache = tmp_path / "empty-offline-cache"+ cache.mkdir()+ output = tmp_path / "upload"+ receipt = HTTP.verify_http(audio, output, request=request,+ operator={"offline": True, "cache_root": str(cache)}, expect_status=None,+ cancel_at_stage="discovery", job_max_runtime_s=90, timeout_s=120, poll_interval_s=0.05)+ assert receipt["expected_status"] is None+ assert receipt["cancellation"]["observed_status"] == "running"+ assert receipt["cancellation"]["observed_stage"] == "discovery"+ assert receipt["cancellation"]["http_status"] == 200+ result = json.loads((output / "result.json").read_text())+ assert HTTP.sha256(output / "jobs" / result["job_id"] / "input.mp3") == HTTP.sha256(audio)+ if delay_cancel:+ assert result["status"] == "failed" and result["reason_code"] == "data_unavailable"+ assert receipt["cancellation"]["confirmed"] is False+ assert receipt["verification_passed"] is False+ assert "did not finish cancelled" in receipt["error"]+ else:+ assert result["status"] == "cancelled"+ assert receipt["cancellation"]["confirmed"] is True+ assert receipt["verification_passed"] is True