Skip to content

Python tool for aircraft Doppler audio geolocation using historical ADS-B flight data (Source-available)

Preserve the frozen nine-arm real recording experiment runner

Cinna Davis309429278978
diff --git a/docs/REAL_METAMORPHIC.md b/docs/REAL_METAMORPHIC.md
new file mode 100644
index 0000000..5f46a3d
--- /dev/null
+++ b/docs/REAL_METAMORPHIC.md
@@ -0,0 +1,143 @@
+# Real recording representation checks
+
+Issue [#61](https://github.com/1337isnot1337/DOPPLERGEIST/issues/61) requires fresh
+end-to-end runs for equivalent representations of actual recordings.
+The runner in `scripts/run_real_metamorphic.py` executes these runs through the
+local HTTP upload service. It accepts a recorded source, a recording-start UTC
+window, and an offline operator configuration.
+
+Each arm starts a new service and job. Intake, discovery, selection, matching,
+search, refinement, decision and serialization run from the uploaded bytes.
+The runner shares only frozen historical indexes and terrain. It does not
+resume audio or search checkpoints. The HTTP verifier checks the actual stage
+artifacts and the persisted terminal result.
+
+## Frozen protocol
+
+The versioned protocol is
+[`verification/metamorphic_protocol_v1.json`](verification/metamorphic_protocol_v1.json).
+It was committed before the development matrix ran. It has nine sequential arms:
+
+| Arm | Change from the baseline |
+| --- | --- |
+| Baseline | MP3, 128 kb/s, 48 kHz, mono |
+| Low bitrate | 64 kb/s |
+| High bitrate | 192 kb/s |
+| Sample rate | 44.1 kHz |
+| Gain | −6 dB |
+| Channels | Two identical channels |
+| Leading silence | 17.25 seconds |
+| Short discovery chunks | 60-second chunks, 30-second overlap |
+| Long discovery chunks | 180-second chunks, 60-second overlap |
+
+All encodings start from the complete source recording's arithmetic channel
+mean at 48 kHz. No arm is encoded from another MP3 arm. The baseline keeps the
+operator's discovery settings. The current defaults are 120/60 seconds.
+
+Only `operator.discovery.chunk_s` and `overlap_s` are exposed chunk controls.
+The protocol explicitly lists unsupported upload, decoder and detector controls.
+It does not insert hooks into the source algorithms.
+
+Each job has a 240-second runtime limit and a 300-second client deadline.
+The encoding/decode command limit is 300 seconds. Pipeline time is at most
+45 minutes across nine client deadlines, plus preparation and cleanup.
+Different budgets can be declared at preparation, before any outcomes exist.
+
+Tolerances are fixed in the protocol: 2 ms for duration and residual alignment,
+1 second for mapped event edges and fitted clocks, 2 Hz for median ridge
+frequency, and 0.05 per matched group for comparable normalized scores.
+Usability also requires nonclipping decoded peaks and gain-corrected PCM
+NRMSE at most 0.15. Group counts, flight associations and score units must agree.
+The runner reports displacement thresholds at 100, 500 and 1,000 metres.
+Accepted locations that move over 1,000 metres receive a separate serious flag.
+
+The PCM calculation reads bounded blocks. Lag checks use twelve fixed short
+windows, not full-hour correlations. A measured lag is reported; it does not
+shift samples to improve the result. Native-rate decoded peaks and identical
+stereo channels are checked separately from the 4 kHz analysis PCM.
+
+## Clock mapping
+
+If leading silence is Δ seconds, subtract Δ from **both** sample-zero UTC
+window endpoints. Subtract Δ from detected audio-relative times for comparison.
+Add Δ to the fitted sample-zero UTC to recover the original recording clock.
+
+For example, 20:17:20–20:17:40 UTC becomes 20:17:02.750–20:17:22.750 UTC
+after 17.25 seconds of padding. The physical time of each recorded sound stays
+the same. The runner retains codec skip/discard metadata, decoded sample counts,
+and residual lag checks. Container start time is not a recording clock.
+
+## Run
+
+Use an installed checkout with FFmpeg and the repository's dependencies.
+The request must have a defensible time window. Use absolute local paths for
+historical indexes and terrain, or run preparation where relative paths resolve.
+Use a stopped, immutable data cache. Active job directories are not data inputs.
+
+```bash
+PYTHONPATH=src python scripts/run_real_metamorphic.py prepare \
+ --audio /absolute/path/actual-recording.wav \
+ --request /absolute/path/request.json \
+ --operator /absolute/path/operator.json \
+ --source-note 'Identify the actual recorded source and its permission/provenance' \
+ --scope-note 'Describe the search scope, clock evidence and evaluation limits' \
+ --output /absolute/path/new-frozen-plan
+
+PYTHONPATH=src python scripts/run_real_metamorphic.py run \
+ --plan /absolute/path/new-frozen-plan/plan.json \
+ --output /absolute/path/new-matrix-run
+```
+
+Preparation writes an immutable plan before encoding. It pins source bytes,
+configuration, historical inputs, terrain, source code, native binaries,
+dependencies, backend environment and FFmpeg version. Runs refuse changed
+inputs. Use `--frozen-input PATH` for an additional immutable local dependency.
+Every output directory must be new.
+
+The runner preserves encoded inputs, exact transformation commands, actual
+job artifacts and observations for every arm. It continues after an ordinary
+arm failure. It preserves pending arms on interruption or changed inputs.
+An unusable encoded arm still reaches the HTTP pipeline. Encoding failures
+remain explicit failures.
+
+`report.json` contains all pairwise comparisons. `report.md` gives a short
+case table. Event matching uses a one-to-one assignment based on mapped time
+overlap and ridge frequency. It reports unmatched ridges/groups, group splits,
+group merges, and ambiguous alternatives. IDs and row positions do not define
+correspondence. Retained distant modes are compared separately.
+
+Each case includes a one-arm reproducer command. The `--case` option creates
+a fresh job and records the other arms as not selected. It cannot pass the
+full-matrix requirement. The `report` command compares preserved observations;
+it never counts as another end-to-end run.
+
+Exit zero requires all nine usable arms to reach the full pipeline and remain
+within the declared comparison tolerances. Exit one preserves incomplete runs
+or observed differences. An inconclusive scientific decision is not itself a
+runner failure when the complete pipeline ran. It remains an inconclusive
+location result.
+
+The full-path count uses the verifier's stronger requirement: every stage must
+be verified and at least one diagnostic candidate must survive. Empty outcomes
+remain in the report. A separate count records all observed stage sequences.
+Neither count requires a supported location decision.
+
+## Scope of the development run
+
+The affordable development control uses the whole released reference recording,
+a source-informed regional box, a narrow source-informed UTC window, and
+incomplete historical flight coverage. It cannot satisfy national acceptance.
+It does not establish independent location accuracy.
+
+No eligible independent truth is supplied in this protocol. Truth errors at
+100/500/1,000 metres remain explicitly unevaluated. Pairwise displacement is
+reported separately. A future eligible national evaluation needs its own
+predeclared request, data, budgets and evaluation-only truth protocol.
+
+Focused contract checks use archived real ridge observations and an optional
+short actual recording. They do not replace any full-recording arm:
+
+```bash
+DOPPLERGEIST_REAL_METAMORPHIC_AUDIO=/absolute/path/actual-short-recording.mp3 \
+ PYTHONPATH=src pytest -q tests/test_real_metamorphic_runner.py
+```
diff --git a/docs/verification/metamorphic_protocol_v1.json b/docs/verification/metamorphic_protocol_v1.json
new file mode 100644
index 0000000..ed9ba50
--- /dev/null
+++ b/docs/verification/metamorphic_protocol_v1.json
@@ -0,0 +1,71 @@
+{
+ "schema_version": 1,
+ "protocol_id": "real-recording-metamorphic-v1",
+ "issue": 61,
+ "source_policy": "One complete actual recording; arithmetic channel mean; no generated events or tracks; no truth supplied to search.",
+ "canonical_pcm": {"sample_rate_hz": 48000, "channels": 1, "sample_format": "f32le"},
+ "baseline": {"bitrate_kbps": 128, "sample_rate_hz": 48000, "channels": 1, "gain_db": 0.0, "leading_silence_s": 0.0},
+ "cases": [
+ {"id": "baseline", "changes": {}},
+ {"id": "bitrate_64", "changes": {"bitrate_kbps": 64}},
+ {"id": "bitrate_192", "changes": {"bitrate_kbps": 192}},
+ {"id": "rate_44100", "changes": {"sample_rate_hz": 44100}},
+ {"id": "gain_minus_6db", "changes": {"gain_db": -6.0}},
+ {"id": "identical_stereo", "changes": {"channels": 2}},
+ {"id": "leading_17_25s", "changes": {"leading_silence_s": 17.25}},
+ {"id": "chunks_60_30", "changes": {"discovery": {"chunk_s": 60.0, "overlap_s": 30.0}}},
+ {"id": "chunks_180_60", "changes": {"discovery": {"chunk_s": 180.0, "overlap_s": 60.0}}}
+ ],
+ "budgets": {"job_max_runtime_s": 240.0, "http_timeout_s": 300.0, "encoding_timeout_s": 300.0},
+ "tolerances": {
+ "decoded_duration_error_s": 0.002,
+ "residual_alignment_lag_s": 0.002,
+ "decoded_absolute_peak_exclusive": 1.0,
+ "gain_corrected_pcm_nrmse": 0.15,
+ "mapped_event_edge_s": 1.0,
+ "mapped_clock_s": 1.0,
+ "median_ridge_frequency_hz": 2.0,
+ "normalized_score_per_matched_group": 0.05,
+ "displacement_thresholds_m": [100.0, 500.0, 1000.0],
+ "accepted_jump_m": 1000.0,
+ "distant_mode_m": 1000.0,
+ "event_group_counts": "equal",
+ "associated_flights": "equal after one-to-one event mapping",
+ "score_units": "equal"
+ },
+ "pcm_comparison": {
+ "sample_rate_hz": 4000,
+ "block_samples": 65536,
+ "lag_windows": 12,
+ "lag_window_s": 8.0,
+ "lag_search_radius_s": 0.1,
+ "lag_rule": "Evenly spaced fixed windows; report each measured lag; never shift comparisons by fitted lag.",
+ "gain_rule": "Remove only the declared gain for comparison; no fitted normalization.",
+ "reference": "Decoded canonical mono PCM before MP3 encoding"
+ },
+ "event_correspondence": {
+ "minimum_overlap_fraction_of_shorter_event": 0.5,
+ "maximum_matching_median_frequency_hz": 25.0,
+ "assignment": "Minimum-cost one-to-one assignment, maximizing eligible pairs first; cost is one minus overlap fraction plus median frequency difference divided by 25 Hz.",
+ "ridge_comparison": "Interpolate the second mapped ridge at first-ridge times within their overlap; do not extrapolate.",
+ "groups": "Report unmatched ridges and groups, group splits and group merges; IDs and row order are not correspondence."
+ },
+ "time_mapping": {
+ "padding_request": "Subtract leading_silence_s from BOTH sample-zero UTC window endpoints.",
+ "event_mapping": "Subtract leading_silence_s from audio-relative event and ridge times.",
+ "clock_mapping": "Add leading_silence_s to fitted sample-zero UTC to recover the original recording clock.",
+ "codec_delay": "Use first decoded sample after ffmpeg skip/discard; retain first-packet side data, decoded sample counts and residual lag checks."
+ },
+ "execution": {
+ "order": "Protocol order, sequentially; baseline first.",
+ "fresh_work": "Each arm starts a fresh local HTTP service and job with original upload bytes; no saved intake, discovery, search or decision is reused.",
+ "shared_data": "Only pinned immutable historical indexes and terrain may be shared; require an offline operator.",
+ "failure_policy": "Retain encoding, usability, verification, terminal and incomplete outcomes. Continue other cases after ordinary failures; stop and preserve pending cases on interruption or changed pinned inputs.",
+ "truth_policy": "No eligible independent truth is supplied in this protocol. Truth error is explicitly not evaluated. Pairwise displacement is not accuracy."
+ },
+ "chunk_controls": {
+ "supported": ["operator.discovery.chunk_s", "operator.discovery.overlap_s"],
+ "unsupported": ["HTTP upload chunk size", "decoder block size", "detector spectral window/hop via the upload operator", "search scheduling chunks as an equivalent audio representation"]
+ },
+ "outcome_policy": "Compare all status/reason, coverage, work-completion and uncertainty-provenance transitions. Compare diagnostic candidates separately from accepted locations. Accepted jumps above 1000 m are serious findings. Regional source-informed runs do not satisfy national acceptance."
+}
diff --git a/scripts/run_real_metamorphic.py b/scripts/run_real_metamorphic.py
new file mode 100644
index 0000000..598d0f8
--- /dev/null
+++ b/scripts/run_real_metamorphic.py
@@ -0,0 +1,831 @@
+#!/usr/bin/env python3
+"""Run fresh HTTP jobs for a predeclared real-recording representation matrix.
+
+Prepare freezes configuration, inputs and implementation before any arm runs.
+Run preserves failed, inconclusive and unusable arms. Report only compares them.
+"""
+from __future__ import annotations
+
+import argparse
+from copy import deepcopy
+from datetime import datetime, timedelta, timezone
+from importlib import metadata
+from importlib.machinery import EXTENSION_SUFFIXES
+import inspect
+import itertools
+import json
+import math
+import os
+from pathlib import Path
+import shlex
+import signal
+import subprocess
+import sys
+import tempfile
+import threading
+
+import numpy as np
+from pyproj import Geod
+from scipy import signal as scipy_signal
+from scipy.optimize import linear_sum_assignment
+
+import dopplergeist
+from dopplergeist.contracts import validate_request
+from dopplergeist.discovery import DiscoverySettings
+from dopplergeist.intake import IntakeRecording, intake_recording, probe_media
+import verify_upload_http as http_verifier
+
+
+ROOT = Path(__file__).resolve().parents[1]
+PROTOCOL = ROOT / "docs/verification/metamorphic_protocol_v1.json"
+GEOD = Geod(ellps="WGS84")
+
+
+def read_json(path, default=None):
+ path = Path(path)
+ return json.loads(path.read_text()) if path.is_file() else default
+
+
+def write_json(path, value):
+ """Atomic progress reports; immutable plans use exclusive creation."""
+ path = Path(path)
+ temporary = path.with_suffix(path.suffix + ".tmp")
+ temporary.write_text(json.dumps(value, indent=2, allow_nan=False) + "\n")
+ temporary.replace(path)
+
+
+def identity(path):
+ path = Path(path).absolute()
+ resolved = path.resolve(strict=True)
+ return {"path": str(path), "resolved_path": str(resolved), "bytes": path.stat().st_size,
+ "sha256": http_verifier.sha256(path)}
+
+
+def parse_utc(value):
+ parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
+ if parsed.tzinfo is None:
+ raise ValueError("UTC endpoints must include a timezone")
+ return parsed.astimezone(timezone.utc)
+
+
+def format_utc(value):
+ return value.isoformat().replace("+00:00", "Z")
+
+
+def shifted_request(request, silence_s):
+ result = deepcopy(request)
+ window = result.get("time_window")
+ if not isinstance(window, dict):
+ raise ValueError("The matrix needs a defensible sample-zero UTC window")
+ for key in ("start_utc", "end_utc"):
+ window[key] = format_utc(parse_utc(window[key]) - timedelta(seconds=silence_s))
+ return result
+
+
+def implementation():
+ from dopplergeist import data_access, matching, native_acoustic, native_cpa
+
+ package = Path(dopplergeist.__file__).resolve().parent
+ result = {
+ "package_files": {p.relative_to(package).as_posix(): http_verifier.sha256(p)
+ for p in sorted(package.rglob("*")) if p.is_file() and (
+ p.suffix in {".py", ".c"} or any(p.name.endswith(suffix) for suffix in EXTENSION_SUFFIXES))},
+ "runner_sha256": http_verifier.sha256(__file__),
+ "verifier_sha256": http_verifier.sha256(http_verifier.__file__),
+ "python": sys.version, "executable": sys.executable,
+ "packages": {name: metadata.version(name) for name in
+ ("numpy", "scipy", "pandas", "pyproj", "rasterio", "pyarrow", "matplotlib")},
+ "backends": {"ridge": matching.RIDGE_FIT_BACKEND, "acoustic": native_acoustic.ACOUSTIC_BACKEND,
+ "cpa": native_cpa.CPA_BACKEND,
+ "index_payload": "native" if data_access._native_payload_columns is not None else "python"},
+ "environment": {name: value for name, value in sorted(os.environ.items())
+ if name in {"DOPPLERGEIST_CPA_BACKEND", "DOPPLERGEIST_ACOUSTIC_BACKEND",
+ "DOPPLERGEIST_RIDGE_BACKEND",
+ "OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS",
+ "NUMEXPR_NUM_THREADS", "VECLIB_MAXIMUM_THREADS", "PYTHONHASHSEED"}},
+ }
+ for name in ("ffmpeg", "ffprobe"):
+ result[name] = subprocess.run([name, "-version"], check=True,
+ capture_output=True, text=True, timeout=10).stdout.splitlines()[0]
+ return result
+
+
+def operator_inputs(operator, additional):
+ """Pin local data named by the operator, without following active job state."""
+ if operator.get("offline", True) is not True:
+ raise ValueError("Use an offline operator with frozen historical data and terrain")
+ paths = set()
+ for value in operator.get("flight_indexes", []):
+ paths.add(Path(value).resolve(strict=True))
+ ground = operator.get("terrain", {})
+ for value in ground.get("rasters", []):
+ paths.add(Path(value).resolve(strict=True))
+ if ground.get("geoid_raster"):
+ paths.add(Path(ground["geoid_raster"]).resolve(strict=True))
+ if operator.get("cache_root"):
+ cache = Path(operator["cache_root"]).resolve(strict=True)
+ paths.update(p.absolute() for p in cache.rglob("*") if p.is_file())
+ paths.update(Path(p).resolve(strict=True) for p in additional)
+ if not paths:
+ raise ValueError("No immutable historical/terrain inputs were supplied")
+ return [identity(path) for path in sorted(paths)]
+
+
+def prepare(audio, request, operator, output, *, source_note, scope_note,
+ additional_inputs=(), job_max_runtime_s=None, http_timeout_s=None):
+ audio = Path(audio).resolve(strict=True)
+ operator = deepcopy(operator)
+ for name in ("flight_indexes",):
+ if name in operator:
+ operator[name] = [str(Path(p).resolve(strict=True)) for p in operator[name]]
+ ground = operator.get("terrain", {})
+ if "rasters" in ground:
+ ground["rasters"] = [str(Path(p).resolve(strict=True)) for p in ground["rasters"]]
+ if ground.get("geoid_raster"):
+ ground["geoid_raster"] = str(Path(ground["geoid_raster"]).resolve(strict=True))
+ if operator.get("cache_root"):
+ operator["cache_root"] = str(Path(operator["cache_root"]).resolve(strict=True))
+ if set(operator.get("discovery", {})) - {"chunk_s", "overlap_s"}:
+ raise ValueError("The upload operator exposes only discovery chunk_s and overlap_s")
+ protocol = read_json(PROTOCOL)
+ for key, override in (("job_max_runtime_s", job_max_runtime_s),
+ ("http_timeout_s", http_timeout_s)):
+ if override is not None:
+ if not math.isfinite(override) or override <= 0:
+ raise ValueError(f"{key} must be finite and positive")
+ protocol["budgets"][key] = float(override)
+ if protocol["budgets"]["job_max_runtime_s"] >= protocol["budgets"]["http_timeout_s"]:
+ raise ValueError("The client deadline must exceed the job runtime limit")
+ if not source_note.strip() or not scope_note.strip():
+ raise ValueError("Declare the real source and the evaluation scope")
+ request = dict(validate_request(dict(request, audio_path=str(audio))))
+ request.pop("audio_path")
+ shifted_request(request, 0)
+ DiscoverySettings(**operator.get("discovery", {}))
+ inputs = operator_inputs(operator, additional_inputs)
+ cases = []
+ for case in protocol["cases"]:
+ settings = dict(protocol["baseline"], **case["changes"])
+ arm_operator = deepcopy(operator)
+ if "discovery" in settings:
+ arm_operator["discovery"] = settings.pop("discovery")
+ DiscoverySettings(**arm_operator.get("discovery", {}))
+ cases.append({"id": case["id"], "encoding": settings,
+ "request": shifted_request(request, settings["leading_silence_s"]),
+ "operator": arm_operator})
+ plan = {"schema_version": 1, "prepared_utc": http_verifier.now(),
+ "protocol": protocol, "protocol_sha256": http_verifier.sha256(PROTOCOL),
+ "source": identity(audio), "source_note": source_note, "scope_note": scope_note,
+ "request": request, "operator": operator, "inputs": inputs, "cases": cases,
+ "additional_inputs": [str(Path(p).resolve(strict=True)) for p in additional_inputs],
+ "implementation": implementation(),
+ "source_commit": subprocess.run(["git", "rev-parse", "HEAD"], cwd=ROOT,
+ check=True, capture_output=True, text=True).stdout.strip(),
+ "truth_evaluation": {"eligible": False, "evaluated": False,
+ "reason": "No independent truth supplied to this protocol"},
+ "national_gate_satisfied": False}
+ output = Path(output).resolve()
+ output.mkdir(parents=True, exist_ok=False)
+ with (output / "plan.json").open("x") as stream:
+ json.dump(plan, stream, indent=2, allow_nan=False)
+ stream.write("\n")
+ (output / "plan.sha256").write_text(http_verifier.sha256(output / "plan.json") + "\n")
+ return output / "plan.json"
+
+
+def check_frozen(plan_path, plan):
+ expected = plan_path.with_name("plan.sha256").read_text().strip()
+ if http_verifier.sha256(plan_path) != expected:
+ raise ValueError("The frozen plan changed; prepare a new experiment")
+ if implementation() != plan["implementation"]:
+ raise ValueError("Implementation, runtime or backend configuration changed after preparation")
+ if identity(plan["source"]["path"]) != plan["source"]:
+ raise ValueError("The recorded source changed")
+ if operator_inputs(plan["operator"], plan["additional_inputs"]) != plan["inputs"]:
+ raise ValueError("Pinned data bytes, paths or cache file set changed")
+
+
+def run_command(command, directory, name, timeout_s, *, env=None):
+ write_json(directory / f"{name}-command.json", command)
+ with (directory / f"{name}.log").open("wb") as log:
+ process = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT,
+ env=env, start_new_session=True)
+ try:
+ returncode = process.wait(timeout=timeout_s)
+ if returncode:
+ raise subprocess.CalledProcessError(returncode, command)
+ except BaseException:
+ # Intake starts FFmpeg. Stop this command's whole process group.
+ try:
+ os.killpg(process.pid, signal.SIGTERM)
+ except ProcessLookupError:
+ pass
+ try:
+ process.wait(timeout=2)
+ except subprocess.TimeoutExpired:
+ os.killpg(process.pid, signal.SIGKILL)
+ process.wait()
+ raise
+
+
+def preflight_decode(audio, directory, protocol):
+ """Use actual intake in a bounded child; HTTP intake is still fresh."""
+ command = [sys.executable, "-c",
+ "import sys; from dopplergeist.intake import intake_recording; "
+ "intake_recording(sys.argv[1], sys.argv[2], sample_rate_hz=int(sys.argv[3]))",
+ str(audio), str(directory), str(protocol["pcm_comparison"]["sample_rate_hz"])]
+ env = dict(os.environ, PYTHONPATH=str(Path(dopplergeist.__file__).resolve().parent.parent)
+ + os.pathsep + os.environ.get("PYTHONPATH", ""))
+ run_command(command, directory.parent, directory.name + "-decode",
+ protocol["budgets"]["encoding_timeout_s"], env=env)
+ return IntakeRecording(directory / "recording.f32", read_json(directory / "recording.json"),
+ read_json(directory / "intake.json"))
+
+
+def make_canonical(source, directory, protocol):
+ probe = probe_media(source)
+ audio = next(s for s in probe["streams"] if s["codec_type"] == "audio")
+ channels = int(audio["channels"])
+ downmix = "pan=mono|c0=" + "+".join(f"{1/channels:.17g}*c{i}" for i in range(channels))
+ path = directory / "canonical.wav"
+ command = ["ffmpeg", "-nostdin", "-v", "error", "-xerror", "-i", str(source),
+ "-map", "0:a:0", "-vn", "-sn", "-dn", "-af", downmix, "-ac", "1",
+ "-ar", str(protocol["canonical_pcm"]["sample_rate_hz"]),
+ "-c:a", "pcm_f32le", "-map_metadata", "-1", "-rf64", "auto", str(path)]
+ run_command(command, directory, "canonical", protocol["budgets"]["encoding_timeout_s"])
+ reference = preflight_decode(path, directory / "reference-pcm", protocol)
+ return path, reference
+
+
+def encode(canonical, directory, settings, protocol):
+ filters = []
+ if settings["gain_db"]:
+ filters.append(f"volume={10**(settings['gain_db']/20):.17g}:precision=double")
+ if settings["leading_silence_s"]:
+ samples = round(settings["leading_silence_s"] * protocol["canonical_pcm"]["sample_rate_hz"])
+ filters.append(f"adelay={samples}S:all=1")
+ if settings["channels"] == 2:
+ filters.append("pan=stereo|c0=c0|c1=c0")
+ path = directory / "input.mp3"
+ command = ["ffmpeg", "-nostdin", "-v", "error", "-xerror", "-i", str(canonical),
+ "-map", "0:a:0", "-vn", "-sn", "-dn"]
+ if filters:
+ command += ["-af", ",".join(filters)]
+ command += ["-ac", str(settings["channels"]), "-ar", str(settings["sample_rate_hz"]),
+ "-c:a", "libmp3lame", "-b:a", f"{settings['bitrate_kbps']}k",
+ "-write_xing", "1", "-map_metadata", "-1", str(path)]
+ run_command(command, directory, "encode", protocol["budgets"]["encoding_timeout_s"])
+ return path
+
+
+def native_decode_metrics(audio, directory, protocol):
+ """Scan blocks of a temporary native-rate decode; keep no second large PCM."""
+ probe = probe_media(audio)
+ stream = next(s for s in probe["streams"] if s["codec_type"] == "audio")
+ channels = int(stream["channels"])
+ command = ["ffmpeg", "-nostdin", "-v", "error", "-xerror", "-i", str(audio),
+ "-map", "0:a:0", "-f", "f32le", "-c:a", "pcm_f32le", "pipe:1"]
+ peak, frames, channel_difference = 0.0, 0, 0.0
+ with tempfile.TemporaryFile(dir=directory) as raw, (directory / "native-decode.log").open("wb") as log:
+ subprocess.run(command, stdout=raw, stderr=log, check=True,
+ timeout=protocol["budgets"]["encoding_timeout_s"])
+ if raw.tell() % (4 * channels):
+ raise ValueError("Native decode has an incomplete audio frame")
+ raw.seek(0)
+ while len(block := np.fromfile(raw, dtype="<f4", count=65536 * channels)):
+ if not np.isfinite(block).all():
+ raise ValueError("Native decode contains nonfinite samples")
+ values = block.reshape(-1, channels)
+ peak = max(peak, float(np.max(np.abs(values))))
+ if channels == 2:
+ channel_difference = max(channel_difference, float(np.max(np.abs(values[:, 0] - values[:, 1]))))
+ frames += len(values)
+ packets = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "a:0",
+ "-read_intervals", "%+#1", "-show_packets", "-of", "json", str(audio)],
+ check=True, capture_output=True, text=True, timeout=10)
+ return {"sample_rate_hz": int(stream["sample_rate"]), "channels": channels,
+ "sample_count": frames, "duration_s": frames / int(stream["sample_rate"]),
+ "absolute_peak": peak, "identical_channel_max_difference": channel_difference if channels == 2 else None,
+ "first_packet": json.loads(packets.stdout).get("packets", [])[:1],
+ "container_start_time": probe["format"].get("start_time"),
+ "container_duration": probe["format"].get("duration"), "decode_command": command}
+
+
+def compare_pcm(reference_path, variant_path, settings, protocol):
+ config, tol = protocol["pcm_comparison"], protocol["tolerances"]
+ sr = config["sample_rate_hz"]
+ left = np.memmap(reference_path, dtype="<f4", mode="r")
+ right = np.memmap(variant_path, dtype="<f4", mode="r")
+ padding = round(settings["leading_silence_s"] * sr)
+ gain = 10 ** (settings["gain_db"] / 20)
+ count = min(len(left), max(0, len(right) - padding))
+ if count == 0:
+ raise ValueError("No recorded samples remain after the declared padding")
+ squared_error, energy = 0.0, 0.0
+ for first in range(0, count, config["block_samples"]):
+ last = min(count, first + config["block_samples"])
+ a = np.asarray(left[first:last], dtype=np.float64)
+ b = np.asarray(right[first+padding:last+padding], dtype=np.float64) / gain
+ squared_error += float(np.dot(a-b, a-b))
+ energy += float(np.dot(a, a))
+ duration_error = (len(right) - padding - len(left)) / sr
+ nrmse = math.sqrt(squared_error / energy) if energy > 0 else None
+ radius = round(config["lag_search_radius_s"] * sr)
+ width = min(round(config["lag_window_s"] * sr), count - 2*radius)
+ lags = []
+ if width > 0:
+ for first in np.linspace(radius, count-radius-width, config["lag_windows"]).astype(int):
+ a = np.asarray(left[first:first+width], dtype=np.float64)
+ a = a - a.mean()
+ if float(np.dot(a, a)) <= 1e-14:
+ lags.append({"start_s": int(first)/sr, "lag_s": None, "reason": "no_alignment_energy"})
+ continue
+ b = np.asarray(right[first+padding-radius:first+padding+width+radius], dtype=np.float64) / gain
+ values = scipy_signal.correlate(b, a, mode="valid", method="fft")
+ sums, squares = np.r_[0.0, np.cumsum(b)], np.r_[0.0, np.cumsum(b*b)]
+ variance = squares[width:] - squares[:-width] - (sums[width:] - sums[:-width])**2/width
+ normalized = values / np.sqrt(np.maximum(variance, np.finfo(float).tiny))
+ lag = (int(np.argmax(normalized)) - radius) / sr
+ lags.append({"start_s": int(first)/sr, "lag_s": lag})
+ measured = [abs(row["lag_s"]) for row in lags if row["lag_s"] is not None]
+ max_lag = max(measured) if measured else None
+ return {"reference_samples": len(left), "variant_samples": len(right),
+ "compared_samples": count, "padding_samples": padding, "declared_gain": gain,
+ "duration_error_s": duration_error, "gain_corrected_nrmse": nrmse,
+ "fixed_window_lags": lags, "maximum_absolute_lag_s": max_lag,
+ "passed": (abs(duration_error) <= tol["decoded_duration_error_s"]
+ and nrmse is not None and nrmse <= tol["gain_corrected_pcm_nrmse"]
+ and max_lag is not None and max_lag <= tol["residual_alignment_lag_s"])}
+
+
+class MemorySampler:
+ """Sample only this runner's descendants, never another worker's jobs."""
+ def __init__(self):
+ self.stop = threading.Event()
+ self.peak = 0
+ self.samples = 0
+ self.thread = threading.Thread(target=self.sample, daemon=True)
+
+ def sample(self):
+ while not self.stop.is_set():
+ pending, visited, total = [os.getpid()], set(), 0
+ while pending:
+ pid = pending.pop()
+ if pid in visited:
+ continue
+ visited.add(pid)
+ try:
+ # The HTTP service can spawn a worker from a non-main thread.
+ for children in Path(f"/proc/{pid}/task").glob("*/children"):
+ pending += [int(x) for x in children.read_text().split()]
+ if pid != os.getpid():
+ for line in Path(f"/proc/{pid}/status").read_text().splitlines():
+ if line.startswith("VmRSS:"):
+ total += int(line.split()[1]) * 1024
+ except (OSError, ValueError):
+ pass
+ self.peak = max(self.peak, total)
+ self.samples += 1
+ self.stop.wait(0.1)
+
+ def __enter__(self):
+ self.thread.start()
+ return self
+
+ def __exit__(self, *_):
+ self.stop.set()
+ self.thread.join()
+
+ def result(self):
+ return {"sampled_descendant_peak_rss_mib": self.peak/(1024**2) if self.peak else None,
+ "interval_s": 0.1, "samples": self.samples,
+ "scope": "Sum of local service and worker descendant RSS; sampled lower bound, not unique physical memory"}
+
+
+def mapped_events(events, padding):
+ result = deepcopy(events)
+ for event in result:
+ event["start_s"] -= padding
+ event["end_s"] -= padding
+ for sample in event["ridge"]:
+ sample["time_s"] -= padding
+ return result
+
+
+def observe_case(directory, case, receipt):
+ job_id = receipt.get("job_id")
+ job = directory / "http" / "jobs" / job_id if job_id else None
+ final = read_json(directory / "http/result.json", {})
+ if not final and job is not None:
+ final = read_json(job / "result.json", {})
+ discovery = read_json(job / "discovery/discovery.json", {}) if job else {}
+ events = final.get("events", discovery.get("events", []))
+ search = read_json(job / "search/search-report.json", {}) if job else {}
+ refined = read_json(job / "refined-candidates.json", {}) if job else {}
+ decision = read_json(job / "decision.json", {}) if job else {}
+ evidence = read_json(directory / "http/pipeline-evidence.json", {})
+ padding = case["encoding"]["leading_silence_s"]
+ candidates = []
+ groups = {event["event_id"]: event.get("harmonic_group_id", event["event_id"]) for event in events}
+ for i, candidate in enumerate(final.get("candidates", [])):
+ row = refined.get("ranked", [])[i] if i < len(refined.get("ranked", [])) else {}
+ copied = deepcopy(candidate)
+ copied["mapped_recording_start_utc"] = format_utc(
+ parse_utc(candidate["recording_start_utc"]) + timedelta(seconds=padding))
+ copied["matched_group_count"] = len({groups[e] for e in candidate["event_ids"] if e in groups})
+ copied["normalized_score"] = candidate["score"] / max(1, copied["matched_group_count"])
+ sensitivity = row.get("sensitivity", {})
+ copied["uncertainty_provenance"] = {key: sensitivity.get(key) for key in
+ ("kind", "method", "sources", "complete", "boundary_limited")}
+ copied["uncertainty_radius_m"] = sensitivity.get("radius_m")
+ copied["mode_id"] = row.get("mode_id")
+ candidates.append(copied)
+ keys = ("status", "tiles_total", "tiles_completed", "tiles_resumed", "unsearched_tiles",
+ "grid_points_searched", "refinement_points_searched", "refinement_points_skipped",
+ "refinement_stages_resumed", "model_assessment_complete", "model_support_complete",
+ "data_scope_complete", "all_event_groups_considered", "event_groups_proposed",
+ "alternative_modes_exhaustively_assessed", "elapsed_s", "process_peak_rss_mib",
+ "bounded_concurrency")
+ return {"job_id": job_id, "job_directory": str(job) if job else None,
+ "status": final.get("status"), "reason_code": final.get("reason_code"),
+ "stage": final.get("stage"), "accepted_location": final.get("status") == "succeeded",
+ "verification_passed": receipt.get("verification_passed", False),
+ "full_pipeline_observed": receipt.get("full_pipeline_observed", False),
+ "full_pipeline_requirements_met": receipt.get("full_pipeline_requirements_met", False),
+ "stages_verified": evidence.get("stages_verified", []),
+ "http_elapsed_s": receipt.get("elapsed_s"), "interrupted": receipt.get("interrupted", False),
+ "verification_errors": {key: receipt[key] for key in
+ ("error", "finalization_error", "requirement_error", "pipeline_evidence_errors") if key in receipt},
+ "mapped_events": mapped_events(events, padding),
+ "ridge_count": len(events), "group_count": len(set(groups.values())),
+ "candidates": candidates, "coverage": final.get("coverage"),
+ "work": {key: search.get(key) for key in keys},
+ "finite_search_evidence": search.get("finite_search_evidence"),
+ "decision_provenance": decision.get("diagnostics"),
+ "acquisition": read_json(job / "data-provenance.json", {}) if job else {}}
+
+
+def assign_pairs(left_count, right_count, eligible):
+ """Maximum-cardinality, then minimum-cost one-to-one correspondence."""
+ if not left_count or not right_count:
+ return []
+ penalty = 10 * (left_count + right_count + 1)
+ costs = np.full((left_count, right_count+left_count), float(penalty))
+ costs[:, :right_count] = 3 * penalty
+ for (i, j), row in eligible.items():
+ costs[i, j] = row["cost"]
+ rows, columns = linear_sum_assignment(costs)
+ return [(int(i), int(j)) for i, j in zip(rows, columns) if (i, j) in eligible]
+
+
+def event_correspondence(left, right, protocol):
+ rules = protocol["event_correspondence"]
+ eligible = {}
+ for i, a in enumerate(left):
+ for j, b in enumerate(right):
+ first, last = max(a["start_s"], b["start_s"]), min(a["end_s"], b["end_s"])
+ overlap = max(0.0, last-first) / min(a["end_s"]-a["start_s"], b["end_s"]-b["start_s"])
+ if overlap < rules["minimum_overlap_fraction_of_shorter_event"]:
+ continue
+ ta, fa = np.array([(r["time_s"], r["frequency_hz"]) for r in a["ridge"]]).T
+ tb, fb = np.array([(r["time_s"], r["frequency_hz"]) for r in b["ridge"]]).T
+ mask = (ta >= tb[0]) & (ta <= tb[-1])
+ if not mask.any():
+ continue
+ frequency = float(np.median(np.abs(fa[mask] - np.interp(ta[mask], tb, fb))))
+ if frequency > rules["maximum_matching_median_frequency_hz"]:
+ continue
+ eligible[i, j] = {"left_event_id": a["event_id"], "right_event_id": b["event_id"],
+ "left_group_id": a.get("harmonic_group_id", a["event_id"]),
+ "right_group_id": b.get("harmonic_group_id", b["event_id"]),
+ "start_delta_s": b["start_s"]-a["start_s"],
+ "end_delta_s": b["end_s"]-a["end_s"],
+ "overlap_fraction": overlap, "median_frequency_delta_hz": frequency,
+ "compared_ridge_samples": int(mask.sum()),
+ "cost": 1-overlap + frequency/rules["maximum_matching_median_frequency_hz"]}
+ matches = [eligible[pair] for pair in assign_pairs(len(left), len(right), eligible)]
+ left_ids = {row["left_event_id"] for row in matches}
+ right_ids = {row["right_event_id"] for row in matches}
+ forward, reverse, possible_forward, possible_reverse = {}, {}, {}, {}
+ for rows, fwd, rev in ((matches, forward, reverse),
+ (eligible.values(), possible_forward, possible_reverse)):
+ for row in rows:
+ fwd.setdefault(row["left_group_id"], set()).add(row["right_group_id"])
+ rev.setdefault(row["right_group_id"], set()).add(row["left_group_id"])
+ return {"matches": matches,
+ "unmatched_left": [e["event_id"] for e in left if e["event_id"] not in left_ids],
+ "unmatched_right": [e["event_id"] for e in right if e["event_id"] not in right_ids],
+ "unmatched_left_groups": sorted({e.get("harmonic_group_id", e["event_id"]) for e in left} - forward.keys()),
+ "unmatched_right_groups": sorted({e.get("harmonic_group_id", e["event_id"]) for e in right} - reverse.keys()),
+ "split_groups": {key: sorted(value) for key, value in forward.items() if len(value) > 1},
+ "merged_groups": {key: sorted(value) for key, value in reverse.items() if len(value) > 1},
+ "possible_split_or_ambiguous_groups": {key: sorted(value) for key, value in possible_forward.items() if len(value) > 1},
+ "possible_merge_or_ambiguous_groups": {key: sorted(value) for key, value in possible_reverse.items() if len(value) > 1}}
+
+
+def distance(a, b):
+ return float(GEOD.inv(a["lon"], a["lat"], b["lon"], b["lat"])[2])
+
+
+def associations_equal(a, b, mapping):
+ first = list(zip(a["event_ids"], a["flight_ids"]))
+ if any(event not in mapping for event, _ in first):
+ return False
+ return sorted((mapping[event], flight) for event, flight in first) == sorted(zip(b["event_ids"], b["flight_ids"]))
+
+
+def compare_cases(left, right, protocol):
+ a, b = left["observation"], right["observation"]
+ tol = protocol["tolerances"]
+ correspondence = event_correspondence(a["mapped_events"], b["mapped_events"], protocol)
+ mapping = {m["left_event_id"]: m["right_event_id"] for m in correspondence["matches"]}
+ result = {"left": left["id"], "right": right["id"], "event_correspondence": correspondence,
+ "representation_usable": bool(left.get("usability", {}).get("passed") and right.get("usability", {}).get("passed")),
+ "complete_pipelines": bool(a["full_pipeline_requirements_met"] and b["full_pipeline_requirements_met"]),
+ "status_transition": [a["status"], b["status"]],
+ "reason_transition": [a["reason_code"], b["reason_code"]],
+ "group_count_delta": b["group_count"] - a["group_count"],
+ "ridge_count_delta": b["ridge_count"] - a["ridge_count"],
+ "coverage_equal": a["coverage"] == b["coverage"],
+ "coverage_transition": [a["coverage"], b["coverage"]],
+ "work_transition": [a["work"], b["work"]],
+ "decision_provenance_equal": a["decision_provenance"] == b["decision_provenance"],
+ "diagnostic_candidate_comparison": None, "accepted_location_displacement_m": None,
+ "serious_accepted_jump": False, "violations": []}
+ checks = {
+ "status_changed": a["status"] != b["status"],
+ "reason_changed": a["reason_code"] != b["reason_code"],
+ "group_count_changed": a["group_count"] != b["group_count"],
+ "ridge_correspondence_incomplete": bool(correspondence["unmatched_left"] or correspondence["unmatched_right"]),
+ "group_split_or_merge": bool(correspondence["split_groups"] or correspondence["merged_groups"]),
+ "mapped_event_edges": any(max(abs(m["start_delta_s"]), abs(m["end_delta_s"])) > tol["mapped_event_edge_s"]
+ for m in correspondence["matches"]),
+ "mapped_ridge_frequency": any(m["median_frequency_delta_hz"] > tol["median_ridge_frequency_hz"]
+ for m in correspondence["matches"]),
+ }
+ if a["candidates"] and b["candidates"]:
+ best_a, best_b = a["candidates"][0], b["candidates"][0]
+ displacement = distance(best_a, best_b)
+ clock_delta = abs((parse_utc(best_b["mapped_recording_start_utc"])
+ - parse_utc(best_a["mapped_recording_start_utc"])).total_seconds())
+ same_associations = associations_equal(best_a, best_b, mapping)
+ same_units = best_a["score_kind"] == best_b["score_kind"]
+ comparable = same_associations and same_units and best_a["matched_group_count"] == best_b["matched_group_count"]
+ score_delta = abs(best_a["normalized_score"] - best_b["normalized_score"]) if comparable else None
+ result["diagnostic_candidate_comparison"] = {
+ "displacement_m": displacement, "within_threshold_m": {
+ str(int(limit)): displacement <= limit for limit in tol["displacement_thresholds_m"]},
+ "mapped_clock_delta_s": clock_delta, "associated_flights_equal": same_associations,
+ "score_units_equal": same_units, "normalized_score_comparable": comparable,
+ "normalized_score_delta": score_delta,
+ "uncertainty_provenance_equal": best_a["uncertainty_provenance"] == best_b["uncertainty_provenance"],
+ "uncertainty_radius_m": [best_a["uncertainty_radius_m"], best_b["uncertainty_radius_m"]]}
+ checks.update(associated_flights_changed=not same_associations, score_units_changed=not same_units,
+ normalized_score_changed=score_delta is not None and score_delta > tol["normalized_score_per_matched_group"],
+ fitted_clock_changed=clock_delta > tol["mapped_clock_s"],
+ diagnostic_candidate_jump=displacement > tol["accepted_jump_m"],
+ uncertainty_provenance_changed=best_a["uncertainty_provenance"] != best_b["uncertainty_provenance"])
+ if a["accepted_location"] and b["accepted_location"]:
+ result["accepted_location_displacement_m"] = displacement
+ result["serious_accepted_jump"] = displacement > tol["accepted_jump_m"]
+ eligible = {}
+ for i, first in enumerate(a["candidates"]):
+ for j, second in enumerate(b["candidates"]):
+ if associations_equal(first, second, mapping):
+ separation = distance(first, second)
+ clock = abs((parse_utc(first["mapped_recording_start_utc"])
+ - parse_utc(second["mapped_recording_start_utc"])).total_seconds())
+ same_units = first["score_kind"] == second["score_kind"]
+ if separation <= tol["distant_mode_m"] and clock <= tol["mapped_clock_s"] and same_units:
+ eligible[i, j] = {"cost": separation/tol["distant_mode_m"] + clock/tol["mapped_clock_s"],
+ "distance_m": separation, "mapped_clock_delta_s": clock,
+ "score_kind": first["score_kind"],
+ "normalized_score_delta": abs(first["normalized_score"]-second["normalized_score"])
+ if first["matched_group_count"] == second["matched_group_count"] else None,
+ "uncertainty_provenance_equal": first["uncertainty_provenance"] == second["uncertainty_provenance"]}
+ pairs = assign_pairs(len(a["candidates"]), len(b["candidates"]), eligible)
+ result["retained_mode_correspondence"] = [
+ dict(left_index=i, right_index=j, **{key: value for key, value in eligible[i, j].items() if key != "cost"})
+ for i, j in pairs]
+ result["unmatched_retained_left_indices"] = sorted(set(range(len(a["candidates"]))) - {i for i, _ in pairs})
+ result["unmatched_retained_right_indices"] = sorted(set(range(len(b["candidates"]))) - {j for _, j in pairs})
+ checks["retained_mode_score_changed"] = any(row["normalized_score_delta"] is not None
+ and row["normalized_score_delta"] > tol["normalized_score_per_matched_group"]
+ for row in result["retained_mode_correspondence"])
+ checks["retained_mode_uncertainty_provenance_changed"] = any(
+ not row["uncertainty_provenance_equal"] for row in result["retained_mode_correspondence"])
+ distant = []
+ for label, source, matched in (("left", a, {i for i, _ in pairs}),
+ ("right", b, {j for _, j in pairs})):
+ for index, candidate in enumerate(source["candidates"]):
+ offset = distance(source["candidates"][0], candidate)
+ if offset >= tol["distant_mode_m"]:
+ distant.append({"side": label, "candidate_index": index,
+ "distance_from_winner_m": offset, "retained_within_1000m": index in matched})
+ result["distant_modes"] = distant
+ checks["distant_mode_lost"] = any(not row["retained_within_1000m"] for row in distant)
+ elif bool(a["candidates"]) != bool(b["candidates"]):
+ checks["candidate_presence_changed"] = True
+ result["violations"] = [name for name, failed in checks.items() if failed]
+ return result
+
+
+def build_report(plan_path, plan, state):
+ cases = state["cases"]
+ observed = [row for row in cases if row.get("observation")]
+ pairs = []
+ for a, b in itertools.combinations(observed, 2):
+ try:
+ pairs.append(compare_cases(a, b, plan["protocol"]))
+ except Exception as exc:
+ pairs.append({"left": a["id"], "right": b["id"],
+ "error": f"{type(exc).__name__}: {exc}",
+ "violations": ["comparison_error"], "serious_accepted_jump": False})
+ completed = [row for row in cases if row["state"] == "finished"]
+ passed = [row for row in completed if row.get("observation", {}).get("verification_passed")]
+ stages = [row for row in passed if row["observation"]["full_pipeline_observed"]]
+ full = [row for row in passed if row["observation"]["full_pipeline_requirements_met"]]
+ violations = [pair for pair in pairs if pair["violations"]]
+ return {"schema_version": 1, "plan": str(plan_path), "plan_sha256": http_verifier.sha256(plan_path),
+ "updated_utc": http_verifier.now(), "source_note": plan["source_note"], "scope_note": plan["scope_note"],
+ "declared_case_count": len(plan["cases"]), "finished_cases": len(completed),
+ "verified_cases": len(passed), "complete_stage_cases": len(stages), "full_pipeline_cases": len(full),
+ "run_state": state["state"], "error": state.get("error"), "cases": cases, "pairwise": pairs,
+ "pairs_with_observed_differences": len(violations),
+ "serious_accepted_jumps": [p for p in pairs if p["serious_accepted_jump"]],
+ "all_complete_usable_and_within_tolerances": (state["state"] == "finished"
+ and len(full) == len(plan["cases"])
+ and all(row.get("usability", {}).get("passed") for row in cases) and not violations),
+ "truth_evaluation": plan["truth_evaluation"], "national_gate_satisfied": False,
+ "interpretation": "Inconclusive decisions remain inconclusive. Pairwise agreement is not location accuracy. Early exits do not complete the pipeline requirement."}
+
+
+def save_report(plan_path, plan, state, output):
+ report = build_report(plan_path, plan, state)
+ write_json(output / "report.json", report)
+ lines = ["# Real recording representation matrix", "", plan["scope_note"], "",
+ f"Full pipeline observed in {report['full_pipeline_cases']}/{report['declared_case_count']} declared cases.",
+ "No independent truth error was evaluated. National acceptance remains open.", "",
+ "| Case | State | Status / reason | Groups / ridges | Usable | Full pipeline | HTTP seconds |",
+ "| --- | --- | --- | --- | --- | --- | --- |"]
+ for row in state["cases"]:
+ obs = row.get("observation", {})
+ seconds = obs.get("http_elapsed_s")
+ lines.append(f"| {row['id']} | {row['state']} | {obs.get('status')} / {obs.get('reason_code')} | "
+ f"{obs.get('group_count')} / {obs.get('ridge_count')} | {row.get('usability', {}).get('passed')} | "
+ f"{obs.get('full_pipeline_requirements_met')} | {round(seconds, 2) if seconds is not None else 'unavailable'} |")
+ lines += ["", f"Pairs with observed differences: {report['pairs_with_observed_differences']}.",
+ f"Serious accepted location jumps: {len(report['serious_accepted_jumps'])}.", "",
+ "The JSON report retains mapped ridges, correspondence, diagnostic candidates,",
+ "coverage/work transitions, uncertainty provenance, and per-arm reproducers."]
+ if state.get("error"):
+ lines += ["", "Run error: " + state["error"]]
+ (output / "report.md").write_text("\n".join(lines) + "\n")
+ return report
+
+
+def run(plan_path, output, *, selected_case=None):
+ plan_path, output = Path(plan_path).resolve(strict=True), Path(output).resolve()
+ plan = read_json(plan_path)
+ check_frozen(plan_path, plan)
+ if "job_max_runtime_s" not in inspect.signature(http_verifier.verify_http).parameters:
+ raise ValueError("This runner needs the HTTP verifier's job_max_runtime_s API")
+ if selected_case and selected_case not in {case["id"] for case in plan["cases"]}:
+ raise ValueError("Unknown case identifier")
+ output.mkdir(parents=True, exist_ok=False)
+ state = {"state": "running", "runner_pid": os.getpid(), "started_utc": http_verifier.now(),
+ "cases": [{"id": case["id"], "state": "pending"} for case in plan["cases"]]}
+ write_json(output / "state.json", state)
+ save_report(plan_path, plan, state, output)
+
+ def interrupted(signum, _frame):
+ raise KeyboardInterrupt(f"Received {signal.Signals(signum).name}")
+
+ previous = signal.signal(signal.SIGTERM, interrupted)
+ active = None
+ try:
+ canonical, reference = make_canonical(plan["source"]["path"], output, plan["protocol"])
+ write_json(output / "canonical.json", identity(canonical))
+ for case, row in zip(plan["cases"], state["cases"]):
+ if selected_case and case["id"] != selected_case:
+ row["state"] = "not_selected"
+ continue
+ check_frozen(plan_path, plan)
+ active = row
+ directory = output / case["id"]
+ directory.mkdir()
+ row.update(state="running", directory=str(directory),
+ reproducer=shlex.join([sys.executable, str(Path(__file__).resolve()), "run",
+ "--plan", str(plan_path), "--output", str(directory)+"-reproducer", "--case", case["id"]]))
+ write_json(output / "state.json", state)
+ try:
+ audio = encode(canonical, directory, case["encoding"], plan["protocol"])
+ row["encoded_input"] = identity(audio)
+ try:
+ decoded = preflight_decode(audio, directory / "preflight", plan["protocol"])
+ native = native_decode_metrics(audio, directory, plan["protocol"])
+ pcm = compare_pcm(reference.samples_path, decoded.samples_path, case["encoding"], plan["protocol"])
+ row["usability"] = {"pcm": pcm, "native_decode": native,
+ "passed": bool(pcm["passed"] and native["absolute_peak"] < plan["protocol"]["tolerances"]["decoded_absolute_peak_exclusive"]
+ and (native["channels"] != 2 or native["identical_channel_max_difference"] == 0.0))}
+ except Exception as exc:
+ row["usability"] = {"passed": False, "error": f"{type(exc).__name__}: {exc}"}
+ baseline_input = state["cases"][0].get("encoded_input")
+ if case["id"].startswith("chunks_") and baseline_input:
+ same_input = row["encoded_input"]["sha256"] == baseline_input["sha256"]
+ row["usability"]["chunk_input_matches_baseline"] = same_input
+ row["usability"]["passed"] = bool(row["usability"]["passed"] and same_input)
+ write_json(directory / "representation.json", {"encoding": case["encoding"],
+ "encoded_input": row["encoded_input"], "usability": row["usability"],
+ "request": case["request"], "operator": case["operator"]})
+ # Failed usability checks do not remove encoded arms or cause
+ # outcome-dependent settings changes. The upload remains fresh.
+ with MemorySampler() as memory:
+ receipt = http_verifier.verify_http(audio, directory / "http",
+ request=case["request"], operator=case["operator"], expect_status=None,
+ job_max_runtime_s=plan["protocol"]["budgets"]["job_max_runtime_s"],
+ timeout_s=plan["protocol"]["budgets"]["http_timeout_s"],
+ require_full_pipeline=False)
+ row["memory"] = memory.result()
+ row["observation"] = observe_case(directory, case, receipt)
+ row["state"] = "finished"
+ if receipt.get("interrupted"):
+ raise KeyboardInterrupt("HTTP verification was interrupted")
+ except Exception as exc:
+ row.update(state="error", error=f"{type(exc).__name__}: {exc}")
+ if (directory / "http/http-verification.json").is_file() and "observation" not in row:
+ try:
+ row["observation"] = observe_case(directory, case,
+ read_json(directory / "http/http-verification.json"))
+ except Exception as parse_error:
+ row["observation_error"] = f"{type(parse_error).__name__}: {parse_error}"
+ finally:
+ write_json(output / "state.json", state)
+ save_report(plan_path, plan, state, output)
+ print(json.dumps({"case": row["id"], "state": row["state"],
+ "status": row.get("observation", {}).get("status"),
+ "reason": row.get("observation", {}).get("reason_code"),
+ "representation_usable": row.get("usability", {}).get("passed")}), flush=True)
+ active = None
+ check_frozen(plan_path, plan)
+ state["state"] = "finished"
+ except BaseException as exc:
+ state.update(state="interrupted" if isinstance(exc, KeyboardInterrupt) else "error",
+ error=f"{type(exc).__name__}: {exc}")
+ if active is not None and active["state"] == "running":
+ active.update(state="interrupted", error=state["error"])
+ finally:
+ signal.signal(signal.SIGTERM, previous)
+ write_json(output / "state.json", state)
+ report = save_report(plan_path, plan, state, output)
+ return report
+
+
+def main(argv=None):
+ parser = argparse.ArgumentParser(description=__doc__)
+ commands = parser.add_subparsers(dest="command", required=True)
+ pre = commands.add_parser("prepare", help="Freeze a protocol before executing its cases")
+ pre.add_argument("--audio", type=Path, required=True)
+ pre.add_argument("--request", type=Path, required=True)
+ pre.add_argument("--operator", type=Path, required=True)
+ pre.add_argument("--output", type=Path, required=True)
+ pre.add_argument("--source-note", required=True)
+ pre.add_argument("--scope-note", required=True)
+ pre.add_argument("--frozen-input", type=Path, action="append", default=[])
+ pre.add_argument("--job-max-runtime-s", type=float)
+ pre.add_argument("--http-timeout-s", type=float)
+ execute = commands.add_parser("run", help="Execute fresh jobs; never resume old scientific work")
+ execute.add_argument("--plan", type=Path, required=True)
+ execute.add_argument("--output", type=Path, required=True)
+ execute.add_argument("--case", help="One-arm reproducer; never counts as the full matrix")
+ report = commands.add_parser("report", help="Compare preserved observations without executing the pipeline")
+ report.add_argument("--plan", type=Path, required=True)
+ report.add_argument("--output", type=Path, required=True)
+ args = parser.parse_args(argv)
+ try:
+ if args.command == "prepare":
+ path = prepare(args.audio, read_json(args.request), read_json(args.operator), args.output,
+ source_note=args.source_note, scope_note=args.scope_note, additional_inputs=args.frozen_input,
+ job_max_runtime_s=args.job_max_runtime_s, http_timeout_s=args.http_timeout_s)
+ print(path)
+ return 0
+ if args.command == "run":
+ result = run(args.plan, args.output, selected_case=args.case)
+ else:
+ plan_path = args.plan.resolve(strict=True)
+ plan = read_json(plan_path)
+ if http_verifier.sha256(plan_path) != plan_path.with_name("plan.sha256").read_text().strip():
+ raise ValueError("The frozen plan changed")
+ if http_verifier.sha256(__file__) != plan["implementation"]["runner_sha256"]:
+ raise ValueError("Use the frozen runner to reproduce these comparisons")
+ result = save_report(plan_path, plan, read_json(args.output / "state.json"), args.output)
+ print(json.dumps({key: result[key] for key in
+ ("run_state", "finished_cases", "full_pipeline_cases", "pairs_with_observed_differences",
+ "all_complete_usable_and_within_tolerances")}))
+ return 0 if result["all_complete_usable_and_within_tolerances"] else 1
+ except (OSError, ValueError, TypeError, subprocess.SubprocessError) as exc:
+ parser.error(str(exc))
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/scripts/verify_upload_http.py b/scripts/verify_upload_http.py
index c17bd04..1c62af3 100644
--- a/scripts/verify_upload_http.py
+++ b/scripts/verify_upload_http.py
@@ -6,7 +6,8 @@ The receipt separately preserves the producer's scientific status. An expected
inconclusive response is never counted as supported localization or accuracy.
With --require-full-pipeline, verification also requires every pipeline stage
and at least one returned candidate; an early exit cannot pass that requirement.
-The existing browser verifier is independent and remains unchanged.
+The browser verifier checks rendering separately. Optional runtime, cancellation,
+and restart controls exercise actual terminal behavior without replacing workers.
"""
from __future__ import annotations
@@ -32,6 +33,9 @@ from dopplergeist.contracts import utc_timestamp, validate_recording, validate_r
from dopplergeist.jobs import TERMINAL, write_json
+CANCELLABLE_STAGES = ("intake", "discovery", "acquisition", "search", "refinement", "decision")
+
+
def _implementation_files(package):
"""Independently cover Python sources, native C, and loaded-format binaries."""
return sorted(path for path in package.iterdir()
@@ -178,7 +182,8 @@ def wait_for_service(process, log_path, deadline):
time.sleep(min(0.05, remaining(deadline)))
-def poll_job(client, job_id, history_path, *, interval_s):
+def poll_job(client, job_id, history_path, *, interval_s, cancel_at_stage=None, cancellation=None):
+ cancellation = {} if cancellation is None else cancellation
with history_path.open("x") as history:
previous = None
while True:
@@ -191,10 +196,75 @@ def poll_job(client, job_id, history_path, *, interval_s):
history.flush()
previous = value
if value["status"] in TERMINAL:
+ if cancel_at_stage is not None and not cancellation:
+ raise ValueError(f"Job ended before cancellation stage {cancel_at_stage!r} was observed")
return value
+ if (cancel_at_stage is not None and not cancellation
+ and value["status"] == "running" and value.get("stage") == cancel_at_stage):
+ cancellation.update(observed_status=value["status"], observed_stage=value["stage"],
+ requested_utc=now())
+ code, reply = client.request("POST", f"/api/jobs/{job_id}/cancel")
+ cancellation["http_status"] = code
+ if code != 200:
+ raise ValueError(f"Cancellation returned HTTP {code}")
+ validate_result(reply)
+ if reply["job_id"] != job_id:
+ raise ValueError("Cancellation returned a different job")
+ cancellation["response_status"] = reply["status"]
time.sleep(min(interval_s, remaining(client.deadline)))
+def inspect_restart(output, command, environment, final, source_sha256, deadline,
+ termination, report, *, repeat_cancel=False):
+ """Restart only this verifier's service and read its completed job again."""
+ directory = output / "jobs" / final["job_id"]
+ result_digest = sha256(directory / "result.json")
+ process = None
+ report.update(passed=False, service={"shutdown_signal": None, "forced_kill": False, "returncode": None})
+ try:
+ with (output / "restart-service.log").open("xb") as log:
+ try:
+ termination.interruptible = False
+ process = subprocess.Popen(command, stdout=log, stderr=subprocess.STDOUT,
+ env=environment, start_new_session=True)
+ report["service"]["pid"] = process.pid
+ termination.interruptible = True
+ termination.raise_if_interrupted()
+ client = wait_for_service(process, output / "restart-service.log", deadline)
+ report["service"]["port"] = client.port
+ job_path = f"/api/jobs/{final['job_id']}"
+ if client.get(job_path) != final:
+ raise ValueError("Restart changed the completed API result")
+ if repeat_cancel:
+ code, cancelled = client.request("POST", job_path + "/cancel")
+ if code != 200 or cancelled != final:
+ raise ValueError("Repeated cancellation changed the completed result")
+ report["terminal_cancel_idempotent"] = True
+ code, _ = client.request("GET", job_path + "/artifacts/input.mp3",
+ destination=output / "downloaded-original-after-restart.mp3")
+ if code != 200 or sha256(output / "downloaded-original-after-restart.mp3") != source_sha256:
+ raise ValueError("Restart changed the original MP3 download")
+ code, _ = client.request("GET", job_path + "/artifacts/result.json",
+ destination=output / "downloaded-result-after-restart.json")
+ if code != 200 or sha256(output / "downloaded-result-after-restart.json") != result_digest:
+ raise ValueError("Restart changed the saved result bytes")
+ if client.get(job_path) != final:
+ raise ValueError("Terminal API result changed during restart readback")
+ report.update(terminal_readback_verified=True, original_download_verified=True,
+ saved_result_bytes_verified=True)
+ finally:
+ termination.interruptible = False
+ stop_service(process, report, termination)
+ if report["service"]["returncode"] != 0 or report["service"]["forced_kill"]:
+ raise ValueError("Restarted service did not stop cleanly")
+ if sha256(directory / "result.json") != result_digest or sha256(directory / "input.mp3") != source_sha256:
+ raise ValueError("Restart shutdown changed completed artifacts")
+ report["passed"] = True
+ finally:
+ termination.interruptible = True
+ termination.raise_if_interrupted()
+
+
def inspect_job(directory, final, wire, operator, original, implementation):
"""Check actual persisted evidence without changing it or opening truth."""
read = lambda path: json.loads((directory / path).read_text())
@@ -1029,13 +1099,25 @@ def inspect_pipeline_evidence(directory, final, wire, operator):
def verify_http(audio, output, *, request=None, operator=None, expect_status,
expect_reason=None, timeout_s=600.0, poll_interval_s=0.25,
- require_full_pipeline=False):
- if expect_status not in TERMINAL:
- raise ValueError("Declare an expected terminal status before execution")
+ require_full_pipeline=False, job_max_runtime_s=None,
+ cancel_at_stage=None, verify_restart=False):
+ """Verify an actual upload; None observes any terminal status for callers.
+
+ A missing job runtime preserves the original timeout_s worker allowance.
+ Give the client a larger timeout to observe an intentional worker time limit.
+ """
+ if expect_status is not None and expect_status not in TERMINAL:
+ raise ValueError("Expected status must be terminal or None for observation")
if type(require_full_pipeline) is not bool:
raise ValueError("require_full_pipeline must be a boolean")
- for name, value in (("timeout_s", timeout_s), ("poll_interval_s", poll_interval_s)):
- if not math.isfinite(value) or value <= 0:
+ if type(verify_restart) is not bool:
+ raise ValueError("verify_restart must be a boolean")
+ if cancel_at_stage is not None and cancel_at_stage not in CANCELLABLE_STAGES:
+ raise ValueError("Cancellation stage must be an actual running pipeline stage")
+ job_runtime = timeout_s if job_max_runtime_s is None else job_max_runtime_s
+ for name, value in (("timeout_s", timeout_s), ("poll_interval_s", poll_interval_s),
+ ("job_max_runtime_s", job_runtime)):
+ if isinstance(value, bool) or not math.isfinite(value) or value <= 0:
raise ValueError(f"{name} must be finite and positive")
audio = Path(audio).resolve(strict=True)
if audio.suffix.lower() != ".mp3" or not audio.is_file():
@@ -1055,7 +1137,9 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,
deadline = began + timeout_s
receipt = {"schema_version": 1, "scope": "actual_local_http_upload_verification",
"started_utc": now(), "expected_status": expect_status, "expected_reason": expect_reason,
- "timeout_s": timeout_s, "source_sha256": sha256(audio), "source_bytes": audio.stat().st_size,
+ "timeout_s": timeout_s, "job_max_runtime_s": job_runtime,
+ "cancel_at_stage": cancel_at_stage, "verify_restart": verify_restart,
+ "source_sha256": sha256(audio), "source_bytes": audio.stat().st_size,
"request": wire, "implementation_files": implementation, "verifier_sha256": sha256(__file__),
"python_executable": sys.executable, "python_version": sys.version,
"verification_passed": False, "http_checks_passed": False, "pipeline_evidence_passed": False,
@@ -1073,7 +1157,7 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,
termination.raise_if_interrupted()
command = [sys.executable, "-m", "dopplergeist.cli", "serve", "--port", "0",
"--jobs", str(output / "jobs"), "--config", str(output / "operator.json"),
- "--max-runtime", str(timeout_s)]
+ "--max-runtime", str(job_runtime)]
receipt["service"]["command"] = command
env = dict(os.environ, PYTHONPATH=str(package.parent) + os.pathsep + os.environ.get("PYTHONPATH", ""),
MPLBACKEND="Agg")
@@ -1085,7 +1169,10 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,
termination.raise_if_interrupted()
client = wait_for_service(process, output / "service.log", deadline)
receipt["service"]["port"] = client.port
- write_json(output / "capabilities.json", client.get("/api/capabilities"))
+ capabilities = client.get("/api/capabilities")
+ write_json(output / "capabilities.json", capabilities)
+ if capabilities.get("max_runtime_s") != job_runtime:
+ raise ValueError("Service did not retain the requested job runtime")
with audio.open("rb") as stream:
code, queued = client.request("POST", "/api/jobs", body=stream, headers={
"Content-Type": "application/octet-stream", "Content-Length": str(audio.stat().st_size),
@@ -1098,11 +1185,20 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,
if uuid.UUID(job_id).hex != job_id:
raise ValueError("Service returned an invalid job identifier")
receipt["job_id"] = job_id
- final = poll_job(client, job_id, output / "history.jsonl", interval_s=poll_interval_s)
+ cancellation = {}
+ if cancel_at_stage is not None:
+ receipt["cancellation"] = cancellation
+ final = poll_job(client, job_id, output / "history.jsonl", interval_s=poll_interval_s,
+ cancel_at_stage=cancel_at_stage, cancellation=cancellation)
write_json(output / "result.json", final)
receipt.update(status=final["status"], reason_code=final.get("reason_code"),
producer_reported_success=final["status"] == "succeeded",
candidate_count=len(final.get("candidates", [])), event_count=len(final.get("events", [])))
+ if cancel_at_stage is not None:
+ cancellation["confirmed"] = (cancellation.get("response_status") == "cancelled"
+ and final["status"] == "cancelled")
+ if not cancellation["confirmed"]:
+ raise ValueError("Cancellation was requested, but the job did not finish cancelled")
listing = client.get(f"/api/jobs/{job_id}/artifacts")
write_json(output / "artifacts.json", listing)
if sum(item["path"] == "input.mp3" for item in listing["artifacts"]) != 1:
@@ -1116,8 +1212,18 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,
receipt["checks"] = inspect_job(output / "jobs" / job_id, final, wire, operator, audio, implementation)
receipt["checks"].update(downloaded_original_verified=True, downloaded_result_verified=True)
receipt["http_checks_passed"] = True
- receipt["expectation_matched"] = (final["status"] == expect_status
+ receipt["expectation_matched"] = ((expect_status is None or final["status"] == expect_status)
and (expect_reason is None or final.get("reason_code") == expect_reason))
+ if verify_restart:
+ termination.interruptible = False
+ stop_service(process, receipt, termination)
+ termination.interruptible = True
+ termination.raise_if_interrupted()
+ if receipt["service"]["returncode"] != 0 or receipt["service"]["forced_kill"]:
+ raise ValueError("Initial service did not stop cleanly before readback")
+ receipt["restart_readback"] = {}
+ inspect_restart(output, command, env, final, receipt["source_sha256"], deadline,
+ termination, receipt["restart_readback"], repeat_cancel=bool(cancellation))
except Exception as exc:
receipt["error"] = f"{type(exc).__name__}: {exc}"
receipt["timed_out"] = isinstance(exc, TimeoutError)
@@ -1156,7 +1262,8 @@ def verify_http(audio, output, *, request=None, operator=None, expect_status,
and receipt.get("pipeline_evidence_preserved_after_inspection") and receipt["implementation_unchanged"]
and receipt.get("terminal_result_preserved_after_shutdown")
and receipt["service"]["returncode"] == 0 and not receipt["service"]["forced_kill"]
- and not receipt.get("shutdown_error") and not receipt["interrupted"])
+ and not receipt.get("shutdown_error") and not receipt["interrupted"]
+ and (not verify_restart or receipt.get("restart_readback", {}).get("passed")))
if require_full_pipeline and not receipt["full_pipeline_requirements_met"]:
receipt["requirement_error"] = (
"Full pipeline verification requires verified acquisition, search, refinement, "
@@ -1193,6 +1300,12 @@ def main(argv=None):
parser.add_argument("--require-full-pipeline", action="store_true",
help="Fail unless every pipeline stage is verified and at least one candidate is returned")
parser.add_argument("--timeout-s", type=float, default=600)
+ parser.add_argument("--job-max-runtime-s", type=float,
+ help="Worker time budget; defaults to --timeout-s. Use a smaller value to verify work_limit")
+ parser.add_argument("--cancel-at-stage", choices=CANCELLABLE_STAGES,
+ help="Request actual API cancellation after this running stage is observed")
+ parser.add_argument("--verify-restart", action="store_true",
+ help="Restart this verifier's service and verify unchanged completed artifacts through HTTP")
parser.add_argument("--poll-interval-s", type=float, default=0.25)
args = parser.parse_args(argv)
if args.end_utc and not args.start_utc:
@@ -1206,7 +1319,8 @@ def main(argv=None):
receipt = verify_http(args.audio, args.output, request=request, operator=operator,
expect_status=args.expect_status, expect_reason=args.expect_reason,
timeout_s=args.timeout_s, poll_interval_s=args.poll_interval_s,
- require_full_pipeline=args.require_full_pipeline)
+ require_full_pipeline=args.require_full_pipeline, job_max_runtime_s=args.job_max_runtime_s,
+ cancel_at_stage=args.cancel_at_stage, verify_restart=args.verify_restart)
except (OSError, ValueError, TypeError) as exc:
parser.error(str(exc))
print(json.dumps({"report": str(args.output.resolve() / "http-verification.json"),
diff --git a/tests/test_real_metamorphic_runner.py b/tests/test_real_metamorphic_runner.py
new file mode 100644
index 0000000..d87d7c2
--- /dev/null
+++ b/tests/test_real_metamorphic_runner.py
@@ -0,0 +1,95 @@
+"""Representation/mapping contracts using archived real ridges and recorded audio."""
+from copy import deepcopy
+import importlib.util
+import json
+import os
+from pathlib import Path
+import sys
+
+import pytest
+
+
+ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT / "scripts"))
+spec = importlib.util.spec_from_file_location("real_metamorphic", ROOT / "scripts/run_real_metamorphic.py")
+runner = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(runner)
+PROTOCOL = json.loads(runner.PROTOCOL.read_text())
+
+
+def recorded_events():
+ return json.loads((ROOT / "tests/data/shared-ridges-real.json").read_text())["shared"]["events"]
+
+
+def test_padding_preserves_both_absolute_utc_endpoints_and_real_ridge_times():
+ request = {"schema_version": 1, "time_window": {
+ "start_utc": "2026-09-08T20:17:20Z", "end_utc": "2026-09-08T20:17:40Z",
+ "source": "user", "source_detail": "Source-informed reference development control"}}
+ shifted = runner.shifted_request(request, 17.25)
+ assert shifted["time_window"]["start_utc"] == "2026-09-08T20:17:02.750000Z"
+ assert shifted["time_window"]["end_utc"] == "2026-09-08T20:17:22.750000Z"
+ for key in ("start_utc", "end_utc"):
+ assert (runner.parse_utc(request["time_window"][key])
+ - runner.parse_utc(shifted["time_window"][key])).total_seconds() == 17.25
+ events = recorded_events()
+ padded = runner.mapped_events(events, -17.25)
+ assert runner.mapped_events(padded, 17.25) == events
+ assert request["time_window"]["start_utc"] == "2026-09-08T20:17:20Z"
+
+
+def test_recorded_ridges_use_one_to_one_correspondence_not_identifiers_or_order():
+ left = recorded_events()
+ right = deepcopy(list(reversed(left)))
+ originals = {}
+ for i, event in enumerate(right):
+ originals[event["event_id"]] = f"renamed-{i}"
+ event["event_id"] = f"renamed-{i}"
+ event["harmonic_group_id"] = f"renamed-group-{i}"
+ result = runner.event_correspondence(left, right, PROTOCOL)
+ assert {row["left_event_id"]: row["right_event_id"] for row in result["matches"]} == originals
+ assert len({row["right_event_id"] for row in result["matches"]}) == len(left)
+ assert not result["unmatched_left"] and not result["unmatched_right"]
+ assert all(row["median_frequency_delta_hz"] == 0 for row in result["matches"])
+ missing = runner.event_correspondence(left, right[:1], PROTOCOL)
+ assert len(missing["matches"]) == 1
+ assert len(missing["unmatched_left"]) == 1
+ assert not missing["unmatched_right"]
+
+
+def test_cache_freeze_detects_an_added_alias_to_existing_real_input(tmp_path):
+ source = ROOT / "tests/data/shared-ridges-real.json"
+ cache = tmp_path / "immutable-cache"
+ cache.mkdir()
+ (cache / "first.json").symlink_to(source)
+ operator = {"cache_root": str(cache), "offline": True}
+ before = runner.operator_inputs(operator, [])
+ (cache / "another-name.json").symlink_to(source)
+ after = runner.operator_inputs(operator, [])
+ assert before != after
+ assert len(before) == 1 and len(after) == 2
+ assert len({row["resolved_path"] for row in after}) == 1
+
+
+def test_actual_recorded_clip_encoding_and_padding_contract(tmp_path):
+ source = os.environ.get("DOPPLERGEIST_REAL_METAMORPHIC_AUDIO")
+ if not source:
+ pytest.skip("Set DOPPLERGEIST_REAL_METAMORPHIC_AUDIO to a short actual recording")
+ canonical, reference = runner.make_canonical(source, tmp_path, PROTOCOL)
+ for name in ("baseline", "identical_stereo", "leading_17_25s", "rate_44100", "gain_minus_6db"):
+ case = next(row for row in PROTOCOL["cases"] if row["id"] == name)
+ settings = dict(PROTOCOL["baseline"], **case["changes"])
+ directory = tmp_path / name
+ directory.mkdir()
+ encoded = runner.encode(canonical, directory, settings, PROTOCOL)
+ decoded = runner.intake_recording(encoded, directory / "preflight", sample_rate_hz=4000)
+ metrics = runner.native_decode_metrics(encoded, directory, PROTOCOL)
+ comparison = runner.compare_pcm(reference.samples_path, decoded.samples_path, settings, PROTOCOL)
+ assert metrics["channels"] == settings["channels"]
+ assert metrics["sample_rate_hz"] == settings["sample_rate_hz"]
+ assert abs(comparison["duration_error_s"]) <= PROTOCOL["tolerances"]["decoded_duration_error_s"]
+ assert comparison["maximum_absolute_lag_s"] <= PROTOCOL["tolerances"]["residual_alignment_lag_s"]
+ assert comparison["padding_samples"] == round(settings["leading_silence_s"]*4000)
+ if settings["channels"] == 2:
+ assert metrics["identical_channel_max_difference"] == 0
+ assert metrics["first_packet"]
+ # Transform contracts are not full-pipeline or one-hour matrix evidence.
diff --git a/tests/test_real_upload_terminal_controls.py b/tests/test_real_upload_terminal_controls.py
new file mode 100644
index 0000000..d2fc631
--- /dev/null
+++ b/tests/test_real_upload_terminal_controls.py
@@ -0,0 +1,66 @@
+"""Actual upload cancellation controls; require an existing real-recording job.
+
+The race control delays only the outgoing cancel request until the real worker
+finishes. All audio, worker execution and HTTP responses remain actual inputs.
+"""
+import importlib.util
+import json
+import os
+from pathlib import Path
+import time
+
+import pytest
+
+
+ROOT = Path(__file__).parents[1]
+SPEC = importlib.util.spec_from_file_location("verify_real_terminal_controls",
+ ROOT / "scripts/verify_upload_http.py")
+HTTP = importlib.util.module_from_spec(SPEC)
+SPEC.loader.exec_module(HTTP)
+
+
[email protected]("delay_cancel", [False, True], ids=["cancel-running", "completion-wins-race"])
+def test_actual_cancellation_must_be_confirmed_even_without_expected_status(tmp_path, monkeypatch, delay_cancel):
+ saved = os.environ.get("DOPPLERGEIST_REAL_UPLOAD_JOB")
+ if not saved:
+ pytest.skip("Set DOPPLERGEIST_REAL_UPLOAD_JOB to a real job with MP3 and recording-start window")
+ source = Path(saved)
+ request = json.loads((source / "request.json").read_text())
+ request.pop("audio_path")
+ assert request.get("time_window"), "The actual recording needs its existing recording-start window"
+ audio = source / "input.mp3"
+ assert HTTP.sha256(audio) == json.loads((source / "input.json").read_text())["sha256"]
+ real_request = HTTP.LocalClient.request
+
+ def request_after_real_completion(client, method, path, **kwargs):
+ if delay_cancel and method == "POST" and path.endswith("/cancel"):
+ while True:
+ status, current = real_request(client, "GET", path.rsplit("/", 1)[0])
+ assert status == 200
+ if current["status"] in HTTP.TERMINAL:
+ break
+ time.sleep(min(0.02, HTTP.remaining(client.deadline)))
+ return real_request(client, method, path, **kwargs)
+
+ monkeypatch.setattr(HTTP.LocalClient, "request", request_after_real_completion)
+ cache = tmp_path / "empty-offline-cache"
+ cache.mkdir()
+ output = tmp_path / "upload"
+ receipt = HTTP.verify_http(audio, output, request=request,
+ operator={"offline": True, "cache_root": str(cache)}, expect_status=None,
+ cancel_at_stage="discovery", job_max_runtime_s=90, timeout_s=120, poll_interval_s=0.05)
+ assert receipt["expected_status"] is None
+ assert receipt["cancellation"]["observed_status"] == "running"
+ assert receipt["cancellation"]["observed_stage"] == "discovery"
+ assert receipt["cancellation"]["http_status"] == 200
+ result = json.loads((output / "result.json").read_text())
+ assert HTTP.sha256(output / "jobs" / result["job_id"] / "input.mp3") == HTTP.sha256(audio)
+ if delay_cancel:
+ assert result["status"] == "failed" and result["reason_code"] == "data_unavailable"
+ assert receipt["cancellation"]["confirmed"] is False
+ assert receipt["verification_passed"] is False
+ assert "did not finish cancelled" in receipt["error"]
+ else:
+ assert result["status"] == "cancelled"
+ assert receipt["cancellation"]["confirmed"] is True
+ assert receipt["verification_passed"] is True