Skip to content

armgit / Cap

Public Imported from GitHub
0

Open source Loom alternative. Beautiful, shareable screen recordings.

test: align final suite expectations with merged fixes

Richie McIlroyeb7c88a77852
diff --git a/crates/verify/expectations.toml b/crates/verify/expectations.toml
index 7f001a2a8..5b1ec5e64 100644
--- a/crates/verify/expectations.toml
+++ b/crates/verify/expectations.toml
@@ -303,9 +303,10 @@ link = ""
[[xfail]]
case = "export/instant/output/mp4/native"
class = "sync-system"
-bound = { count = 1, offset-ms = [5, 34] }
-reason = "NEW: the gate's per-pipeline zero (see export/instant-gated-*): system audio is late by up to one video frame (8.5 ms on Linux; 7.5 ms on macOS and Windows since #2635 keeps the system markers in the mix)"
-link = ""
+flaky = true
+bound = { count = 1, offset-ms = [-2.7, 28.3], mic-offset-ms = [5, 34], system-minus-mic-ms = [-7.7, -5.7] }
+reason = "Instant mixer input timing and the gate phase produce a -6.7 ms system-minus-mic mode, within 1 ms. Only this inter-source signature with the microphone inside its unchanged 5 to 34 ms bound is excused. Coordinator decision 09:20, 2026-10-16."
+link = "https://github.com/CapSoftware/Cap/tree/fix/instant-mixer-input-alignment"
[[xfail]]
case = "export/instant/output/mp4/native"
@@ -679,19 +680,11 @@ bound = { count = 1 }
reason = "NEW (bugs B1 and C, see loss-time): nothing recoverable (no frames)"
link = "https://github.com/CapSoftware/Cap/pull/2643"
-[[xfail]]
-case = "crash/recorder-kill/inproc/*"
-class = "dropped-complete"
-flaky = true
-bound = { count = 1, frames = [1, 60], outside-pending = 0 }
-reason = "Bug A: kill timing can expose a physically complete fragment still listed as pending before the next manifest rescan. Only frames inside those complete pending fragments are excused; at most one 60-frame fragment. Coordinator family decision 2026-10-16. Remove when #2640 lands."
-link = "https://github.com/CapSoftware/Cap/pull/2640"
-
[[xfail]]
case = "export/instant/output/mp4/native"
class = "sync-system"
platforms = ["macos"]
flaky = true
-bound = { count = 1, offset-ms = [-70, -30], mic-offset-ms = [5, 34] }
-reason = "Instant mixer intermittently places system audio about 60 ms before the microphone. Control 38858394655: system -46.1 ms, mic +13.9 ms; candidate 38861420654 and repeat 2: system -45.9 ms. Only this negative system offset with the microphone inside its unchanged bound is excused. Coordinator decision C, 2026-10-16."
-link = "https://github.com/CapSoftware/Cap/blob/ci/verify-ci/crates/verify/investigations/instant-system-audio.md"
+bound = { count = 1, offset-ms = [-56, -25], mic-offset-ms = [5, 34], system-minus-mic-ms = [-61, -59] }
+reason = "Instant mixer input timing produces a -60 ms system-minus-mic mode, within 1 ms. Only this inter-source signature with the microphone inside its unchanged 5 to 34 ms bound is excused. Coordinator decision 09:20, 2026-10-16."
+link = "https://github.com/CapSoftware/Cap/tree/fix/instant-mixer-input-alignment"
diff --git a/crates/verify/investigations/instant-system-audio.md b/crates/verify/investigations/instant-system-audio.md
index 019e31a55..199adc361 100644
--- a/crates/verify/investigations/instant-system-audio.md
+++ b/crates/verify/investigations/instant-system-audio.md
@@ -1,7 +1,7 @@
# Instant system audio investigation
Opened 2026-10-16 by the suite integration executor under coordinator decision C.
-Status: open, product cause not yet isolated. This track is separate from AVFoundation decoding.
+Status: product cause confirmed by the mixer arrival-schedule probe. The fix is tracked on [fix/instant-mixer-input-alignment](https://github.com/CapSoftware/Cap/tree/fix/instant-mixer-input-alignment). This track is separate from AVFoundation decoding.
| Evidence | System offset | Mic offset |
| --- | --- | --- |
@@ -11,8 +11,8 @@ Status: open, product cause not yet isolated. This track is separate from AVFoun
All use fresh fixtures. The coordinator also records #2635 evidence of the mixer placing system audio about 60 ms ahead of the microphone in about half of Instant recordings. The native case reads the pipeline's output.mp4 directly, independently of the product decoder.
-The normal offset bound remains 5 to 34 ms. A separate macOS flaky expectation permits only system offsets -70 to -30 ms, with a measured microphone offset inside its existing 5 to 34 ms bound. Missing measurements, other sources, other offsets, and other classes remain failures.
+The normal offset bound remains 5 to 34 ms. Coordinator decisions at 09:20 and 10:05 replace the default Instant system-sync expectation with two bounded flaky signatures: system minus microphone at -60.0 or -6.7 ms, within 1 ms, with a measured microphone offset inside 5 to 34 ms. The -60 ms signature remains macOS-only; the -6.7 ms signature applies on all platforms. Missing measurements, other offsets and other classes remain failures.
-Next investigation: correlate source capture timestamps, AudioGapTracker overlap/catch-up decisions and mixer output positions on fresh matched runs. Preserve both the negative and positive states. Propose a product fix with matched proof separately; this suite change does not decide that fix or its acceptance.
+AudioMixer feeds each input to amix without per-input timing, making each first sample its own zero. Scheduler-dependent 53.3 ms startup silence partly masks the resulting 60 ms source displacement. The separate product track places each input relative to the mixer start and makes padding timestamp-based; its acceptance is outside this suite integration.
Initial code trace: `crates/verify/src/pipeline.rs` builds the Instant audio pipeline with the same epoch anchoring as the product. `crates/recording/src/output_pipeline/core.rs` attaches sources to `AudioMixer` and then applies `AudioGapTracker` overlap handling to its output. Both the mixer source alignment and later overlap trimming remain candidates. No local capture or benchmark was started while PID 72210 was live.
diff --git a/crates/verify/src/report.rs b/crates/verify/src/report.rs
index 5c382c964..88135075b 100644
--- a/crates/verify/src/report.rs
+++ b/crates/verify/src/report.rs
@@ -600,25 +600,55 @@ mod tests {
#[test]
fn instant_system_offset_expectation_requires_its_exact_signature() {
let entries = cap_verify_expect::parse(include_str!("../expectations.toml")).unwrap();
- let entry = entries
+ let entries: Vec<_> = entries
.iter()
- .find(|entry| entry.class == "sync-system" && entry.flaky)
- .unwrap();
+ .filter(|entry| entry.class == "sync-system" && entry.flaky)
+ .collect();
+ assert_eq!(entries.len(), 2);
let metrics = |offset, mic| {
BTreeMap::from([
("count".into(), cap_verify_expect::Range::at(1.0)),
("offset-ms".into(), cap_verify_expect::Range::at(offset)),
("mic-offset-ms".into(), cap_verify_expect::Range::at(mic)),
+ (
+ "system-minus-mic-ms".into(),
+ cap_verify_expect::Range::at(((offset - mic) * 10.0_f64).round() / 10.0),
+ ),
])
};
- assert!(cap_verify_expect::within(&entry.bound, Some(&metrics(-46.1, 13.9))).is_ok());
- for (offset, mic) in [(-71.0, 13.9), (-29.0, 13.9), (-46.1, 35.0), (-46.1, -10.0)] {
- assert!(cap_verify_expect::within(&entry.bound, Some(&metrics(offset, mic))).is_err());
+ let accepts = |metrics: &BTreeMap<String, cap_verify_expect::Range>| {
+ entries
+ .iter()
+ .any(|entry| cap_verify_expect::within(&entry.bound, Some(metrics)).is_ok())
+ };
+ for (offset, mic) in [(-46.1, 13.9), (7.8, 14.5), (-47.1, 13.9), (8.8, 14.5)] {
+ assert!(accepts(&metrics(offset, mic)));
+ }
+ for (offset, mic) in [
+ (-47.2, 13.9),
+ (-45.0, 13.9),
+ (6.7, 14.5),
+ (8.9, 14.5),
+ (-25.0, 35.0),
+ (-56.0, 4.0),
+ (0.0, 14.5),
+ ] {
+ assert!(!accepts(&metrics(offset, mic)));
}
- let mut absent = metrics(-46.1, 13.9);
- absent.remove("mic-offset-ms");
- assert!(cap_verify_expect::within(&entry.bound, Some(&absent)).is_err());
- assert!(entry.matches("export/instant/output/mp4/native", "macos"));
- assert!(!entry.matches("export/instant/output/mp4/native", "linux"));
+ for required in ["mic-offset-ms", "system-minus-mic-ms"] {
+ let mut absent = metrics(-46.1, 13.9);
+ absent.remove(required);
+ assert!(!accepts(&absent));
+ }
+ for entry in &entries {
+ assert!(entry.matches("export/instant/output/mp4/native", "macos"));
+ }
+ assert_eq!(
+ entries
+ .iter()
+ .filter(|entry| entry.matches("export/instant/output/mp4/native", "linux"))
+ .count(),
+ 1
+ );
}
}
diff --git a/crates/verify/tests/export.rs b/crates/verify/tests/export.rs
index 9c42e35ae..cd40593b4 100644
--- a/crates/verify/tests/export.rs
+++ b/crates/verify/tests/export.rs
@@ -799,10 +799,15 @@ fn instant_case(spec: cap_verify::pipeline::InstantSpec) -> Option<String> {
}
if av.abs() > 0.005 {
if source == AudioSource::System
- && av < 0.0
+ && case.id == "export/instant/output/mp4/native"
&& let Some(mic_offset_ms) = mic_offset_ms
{
case.measure(&sync_class, "mic-offset-ms", mic_offset_ms);
+ case.measure(
+ &sync_class,
+ "system-minus-mic-ms",
+ ((av * 1e3 - mic_offset_ms) * 10.0).round() / 10.0,
+ );
}
case.measure(&sync_class, "offset-ms", (av * 1e4).round() / 10.0);
case.problem(