Make your computer a home for your AI partner. Avibe is the local-first Agent OS: Vibey, its built-in AI partner, works out of the box, and Claude Code, Codex & OpenCode run unchanged beside it — reach them from your browser, phone, or any chat app.
Python agentagent-osaiai-agentsai-partnerchatopsclaudeclaude-codecodexdevtoolsdiscord-botlark-botllmlocal-firstlong-term-memoryopencodeslack-botvibe-codingvibeywechat
test(scenarios): resolve every catalog's evidence citations, with multi-case evidence and the dead primary citations (#2570)
Call White820ec641d055
diff --git a/standards/scenario-testing/STANDARD.md b/standards/scenario-testing/STANDARD.mdindex d2f21beae..8ee15ded7 100644--- a/standards/scenario-testing/STANDARD.md+++ b/standards/scenario-testing/STANDARD.md@@ -184,7 +184,11 @@ Every scenario-backed automated test should expose the scenario ID in a searchabEvery citation a row makes must name a **case**, and that name must resolve toexactly one case the suite collects in the cited file — in whichever key the row-states its evidence, `test` and any layer-specific key alike. A citation that+states its evidence, `test` and any layer-specific key alike. When several cases+prove a row's contract together, `test` lists each of them, and each entry+resolves to exactly one case on its own; a class or a file standing in for the+group is not a citation of any of them. Only `test` takes a list:+`partial_evidence.test` names one case. A citation thatnames only a file is not evidence for the row that makes it: the file stayscollected for whatever else cites it, so deleting the one case that answeredthis row leaves the row reading `covered` with nothing executable tied to it.diff --git a/tests/scenarios/harness_failure_recovery/catalog.yaml b/tests/scenarios/harness_failure_recovery/catalog.yamlindex c233eedba..2382a44ea 100644--- a/tests/scenarios/harness_failure_recovery/catalog.yaml+++ b/tests/scenarios/harness_failure_recovery/catalog.yaml@@ -37,12 +37,17 @@ scenarios:backend: sharedtest: tests/test_scheduled_tasks.py::test_drain_serializes_executions_per_session- id: HFR-004- name: restart recovers held run queues without flushing user-owned queue heads+ name: >-+ restart recovers a Delivery-owned Run once, and never sends user-owned queue input+ or a Run queued behind a stopped user headstatus: coveredkind: restartlayer: scenariobackend: shared- test: tests/test_scheduled_tasks.py::test_restart_recovers_persisted_workbench_run_queue_after_older_owner_settles+ test:+ - tests/test_scheduled_tasks.py::test_restart_recovers_delivery_owned_run_without_unrelated_run_arbitration+ - tests/test_scheduled_tasks.py::test_restart_does_not_flush_user_queue_ahead_of_held_agent_run+ - tests/test_scheduled_tasks.py::test_restart_does_not_auto_send_pure_user_queue- id: HFR-005name: live long-running owner is never stolen by agestatus: covered@@ -191,12 +196,12 @@ scenarios:backend: sharedtest: tests/test_scheduled_tasks.py::test_sweep_never_terminalizes_a_transport_reason_with_no_timestamp- id: HFR-026- name: live ownership exempts an aged queue hold from its TTL+ name: only explicit Turn attribution owns running work; queue-hold metadata owns nothingstatus: coveredkind: safetylayer: scenariobackend: shared- test: tests/test_scheduled_tasks.py::test_sweep_spares_an_aged_queue_hold_a_live_turn_still_owns+ test: tests/test_scheduled_tasks.py::test_legacy_hold_metadata_does_not_change_turn_or_run_ownership- id: HFR-027name: gate-owned Workbench turn settles its own Run when no result arrivesstatus: covered@@ -205,12 +210,12 @@ scenarios:backend: sharedtest: tests/test_scheduled_tasks.py::test_workbench_turn_settles_its_agent_run_when_no_result_arrives- id: HFR-028- name: swept Run's persisted queue segment is retired immediately+ name: cancelling a Delivery-owned Run retires its queued Deliverystatus: coveredkind: failure_recoverylayer: scenariobackend: shared- test: tests/test_scheduled_tasks.py::test_sweep_retires_the_queue_segment_of_the_run_it_expired+ test: tests/test_scheduled_tasks.py::test_canceling_a_delivery_owned_run_retires_its_exact_queue_row- id: HFR-029name: backend runtime refresh settles its Run as a refresh, not a user stopstatus: covered@@ -387,12 +392,12 @@ scenarios:backend: sharedtest: tests/test_agent_run_target.py::test_resolve_agent_run_target_tolerates_concurrent_row_insert- id: HFR-054- name: an unresolvable pinned session pauses its definition and notifies once+ name: an unresolvable pinned session notifies on its first failure and pauses its definition after threestatus: coveredkind: failure_recoverylayer: scenariobackend: shared- test: tests/test_scheduled_tasks.py::test_execute_task_pauses_and_notifies_when_pinned_session_is_missing+ test: tests/test_scheduled_tasks.py::test_execute_task_notifies_then_pauses_after_three_unresolvable_failures- id: HFR-055name: a create_once definition rebinds instead of failing foreverstatus: covered@@ -438,7 +443,10 @@ scenarios:backend: sharedtest: tests/test_harness_failure_visibility.py::test_complete_does_not_clobber_an_already_terminal_run- id: HFR-061- name: the coalesced completer does not clobber an already-settled success+ name: >-+ Turn settlement does not clobber an already-settled success, a Stop that lands under+ a settlement writer stays canceled with no failure notice, and an uncancelled run+ still settles failed with its noticestatus: coveredkind: safetylayer: contract@@ -497,7 +505,11 @@ scenarios:# They take no ids of their own because PR6's block is full at 099 and 100+ is# contracted to PR2 by the merged plan (docs/plans/harness-run-reliability.md# §10.7; landed in #1044, PR2 not started per its status header).- test: tests/test_harness_failure_visibility.py::test_complete_coalesced_does_not_clobber_an_already_succeeded_run+ test:+ - tests/test_harness_failure_visibility.py::test_turn_participant_settlement_does_not_clobber_an_already_succeeded_run+ - tests/test_harness_failure_visibility.py::test_a_cancel_landing_under_the_coalesced_completer_is_not_overwritten+ - tests/test_harness_failure_visibility.py::test_a_cancel_landing_under_settle_run_terminal_is_not_overwritten+ - tests/test_harness_failure_visibility.py::test_the_cancel_cas_does_not_leave_an_uncancelled_run_unsettled- id: HFR-062name: one success does not erase recent failure history in the CLIstatus: covered@@ -3705,8 +3717,8 @@ scenarios:test: tests/test_scheduled_tasks.py::test_an_adopted_reservation_is_dropped_from_the_retry_record_without_being_touched- id: HFR-280name: >-- a Workbench PATCH cannot mutate a session archived inside its window, and the- archive outranks the backend lock+ a Workbench PATCH and a concurrent archive of the same session serialize: the archive+ waits for the PATCH, so a rename or re-route never lands on an already-archived rowstatus: coveredkind: safetylayer: contract@@ -3735,7 +3747,13 @@ scenarios:# Both fail against the pre-fix service: the rename lands silently (DID NOT RAISE),# the re-route answers SessionBackendLockedError. Same _commit_competing_bind_after# harness and the same real _archive_write payload as HFR-252.- test: tests/test_sqlite_sessions_store.py::test_update_session_cannot_rename_a_session_archived_inside_its_window+ # Later the writer reservation removed the read/archive/write window itself: the+ # PATCH and the archive now serialize, so there is no race left to refuse. Both+ # halves are proved by test_update_session_serializes_rename_before_archive and+ # test_update_session_serializes_route_change_before_archive.+ test:+ - tests/test_sqlite_sessions_store.py::test_update_session_serializes_rename_before_archive+ - tests/test_sqlite_sessions_store.py::test_update_session_serializes_route_change_before_archive- id: HFR-281name: >-a completion hook queued inside a stamp names an Agent the catalog still has@@ -4258,7 +4276,7 @@ scenarios:backend: codex# A definitive process death makes the old local Turn fence non-authoritative;# only unknown ownership or an active Activity can still veto replacement.- test: tests/test_codex_agent.py::CodexTransportCwdStalenessTests::test_hfr_473_dead_transport_restarts_past_stale_turn_ownership+ test: tests/test_codex_agent.py::CodexGenerationAcquisitionTests::test_hfr_473_dead_transport_restarts_past_stale_turn_ownership- id: HFR-474name: upgrade recovery collapses legacy restart notices by durable Turn ownershipstatus: covereddiff --git a/tests/scenarios/memory_upgrade_compatibility/test_catalog.py b/tests/scenarios/memory_upgrade_compatibility/test_catalog.pyindex c6116094a..3a195a7ef 100644--- a/tests/scenarios/memory_upgrade_compatibility/test_catalog.py+++ b/tests/scenarios/memory_upgrade_compatibility/test_catalog.py@@ -23,11 +23,15 @@ def test_memory_upgrade_scenario_ids_resolve_to_executable_evidence():ids = {item["id"] for item in scenarios}assert len(ids) == len(scenarios)for scenario in scenarios:- path, name = scenario["test"].split("::")- assert path in catalog["capability"]["canonical_tests"]- assert path in entry["scenario_tests"] or path in entry["unit_or_contract_tests"]- functions = [node for node in ast.parse((ROOT / path).read_text()).body if isinstance(node, ast.FunctionDef)]- function, = [node for node in functions if node.name == name]- assert scenario["id"] in (ast.get_docstring(function) or "")+ # A row whose contract several cases prove lists each of them.+ references = scenario["test"] if isinstance(scenario["test"], list) else [scenario["test"]]+ assert references+ for reference in references:+ path, name = reference.split("::")+ assert path in catalog["capability"]["canonical_tests"]+ assert path in entry["scenario_tests"] or path in entry["unit_or_contract_tests"]+ functions = [node for node in ast.parse((ROOT / path).read_text()).body if isinstance(node, ast.FunctionDef)]+ function, = [node for node in functions if node.name == name]+ assert scenario["id"] in (ast.get_docstring(function) or "")for observation in observations["observations"]:assert set(observation["affects"]) <= idsdiff --git a/tests/scenarios/message_delivery/catalog.yaml b/tests/scenarios/message_delivery/catalog.yamlindex a5b2a14c4..a339521e7 100644--- a/tests/scenarios/message_delivery/catalog.yaml+++ b/tests/scenarios/message_delivery/catalog.yaml@@ -103,7 +103,7 @@ scenarios:status: coveredkind: recoverylayer: contract- test: tests/test_session_delivery_fsm.py::test_lost_accepted_receipt_restarts_unknown_without_resteer+ test: tests/test_session_delivery_fsm.py::test_lost_accepted_receipt_materializes_from_exact_restart_evidence- id: MESSAGE-DELIVERY-012name: Definitive P1 refusal and idle drain start one fallback Turnstatus: covered@@ -121,7 +121,7 @@ scenarios:status: coveredkind: concurrencylayer: contract- test: tests/test_session_delivery_fsm.py::test_p0_terminal_restart_claims_successor_once_after_old_terminal+ test: tests/test_session_delivery_fsm.py::test_content_p0_preserves_open_hold_and_claims_successor_once- id: MESSAGE-DELIVERY-015name: Concurrent idle P3 admissions choose one Turn owner and retain one FIFO loserstatus: covered@@ -247,7 +247,10 @@ scenarios:status: coveredkind: happy_pathlayer: contract- test: tests/test_delivery_ack_reaction.py::AdmissionAckTests+ test:+ - tests/test_delivery_ack_reaction.py::AdmissionAckTests::test_steered_input_reports_a_receipt+ - tests/test_delivery_ack_reaction.py::AdmissionAckTests::test_queued_and_pending_steer_report_the_queued_receipt+ - tests/test_delivery_ack_reaction.py::AdmissionAckTests::test_unconfirmed_and_undelivered_states_are_distinguishable- id: MESSAGE-DELIVERY-302name: A queued admission receipt is replaced by the running turn indicatorstatus: covered@@ -271,7 +274,9 @@ scenarios:status: coveredkind: recoverylayer: contract- test: tests/test_delivery_ack_reaction.py::ReactionTargetSurvivalTests+ test:+ - tests/test_delivery_ack_reaction.py::ReactionTargetSurvivalTests::test_quick_reply_echo_is_captured_from_the_ingress_context+ - tests/test_delivery_ack_reaction.py::ReactionTargetSurvivalTests::test_hydrated_context_gets_its_reaction_target_back- id: MESSAGE-DELIVERY-306name: A native-less quick reply reacts on its persisted echo targetstatus: covereddiff --git a/tests/scenarios/model_hub/test_model_hub_catalog.py b/tests/scenarios/model_hub/test_model_hub_catalog.pyindex 17dcb2147..446f3437a 100644--- a/tests/scenarios/model_hub/test_model_hub_catalog.py+++ b/tests/scenarios/model_hub/test_model_hub_catalog.py@@ -163,12 +163,16 @@ def test_mh_catalog_001_scenario_catalog_is_complete_and_executable() -> None:if not status["test_required"]:assert test_ref is Noneelse:- _resolve_evidence_reference(- scenario_id=scenario["id"],- reference=test_ref,- canonical_tests=canonical_tests,- expected_fail=status["expected_fail"],- )+ # A row whose contract several cases prove lists each of them.+ references = test_ref if isinstance(test_ref, list) else [test_ref]+ assert references, f"Scenario {scenario['id']} lists no evidence"+ for reference in references:+ _resolve_evidence_reference(+ scenario_id=scenario["id"],+ reference=reference,+ canonical_tests=canonical_tests,+ expected_fail=status["expected_fail"],+ )partial_evidence = scenario.get("partial_evidence")if partial_evidence is not None:diff --git a/tests/scenarios/permissions/catalog.yaml b/tests/scenarios/permissions/catalog.yamlindex eff1d6803..252be08c6 100644--- a/tests/scenarios/permissions/catalog.yaml+++ b/tests/scenarios/permissions/catalog.yaml@@ -238,7 +238,18 @@ scenarios:status: coveredkind: authorizationlayer: scenario- test: tests/scenarios/permissions/test_editor_reservation_and_fork.py+ # Every case named for this row.+ test:+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_editor_reserves_only_a_project_and_agent_it_may_use+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_reservation_requires_agent_selection_authority+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_unauthorized_callers_never_reserve+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_standalone_reservation_keeps_runtime_management_boundary+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_local_callers_keep_owner_reservation_semantics+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_fork_of_a_permitted_source_and_agent_succeeds+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_fork_inherited_agent_needs_selection_authority+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_fork_override_agent_needs_selection_authority+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_fork_checks_source_and_destination_independently+ - tests/scenarios/permissions/test_editor_reservation_and_fork.py::test_permissions_023_fork_preserves_existing_source_answersgiven: An Editor holds Project and Agent selection authority for only part of the instancewhen: The caller reserves a Session and forks an existing one with inherited or overridden Agentsthen: Source and destination are checked independently, unauthorized callers never reserve, and local callers keep Owner semantics@@ -248,7 +259,16 @@ scenarios:status: coveredkind: authorizationlayer: scenario- test: tests/scenarios/permissions/test_deferred_authority_provenance.py+ # Every case named for this row.+ test:+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_a_vault_request_records_the_authority_that_made_it+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_a_caller_cannot_forge_the_recorded_authority+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_outward_request_views_hide_the_recorded_authority+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_auto_resume_runs_under_the_requesting_caller+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_a_local_requests_auto_resume_stays_local+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_a_request_cannot_name_a_session_its_caller_cannot_resume+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_a_reachable_target_still_records_the_request+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_024_a_request_that_names_no_session_is_unchangedgiven: A remote caller opens a Vault access request that is decided long after the call returnswhen: The request is read back, audited, and auto-resumed by the scheduled callbackthen: The row keeps the trusted snapshot for the resumed turn, every outward view and the audit trail hide it, and a local caller's row is byte-for-byte unchanged@@ -268,7 +288,27 @@ scenarios:status: coveredkind: authorizationlayer: scenario- test: tests/scenarios/permissions/test_cli_invocation_authority.py+ # Every case named for this row.+ test:+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_every_cli_command_declares_a_role_floor+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_authority_comes_from_the_caller_env+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_declared_remote_caller_without_provenance_fails_closed+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_personal_pairing_admits_a_snapshot_without_organization_claims+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_unbound_background_editor_cannot_become_local_owner+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_snapshot_from_another_pairing_is_anonymous_remote+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_stale_snapshot_is_refused_and_writes_nothing+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_stale_snapshot_cannot_reach_a_command_that_forwards_it+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_each_namespace_admits_at_its_own_floor+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_refused_command_leaves_the_record_it_would_have_written+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_an_existing_resource_guard_now_sees_the_invocation_caller+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_an_explicit_context_still_outranks_the_invocation+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_authority_lasts_exactly_one_invocation+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_below_floor_command_is_refused_while_a_remote_owner_is_not+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_internal_activation_entry_points_stay_owner_work+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_guided_remote_setup_refuses_before_a_pairing_key_is_solicited+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_an_admitted_restart_hands_the_supervisor_no_caller+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_deferred_activation_runs_as_the_installation+ - tests/scenarios/permissions/test_cli_invocation_authority.py::test_permissions_026_a_runtime_owned_process_starts_without_the_callergiven: The host injects a remote caller's signed snapshot into the Agent subprocess environmentwhen: The real CLI entry point dispatches commands across every parser leaf and caller rolethen: Every command declares a role floor, below-floor commands are refused before any effect, unusable provenance fails closed as anonymous remote, and the authority lasts exactly one invocation@@ -278,12 +318,23 @@ scenarios:status: coveredkind: authorizationlayer: scenario- test: tests/scenarios/permissions/test_deferred_authority_provenance.py+ # Every case named for this row, and the share route's visitor case.+ test:+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_a_dispatching_show_event_records_its_author+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_a_non_dispatching_event_hides_the_author_from_the_transcript+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_the_cli_reserves_a_dispatching_event_under_its_own_caller+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_the_cli_reserves_nothing_for_an_event_that_starts_no_turn+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_an_admitted_reservation_is_still_admitted_at_execution+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_revoking_access_after_the_reservation_stops_the_turn+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_an_unreachable_session_is_refused_before_the_reservation+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_local_and_management_callers_keep_the_same_targets+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_the_cli_fails_the_command_when_its_reservation_is_refused+ - tests/scenarios/permissions/test_deferred_authority_provenance.py::test_permissions_027_an_offline_cli_dispatch_still_runs_as_its_caller+ - tests/test_ui_show_pages.py::test_public_show_page_dispatch_records_the_visitor_that_admitted_itgiven: A Show annotation or intent starts an Agent turn through a delivery reservation rather than the message that woke the daemonwhen: Remote and local authors append dispatching and non-dispatching events, including the CLI reservation replayed by the live UI post and a share visitor posting to /pthen: The reserved delivery carries the author's snapshot for the resumed turn, the event echo and transcript stream hide it, the CLI caller's authority survives the replay, and the share route resolves its visitor once so admission, display author and recorded authority cannot describe different peoplerelated_tests:- - tests/test_ui_show_pages.py::test_public_show_page_dispatch_records_the_visitor_that_admitted_it- tests/test_ui_show_pages.py::test_public_show_page_dispatch_fails_closed_without_a_resolved_visitor- id: PERMISSIONS-028@@ -291,7 +342,33 @@ scenarios:status: coveredkind: authorizationlayer: scenario- test: tests/scenarios/permissions/test_effective_target_preflight.py+ # Every case named for this row.+ test:+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_the_open_project_binds_the_signed_editor+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_every_producer_refuses_an_unreachable_target+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_legacy_key_names_the_same_session+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_the_callers_own_session_is_an_effective_target+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_callback_is_admitted_before_the_run_is_reserved+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_forbidden_project_target_is_refused+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_an_unusable_agent_refuses_the_turn+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_permitted_editor_still_queues_the_turn+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_runtime_management_and_local_callers_are_unchanged+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_an_update_reauthorizes_its_effective_target+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_an_inherited_update_target_is_checked_too+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_permitted_destination_still_takes_the_work+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_per_run_work_with_no_scope_is_a_stream_of_standalone_sessions+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_replacement_target_is_the_one_that_is_admitted+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_replacement_cannot_land_where_the_caller_cannot_chat+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_per_run_work_admits_the_agent_every_fire_will_use+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_an_admitted_per_run_target_keeps_the_agent_it_resolved+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_bound_session_still_keeps_the_agents_authority+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_reserved_replacement_still_follows_the_session_it_makes+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_clearing_the_agent_on_per_run_work_resolves_one+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_the_callers_that_could_still_retarget_per_run_work+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_a_fresh_legacy_target_is_a_placement_not_a_blank+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_fresh_legacy_work_still_opens_for_the_callers_that_could+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_an_update_onto_a_fresh_legacy_target_is_admitted_too+ - tests/scenarios/permissions/test_effective_target_preflight.py::test_permissions_028_saved_control_keeps_the_definitions_own_authoritygiven: A signed caller names a Session it may not drive, by Session id, by legacy scope key whose thread exists or has never been opened, or by defaulting to its own, or repoints a stored definition at a destination it may or may not usewhen: A Run, a callback route, and a Task or Watch definition are produced or retargeted through the real CLIthen: Each producer refuses before writing, leaving Run definition Session and Scope counts and the stored row unchanged, a target whose Session does not exist yet is admitted as the placement it is rather than skipped for having no row, an edit that replaces its Session is judged on the destination it chooses rather than the one it abandons, work that becomes per-run admits the Agent each future fire will select with its follow-the-Session marker read from the edit's final policy, and permitted Editors Member Owner and local callers keep the reach they had while saved automation control keeps the definition's own authoritydiff --git a/tests/scenarios/permissions/test_cli_invocation_authority.py b/tests/scenarios/permissions/test_cli_invocation_authority.pyindex c44b0606a..22ea659f8 100644--- a/tests/scenarios/permissions/test_cli_invocation_authority.py+++ b/tests/scenarios/permissions/test_cli_invocation_authority.py@@ -238,6 +238,7 @@ def test_permissions_026_a_personal_pairing_admits_a_snapshot_without_organizatidef test_permissions_026_unbound_background_editor_cannot_become_local_owner(invoke, monkeypatch, capsys):+ """PERMISSIONS-026: a background command carrying an Editor's snapshot stays that Editor."""from core.caller_context import background_command_envenv = background_command_env(diff --git a/tests/scenarios/permissions/test_effective_target_preflight.py b/tests/scenarios/permissions/test_effective_target_preflight.pyindex 6697a8f73..1e98d60dc 100644--- a/tests/scenarios/permissions/test_effective_target_preflight.py+++ b/tests/scenarios/permissions/test_effective_target_preflight.py@@ -300,7 +300,7 @@ def _placed_argv(kind_of_work, placement, scope_id, *, name="p"):def test_permissions_028_the_open_project_binds_the_signed_editor():- """The fixture's positive control is the caller the Project actually names."""+ """PERMISSIONS-028: the fixture's positive control is the caller the Project actually names."""assert _editor_email_snapshot_matches_open_project()diff --git a/tests/scenarios/test_catalog_citations.py b/tests/scenarios/test_catalog_citations.pyindex a624b5683..6f5100efe 100644--- a/tests/scenarios/test_catalog_citations.py+++ b/tests/scenarios/test_catalog_citations.py@@ -1,25 +1,41 @@-"""Every case a scenario catalog's pointer keys name still exists, in every catalog.--``related_tests`` and ``supporting_tests`` point a reader from a scenario to-neighbouring cases. They claim no coverage, so nothing read them, and a renamed-or deleted test left the pointer leading nowhere: six had, in two catalogs.-Evidence keys are another check's subject: MH-CATALOG-001 resolves Model Hub's,-and the UI gate resolves every catalog's UI evidence.--A pointer is a list. Each entry names a file, which must exist, and may name a-case in it: ``file::Class::method``, or ``file::name`` naming a module-level-test function, a test class, or a test method of a class in the file. pytest's-default naming rules decide what is a test, so a helper or a plain class is-not. This reads source and matches exact names; it does not emulate-collection-time exclusions such as ``__test__ = False``, which #2565 tracks.--UI pointers are resolved against vitest's own collection by+"""Every case a scenario catalog cites still exists, in every catalog and every key.++A row's citations are how a reader gets from a scenario to the code that runs+it: ``test`` and ``partial_evidence`` name the evidence, and the pointer keys+``related_tests`` and ``supporting_tests`` name neighbouring cases. Only the+Model Hub catalog checked its own evidence keys (MH-CATALOG-001), so renamed+or deleted tests left nine rows across two catalogs reading ``covered`` while+the case they cited no longer existed. Every catalog's legend defines+``covered`` as automated coverage that exists, so a ``covered`` row must cite it.++A pytest citation resolves when its file holds a case pytest collects:+``file::Class::method``, or ``file::name`` naming a module-level test, a test+class, or a test method of a class in that file, which is how several catalogs+cite unittest cases. A helper or a plain class is no case. A parametrized id+resolves through its function. An evidence key names exactly one case per+entry (STANDARD.md §4): a bare file or a test class stays collected for+whatever else cites it, so it evidences no row in particular, and a row whose+contract several cases prove lists each of them under ``test``;+``partial_evidence.test`` names one. A cited Python file must be one pytest's+default discovery collects (``test_*.py`` or ``*_test.py``), and an evidence+case must not be skipped unconditionally on itself, its class or its module.+Pointer keys may name a bare file, a test class, a function or a method, which+must exist.++This reads source and applies pytest's default naming rules; it does not+emulate every collection-time exclusion, such as ``__test__ = False``, a class+with ``__init__``, or deselection in a hook; it matches exact names only; and it+leaves conditional skips (``skipif``) to the environment. Resolving against+pytest's own collection, and execution status against CI's results, is #2565.++UI (vitest) citations are resolved against vitest's own collection by``ui/scripts/validate-scenario-catalog.mjs``; here only their file is checked."""from __future__ import annotationsimport ast+import refrom functools import cachefrom pathlib import Path@@ -30,41 +46,147 @@ REPO_ROOT = Path(__file__).resolve().parents[2]SCENARIO_ROOT = REPO_ROOT / "tests" / "scenarios"_UI_SUFFIXES = {".ts", ".tsx", ".mts", ".mjs"}_POINTER_KEYS = ("related_tests", "supporting_tests")+# pytest's default ``python_files``, which this repository does not override.+_DISCOVERED = re.compile(r"(test_.*|.*_test)\.py")@cache-def _test_names(path: Path) -> frozenset[str]:- """Every name in ``path`` that is a test pytest's default rules collect, or a test class.+def _collectable_cases(path: Path) -> tuple[frozenset[str], dict[str, frozenset[str]]]:+ """The test functions pytest collects from ``path``, and each test class's test methods.- A module-level ``test*`` function; a ``Test*`` class or a ``unittest.TestCase``- subclass, directly or through another class in the file; and each ``test*``- method of such a class, by itself and as ``Class::method``.+ pytest's default naming rules, which this repository does not change: a+ module-level ``test*`` function, and a ``test*`` method of a ``Test*`` class or+ of a ``unittest.TestCase`` subclass, directly or through another class in the+ file. Collection-time exclusions are out of its reach (#2565)."""tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))classes = {node.name: node for node in tree.body if isinstance(node, ast.ClassDef)}+ def base_names(node: ast.ClassDef) -> list[str]:+ return [base.attr if isinstance(base, ast.Attribute) else getattr(base, "id", "") for base in node.bases]+def is_test_case(name: str, seen: frozenset[str] = frozenset()) -> bool:node = classes.get(name)if node is None or name in seen:return False- bases = [base.attr if isinstance(base, ast.Attribute) else getattr(base, "id", "") for base in node.bases]- return any(base.endswith("TestCase") or is_test_case(base, seen | {name}) for base in bases)+ return any(+ base.endswith("TestCase") or is_test_case(base, seen | {name}) for base in base_names(node)+ )- def test_functions(body: list[ast.stmt]) -> set[str]:- return {+ def test_names(body: list[ast.stmt]) -> frozenset[str]:+ return frozenset(node.namefor node in bodyif isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name.startswith("test")- }+ )++ test_classes = {+ name: test_names(node.body)+ for name, node in classes.items()+ if name.startswith("Test") or is_test_case(name)+ }+ return test_names(tree.body), test_classes+++def _is_unconditional_skip(marker: ast.expr) -> bool:+ """``pytest.mark.skip`` or ``unittest.skip``, bare or called; ``skipif`` is a condition and passes."""++ target = marker.func if isinstance(marker, ast.Call) else marker+ return (target.attr if isinstance(target, ast.Attribute) else getattr(target, "id", "")) == "skip"+++@cache+def _unconditionally_skipped(path: Path) -> tuple[bool, frozenset[str]]:+ """Whether the module's ``pytestmark`` skips it, and the functions and classes a skip decorates."""- names = test_functions(tree.body)- for name, node in classes.items():- if name.startswith("Test") or is_test_case(name):- names.add(name)- for method in test_functions(node.body):- names.update({method, f"{name}::{method}"})- return frozenset(names)+ tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))+ module = False+ for node in tree.body:+ if isinstance(node, ast.Assign) and any(getattr(target, "id", "") == "pytestmark" for target in node.targets):+ marks = node.value.elts if isinstance(node.value, (ast.List, ast.Tuple)) else [node.value]+ module = module or any(_is_unconditional_skip(mark) for mark in marks)+ decorated = {+ f"{owner.name}::{node.name}" if owner is not None else node.name+ for owner, node in (+ [(None, node) for node in tree.body]+ + [(cls, node) for cls in tree.body if isinstance(cls, ast.ClassDef) for node in cls.body]+ )+ if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef))+ and any(_is_unconditional_skip(decorator) for decorator in node.decorator_list)+ }+ return module, frozenset(decorated)+++def _never_runs(path: Path, name: str) -> bool:+ """Whether an unconditional skip on the module, the case's class or the case itself disables it.++ A short ``name`` is a method of the one class holding it, which the caller has+ already established. Conditional skips (``skipif``) are platform scoping and+ depend on the environment; deciding them belongs to CI's results (#2565).+ """++ module, decorated = _unconditionally_skipped(path)+ if module:+ return True+ parts = name.partition("[")[0].split("::")+ if len(parts) == 2:+ return parts[0] in decorated or "::".join(parts) in decorated+ (case,) = parts+ if case in decorated:+ return True+ _, classes = _collectable_cases(path)+ owner = next((cls for cls, methods in classes.items() if case in methods), None)+ return owner is not None and (owner in decorated or f"{owner}::{case}" in decorated)+++def _named_cases(path: Path, name: str) -> tuple[int, bool]:+ """How many collected cases ``name`` names in ``path``, and whether it names a test class."""++ functions, classes = _collectable_cases(path)+ parts = name.partition("[")[0].split("::")+ if len(parts) == 2:+ return int(parts[1] in classes.get(parts[0], frozenset())), False+ if len(parts) != 1:+ return 0, False+ (case,) = parts+ return int(case in functions) + sum(case in methods for methods in classes.values()), case in classes+++def _citations(row: dict) -> tuple[list[tuple[str, object]], list[str]]:+ """Each (key, reference) the row cites a test file or case with, and each malformed key.++ A malformed key is reported rather than skipped: read as absent, ``test: []``+ or a ``partial_evidence`` with no reference would cite nothing and pass.+ """++ cited: list[tuple[str, object]] = []+ malformed: list[str] = []+ evidence = row.get("test")+ if evidence is None and row.get("status") == "covered":+ malformed.append("is covered but cites no test")+ if isinstance(evidence, list):+ # A row whose contract several cases prove names each of them.+ if not evidence:+ malformed.append("test lists no case")+ cited.extend(("test", reference) for reference in evidence)+ elif evidence is not None:+ cited.append(("test", evidence))+ if "partial_evidence" in row:+ partial = row["partial_evidence"]+ if isinstance(partial, dict) and isinstance(partial.get("test"), list):+ malformed.append("partial_evidence lists cases; it names one, and only test may list several")+ elif isinstance(partial, dict) and "test" in partial:+ cited.append(("partial_evidence", partial["test"]))+ else:+ malformed.append("partial_evidence names no test")+ for key in _POINTER_KEYS:+ pointers = row.get(key)+ if isinstance(pointers, list):+ cited.extend((key, reference) for reference in pointers)+ elif pointers is not None:+ malformed.append(f"{key} is not a list")+ return cited, malformeddef _catalogs() -> list[Path]:@@ -74,29 +196,40 @@ def _catalogs() -> list[Path]:@pytest.mark.parametrize("catalog", _catalogs(), ids=lambda path: path.parent.name)-def test_every_pointer_names_a_file_or_case_that_exists(catalog: Path) -> None:+def test_every_catalog_citation_resolves_to_a_case(catalog: Path) -> None:rows = yaml.safe_load(catalog.read_text(encoding="utf-8")).get("scenarios") or []dead: list[str] = []for row in rows:- for key in _POINTER_KEYS:- pointers = row.get(key)- if pointers is None:+ cited, malformed = _citations(row)+ dead.extend(f"{row['id']} {problem}" for problem in malformed)+ for key, reference in cited:+ where = f"{row['id']} {key}: {reference}"+ if not isinstance(reference, str):+ dead.append(f"{row['id']} {key}: {reference!r} is not a reference")+ continue+ path_text, separator, name = reference.partition("::")+ path = REPO_ROOT / path_text+ pointer = key in _POINTER_KEYS+ if not pointer and not name.strip():+ dead.append(f"{where} (evidence must name a case, not only its file)")+ continue+ if not path.is_file():+ dead.append(f"{where} (no such file)")+ continue+ if path.suffix == ".py" and not _DISCOVERED.fullmatch(path.name):+ dead.append(f"{where} (pytest's default discovery never collects {path.name})")continue- if not isinstance(pointers, list):- dead.append(f"{row['id']} {key} is not a list")+ if not separator or path.suffix in _UI_SUFFIXES:continue- for reference in pointers:- where = f"{row['id']} {key}: {reference!r}"- if not isinstance(reference, str):- dead.append(f"{where} (not a reference)")- continue- path_text, separator, name = reference.partition("::")- path = REPO_ROOT / path_text- if not path.is_file():- dead.append(f"{where} (no such file)")- elif separator and not name.strip():- dead.append(f"{where} (names an empty case)")- elif separator and path.suffix not in _UI_SUFFIXES:- if name.partition("[")[0] not in _test_names(path):- dead.append(f"{where} (no such test in the file)")- assert not dead, f"{catalog.relative_to(REPO_ROOT)} has pointers that lead nowhere:\n" + "\n".join(dead)+ count, names_class = _named_cases(path, name)+ if pointer:+ # A pointer claims no evidence, so a class or a group of cases is a fine destination.+ if not count and not names_class:+ dead.append(f"{where} (no such case in the file)")+ elif names_class:+ dead.append(f"{where} (names a test class, not one case; cite each case that proves the row)")+ elif count != 1:+ dead.append(f"{where} (names {count} cases, expected exactly one; cite it as Class::method)")+ elif _never_runs(path, name):+ dead.append(f"{where} (is skipped unconditionally, so it never runs)")+ assert not dead, f"{catalog.relative_to(REPO_ROOT)} has citations that resolve to no case:\n" + "\n".join(dead)diff --git a/tests/test_codex_agent.py b/tests/test_codex_agent.pyindex 161807779..1590150a6 100644--- a/tests/test_codex_agent.py+++ b/tests/test_codex_agent.py@@ -6628,6 +6628,7 @@ class CodexGenerationAcquisitionTests(unittest.IsolatedAsyncioTestCase):self.assertTrue(previous.generation.retiring)async def test_hfr_473_dead_transport_restarts_past_stale_turn_ownership(self):+ """HFR-473: a definitively dead transport is replaced past its stale Turn fence."""agent = self._agent()with tempfile.TemporaryDirectory() as cwd:dead = SimpleNamespace(diff --git a/tests/test_delivery_ack_reaction.py b/tests/test_delivery_ack_reaction.pyindex cb645a938..327425f62 100644--- a/tests/test_delivery_ack_reaction.py+++ b/tests/test_delivery_ack_reaction.py@@ -98,6 +98,7 @@ def _svc(im, *, reactions=True):class AdmissionAckTests(unittest.IsolatedAsyncioTestCase):async def test_steered_input_reports_a_receipt(self):+ """MESSAGE-DELIVERY-301: input steered into the running turn reports that to its sender."""im = _FakeIM()svc = _svc(im)@@ -107,6 +108,7 @@ class AdmissionAckTests(unittest.IsolatedAsyncioTestCase):self.assertEqual(im.calls, [("add", "m1", STEERED_REACTION_EMOJI)])async def test_queued_and_pending_steer_report_the_queued_receipt(self):+ """MESSAGE-DELIVERY-301: queued input, and input still waiting to steer, report the queued receipt."""for state in ("queued", "pending_steer"):with self.subTest(state=state):im = _FakeIM()@@ -118,6 +120,7 @@ class AdmissionAckTests(unittest.IsolatedAsyncioTestCase):self.assertEqual(im.calls, [("add", "m1", QUEUED_REACTION_EMOJI)])async def test_unconfirmed_and_undelivered_states_are_distinguishable(self):+ """MESSAGE-DELIVERY-301: an unconfirmed steer and an undelivered input read differently."""cases = {"reconciling_steer": UNCONFIRMED_REACTION_EMOJI,"retired": NOT_DELIVERED_REACTION_EMOJI,@@ -408,6 +411,7 @@ class ReactionTargetSurvivalTests(unittest.TestCase):"""The reaction target must survive admission and durable hydration."""def test_quick_reply_echo_is_captured_from_the_ingress_context(self):+ """MESSAGE-DELIVERY-305: the receipt target is captured from the context that took the input."""from core.handlers.message_handler import MessageHandlercontext = _ctx(message_id=None)@@ -421,6 +425,7 @@ class ReactionTargetSurvivalTests(unittest.TestCase):self.assertIsNone(MessageHandler._reaction_target(_ctx()))def test_hydrated_context_gets_its_reaction_target_back(self):+ """MESSAGE-DELIVERY-305: a context hydrated from its Delivery in a new process gets the target back."""from core.handlers.message_handler import MessageHandlerhydrated = _ctx(message_id="delivery-abc")diff --git a/tests/test_harness_failure_visibility.py b/tests/test_harness_failure_visibility.pyindex 0d3f89572..06fd5b1ca 100644--- a/tests/test_harness_failure_visibility.py+++ b/tests/test_harness_failure_visibility.py@@ -233,7 +233,7 @@ def _cancel_mid_write(sqlite_store, run_id: str, *, fire_on: str):def test_a_cancel_landing_under_the_coalesced_completer_is_not_overwritten(tmp_path: Path,) -> None:- """Subordinate to HFR-060/061 — Stop must outrank a stale terminal snapshot.+ """Subordinate to HFR-060/HFR-061 — Stop must outrank a stale terminal snapshot.HFR-061 stopped the coalesced completer clobbering an already-SETTLED row byskipping terminal statuses, but it deliberately let ``canceled`` fall through so a@@ -284,7 +284,7 @@ def test_a_cancel_landing_under_the_coalesced_completer_is_not_overwritten(def test_a_cancel_landing_under_settle_run_terminal_is_not_overwritten(tmp_path: Path,) -> None:- """Subordinate to HFR-060/061 — the same hole in the zombie-settlement writer.+ """Subordinate to HFR-060/HFR-061 — the same hole in the zombie-settlement writer.``settle_run_terminal`` scopes its UPDATE to ``queued|running``, which catches acancel of a QUEUED row because ``cancel_run`` flips that one straight to@@ -399,7 +399,7 @@ def test_a_cancel_landing_before_deferred_owner_reservation_is_not_overwritten(def test_the_cancel_cas_does_not_leave_an_uncancelled_run_unsettled(tmp_path: Path) -> None:- """Subordinate to HFR-060/061 — the guard must not cost an ordinary settlement.+ """Subordinate to HFR-060/HFR-061 — the guard must not cost an ordinary settlement.The negative control for the two races above. A ``cancel_requested`` predicate isthe kind of guard that quietly refuses writes nobody raced — the failure modediff --git a/tests/test_model_hub_l3.py b/tests/test_model_hub_l3.pyindex c2c0c02af..5034aef7a 100644--- a/tests/test_model_hub_l3.py+++ b/tests/test_model_hub_l3.py@@ -8017,7 +8017,9 @@ def test_auth_setup_executable_scenarios_are_registered() -> None:if match:scenario_id = match.group(1)assert scenario_id in registered, (scenario_id, node.name)- assert registered[scenario_id]["test"].endswith("::" + node.name)+ evidence = registered[scenario_id]["test"]+ references = evidence if isinstance(evidence, list) else [evidence]+ assert any(reference.endswith("::" + node.name) for reference in references)@pytest.mark.parametrize("protocol", _MODEL_FREE_OBSERVATION_PROTOCOLS)diff --git a/tests/test_scheduled_tasks.py b/tests/test_scheduled_tasks.pyindex e7391c586..9e5988dcd 100644--- a/tests/test_scheduled_tasks.py+++ b/tests/test_scheduled_tasks.py@@ -3786,7 +3786,7 @@ def test_restart_recovers_delivery_owned_run_without_unrelated_run_arbitration(monkeypatch,tmp_path,) -> None:- """Only the Delivery FIFO can block recovery; an unlinked Run cannot."""+ """HFR-004: only the Delivery FIFO can block recovery; an unlinked Run cannot."""from core.session_turns import (SCHEDULED_PROVENANCE_KEY,@@ -5567,7 +5567,7 @@ def test_legacy_queue_hold_metadata_has_no_sweep_authority(def test_legacy_hold_metadata_does_not_change_turn_or_run_ownership(tmp_path: Path, monkeypatch) -> None:- """Only explicit Turn attribution owns running work; legacy flags own nothing."""+ """HFR-026: only explicit Turn attribution owns running work; legacy flags own nothing."""monkeypatch.setenv("AVIBE_HOME", str(tmp_path))request_store = TaskExecutionStore()@@ -5600,6 +5600,10 @@ def test_legacy_hold_metadata_does_not_change_turn_or_run_ownership(service._sweep_stale_runs()+ # The attributed run outlives its orphan grace because a Turn owns it, and the+ # legacy hold flag gives the other two rows no owner at all.+ assert request_store.get_run(primary)["status"] == "running"+ assert session_turns.owned_agent_run_ids() == {primary}assert request_store.get_run(held)["status"] == "queued"assert request_store.get_run(unowned)["status"] == "queued"@@ -6031,7 +6035,7 @@ def test_sweep_spares_a_hold_parked_behind_a_live_session_turn(def test_canceling_a_delivery_owned_run_retires_its_exact_queue_row(tmp_path: Path, monkeypatch) -> None:- """Run cancellation and exact Delivery retirement share one transaction."""+ """HFR-028: Run cancellation and exact Delivery retirement share one transaction."""from storage.background import attach_agent_run_delivery_in_connectionfrom storage.models import agent_sessionsdiff --git a/ui/scripts/scenarioCatalog.mjs b/ui/scripts/scenarioCatalog.mjsindex 3619710fa..6a33e633b 100644--- a/ui/scripts/scenarioCatalog.mjs+++ b/ui/scripts/scenarioCatalog.mjs@@ -53,7 +53,8 @@ const POINTER_KEYS = ['related_tests', 'supporting_tests'];* the UI half of a row owned by a pytest scenario — and reading only the first* left five fully-written citations unasked, which is the legend hole again in* another key. It cost four of those five: three named cases #1401 deleted on- * 2026-08-14, unnoticed because nothing read them.+ * 2026-08-14, unnoticed because nothing read them. `partial_evidence` is a third+ * such key and is read for the same reason.** `canonical_tests` stays unread, and so does a pointer key's bare file, and* that is the same rule rather than an exception to it: they name a file and no@@ -69,8 +70,16 @@ const POINTER_KEYS = ['related_tests', 'supporting_tests'];export const uiEvidenceRows = (catalog) =>(catalog.scenarios ?? []).flatMap((row) => {const citations = [];- if (typeof row.test === 'string' && isUiEvidence(row.test)) {- const [file, cited = null] = row.test.split('::');+ // A row whose contract several cases prove lists each of them, and each must resolve on its own.+ for (const evidence of Array.isArray(row.test) ? row.test : [row.test]) {+ if (typeof evidence === 'string' && isUiEvidence(evidence)) {+ const [file, cited = null] = evidence.split('::');+ citations.push({ id: row.id, file, cited });+ }+ }+ const partial = row.partial_evidence;+ if (partial && typeof partial.test === 'string' && isUiEvidence(partial.test)) {+ const [file, cited = null] = partial.test.split('::');citations.push({ id: row.id, file, cited });}const contract = row.ui_contract;diff --git a/ui/scripts/scenarioCatalog.test.mjs b/ui/scripts/scenarioCatalog.test.mjsindex 51e98c4e6..646876eed 100644--- a/ui/scripts/scenarioCatalog.test.mjs+++ b/ui/scripts/scenarioCatalog.test.mjs@@ -174,9 +174,17 @@ describe('UI evidence resolution', () => {related_tests: ['ui/src/e.test.tsx::names its neighbour', 'tests/test_e.py::test_e'],supporting_tests: ['ui/src/f.test.tsx::supports it'],},+ // Evidence several cases prove together: each listed case is asked on its own.+ { id: 'MH-LIST', status: 'covered', test: ['ui/src/h.test.tsx::MH-LIST first', 'tests/test_g.py::test_g', 'ui/src/h.test.tsx::MH-LIST second'] },+ {+ id: 'MH-PARTIAL',+ status: 'partial',+ test: 'tests/test_f.py::test_f',+ partial_evidence: { covers: 'the UI half', test: 'ui/src/g.test.tsx::MH-PARTIAL' },+ },{ id: 'MH-GAP', status: 'gap', test: null },];- const asked = ['MH-UI', 'MH-SCRIPT', 'MH-FILE', 'MH-CONTRACT', 'MH-POINTER', 'MH-POINTER'];+ const asked = ['MH-UI', 'MH-SCRIPT', 'MH-FILE', 'MH-CONTRACT', 'MH-POINTER', 'MH-POINTER', 'MH-LIST', 'MH-LIST', 'MH-PARTIAL'];const rowsOf = (catalog) => uiEvidenceRows({ ...catalog, scenarios });expect(rowsOf({ status_legend: { covered: { test_required: true } } }).map((row) => row.id)).toEqual(asked);