Enterprise agent platform for building, deploying, and operating AI teammates.
feat(connector-checks): Tier 2 structural read checks for linear, intercom, deputy, employment-hero and xero (#3754)
xiaolei4953bfec0a4e
diff --git a/docs/operations/connector-smoke-v2.md b/docs/operations/connector-smoke-v2.mdindex 176394dc5..9d830a04f 100644--- a/docs/operations/connector-smoke-v2.md+++ b/docs/operations/connector-smoke-v2.md@@ -199,6 +199,8 @@ The six document connectors (`google-docs`, `google-sheets`, `google-slides`, `p`outlook`, `sharepoint` and `planner` have adopted V2 the same way: each keeps its L3 call and adds structural read checks on the reads that need no object id or query. `sharepoint` passes Graph's `root` site alias as the site id, which needs no fixture, and a `planner` list item asserts only the keys that survive the connector's output projection. A connector whose test account is not connected yet still declares its checks: the run has no target for it and reports no row until the account connects (#3573).+`linear`, `intercom`, `deputy`, `employment-hero` and `xero` have adopted V2 the same way: each keeps its L3 call and adds structural read checks on cheap reads that need no object id, date or query, and exempts every other tool. The checked reads are Linear's team, project, user and issue lists, Intercom's conversation list, Deputy's `Company` list and `Employee` field schema (`deputy_resource_info`, which describes a resource type and is not a list), Employment Hero's organisation list and the default-argument list reads of Xero. The exempted reads need an id (`linear_get_issue`, `intercom_get_contact`, `deputy_get_resource` and the Employment Hero reads that take an organisation id), a date or report parameters, or a free-form query (`deputy_query_resource`, `intercom_search_contacts`), or they scan a whole organisation's invoices (`list_overdue_invoices`, `list_receivables_ageing`, `list_paid_invoices`). A list that pages is requested one item at a time (`limit`, `page_size` or `item_per_page` of 1), and a check asserts the shape of the answer, not a count, so it passes on an empty list. Xero's `GET /Quotes` has no page size, so `list_quotes` asserts its paging facts but no length cap. `employment-hero`'s server has not been exercised against a live account, so its check is the first run of it (#3573).+`jira` has adopted V2 at the Tier 1 bar (#3570). It keeps its L3 call and adds four read checks on the test tenant's seeded read project: `jira_search_issues` with the query `jira.seed.search_jql`, whose first issue is `jira.seed.issue_key`; `jira_list_comments` on that issue, whose first comment is `jira.seed.comment_id`; `jira_get_issue` on it; and `jira_get_current_user`. Its write check is the round trip shown under [Write checks and the sweep](#write-checks-and-the-sweep), with the sweep reading `jira.write_sweep_jql`. Each step that reads or writes issues passes `cloud_id` from `jira.tenant_id` instead of relying on the tool's single-site auto-detection; the tenant pin still refuses the write when the grant reaches any other site. The other seven tools are exempt: three reads the checked tools make unnecessary (they carry over 80% of Jira's calls), and four writes the write-check format cannot express, since each edits or adds to an existing issue rather than creating one it can delete.`slack` has adopted V2 at the Tier 1 bar (#3571), with read checks only. It keeps its L3 call and adds four read checks on the QA workspace's seed, a public channel named `xagent-smoke-seed` with a set topic and one thread in it: `slack_list_channels` finds that channel by name and asserts its id, `slack.seed.channel_id`; `slack_get_channel_history` reads the channel up to the thread's reply, `slack.seed.reply_ts`, and asserts the thread's parent, `slack.seed.thread_ts`; `slack_get_thread_replies` reads that thread; and `slack_get_channel_info` asserts the channel's name and topic. The seed posts the thread last, after its bots joined and it set the topic, each of which leaves a message in the channel; the history read stops at the reply, which `latest` excludes, so a message posted to the channel later shifts nothing the checks assert. A later reply in the thread comes after the one `slack_get_thread_replies` asserts, and the history read asserts only that the parent has a reply count, which grows with it. `slack.seed.thread_ts` and `slack.seed.reply_ts` must be strings, as Slack writes a `ts` (`"1762905600.000100"`): as a tool argument a number is refused before the call (`call_failed`), in an assertion it never equals the string the tools return, and either way it loses digits. A new install of an app not on the Slack Marketplace may call `conversations.history` and `conversations.replies` once a minute and gets at most 15 messages back, so a run calls each of them once with a small limit, and `slack_search_messages`, which calls both again, is exempt. A second run on the same Slack app within about a minute gets a 429 whose `Retry-After` exceeds the 30 s the tool waits, so the throttled reads fail as `assertion_failed` with reason `status_success`, as they do when the seed's channel or thread is gone; leave a minute between runs before reading such a failure as drift. The QA workspace is on Slack's free plan, which hides messages older than 90 days, so the thread must be re-seeded before then, and the new `ts` values replace `slack.seed.thread_ts` and `slack.seed.reply_ts` in the tenant registry. `slack_list_direct_messages` is exempt because a Toby run's bot grant admits public channels only (xorbitsai/xagent-saas#2553), so its listing is empty. No write tool has a check: `slack_post_message` and `slack_upload_file` create what no tool of the server deletes, and whether `slack_join_channel` and the reaction tools get a write check is decided in #3743, since no tool reads the workspace's team id for a tenant pin and the bot grant only reads.diff --git a/src/xagent/web/connector_checks/registry.py b/src/xagent/web/connector_checks/registry.pyindex 853ce9ecc..d7641ca9d 100644--- a/src/xagent/web/connector_checks/registry.py+++ b/src/xagent/web/connector_checks/registry.py@@ -15,6 +15,7 @@ from collections.abc import Mappingfrom ..connector_probes import get_connector_probe, get_unprobeable_reasonfrom .declarations import (+ Assertion,ConnectorChecks,Contains,CreatedRef,@@ -196,6 +197,67 @@ def _declared_l3(app_id: str) -> ReadCheck:return check+def _xero_list_check(list_field: str, *, paged: bool, capped: bool = True) -> ReadCheck:+ """A structural read check on one of Xero's list tools that take no id.++ The tool is ``list_<list_field>``. Every Xero list answers+ ``{status, <list_field>: [...], returned}`` through ``_unpaged_response``+ (with ``total_count``) or ``_paged_response`` (with ``page``, ``page_size``+ and ``has_more``). The items are ``_compact`` views that drop an empty+ field, so only the list and its paging facts are asserted, never an item's+ keys.++ A paged list is asked for one item and capped at one, unless ``capped`` is+ false: ``GET /Quotes`` has no ``pageSize`` parameter (Xero's OpenAPI+ description lists only ``page``), so a QA organisation with two quotes+ answers both.+ """+ tool = f"list_{list_field}"+ if paged:+ facts: tuple[Assertion, ...] = (+ HasKeys((), (list_field, "page", "page_size", "returned", "has_more")),+ IsType(("has_more",), "boolean"),+ # The type first: the judge stops at the first failure, and Length+ # of a value that is not a list would report `length`.+ IsType((list_field,), "array"),+ *((Length((list_field,), max=1),) if capped else ()),+ )+ else:+ facts = (+ HasKeys((), (list_field, "total_count", "returned")),+ IsType(("total_count",), "number"),+ IsType((list_field,), "array"),+ )+ return ReadCheck(+ f"xero.{tool}",+ tool,+ {"page_size": 1} if paged else {},+ assertions=(StatusSuccess(), *facts, IsType(("returned",), "number")),+ )+++def _linear_list_check(+ tool_suffix: str, list_field: str, item_keys: tuple[str, ...]+) -> ReadCheck:+ """A structural read check on one of Linear's list or search tools.++ Each answers ``{status, <list_field>: [...], truncated}``; the keys of an+ item are the ones the tool's GraphQL selection always returns, null or not.+ """+ return ReadCheck(+ f"linear.{tool_suffix}",+ f"linear_{tool_suffix}",+ {"limit": 1},+ assertions=(+ StatusSuccess(),+ HasKeys((), (list_field, "truncated")),+ EachHasKeys((list_field,), item_keys),+ Length((list_field,), max=1),+ IsType(("truncated",), "boolean"),+ ),+ )++def _jira_checks() -> ConnectorChecks:"""Jira's Tier 1 checks (#3570).@@ -1741,6 +1803,223 @@ CONNECTOR_CHECKS: Mapping[str, ConnectorChecks] = {writes_without_cleanup=("zoom_create_meeting",),),),+ "linear": ConnectorChecks(+ "linear",+ reads=(+ _declared_l3("linear"),+ _linear_list_check("list_teams", "teams", ("id", "key", "name")),+ _linear_list_check(+ "list_projects",+ "projects",+ ("id", "name", "status", "progress", "url", "teams"),+ ),+ _linear_list_check(+ "search_users", "users", ("id", "name", "email", "active")+ ),+ _linear_list_check(+ "search_issues",+ "issues",+ (+ "id",+ "identifier",+ "title",+ "priority",+ "url",+ "createdAt",+ "updatedAt",+ "state",+ "assignee",+ "team",+ "project",+ "labels",+ ),+ ),+ ),+ exemptions=_exemptions(+ issue=_TIER_2,+ reads=(+ "linear_get_issue",+ "linear_list_comments",+ "linear_list_labels",+ "linear_list_workflow_states",+ ),+ in_place_edits=("linear_update_issue",),+ writes_without_cleanup=("linear_create_issue", "linear_add_comment"),+ ),+ ),+ "intercom": ConnectorChecks(+ "intercom",+ reads=(+ _declared_l3("intercom"),+ ReadCheck(+ "intercom.list_conversations",+ "intercom_list_conversations",+ # Not the L3 probe's call: its default `state` is "open", which+ # a QA workspace usually has no conversation in, and the item+ # keys would never be asserted.+ {"state": "all", "limit": 1},+ assertions=(+ StatusSuccess(),+ HasKeys(+ (), ("conversations", "total_count", "has_more", "next_cursor")+ ),+ EachHasKeys(+ ("conversations",),+ (+ "id",+ "state",+ "title",+ "subject",+ "preview",+ "created_at",+ "updated_at",+ ),+ ),+ Length(("conversations",), max=1),+ IsType(("has_more",), "boolean"),+ ),+ ),+ ),+ exemptions=_exemptions(+ issue=_TIER_2,+ reads=(+ "intercom_get_contact",+ "intercom_get_conversation",+ "intercom_search_contacts",+ ),+ in_place_edits=("intercom_close_conversation",),+ writes_without_cleanup=(+ "intercom_add_internal_note",+ "intercom_reply_to_conversation",+ ),+ ),+ ),+ "deputy": ConnectorChecks(+ "deputy",+ reads=(+ _declared_l3("deputy"),+ # The resource is a Deputy object type, not an object id.+ ReadCheck(+ "deputy.list_companies",+ "deputy_list_resource",+ {"resource": "Company"},+ assertions=(+ StatusSuccess(),+ HasKeys((), ("records", "truncated")),+ IsType(("records",), "array"),+ IsType(("truncated",), "boolean"),+ ),+ ),+ ReadCheck(+ "deputy.employee_info",+ "deputy_resource_info",+ {"resource": "Employee"},+ assertions=(+ StatusSuccess(),+ HasKeys((), ("info",)),+ IsType(("info",), "object"),+ ),+ ),+ ),+ exemptions=_exemptions(+ issue=_TIER_2,+ reads=("deputy_get_resource", "deputy_query_resource"),+ in_place_edits=("deputy_update_resource",),+ writes_without_cleanup=("deputy_add_employee", "deputy_create_resource"),+ ),+ ),+ "employment-hero": ConnectorChecks(+ "employment-hero",+ reads=(+ _declared_l3("employment-hero"),+ # The list's own `data` envelope, as Employment Hero documents it:+ # {"items": [...], ...}. The server has not run against a live+ # account, so a live answer that differs fails here first.+ ReadCheck(+ "employment-hero.list_organisations",+ "employment_hero_list_organisations",+ {"item_per_page": 1},+ assertions=(+ StatusSuccess(),+ HasKeys((), ("organisations",)),+ IsType(("organisations",), "object"),+ HasKeys(("organisations",), ("items",)),+ IsType(("organisations", "items"), "array"),+ Length(("organisations", "items"), max=1),+ ),+ ),+ ),+ exemptions=_exemptions(+ issue=_TIER_2,+ reads=(+ "employment_hero_get_employee",+ "employment_hero_get_leave_balances",+ "employment_hero_list_employees",+ "employment_hero_list_leave_categories",+ "employment_hero_list_leave_requests",+ "employment_hero_list_team_employees",+ "employment_hero_list_teams",+ "employment_hero_list_timesheet_entries",+ ),+ in_place_edits=("employment_hero_approve_leave_request",),+ writes_without_cleanup=("employment_hero_create_leave_request",),+ ),+ ),+ "xero": ConnectorChecks(+ "xero",+ reads=(+ _declared_l3("xero"),+ _xero_list_check("accounts", paged=False),+ _xero_list_check("contact_groups", paged=False),+ _xero_list_check("items", paged=False),+ _xero_list_check("tax_rates", paged=False),+ _xero_list_check("tracking_categories", paged=False),+ _xero_list_check("contacts", paged=True),+ _xero_list_check("invoices", paged=True),+ _xero_list_check("credit_notes", paged=True),+ _xero_list_check("bank_transactions", paged=True),+ _xero_list_check("payments", paged=True),+ _xero_list_check("quotes", paged=True, capped=False),+ _xero_list_check("manual_journals", paged=True),+ ),+ exemptions=_exemptions(+ issue=_TIER_2,+ reads=(+ "list_aged_payables_by_contact",+ "list_aged_receivables_by_contact",+ "list_invoice_history",+ "list_overdue_invoices",+ "list_paid_invoices",+ "list_profit_and_loss",+ "list_receivables_ageing",+ "list_report_balance_sheet",+ "list_trial_balance",+ ),+ in_place_edits=(+ "update_bank_transaction",+ "update_contact",+ "update_credit_note",+ "update_invoice",+ "update_item",+ "update_manual_journal",+ "update_quote",+ "update_tracking_category",+ "update_tracking_options",+ ),+ writes_without_cleanup=(+ "create_bank_transaction",+ "create_contact",+ "create_credit_note",+ "create_invoice",+ "create_item",+ "create_manual_journal",+ "create_payment",+ "create_quote",+ "create_tracking_category",+ "create_tracking_options",+ ),+ ),+ ),}@@ -1754,8 +2033,6 @@ LEGACY_CONNECTORS: frozenset[str] = frozenset("aws","chartmogul","chrome-devtools",- "deputy",- "employment-hero","facebook","fireflies","freshdesk",@@ -1763,8 +2040,6 @@ LEGACY_CONNECTORS: frozenset[str] = frozenset("google-search-console","granola","instagram",- "intercom",- "linear","linkedin","magento","meta-ads",@@ -1776,7 +2051,6 @@ LEGACY_CONNECTORS: frozenset[str] = frozenset("shopify","stripe","whatsapp",- "xero",})diff --git a/tests/web/connector_checks/test_registry.py b/tests/web/connector_checks/test_registry.pyindex 1538e146e..68f502569 100644--- a/tests/web/connector_checks/test_registry.py+++ b/tests/web/connector_checks/test_registry.py@@ -266,6 +266,57 @@ def test_the_fixture_is_bound_to_a_required_parameter(app_id: str) -> None:assert required <= set(check.arguments), (check.tool, required)+def _linear_pin(+ tool: str, field: str, item_keys: tuple[str, ...]+) -> tuple[str, dict[str, Any], tuple[Assertion, ...]]:+ return (+ tool,+ {"limit": 1},+ (+ StatusSuccess(),+ HasKeys((), (field, "truncated")),+ EachHasKeys((field,), item_keys),+ Length((field,), max=1),+ IsType(("truncated",), "boolean"),+ ),+ )+++def _xero_unpaged_pin(+ field: str,+) -> tuple[str, dict[str, Any], tuple[Assertion, ...]]:+ return (+ f"list_{field}",+ {},+ (+ StatusSuccess(),+ HasKeys((), (field, "total_count", "returned")),+ IsType(("total_count",), "number"),+ IsType((field,), "array"),+ IsType(("returned",), "number"),+ ),+ )+++def _xero_paged_pin(+ field: str, *, capped: bool = True+) -> tuple[str, dict[str, Any], tuple[Assertion, ...]]:+ # `GET /Quotes` has no `pageSize`, so the quotes check caps nothing.+ cap: tuple[Assertion, ...] = (Length((field,), max=1),) if capped else ()+ return (+ f"list_{field}",+ {"page_size": 1},+ (+ StatusSuccess(),+ HasKeys((), (field, "page", "page_size", "returned", "has_more")),+ IsType(("has_more",), "boolean"),+ IsType((field,), "array"),+ *cap,+ IsType(("returned",), "number"),+ ),+ )++# The Tier 2 read checks (issue #3573): the tool, its arguments, and every# assertion, so dropping an item key or a whole assertion fails here._TIER_2_READS: Mapping[str, tuple[str, dict[str, Any], tuple[Assertion, ...]]] = {@@ -458,6 +509,96 @@ _TIER_2_READS: Mapping[str, tuple[str, dict[str, Any], tuple[Assertion, ...]]] =IsType(("truncated",), "boolean"),),),+ "linear.list_teams": _linear_pin(+ "linear_list_teams", "teams", ("id", "key", "name")+ ),+ "linear.list_projects": _linear_pin(+ "linear_list_projects",+ "projects",+ ("id", "name", "status", "progress", "url", "teams"),+ ),+ "linear.search_users": _linear_pin(+ "linear_search_users", "users", ("id", "name", "email", "active")+ ),+ "linear.search_issues": _linear_pin(+ "linear_search_issues",+ "issues",+ (+ "id",+ "identifier",+ "title",+ "priority",+ "url",+ "createdAt",+ "updatedAt",+ "state",+ "assignee",+ "team",+ "project",+ "labels",+ ),+ ),+ "intercom.list_conversations": (+ "intercom_list_conversations",+ {"state": "all", "limit": 1},+ (+ StatusSuccess(),+ HasKeys((), ("conversations", "total_count", "has_more", "next_cursor")),+ EachHasKeys(+ ("conversations",),+ (+ "id",+ "state",+ "title",+ "subject",+ "preview",+ "created_at",+ "updated_at",+ ),+ ),+ Length(("conversations",), max=1),+ IsType(("has_more",), "boolean"),+ ),+ ),+ "deputy.list_companies": (+ "deputy_list_resource",+ {"resource": "Company"},+ (+ StatusSuccess(),+ HasKeys((), ("records", "truncated")),+ IsType(("records",), "array"),+ IsType(("truncated",), "boolean"),+ ),+ ),+ "deputy.employee_info": (+ "deputy_resource_info",+ {"resource": "Employee"},+ (StatusSuccess(), HasKeys((), ("info",)), IsType(("info",), "object")),+ ),+ "employment-hero.list_organisations": (+ "employment_hero_list_organisations",+ {"item_per_page": 1},+ (+ StatusSuccess(),+ HasKeys((), ("organisations",)),+ IsType(("organisations",), "object"),+ HasKeys(("organisations",), ("items",)),+ IsType(("organisations", "items"), "array"),+ Length(("organisations", "items"), max=1),+ ),+ ),+ "xero.list_accounts": _xero_unpaged_pin("accounts"),+ "xero.list_contact_groups": _xero_unpaged_pin("contact_groups"),+ "xero.list_items": _xero_unpaged_pin("items"),+ "xero.list_tax_rates": _xero_unpaged_pin("tax_rates"),+ "xero.list_tracking_categories": _xero_unpaged_pin("tracking_categories"),+ "xero.list_contacts": _xero_paged_pin("contacts"),+ "xero.list_invoices": _xero_paged_pin("invoices"),+ "xero.list_credit_notes": _xero_paged_pin("credit_notes"),+ "xero.list_bank_transactions": _xero_paged_pin("bank_transactions"),+ "xero.list_payments": _xero_paged_pin("payments"),+ "xero.list_quotes": _xero_paged_pin("quotes", capped=False),+ "xero.list_manual_journals": _xero_paged_pin("manual_journals"),}@@ -707,7 +848,20 @@ def test_a_hubspot_check_fails_without_what_every_portal_has(@pytest.mark.parametrize(- "app_id", ["google-ads", "outlook", "planner", "sharepoint", "zendesk", "zoom"]+ "app_id",+ [+ "deputy",+ "employment-hero",+ "google-ads",+ "intercom",+ "linear",+ "outlook",+ "planner",+ "sharepoint",+ "xero",+ "zendesk",+ "zoom",+ ],)def test_a_tier_2_l3_check_is_the_probe_derived_one(app_id: str) -> None:expected = l3_read_check(app_id)diff --git a/tests/web/connector_checks/test_tier_2_checks.py b/tests/web/connector_checks/test_tier_2_checks.pynew file mode 100644index 000000000..14e6f9890--- /dev/null+++ b/tests/web/connector_checks/test_tier_2_checks.py@@ -0,0 +1,457 @@+"""Smoke V2 Tier 2 checks of linear, intercom, deputy, employment-hero and xero (#3573).++The declarations are pinned in ``test_registry.py`` (``_TIER_2_READS``). This+runs each declared call against the real tool with only the vendor request+replaced by a realistic answer, judges what the tool returns with the check's+own assertions, and records the request, so a tool whose output shape drifts+from its declaration, or that stops sending the page size a check relies on,+fails here and not on the QA member.+"""++from __future__ import annotations++import json+from collections.abc import Callable+from types import ModuleType+from typing import Any++import pytest++from xagent.web.connector_checks.assertions import judge_result+from xagent.web.connector_checks.declarations import ReadCheck+from xagent.web.connector_checks.registry import (+ _TIER_2,+ CONNECTOR_CHECKS,+ WRITE_WITHOUT_CLEANUP,+)+from xagent.web.connector_checks.results import FailureClass+from xagent.web.tools.mcp import deputy, employment_hero, intercom, linear, xero++MODULES: dict[str, ModuleType] = {+ "linear": linear,+ "intercom": intercom,+ "deputy": deputy,+ "employment-hero": employment_hero,+ "xero": xero,+}+++def _check(check_id: str) -> ReadCheck:+ app_id = check_id.split(".", 1)[0]+ check = next(c for c in CONNECTOR_CHECKS[app_id].reads if c.id == check_id)+ assert isinstance(check, ReadCheck)+ return check+++def _structural_ids(app_id: str) -> list[str]:+ return [+ c.id+ for c in CONNECTOR_CHECKS[app_id].reads+ if isinstance(c, ReadCheck) and c.id != f"{app_id}.l3_probe"+ ]+++ALL_IDS = [cid for app in MODULES for cid in _structural_ids(app)]+++def _judge(check_id: str, output: str) -> tuple[FailureClass | None, str | None]:+ check = _check(check_id)+ raw = {+ "content": [{"type": "text", "text": output}],+ "structured_content": None,+ "is_error": False,+ }+ return judge_result(raw, check.payload, check.assertions)+++def _run(check_id: str) -> str:+ """The check's own call, made on the real tool: its arguments must fit it."""+ check = _check(check_id)+ tool: Callable[..., str] = getattr(MODULES[check_id.split(".", 1)[0]], check.tool)+ return tool(**check.arguments)+++# --- what is declared ------------------------------------------------------------++[email protected]("app_id", MODULES)+def test_tier_2_declares_reads_only(app_id: str) -> None:+ checks = CONNECTOR_CHECKS[app_id]++ assert checks.writes == () and checks.sweep is None+ assert _structural_ids(app_id)++[email protected]("app_id", MODULES)+def test_every_exemption_names_tier_2_or_says_why_it_has_no_cleanup(+ app_id: str,+) -> None:+ for exemption in CONNECTOR_CHECKS[app_id].exemptions:+ assert exemption.reason == WRITE_WITHOUT_CLEANUP or exemption.reason.endswith(+ f"decided in {_TIER_2}"+ ), (app_id, exemption.tool)+++# --- what the tools really return, and what they send ------------------------------+++class _Capture:+ """The vendor requests of one tool call."""++ def __init__(self) -> None:+ self.requests: list[dict[str, Any]] = []+++def _linear_node(field: str) -> dict[str, Any]:+ nodes: dict[str, dict[str, Any]] = {+ "teams": {"id": "t1", "key": "ENG", "name": "Engineering"},+ "projects": {+ "id": "p1",+ "name": "Roadmap",+ "status": {"id": "s1", "name": "Planned", "type": "planned"},+ "progress": 0.5,+ "url": "https://linear.app/x/project/roadmap",+ "teams": {"nodes": [{"id": "t1", "key": "ENG", "name": "Engineering"}]},+ },+ "users": {+ "id": "u1",+ "name": "Ada",+ "email": "[email protected]",+ "active": True,+ },+ "issues": {+ "id": "i1",+ "identifier": "ENG-1",+ "title": "A bug",+ "priority": 2,+ "url": "https://linear.app/x/issue/ENG-1",+ "createdAt": "2026-01-01T00:00:00Z",+ "updatedAt": "2026-01-02T00:00:00Z",+ "state": {"id": "s2", "name": "Todo", "type": "unstarted"},+ # An unassigned, project-less issue still carries the keys, as null.+ "assignee": None,+ "team": {"id": "t1", "key": "ENG", "name": "Engineering"},+ "project": None,+ "labels": {"nodes": []},+ },+ }+ return nodes[field]+++_LINEAR = {+ "linear.list_teams": "teams",+ "linear.list_projects": "projects",+ "linear.search_users": "users",+ "linear.search_issues": "issues",+}++[email protected]("count", [0, 1])[email protected]("check_id", sorted(_LINEAR))+def test_a_linear_list_output_satisfies_its_declaration_and_asks_for_one_item(+ monkeypatch: pytest.MonkeyPatch, check_id: str, count: int+) -> None:+ field = _LINEAR[check_id]+ seen = _Capture()++ def graphql(query: str, variables: dict[str, Any] | None = None):+ seen.requests.append(variables or {})+ return (+ {field: {"nodes": [_linear_node(field)] * count, "pageInfo": {}}},+ None,+ )++ monkeypatch.setattr(linear, "_graphql", graphql)++ assert _judge(check_id, _run(check_id)) == (None, None)+ assert any(v.get("first") == 1 for v in seen.requests), seen.requests++[email protected]("count", [0, 1])+def test_intercom_list_output_satisfies_its_declaration_and_asks_for_one_item(+ monkeypatch: pytest.MonkeyPatch, count: int+) -> None:+ seen = _Capture()+ conversation = {+ "id": "1",+ "state": "open",+ "title": None,+ "source": {"subject": "Hello", "body": "<p>Hi</p>"},+ "created_at": 1700000000,+ "updated_at": 1700000100,+ }++ def request(method: str, path: str, *, body: dict[str, Any] | None = None):+ seen.requests.append(body or {})+ return {+ "conversations": [conversation] * count,+ "total_count": count,+ "pages": {},+ }++ monkeypatch.setattr(intercom, "_request", request)++ assert _judge(+ "intercom.list_conversations", _run("intercom.list_conversations")+ ) == (+ None,+ None,+ )+ (body,) = seen.requests+ assert body["pagination"]["per_page"] == 1+ # "all" is not a state filter: an open-only call would miss a closed+ # conversation, the only kind a QA workspace may hold.+ assert body["query"]["field"] != "state"++[email protected]("count", [0, 1])+def test_deputy_list_output_satisfies_its_declaration(+ monkeypatch: pytest.MonkeyPatch, count: int+) -> None:+ monkeypatch.setattr(+ deputy, "_request", lambda *a, **k: [{"Id": 1, "CompanyName": "Demo"}] * count+ )++ assert _judge("deputy.list_companies", _run("deputy.list_companies")) == (+ None,+ None,+ )+++def test_deputy_a_null_list_is_an_empty_one_and_still_satisfies_the_declaration(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ # Deputy serializes "no records" as null on some installs.+ monkeypatch.setattr(deputy, "_request", lambda *a, **k: None)++ assert _judge("deputy.list_companies", _run("deputy.list_companies")) == (+ None,+ None,+ )+++def test_deputy_resource_info_output_satisfies_its_declaration(+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ monkeypatch.setattr(+ deputy,+ "_request",+ lambda *a, **k: {"fields": {"Id": "Integer"}, "assocs": {}, "joins": {}},+ )++ assert _judge("deputy.employee_info", _run("deputy.employee_info")) == (None, None)++[email protected]("count", [0, 1])+def test_employment_hero_organisations_output_satisfies_its_declaration_and_asks_for_one(+ monkeypatch: pytest.MonkeyPatch, count: int+) -> None:+ seen = _Capture()++ def request(method: str, path: str, *, params: dict[str, Any] | None = None, **k):+ seen.requests.append(params or {})+ return {+ "data": {+ "items": [{"id": "o1", "name": "Demo", "country": "AU"}] * count,+ "page_index": 1,+ "item_per_page": 1,+ "total_items": count,+ "total_pages": 1,+ }+ }++ monkeypatch.setattr(employment_hero, "_request", request)++ check_id = "employment-hero.list_organisations"+ assert _judge(check_id, _run(check_id)) == (None, None)+ assert [r.get("item_per_page") for r in seen.requests] == [1]+++# What a Xero list is called in the vendor's answer.+_XERO_UNPAGED = {+ "xero.list_accounts": "Accounts",+ "xero.list_contact_groups": "ContactGroups",+ "xero.list_items": "Items",+ "xero.list_tax_rates": "TaxRates",+ "xero.list_tracking_categories": "TrackingCategories",+}+# Every paged list but quotes sends `pageSize`; `GET /Quotes` has none.+_XERO_PAGED_CAPPED = {+ "xero.list_contacts": "Contacts",+ "xero.list_invoices": "Invoices",+ "xero.list_credit_notes": "CreditNotes",+ "xero.list_bank_transactions": "BankTransactions",+ "xero.list_payments": "Payments",+ "xero.list_manual_journals": "ManualJournals",+}+++def _xero_request(+ seen: _Capture, vendor_key: str, count: int, *, pagination: bool+) -> Callable[..., Any]:+ def request(method: str, path: str, *, params: dict[str, Any] | None = None, **k):+ seen.requests.append(params or {})+ answer: dict[str, Any] = {vendor_key: [{}] * count}+ if pagination:+ answer["pagination"] = {+ "page": 1,+ "pageSize": 1,+ "pageCount": 1,+ "itemCount": count,+ }+ return answer++ return request++[email protected]("count", [0, 1])[email protected]("check_id", sorted(_XERO_UNPAGED))+def test_a_xero_unpaged_list_output_satisfies_its_declaration(+ monkeypatch: pytest.MonkeyPatch, check_id: str, count: int+) -> None:+ monkeypatch.setattr(+ xero,+ "_request",+ _xero_request(_Capture(), _XERO_UNPAGED[check_id], count, pagination=False),+ )++ assert _judge(check_id, _run(check_id)) == (None, None)++[email protected]("count", [0, 1])[email protected]("check_id", sorted(_XERO_PAGED_CAPPED))+def test_a_xero_paged_list_output_satisfies_its_declaration_and_asks_for_one_item(+ monkeypatch: pytest.MonkeyPatch, check_id: str, count: int+) -> None:+ seen = _Capture()+ monkeypatch.setattr(+ xero,+ "_request",+ _xero_request(seen, _XERO_PAGED_CAPPED[check_id], count, pagination=True),+ )++ assert _judge(check_id, _run(check_id)) == (None, None)+ assert [r.get("pageSize") for r in seen.requests] == [1]++[email protected]("count", [1, 2, 3])+def test_xero_quotes_answer_a_whole_page_because_the_vendor_has_no_page_size(+ monkeypatch: pytest.MonkeyPatch, count: int+) -> None:+ # `GET /Quotes` takes only `page` and answers no pagination block, so an+ # organisation with several quotes answers all of them to a page_size of 1.+ monkeypatch.setattr(+ xero,+ "_request",+ _xero_request(_Capture(), "Quotes", count, pagination=False),+ )++ assert _judge("xero.list_quotes", _run("xero.list_quotes")) == (None, None)+++# --- what a check rejects ----------------------------------------------------------++[email protected]("check_id", ALL_IDS)+def test_a_check_fails_on_a_tool_error_envelope(check_id: str) -> None:+ # A rejected token or a malformed request: the tool reports it in its own+ # envelope with the MCP error flag unset.+ output = json.dumps({"status": "error", "message": "401 Unauthorized"})++ assert _judge(check_id, output) == (+ FailureClass.ASSERTION_FAILED,+ "status_success",+ )++[email protected]("check_id", ALL_IDS)+def test_a_check_fails_when_the_list_is_missing(check_id: str) -> None:+ assert _judge(check_id, json.dumps({"status": "success"})) == (+ FailureClass.ASSERTION_FAILED,+ "has_keys",+ )+++ ("check_id", "payload", "reason"),+ [+ (+ "linear.list_teams",+ {+ "status": "success",+ "teams": [{"id": "t1", "name": "x"}],+ "truncated": False,+ },+ "each_has_keys",+ ),+ (+ "linear.list_teams",+ {"status": "success", "teams": [], "truncated": "no"},+ "is_type",+ ),+ (+ "intercom.list_conversations",+ {+ "status": "success",+ "conversations": [{"id": "1"}],+ "total_count": 1,+ "has_more": False,+ "next_cursor": None,+ },+ "each_has_keys",+ ),+ (+ "deputy.employee_info",+ {"status": "success", "info": "not an object"},+ "is_type",+ ),+ (+ "employment-hero.list_organisations",+ {"status": "success", "organisations": {"total_items": 0}},+ "has_keys",+ ),+ (+ "employment-hero.list_organisations",+ {"status": "success", "organisations": {"items": [{}, {}]}},+ "length",+ ),+ (+ "xero.list_accounts",+ {"status": "success", "accounts": {}, "total_count": 0, "returned": 0},+ "is_type",+ ),+ (+ "xero.list_invoices",+ {+ "status": "success",+ "invoices": [{}, {}],+ "page": 1,+ "page_size": 1,+ "returned": 2,+ "has_more": False,+ },+ "length",+ ),+ # Not a list at all: the type is judged before the length, so this is+ # reported as what is wrong with it.+ (+ "xero.list_invoices",+ {+ "status": "success",+ "invoices": "none",+ "page": 1,+ "page_size": 1,+ "returned": 0,+ "has_more": False,+ },+ "is_type",+ ),+ ],+)+def test_a_check_fails_on_another_shape(+ check_id: str, payload: dict[str, Any], reason: str+) -> None:+ assert _judge(check_id, json.dumps(payload)) == (+ FailureClass.ASSERTION_FAILED,+ reason,+ )diff --git a/tests/web/connector_checks/test_tool_coverage.py b/tests/web/connector_checks/test_tool_coverage.pyindex 4632a6ff2..4ed99410b 100644--- a/tests/web/connector_checks/test_tool_coverage.py+++ b/tests/web/connector_checks/test_tool_coverage.py@@ -35,8 +35,6 @@ _PINNED_LEGACY = frozenset("aws","chartmogul","chrome-devtools",- "deputy",- "employment-hero","facebook","fireflies","freshdesk",@@ -44,8 +42,6 @@ _PINNED_LEGACY = frozenset("google-search-console","granola","instagram",- "intercom",- "linear","linkedin","magento","meta-ads",@@ -57,7 +53,6 @@ _PINNED_LEGACY = frozenset("shopify","stripe","whatsapp",- "xero",})