Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(data-warehouse): add gainsight cs cta configuration and task tables (#120214)

Tom Owers4a1e16eefc24
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
index e3bd6879c233..f9ca631fff31 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md
@@ -5135,12 +5135,12 @@ Note: Diffed the official Seller API V2 OpenAPI file and also inspected the Buye
## Gainsight CS — gaps
-Today (9): `activity_timeline`, `call_to_action`, `company`, `company_person`, `cta_group`, `email_logs`, `gsuser`, `relationship`, `relationship_person`
+Today (11): `activity_timeline`, `call_to_action`, `company`, `company_person`, `cta_configuration`, `cta_group`, `cta_task`, `email_logs`, `gsuser`, `relationship`, `relationship_person`
Diffed against: <https://support.gainsight.com/gainsight_nxt/API_and_Developer_Docs>
-- [ ] `Get CTA configurations (GET /v2/cockpit/admin/picklist/lite)` — resolve CTA status, priority, type, and reason IDs into readable labels (high)
-- [ ] `Fetch Tasks (GET /v2/cockpit/task/list/{CTA_GSID})` — measure follow-up task completion, due dates, and ownership for synced calls to action (medium)
+- [x] `Get CTA configurations (GET /v2/cockpit/admin/picklist/lite)` — resolve CTA status, priority, type, and reason IDs into readable labels (high)
+- [x] `Fetch Tasks (GET /v2/cockpit/task/list/{CTA_GSID})` — measure follow-up task completion, due dates, and ownership for synced calls to action (medium)
Note: Read the official API index and the Cockpit CTA, Task, Success Plan, Person, and Data Management references. The two gaps use specialized GET routes; tasks need CTA fan-out and picklists need entity and category selection. The custom_objects field already permits extra readable objects through the generic query API, so arbitrary object names are not new gaps. Excluded write APIs, job status, and playbook configuration; tenant access determines generic object availability.
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/canonical_descriptions.py
index 34f9d841bdde..a41f3a990d2d 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/canonical_descriptions.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/canonical_descriptions.py
@@ -7,6 +7,11 @@ _COMPANY_DOCS_URL = (
"https://support.gainsight.com/gainsight_nxt/API_and_Developer_Docs/Company_and_Relationship_API/"
"Company_API_Documentation"
)
+_CTA_DOCS_URL = (
+ "https://support.gainsight.com/gainsight_nxt/API_and_Developer_Docs/Cockpit_API/"
+ "Call_To_Action_(CTA)_API_Documentation"
+)
+_TASK_DOCS_URL = "https://support.gainsight.com/gainsight_nxt/API_and_Developer_Docs/Cockpit_API/Task_APIs"
_TIMELINE_DOCS_URL = "https://support.gainsight.com/gainsight_nxt/API_and_Developer_Docs/Timeline_API/Timeline_APIs"
# Curated from the Gainsight NXT API docs. Each tenant adds its own custom fields on top of these,
@@ -91,4 +96,34 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {
"docs_url": _DOCS_URL,
"columns": _SHARED_COLUMNS,
},
+ "cta_configuration": {
+ "description": "Company and global CTA picklist values: the types, statuses, priorities, and reasons a Call to Action can take.",
+ "docs_url": _CTA_DOCS_URL,
+ "columns": {
+ "gsid": "Unique Gainsight identifier for the picklist value. Matches the type, status, priority, and reason ids on CTAs.",
+ "name": "Label shown for the picklist value.",
+ "active": "Whether the picklist value is active.",
+ "entityType": "Entity the value applies to, such as GLOBAL or COMPANY.",
+ "picklistCategory": "Picklist the value belongs to: CTA_TYPE, CTA_STATUS, CTA_PRIORITY, or CTA_REASON.",
+ "typeId": "Gainsight identifier of the CTA type the value applies to.",
+ "typeName": "Name of the CTA type the value applies to.",
+ },
+ },
+ "cta_task": {
+ "description": "Tasks attached to Calls to Action, with their owners, due dates, and completion state.",
+ "docs_url": _TASK_DOCS_URL,
+ "columns": {
+ "Gsid": "Unique Gainsight identifier for the task.",
+ "CtaId": "Gainsight identifier of the CTA the task belongs to.",
+ "Name": "Task name.",
+ "DueDate": "Date the task is due.",
+ "ClosedDate": "Date the task was closed.",
+ "IsClosed": "Whether the task is closed.",
+ "CtaId__gr.Name": "Name of the CTA the task belongs to.",
+ "OwnerId__gr.Name": "Name of the user who owns the task.",
+ "PriorityId__gr.Name": "Task priority.",
+ "StatusId__gr.Name": "Task status.",
+ "TypeId__gr.Name": "Task type.",
+ },
+ },
}
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/gainsight_cs.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/gainsight_cs.py
index 648b92b6cc7f..f88a61b7f2fd 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/gainsight_cs.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/gainsight_cs.py
@@ -1,9 +1,10 @@
import re
import json
import time
-from collections.abc import Iterator
+from collections.abc import Callable, Iterator
from datetime import UTC, datetime
from typing import Any, Optional
+from urllib.parse import urlencode
import requests
@@ -14,13 +15,24 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.mix
from products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManager
from products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SourceResponse
from products.warehouse_sources.backend.temporal.data_imports.sources.gainsight_cs.settings import (
+ COCKPIT_PRIMARY_KEYS,
+ CTA_CONFIGURATION,
+ CTA_OBJECT_NAME,
+ CTA_TASK,
DATE_DATA_TYPES,
DESCRIBE_PATH,
+ ENDPOINTS,
GAINSIGHT_CS_OBJECTS,
GSID,
MAX_PAGE_SIZE,
OBJECT_NAME_PATTERN,
+ PICKLIST_CATEGORIES,
+ PICKLIST_ENTITY_TYPE,
+ PICKLIST_PATH,
QUERY_PATH,
+ TASK_CTA_ID,
+ TASK_LIST_PATH,
+ TASK_PAGE_SIZE,
)
REQUEST_TIMEOUT = 60
@@ -35,6 +47,8 @@ RESPONSE_CHUNK_BYTES = 256 * 1024
# resumes from the saved offset, so a large object still finishes while a host that never sends a
# short page can't hold the worker for the whole activity timeout.
MAX_PAGES_PER_ATTEMPT = 2000
+# Task pages read for one CTA before the source gives up on a host that never sends a short page.
+MAX_TASK_PAGES_PER_CTA = 100
HOST_NOT_ALLOWED_ERROR = "That Gainsight domain isn't allowed."
RESPONSE_TOO_LARGE_ERROR = "Gainsight returned a response larger than this source accepts."
@@ -84,6 +98,8 @@ class GainsightCsResumeConfig:
# Row index of the next page to request. The Read API pages with limit/offset in the POST body,
# so an offset is all that's needed to pick a sync back up.
offset: int = 0
+ # cta_task only: the first task page to request for the CTA at `offset`.
+ task_page: int = 1
@frozen
@@ -147,6 +163,10 @@ def resolve_object_name(schema_name: str, custom_objects: Optional[str]) -> str:
if known is not None:
return known.object_name
+ if schema_name == CTA_TASK:
+ # Tasks are listed per CTA, so reading them starts with a query over the CTA object.
+ return CTA_OBJECT_NAME
+
if schema_name in parse_custom_objects(custom_objects):
return schema_name
@@ -165,7 +185,7 @@ def parse_custom_objects(custom_objects: Optional[str]) -> list[str]:
names = []
for raw in custom_objects.split(","):
name = raw.strip()
- if name and _OBJECT_NAME_RE.match(name) and name not in GAINSIGHT_CS_OBJECTS:
+ if name and _OBJECT_NAME_RE.match(name) and name not in ENDPOINTS:
names.append(name)
return names
@@ -240,6 +260,41 @@ def _parse_fields(body: Any, object_name: str) -> list[GainsightCsField]:
)
+def _picklist_rows(body: Any) -> list[dict[str, Any]]:
+ """Flatten the picklist response, which keys each category's values by category name."""
+ data = _envelope_data(body, CTA_CONFIGURATION)
+
+ if data is None:
+ return []
+ if not isinstance(data, dict):
+ raise ValueError(f"Gainsight returned an unexpected record shape for '{CTA_CONFIGURATION}'.")
+
+ rows: dict[tuple[str, str], dict[str, Any]] = {}
+ for category, values in data.items():
+ if not isinstance(values, list):
+ continue
+ for value in values:
+ if not isinstance(value, dict) or not value.get("gsid"):
+ continue
+ row = {**value, "picklistCategory": value.get("picklistCategory") or category}
+ rows.setdefault((row["picklistCategory"], row["gsid"]), row)
+ return list(rows.values())
+
+
+def _get_json(session: requests.Session, url: str, access_key: str, params: Optional[dict[str, Any]] = None) -> Any:
+ response = session.get(url, headers=_headers(access_key), params=params, timeout=REQUEST_TIMEOUT, stream=True)
+ _raise_for_redirect(response)
+ response.raise_for_status()
+ return _read_capped_json(response)
+
+
+def _post_json(session: requests.Session, url: str, access_key: str, body: dict[str, Any]) -> Any:
+ response = session.post(url, headers=_headers(access_key), json=body, timeout=REQUEST_TIMEOUT, stream=True)
+ _raise_for_redirect(response)
+ response.raise_for_status()
+ return _read_capped_json(response)
+
+
def describe_object(
session: requests.Session, domain: str, access_key: str, object_name: str
) -> list[GainsightCsField]:
@@ -250,10 +305,13 @@ def describe_object(
rather than hardcoded.
"""
url = f"{_base_url(domain)}{DESCRIBE_PATH.format(object_name=object_name)}"
- response = session.get(url, headers=_headers(access_key), timeout=REQUEST_TIMEOUT, stream=True)
- _raise_for_redirect(response)
- response.raise_for_status()
- return _parse_fields(_read_capped_json(response), object_name)
+ return _parse_fields(_get_json(session, url, access_key), object_name)
+
+
+def _gsid_order_by(fields: list[GainsightCsField]) -> Optional[dict[str, str]]:
+ # Paginating with limit/offset over an unordered result set can skip or repeat rows as records
+ # shift between calls, so order on Gsid when the object reports it as sortable.
+ return {GSID: "asc"} if any(field.name == GSID and field.sortable for field in fields) else None
def _raise_for_redirect(response: requests.Response) -> None:
@@ -300,9 +358,7 @@ def gainsight_cs_source(
select = [field.name for field in fields]
date_fields = frozenset(field.name for field in fields if field.data_type in DATE_DATA_TYPES)
field_names = {field.name for field in fields}
- # Paginating with limit/offset over an unordered result set can skip or repeat rows as records
- # shift between calls, so order on Gsid when the object reports it as sortable.
- order_by = {GSID: "asc"} if any(field.name == GSID and field.sortable for field in fields) else None
+ order_by = _gsid_order_by(fields)
url = f"{_base_url(domain)}{QUERY_PATH.format(object_name=object_name)}"
@@ -322,11 +378,7 @@ def gainsight_cs_source(
if order_by is not None:
body["orderBy"] = order_by
- response = session.post(url, headers=_headers(access_key), json=body, timeout=REQUEST_TIMEOUT, stream=True)
- _raise_for_redirect(response)
- response.raise_for_status()
-
- rows = _rows_from_body(_read_capped_json(response), object_name)
+ rows = _rows_from_body(_post_json(session, url, access_key, body), object_name)
if rows:
yield [_normalize_row(row, date_fields) for row in rows]
@@ -361,14 +413,119 @@ def gainsight_cs_source(
)
-def validate_credentials(
- domain: str, access_key: str, object_name: str, team_id: Optional[int]
+def gainsight_cs_cta_configuration_source(domain: str, access_key: str, team_id: int) -> SourceResponse:
+ _require_allowed_host(domain, team_id)
+
+ session = make_tracked_session(redact_values=(access_key,), allow_redirects=False)
+ url = f"{_base_url(domain)}{PICKLIST_PATH}"
+ params = {"category": ",".join(PICKLIST_CATEGORIES), "et": PICKLIST_ENTITY_TYPE}
+
+ def items() -> Iterator[list[dict[str, Any]]]:
+ _require_allowed_host(domain, team_id)
+ rows = _picklist_rows(_get_json(session, url, access_key, params))
+ if rows:
+ yield rows
+
+ return SourceResponse(
+ name=CTA_CONFIGURATION,
+ items=items,
+ primary_keys=COCKPIT_PRIMARY_KEYS[CTA_CONFIGURATION],
+ )
+
+
+def gainsight_cs_cta_task_source(
+ domain: str,
+ access_key: str,
+ team_id: int,
+ resumable_source_manager: ResumableSourceManager[GainsightCsResumeConfig],
+) -> SourceResponse:
+ _require_allowed_host(domain, team_id)
+
+ session = make_tracked_session(redact_values=(access_key,), allow_redirects=False)
+ order_by = _gsid_order_by(describe_object(session, domain, access_key, CTA_OBJECT_NAME))
+ query_url = f"{_base_url(domain)}{QUERY_PATH.format(object_name=CTA_OBJECT_NAME)}"
+
+ def cta_task_pages(cta_gsid: str, cta_offset: int, first_page: int) -> Iterator[list[dict[str, Any]]]:
+ url = f"{_base_url(domain)}{TASK_LIST_PATH.format(cta_gsid=cta_gsid)}"
+ for page_number in range(first_page, first_page + MAX_TASK_PAGES_PER_CTA):
+ params = {"pn": page_number, "ps": TASK_PAGE_SIZE}
+ rows = _rows_from_body(_get_json(session, url, access_key, params), CTA_TASK)
+ last_page = len(rows) < TASK_PAGE_SIZE
+ if last_page:
+ next_state = GainsightCsResumeConfig(offset=cta_offset + 1)
+ else:
+ next_state = GainsightCsResumeConfig(offset=cta_offset, task_page=page_number + 1)
+ resumable_source_manager.save_state(next_state)
+ if rows:
+ yield [{**row, TASK_CTA_ID: cta_gsid} for row in rows]
+ else:
+ resumable_source_manager.safe_point()
+ if last_page:
+ return
+
+ # The staged cursor points at the next task page, so the next attempt continues this CTA.
+ resumable_source_manager.safe_point()
+ raise GainsightCsPageBudgetExhaustedError(
+ f"Stopped after {MAX_TASK_PAGES_PER_CTA} task pages of CTA '{cta_gsid}'; the next attempt resumes at page {page_number + 1}."
+ )
+
+ def items() -> Iterator[list[dict[str, Any]]]:
+ _require_allowed_host(domain, team_id)
+
+ # The offset counts CTAs, not tasks: it is the index of the CTA whose tasks to fetch next.
+ offset = 0
+ first_task_page = 1
+ if resumable_source_manager.can_resume():
+ resume = resumable_source_manager.load_state()
+ if resume is not None:
+ offset = max(resume.offset, 0)
+ first_task_page = max(resume.task_page, 1)
+
+ for _ in range(MAX_PAGES_PER_ATTEMPT):
+ body: dict[str, Any] = {"select": [GSID], "limit": MAX_PAGE_SIZE, "offset": offset}
+ if order_by is not None:
+ body["orderBy"] = order_by
+
+ ctas = _rows_from_body(_post_json(session, query_url, access_key, body), CTA_OBJECT_NAME)
+ for cta in ctas:
+ cta_gsid = cta.get(GSID)
+ # The Gsid lands in the request path, so only an id-shaped value is sent.
+ if isinstance(cta_gsid, str) and _OBJECT_NAME_RE.match(cta_gsid):
+ yield from cta_task_pages(cta_gsid, offset, first_task_page)
+ offset += 1
+ first_task_page = 1
+
+ if len(ctas) < MAX_PAGE_SIZE:
+ return
+
+ resumable_source_manager.safe_point()
+ raise GainsightCsPageBudgetExhaustedError(
+ f"Stopped after {MAX_PAGES_PER_ATTEMPT} pages of '{CTA_OBJECT_NAME}'; the next attempt resumes at CTA {offset}."
+ )
+
+ return SourceResponse(
+ name=CTA_TASK,
+ items=items,
+ primary_keys=COCKPIT_PRIMARY_KEYS[CTA_TASK],
+ # Cleared only once the pipeline has written every batch, so a failed final write resumes
+ # near the end of the fan-out instead of starting it again.
+ on_complete=resumable_source_manager.clear_state,
+ )
+
+
+def _probe(
+ domain: str,
+ access_key: str,
+ team_id: Optional[int],
+ path: str,
+ not_found_error: str,
+ parse: Callable[[Any], object],
) -> tuple[bool, Optional[str]]:
ok, err = check_domain(domain, team_id)
if not ok:
return False, err
- url = f"{_base_url(domain)}{DESCRIBE_PATH.format(object_name=object_name)}"
+ url = f"{_base_url(domain)}{path}"
try:
response = make_tracked_session(redact_values=(access_key,), allow_redirects=False).get(
url, headers=_headers(access_key), timeout=30, stream=True
@@ -390,15 +547,42 @@ def validate_credentials(
if response.status_code == 404:
response.close()
- return False, f"Gainsight has no object named '{object_name}' on that domain."
+ return False, not_found_error
if response.status_code != 200:
response.close()
return False, f"Gainsight returned an unexpected status ({response.status_code})."
try:
- _parse_fields(_read_capped_json(response), object_name)
+ parse(_read_capped_json(response))
except (ValueError, GainsightCsResponseTooLargeError, GainsightCsResponseTooSlowError) as e:
return False, str(e)
return True, None
+
+
+def validate_credentials(
+ domain: str, access_key: str, object_name: str, team_id: Optional[int]
+) -> tuple[bool, Optional[str]]:
+ return _probe(
+ domain,
+ access_key,
+ team_id,
+ DESCRIBE_PATH.format(object_name=object_name),
+ f"Gainsight has no object named '{object_name}' on that domain.",
+ lambda body: _parse_fields(body, object_name),
+ )
+
+
+def validate_cta_configuration_access(
+ domain: str, access_key: str, team_id: Optional[int]
+) -> tuple[bool, Optional[str]]:
+ query = urlencode({"category": ",".join(PICKLIST_CATEGORIES), "et": PICKLIST_ENTITY_TYPE})
+ return _probe(
+ domain,
+ access_key,
+ team_id,
+ f"{PICKLIST_PATH}?{query}",
+ "Gainsight didn't find the Cockpit configuration API on that domain.",
+ _picklist_rows,
+ )
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/settings.py
index f921ca31ba7d..aa6a968a0b12 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/settings.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/settings.py
@@ -8,11 +8,23 @@ from products.warehouse_sources.backend.types import IncrementalField
# gainsight_cs.py revalidates it against the SSRF host policy before connecting.
QUERY_PATH = "/v1/data/objects/query/{object_name}"
DESCRIBE_PATH = "/v1/meta/services/objects/{object_name}/describe"
+PICKLIST_PATH = "/v2/cockpit/admin/picklist/lite"
+TASK_LIST_PATH = "/v2/cockpit/task/list/{cta_gsid}"
# The Read API returns at most 5000 records per call and pages with limit/offset carried in the
# POST body rather than the query string.
MAX_PAGE_SIZE = 5000
+# The Fetch Task List API pages with pn/ps query params and returns at most 1000 tasks per call.
+TASK_PAGE_SIZE = 1000
+
+# CTA_TYPE is documented to return its dependent picklists as well, so a value can arrive twice and
+# rows are deduped on category and gsid.
+PICKLIST_CATEGORIES = ("CTA_TYPE", "CTA_STATUS", "CTA_PRIORITY", "CTA_REASON")
+# Relationship picklists need a relationship type id per request, so only the company and global
+# configuration is synced.
+PICKLIST_ENTITY_TYPE = "COMPANY"
+
# Gainsight's universal record identifier, present on every standard object.
GSID = "Gsid"
@@ -93,8 +105,23 @@ GAINSIGHT_CS_OBJECTS: dict[str, GainsightCsObjectConfig] = {
),
}
-ENDPOINTS = tuple(GAINSIGHT_CS_OBJECTS.keys())
+CTA_CONFIGURATION = "cta_configuration"
+CTA_TASK = "cta_task"
+# The CTA reference the task rows carry. The Fetch Task List API returns only the CTA name, so the
+# source adds the CTA Gsid it requested under Gainsight's own field name.
+TASK_CTA_ID = "CtaId"
+CTA_OBJECT_NAME = "call_to_action"
+
+# Cockpit tables read through dedicated v2 Cockpit routes instead of the Read API, so they carry no
+# object name and can't be described.
+COCKPIT_PRIMARY_KEYS: dict[str, list[str]] = {
+ CTA_CONFIGURATION: ["picklistCategory", "gsid"],
+ CTA_TASK: [TASK_CTA_ID, GSID],
+}
+
+ENDPOINTS = (*GAINSIGHT_CS_OBJECTS.keys(), *COCKPIT_PRIMARY_KEYS.keys())
INCREMENTAL_FIELDS: dict[str, list[IncrementalField]] = {
- name: list(config.incremental_fields) for name, config in GAINSIGHT_CS_OBJECTS.items()
+ **{name: list(config.incremental_fields) for name, config in GAINSIGHT_CS_OBJECTS.items()},
+ **{name: [] for name in COCKPIT_PRIMARY_KEYS},
}
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/source.py
index ce68b4a6221c..69c526daba20 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/source.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/source.py
@@ -26,12 +26,17 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.gainsight_
RESPONSE_TOO_LARGE_ERROR,
RESPONSE_TOO_SLOW_ERROR,
GainsightCsResumeConfig,
+ gainsight_cs_cta_configuration_source,
+ gainsight_cs_cta_task_source,
gainsight_cs_source,
parse_custom_objects,
resolve_object_name,
validate_credentials as validate_gainsight_cs_credentials,
+ validate_cta_configuration_access,
)
from products.warehouse_sources.backend.temporal.data_imports.sources.gainsight_cs.settings import (
+ CTA_CONFIGURATION,
+ CTA_TASK,
ENDPOINTS,
GAINSIGHT_CS_OBJECTS,
GSID,
@@ -97,6 +102,9 @@ class GainsightCsSource(ResumableSource[GainsightCsSourceConfig, GainsightCsResu
schema_name: Optional[str] = None,
api_version: str | None = None,
) -> tuple[bool, str | None]:
+ if schema_name == CTA_CONFIGURATION:
+ return validate_cta_configuration_access(config.domain, config.access_key, team_id)
+
if schema_name is None:
object_name = CREDENTIAL_PROBE_OBJECT
else:
@@ -116,6 +124,19 @@ class GainsightCsSource(ResumableSource[GainsightCsSourceConfig, GainsightCsResu
resumable_source_manager: ResumableSourceManager[GainsightCsResumeConfig],
inputs: SourceInputs,
) -> SourceResponse:
+ if inputs.schema_name == CTA_CONFIGURATION:
+ return gainsight_cs_cta_configuration_source(
+ domain=config.domain, access_key=config.access_key, team_id=inputs.team_id
+ )
+
+ if inputs.schema_name == CTA_TASK:
+ return gainsight_cs_cta_task_source(
+ domain=config.domain,
+ access_key=config.access_key,
+ team_id=inputs.team_id,
+ resumable_source_manager=resumable_source_manager,
+ )
+
object_name = resolve_object_name(inputs.schema_name, config.custom_objects)
known = GAINSIGHT_CS_OBJECTS.get(inputs.schema_name)
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs.py
index 070b303bc951..f0d304687e39 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs.py
@@ -20,6 +20,7 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.gainsight_
GainsightCsResumeConfig,
_normalize_row,
_parse_fields,
+ _picklist_rows,
_read_capped_json,
_rows_from_body,
normalize_domain,
@@ -191,6 +192,132 @@ class TestPagination:
assert result.requests == []
+class TestCtaConfiguration:
+ def test_flattens_each_category_and_drops_repeats(self) -> None:
+ status = {"gsid": "s1", "name": "New", "picklistCategory": "CTA_STATUS"}
+ body = {
+ "result": True,
+ "data": {
+ "CTA_STATUS": [status, status],
+ "CTA_PRIORITY": [{"gsid": "p1", "name": "High"}],
+ "CTA_REASON": [{"name": "no id"}],
+ },
+ }
+
+ assert _picklist_rows(body) == [status, {"gsid": "p1", "name": "High", "picklistCategory": "CTA_PRIORITY"}]
+
+ def test_requests_every_category_for_companies(self) -> None:
+ driver = SourceDriver(GainsightCsSource(), GainsightCsSourceConfig(domain=DOMAIN, access_key="key"))
+ result = driver.run(
+ "cta_configuration",
+ [ScriptedResponse(json={"result": True, "data": {"CTA_TYPE": [{"gsid": "t1", "name": "Risk"}]}})],
+ )
+
+ assert result.raised is None
+ assert result.paths == ["/v2/cockpit/admin/picklist/lite"]
+ assert result.params("category") == ["CTA_TYPE,CTA_STATUS,CTA_PRIORITY,CTA_REASON"]
+ assert result.params("et") == ["COMPANY"]
+ assert result.rows == [{"gsid": "t1", "name": "Risk", "picklistCategory": "CTA_TYPE"}]
+
+
+class TestCtaTasks:
+ @staticmethod
+ def _driver() -> SourceDriver:
+ return SourceDriver(GainsightCsSource(), GainsightCsSourceConfig(domain=DOMAIN, access_key="key"))
+
+ @staticmethod
+ def _page(rows: list[dict[str, Any]]) -> ScriptedResponse:
+ return ScriptedResponse(json={"result": True, "data": rows})
+
+ def test_walks_every_cta_and_pages_through_its_tasks(self) -> None:
+ describe = ScriptedResponse(json=_describe_body([_field("Gsid")], "call_to_action"))
+ with mock.patch(PAGE_SIZE_PATCH, 3), mock.patch(f"{GAINSIGHT_MODULE}.TASK_PAGE_SIZE", 2):
+ result = self._driver().run(
+ "cta_task",
+ [
+ describe,
+ self._page([{"Gsid": "ctaA"}, {"Gsid": "../x"}, {"Gsid": "ctaB"}]),
+ self._page([{"Gsid": "t1"}, {"Gsid": "t2"}]),
+ self._page([{"Gsid": "t3"}]),
+ self._page([]),
+ self._page([{"Gsid": "ctaC"}]),
+ self._page([{"Gsid": "t4"}]),
+ ],
+ )
+
+ assert result.raised is None
+ assert result.paths == [
+ "/v1/meta/services/objects/call_to_action/describe",
+ "/v1/data/objects/query/call_to_action",
+ "/v2/cockpit/task/list/ctaA",
+ "/v2/cockpit/task/list/ctaA",
+ "/v2/cockpit/task/list/ctaB",
+ "/v1/data/objects/query/call_to_action",
+ "/v2/cockpit/task/list/ctaC",
+ ]
+ assert result.requests[1].json() == {
+ "select": ["Gsid"],
+ "limit": 3,
+ "offset": 0,
+ "orderBy": {"Gsid": "asc"},
+ }
+ assert result.requests[5].json()["offset"] == 3
+ assert result.params("pn") == [None, None, "1", "2", "1", None, "1"]
+ assert result.rows == [
+ {"Gsid": "t1", "CtaId": "ctaA"},
+ {"Gsid": "t2", "CtaId": "ctaA"},
+ {"Gsid": "t3", "CtaId": "ctaA"},
+ {"Gsid": "t4", "CtaId": "ctaC"},
+ ]
+ assert result.saved_states == [
+ GainsightCsResumeConfig(offset=0, task_page=2),
+ GainsightCsResumeConfig(offset=1),
+ GainsightCsResumeConfig(offset=3),
+ GainsightCsResumeConfig(offset=4),
+ ]
+ assert result.final_state == GainsightCsResumeConfig(offset=4)
+ assert result.completed_state is None
+
+ def test_resumes_mid_way_through_a_ctas_tasks(self) -> None:
+ with mock.patch(f"{GAINSIGHT_MODULE}.TASK_PAGE_SIZE", 2):
+ result = self._driver().run(
+ "cta_task",
+ [
+ ScriptedResponse(json=_describe_body([_field("Gsid")], "call_to_action")),
+ self._page([{"Gsid": "ctaD"}, {"Gsid": "ctaE"}]),
+ self._page([{"Gsid": "t9"}]),
+ self._page([{"Gsid": "t10"}]),
+ ],
+ resume_state=GainsightCsResumeConfig(offset=7, task_page=3),
+ )
+
+ assert result.raised is None
+ assert result.requests[1].json()["offset"] == 7
+ assert result.params("pn") == [None, None, "3", "1"]
+ assert result.rows == [{"Gsid": "t9", "CtaId": "ctaD"}, {"Gsid": "t10", "CtaId": "ctaE"}]
+
+ def test_checkpoints_the_next_task_page_when_a_cta_exhausts_its_budget(self) -> None:
+ full_page = self._page([{"Gsid": "t1"}, {"Gsid": "t2"}])
+ with (
+ mock.patch(f"{GAINSIGHT_MODULE}.TASK_PAGE_SIZE", 2),
+ mock.patch(f"{GAINSIGHT_MODULE}.MAX_TASK_PAGES_PER_CTA", 2),
+ ):
+ result = self._driver().run(
+ "cta_task",
+ [
+ ScriptedResponse(json=_describe_body([_field("Gsid")], "call_to_action")),
+ self._page([{"Gsid": "ctaA"}]),
+ full_page,
+ full_page,
+ ],
+ resume_state=GainsightCsResumeConfig(offset=5, task_page=4),
+ )
+
+ assert isinstance(result.raised, GainsightCsPageBudgetExhaustedError)
+ assert result.params("pn") == [None, None, "4", "5"]
+ assert result.final_state == GainsightCsResumeConfig(offset=5, task_page=6)
+
+
class TestCappedResponses:
def _streaming(self, chunks: list[bytes]) -> mock.MagicMock:
response = mock.MagicMock()
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs_source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs_source.py
index 6c4c3927231a..8c54ba83a672 100644
--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs_source.py
+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gainsight_cs/tests/test_gainsight_cs_source.py
@@ -1,3 +1,5 @@
+from parameterized import parameterized
+
from products.warehouse_sources.backend.temporal.data_imports.sources.common.testing import (
ScriptedResponse,
SourceDriver,
@@ -28,6 +30,27 @@ class TestGainsightCsSourceCredentials:
assert result == (True, None)
assert [request.path for request in network.requests_log] == ["/v1/meta/services/objects/health__gc/describe"]
+ @parameterized.expand(
+ [
+ (
+ "cta_configuration",
+ {"result": True, "data": {"CTA_STATUS": [{"gsid": "s1", "name": "New"}]}},
+ "/v2/cockpit/admin/picklist/lite",
+ ),
+ (
+ "cta_task",
+ {"result": True, "data": [{"objectName": "call_to_action", "fields": [{"fieldName": "Gsid"}]}]},
+ "/v1/meta/services/objects/call_to_action/describe",
+ ),
+ ]
+ )
+ def test_probes_the_cockpit_route_a_cockpit_table_reads(self, schema_name: str, body: dict, path: str) -> None:
+ with scripted_network([ScriptedResponse(json=body)]) as network:
+ result = GainsightCsSource().validate_credentials(_config(), team_id=1, schema_name=schema_name)
+
+ assert result == (True, None)
+ assert [request.path for request in network.requests_log] == [path]
+
def test_probes_company_when_no_schema_is_named(self) -> None:
fields = {"result": True, "data": [{"objectName": "company", "fields": [{"fieldName": "Gsid"}]}]}
with scripted_network([ScriptedResponse(json=fields)]) as network: