:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(warehouse_sources): support teachable API version v2 (#126954)
Tom Owersf9e933d2f00c
diff --git a/.agents/skills/warehouse-source-new-version/SKILL.md b/.agents/skills/warehouse-source-new-version/SKILL.mdindex 002131c178fb..637bafceca4c 100644--- a/.agents/skills/warehouse-source-new-version/SKILL.md+++ b/.agents/skills/warehouse-source-new-version/SKILL.md@@ -64,7 +64,7 @@ The opposite gap also occurs: a reference rendered from one version-blind OpenAP- **Discovery and probe paths receive the pin — consume it.** The framework passes the resolved pin as the `api_version` parameter of `get_schemas`, `validate_credentials`, `get_endpoint_permissions`, and the webhook management methods. A multi-version source MUST build its discovery/probe/webhook clients from that parameter, not from `default_version` or a hardcoded header — otherwise a pinned source discovers/reconciles under the wrong version and its tables can disappear, duplicate, or fail reconciliation. Resolve it with `self.resolve_api_version(api_version)` — callers with a row pass an already-resolved value (mirroring `SourceInputs.api_version`), so the source-side resolve only covers pre-creation calls that pass `None`. Ignoring the parameter is only correct when you can state why the version makes no difference to that path.- Watch for version-dependent column hints/schemas: e.g. Stripe's `external_table_definitions` were built for specific versions. When adding a version whose response shapes differ, gate the canonical column hints to the versions they were built for and let newer versions auto-infer the schema from the data (a set of hint-compatible versions checked where hints are applied). For `has_managed_hogql_schema=True` sources this includes the read path: `hogql_definition`'s canonical column mapping is version-blind, so renamed columns need the canonical schema/descriptions updated too.4. **Keep old versions working**: do not delete or alter the request path for previously supported versions. Removing a version is an explicit future decision, not part of a version-add PR.-5. **Tests**: extend the source's tests so both the old and new versions are exercised — at minimum that the version label reaches the client/request layer for each supported version (mock the boundary; parameterize over versions). The registry invariant test picks up declaration mistakes automatically. Don't re-test the base-class `resolve_api_version` contract (`test_source_versions.py` covers every source). When versions diverge, shape fixtures per version from the vendor docs — a v1-shaped mock under a v2 pin proves nothing.+5. **Tests**: extend the source's tests so both the old and new versions are exercised — at minimum that the version label reaches the client/request layer for each supported version (mock the boundary; parameterize over versions). The registry invariant test picks up declaration mistakes automatically. Don't re-test the base-class `resolve_api_version` contract (`test_source_versions.py` covers every source). When versions diverge, shape fixtures per version from the vendor docs — a v1-shaped mock under a v2 pin proves nothing. A test that passes no `api_version` runs on the default, so a default flip moves every existing legacy-path test onto the new path, where it fails or silently tests the wrong wire. Pin those tests to the old label explicitly.6. **One PR per source.** Conventional title: `feat(warehouse_sources): support <vendor> API version <label>` — the scope is always `warehouse_sources` (the product), never the source dir/vendor name.## Deprecating a versiondiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.pyindex 3231385dfc41..624e84039c90 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.py@@ -10,6 +10,8 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"id": "Unique identifier for the user.","name": "The user's full name.","email": "The user's email address.",+ "role": "The user's role at the school: student, owner, author or affiliate. API v2 only.",+ "created_at": "When the user was created, in ISO8601 format. API v2 only.",},},"courses": {@@ -34,6 +36,9 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"expires_at": "When the enrollment expires, or null if it never expires.","completed_at": "When the student completed the course, or null if not yet finished.","percent_complete": "Percentage of the course the student has marked complete.",+ "sale_id": "Identifier of the sale that granted the enrollment, or null if no sale granted it. API v2 only.",+ "status": "Enrollment status: active, completed, expired or disabled. API v2 only.",+ "progress": "The student's percent_complete and completed_at for the course. API v2 only.",},},"transactions": {@@ -88,6 +93,7 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"enrolled_at": "When the student enrolled in the course, in ISO8601 format.","lecture_sections": "Progress for each lecture section, with the completion state of each lecture in it.","percent_complete": "Percentage of the course the student has marked complete.",+ "sections": "Progress for each course section, with the completion state of each lecture in it. API v2 only.",},},"lectures": {@@ -101,6 +107,7 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"position": "Position of the lecture within its section (1 is first).","lecture_section_id": "Unique identifier of the section that contains the lecture.","attachments": "Content attached to the lecture: text, video, image, PDF, quiz, code and other blocks.",+ "section": "The section that contains the lecture. API v2 only.",},},"quizzes": {diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.pyindex edae05e75db0..3e8c9a4d3313 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.py@@ -8,6 +8,10 @@ from products.warehouse_sources.backend.types import IncrementalFieldTEACHABLE_BASE_URL = "https://developers.teachable.com"+TEACHABLE_API_VERSION_V1 = "v1"+TEACHABLE_API_VERSION_V2 = "v2"+TEACHABLE_API_VERSIONS = (TEACHABLE_API_VERSION_V1, TEACHABLE_API_VERSION_V2)+# Teachable documents a 20-row default page size (5 for /pricing_plans). We request more via# `per`; if the server clamps it, pagination still terminates correctly because the paginators# stop on `meta.number_of_pages` / `meta.has_more_results` / an empty page, never on the@@ -19,8 +23,11 @@ DEFAULT_PAGE_SIZE = 100# incremental sync so late-arriving boundary rows aren't skipped (the merge dedupes on `id`).TRANSACTIONS_INCREMENTAL_LOOKBACK_SECONDS = 300+# v2 pages with `page`/`per_page` and caps `per_page` at 100.+V2_PAGE_SIZE = 100+-@dataclass+@dataclass(frozen=True)class TeachableEndpointConfig:name: strpath: str@@ -31,6 +38,11 @@ class TeachableEndpointConfig:partition_key: str | None = Nonepage_size: int = DEFAULT_PAGE_SIZEfanout: DependentEndpointConfig | None = None+ # Path on the v2 wire. None keeps the table on its v1 path under a v2 pin: v2 marks those+ # endpoints beta and a school may not have access to them yet.+ v2_path: str | None = None+ v2_sort_by: str | None = None+ v2_primary_key: list[str] | None = NoneTEACHABLE_ENDPOINTS: dict[str, TeachableEndpointConfig] = {@@ -41,6 +53,8 @@ TEACHABLE_ENDPOINTS: dict[str, TeachableEndpointConfig] = {primary_key=["id"],# The list response only carries id/name/email — no timestamp to filter or partition# on, so full refresh with no partitioning.+ v2_path="/v2/users",+ v2_sort_by="id",),"courses": TeachableEndpointConfig(name="courses",@@ -48,6 +62,8 @@ TEACHABLE_ENDPOINTS: dict[str, TeachableEndpointConfig] = {data_selector="courses",primary_key=["id"],# Course rows have no timestamp field and no server-side change filter — full refresh.+ v2_path="/v2/products/courses",+ v2_sort_by="created_at",),"course_enrollments": TeachableEndpointConfig(name="course_enrollments",@@ -69,6 +85,11 @@ TEACHABLE_ENDPOINTS: dict[str, TeachableEndpointConfig] = {include_from_parent=["id"],parent_field_renames={"id": "course_id"},),+ v2_path="/v2/products/courses/{course_id}/enrollments",+ v2_sort_by="enrolled_at",+ # v2 returns one enrollment per sale, so a user can hold several in the same course.+ # `sale_id` is null for an enrollment that no sale granted.+ v2_primary_key=["course_id", "user_id", "sale_id"],),"transactions": TeachableEndpointConfig(name="transactions",@@ -96,6 +117,7 @@ TEACHABLE_ENDPOINTS: dict[str, TeachableEndpointConfig] = {data_selector="course_progress",primary_key=["course_id", "user_id"],partition_key="enrolled_at",+ v2_path="/v2/products/courses/{course_id}/users/{user_id}/progress",),"lectures": TeachableEndpointConfig(name="lectures",@@ -103,6 +125,9 @@ TEACHABLE_ENDPOINTS: dict[str, TeachableEndpointConfig] = {path="/v1/courses/{course_id}/lectures/{lecture_id}",data_selector="lecture",primary_key=["course_id", "id"],+ # v2 has a lecture list endpoint, so the course curriculum walk is not needed there.+ v2_path="/v2/products/courses/{course_id}/lectures",+ v2_sort_by="created_at",),"quizzes": TeachableEndpointConfig(name="quizzes",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.pyindex ffd955de1029..6e13ea6e890a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.py@@ -25,6 +25,8 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.generated_from products.warehouse_sources.backend.temporal.data_imports.sources.teachable.settings import (ENDPOINTS,INCREMENTAL_FIELDS,+ TEACHABLE_API_VERSION_V1,+ TEACHABLE_API_VERSION_V2,TRANSACTIONS_INCREMENTAL_LOOKBACK_SECONDS,)from products.warehouse_sources.backend.temporal.data_imports.sources.teachable.teachable import (@@ -40,8 +42,10 @@ class TeachableSource(ResumableSource[TeachableSourceConfig, TeachableResumeConfcodebase_detection = detect(env=("^TEACHABLE_",))lists_tables_without_credentials = True # static endpoint catalog — safe for public docs- # Teachable's API only exposes an unversioned `/v1/` path with no documented version- # choice, so we leave the framework's unversioned default in place.+ # v2 moves users, courses, enrollments, course progress and lectures onto the v2 wire. The+ # other tables stay on v1 under a v2 pin, so the table set is the same for both versions.+ supported_versions = (TEACHABLE_API_VERSION_V1, TEACHABLE_API_VERSION_V2)+ default_version = TEACHABLE_API_VERSION_V2api_docs_url = "https://docs.teachable.com/"@property@@ -51,7 +55,7 @@ class TeachableSource(ResumableSource[TeachableSourceConfig, TeachableResumeConfdef get_non_retryable_errors(self) -> dict[str, str | None]:return {"401 Client Error: Unauthorized for url": "Invalid Teachable API key. Please create a new key in your school admin under Settings > API and reconnect.",- "403 Client Error: Forbidden for url": "Teachable rejected the API key. Check that the key is active and that your school is on the Growth plan or higher (required for API access).",+ "403 Client Error: Forbidden for url": "Teachable rejected the API key. Check that the key is active and can read every table you sync, and that your school is on the Growth plan or higher (required for API access). For Teachable API v2, you may also need to ask Teachable Support to turn on API v2 for your school.",}def get_canonical_descriptions(self) -> CanonicalDescriptions:@@ -85,7 +89,7 @@ class TeachableSource(ResumableSource[TeachableSourceConfig, TeachableResumeConfschema_name: Optional[str] = None,api_version: str | None = None,) -> tuple[bool, str | None]:- return validate_teachable_credentials(config.api_key)+ return validate_teachable_credentials(config.api_key, self.resolve_api_version(api_version), schema_name)def get_resumable_source_manager(self, inputs: SourceInputs) -> ResumableSourceManager[TeachableResumeConfig]:return ResumableSourceManager[TeachableResumeConfig](inputs, TeachableResumeConfig)@@ -101,6 +105,7 @@ class TeachableSource(ResumableSource[TeachableSourceConfig, TeachableResumeConfendpoint=inputs.schema_name,team_id=inputs.team_id,job_id=inputs.job_id,+ api_version=self.resolve_api_version(inputs.api_version),resumable_source_manager=resumable_source_manager,should_use_incremental_field=inputs.should_use_incremental_field,db_incremental_field_last_value=inputs.db_incremental_field_last_valuediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.pyindex 8e218cec64ca..0658ec79a8b2 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.py@@ -37,13 +37,16 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.resfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SortMode, SourceResponsefrom products.warehouse_sources.backend.temporal.data_imports.sources.teachable.settings import (DEFAULT_PAGE_SIZE,+ TEACHABLE_API_VERSION_V2,+ TEACHABLE_API_VERSIONS,TEACHABLE_BASE_URL,TEACHABLE_ENDPOINTS,+ V2_PAGE_SIZE,TeachableEndpointConfig,)[email protected](frozen=True)class TeachableResumeConfig:"""Paginator checkpoint: `page` for page-numbered endpoints, `search_after` for /users."""@@ -173,17 +176,59 @@ class TeachableUsersPaginator(BasePaginator):self._has_next_page = True-def validate_credentials(api_key: str) -> tuple[bool, str | None]:- response = make_tracked_session(redact_values=(api_key,), allow_redirects=False).get(- f"{TEACHABLE_BASE_URL}/v1/courses",- headers={"apiKey": api_key, "Accept": "application/json"},- params={"per": 1},- timeout=10,- )+def _v2_page_paginator() -> PageNumberPaginator:+ return PageNumberPaginator(base_page=1, page_param="page", total_path="meta.total_pages")+++def _v2_list_params(config: TeachableEndpointConfig) -> dict[str, Any]:+ # An explicit ascending sort keeps page boundaries stable while rows are added mid-sync.+ return {"per_page": V2_PAGE_SIZE, "sort_by": config.v2_sort_by, "sort_direction": "asc"}+++def _require_supported_version(api_version: str) -> None:+ if api_version not in TEACHABLE_API_VERSIONS:+ raise ValueError(f"Unsupported Teachable API version: {api_version!r}")+++def _probes(api_version: str, schema_name: Optional[str]) -> list[tuple[str, dict[str, Any]]]:+ v1_probe: tuple[str, dict[str, Any]] = ("/v1/courses", {"per": 1})+ if api_version != TEACHABLE_API_VERSION_V2:+ return [v1_probe]+ courses_probe = (cast(str, TEACHABLE_ENDPOINTS["courses"].v2_path), {"per_page": 1})+ users_probe = (cast(str, TEACHABLE_ENDPOINTS["users"].v2_path), {"per_page": 1})+ config = TEACHABLE_ENDPOINTS.get(schema_name) if schema_name else None+ if config is None:+ # v2 keys carry per-resource scopes, so a key that can read either top-level list is usable.+ return [courses_probe, users_probe]+ if config.v2_path is None:+ return [v1_probe]+ # Every other v2 table walks the course list first.+ return [users_probe] if schema_name == "users" else [courses_probe]+++def validate_credentials(api_key: str, api_version: str, schema_name: Optional[str] = None) -> tuple[bool, str | None]:+ _require_supported_version(api_version)+ session = make_tracked_session(redact_values=(api_key,), allow_redirects=False)+ for path, params in _probes(api_version, schema_name):+ response = session.get(+ f"{TEACHABLE_BASE_URL}{path}",+ headers={"apiKey": api_key, "Accept": "application/json"},+ params=params,+ timeout=10,+ )+ if response.status_code != 403:+ breakif response.status_code == 200:return True, Noneif response.status_code == 401:return False, "Invalid Teachable API key"+ if response.status_code == 403 and path.startswith("/v2/"):+ return (+ False,+ "Teachable rejected the API key. Check that the key is active and can read the tables you want to "+ "sync, and that your school is on the Growth plan or higher. Teachable API v2 is in beta, so you "+ "may also need to ask Teachable Support to turn on API v2 for your school.",+ )if response.status_code == 403:return (False,@@ -427,6 +472,91 @@ def _course_progress_rows(client: RESTClient, safe_point: SafePoint) -> NestedRoyield [{**progress, "course_id": course_id, "user_id": user_id}]+def _ref_id(value: Any) -> Optional[int]:+ if isinstance(value, dict) and isinstance(value.get("id"), int):+ return value["id"]+ return None+++def _iter_v2_course_ids(client: RESTClient, safe_point: SafePoint) -> Iterator[int]:+ config = TEACHABLE_ENDPOINTS["courses"]+ for courses in client.paginate(+ path=cast(str, config.v2_path),+ params=_v2_list_params(config),+ paginator=_v2_page_paginator(),+ data_selector="data",+ data_selector_required=True,+ ):+ safe_point()+ for course_id in (_ref_id(course) for course in courses):+ if course_id is not None:+ yield course_id+++def _iter_v2_course_pages(+ client: RESTClient, endpoint: str, course_id: int, safe_point: SafePoint+) -> Iterator[list[dict[str, Any]]]:+ config = TEACHABLE_ENDPOINTS[endpoint]+ for page in client.paginate(+ path=cast(str, config.v2_path).format(course_id=course_id),+ params=_v2_list_params(config),+ paginator=_v2_page_paginator(),+ data_selector="data",+ data_selector_required=True,+ hooks=_IGNORE_NOT_FOUND,+ ):+ safe_point()+ yield [row for row in page if isinstance(row, dict)]+++def _v2_enrollment_rows(client: RESTClient, safe_point: SafePoint) -> NestedRows:+ for course_id in _iter_v2_course_ids(client, safe_point):+ for enrollments in _iter_v2_course_pages(client, "course_enrollments", course_id, safe_point):+ # v2 nests the user and sale references, so their ids are lifted out for the primary key.+ rows = [+ {+ **enrollment,+ "course_id": course_id,+ "user_id": _ref_id(enrollment.get("user")),+ "sale_id": _ref_id(enrollment.get("sale")),+ }+ for enrollment in enrollments+ ]+ if rows:+ yield rows+++def _v2_course_progress_rows(client: RESTClient, safe_point: SafePoint) -> NestedRows:+ path_template = cast(str, TEACHABLE_ENDPOINTS["course_progress"].v2_path)+ for course_id in _iter_v2_course_ids(client, safe_point):+ seen_user_ids: set[int] = set()+ for enrollments in _iter_v2_course_pages(client, "course_enrollments", course_id, safe_point):+ for user_id in (_ref_id(enrollment.get("user")) for enrollment in enrollments):+ if user_id is None or user_id in seen_user_ids:+ continue+ seen_user_ids.add(user_id)+ path = path_template.format(course_id=course_id, user_id=user_id)+ body = _get_object(client, path)+ if body is None:+ safe_point()+ continue+ yield [{**_required(body, "data", path), "course_id": course_id, "user_id": user_id}]+++def _v2_lecture_rows(client: RESTClient, safe_point: SafePoint) -> NestedRows:+ for course_id in _iter_v2_course_ids(client, safe_point):+ for lectures in _iter_v2_course_pages(client, "lectures", course_id, safe_point):+ if lectures:+ yield [{**lecture, "course_id": course_id} for lecture in lectures]+++V2_NESTED_ROWS: dict[str, Callable[[RESTClient, SafePoint], NestedRows]] = {+ "course_enrollments": _v2_enrollment_rows,+ "course_progress": _v2_course_progress_rows,+ "lectures": _v2_lecture_rows,+}++# These tables sit two or more levels under /v1/courses, which the dependent-resource helper# can't express. Nothing is resumable: a hand-off restarts the walk, and every request is# followed by a yield or a safe point.@@ -443,12 +573,13 @@ def _make_source_response(items_fn: Callable[[], Iterable[Any]],sort_mode: SortMode = "asc",supports_resume: bool = True,+ primary_keys: list[str] | None = None,) -> SourceResponse:return SourceResponse(name=endpoint_config.name,items=items_fn,supports_resume=supports_resume,- primary_keys=endpoint_config.primary_key,+ primary_keys=primary_keys or endpoint_config.primary_key,sort_mode=sort_mode,partition_count=1,partition_size=1,@@ -458,18 +589,94 @@ def _make_source_response()+def _teachable_v2_source(+ api_key: str,+ endpoint: str,+ team_id: int,+ job_id: str,+ resumable_source_manager: Optional[ResumableSourceManager[TeachableResumeConfig]],+) -> SourceResponse:+ endpoint_config = TEACHABLE_ENDPOINTS[endpoint]+ primary_keys = endpoint_config.v2_primary_key or endpoint_config.primary_key++ nested_rows = V2_NESTED_ROWS.get(endpoint)+ if nested_rows is not None:+ client = _rest_client(api_key)+ safe_point: SafePoint = (+ resumable_source_manager.safe_point if resumable_source_manager is not None else lambda: None+ )+ return _make_source_response(+ endpoint_config,+ lambda: nested_rows(client, safe_point),+ supports_resume=False,+ primary_keys=primary_keys,+ )++ config: RESTAPIConfig = {+ "client": _rest_api_client_config(api_key),+ "resource_defaults": {},+ "resources": [+ {+ "name": endpoint_config.name,+ "table_name": endpoint_config.name,+ "write_disposition": "replace",+ "endpoint": {+ "path": cast(str, endpoint_config.v2_path),+ "params": _v2_list_params(endpoint_config),+ "data_selector": "data",+ "data_selector_required": True,+ "paginator": _v2_page_paginator(),+ },+ "table_format": "delta",+ }+ ],+ }++ initial_paginator_state: Optional[dict[str, Any]] = None+ resume_hook: Optional[Callable[[Optional[dict[str, Any]]], None]] = None+ if resumable_source_manager is not None:+ if resumable_source_manager.can_resume():+ resume_config = resumable_source_manager.load_state()+ # A `search_after` cursor comes from the v1 /users walk and means nothing to v2 paging.+ if resume_config is not None and resume_config.page is not None:+ initial_paginator_state = {"page": resume_config.page}++ def save_checkpoint(state: Optional[dict[str, Any]]) -> None:+ if resumable_source_manager is None or not state or state.get("page") is None:+ return+ resumable_source_manager.save_state(TeachableResumeConfig(page=state["page"]))++ resume_hook = save_checkpoint++ resource = rest_api_resource(+ config,+ team_id,+ job_id,+ None,+ resume_hook=resume_hook,+ initial_paginator_state=initial_paginator_state,+ )+ return _make_source_response(endpoint_config, lambda: resource, primary_keys=primary_keys)++def teachable_source(api_key: str,endpoint: str,team_id: int,job_id: str,+ api_version: str,resumable_source_manager: Optional[ResumableSourceManager[TeachableResumeConfig]] = None,should_use_incremental_field: bool = False,db_incremental_field_last_value: Optional[Any] = None,incremental_field: str | None = None,) -> SourceResponse:+ _require_supported_version(api_version)endpoint_config = TEACHABLE_ENDPOINTS[endpoint]+ # Every table on the v2 wire is full refresh, so the incremental inputs only reach v1 paths.+ if api_version == TEACHABLE_API_VERSION_V2 and endpoint_config.v2_path is not None:+ return _teachable_v2_source(api_key, endpoint, team_id, job_id, resumable_source_manager)+nested_rows = NESTED_FANOUT_ROWS.get(endpoint)if nested_rows is not None:client = _rest_client(api_key)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.pyindex c37bb75742cb..a049c20851ef 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.py@@ -16,6 +16,10 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.tesfrom products.warehouse_sources.backend.temporal.data_imports.sources.generated_configs.teachable import (TeachableSourceConfig,)+from products.warehouse_sources.backend.temporal.data_imports.sources.teachable.settings import (+ TEACHABLE_API_VERSION_V1 as V1,+ TEACHABLE_API_VERSION_V2 as V2,+)from products.warehouse_sources.backend.temporal.data_imports.sources.teachable.source import TeachableSourcefrom products.warehouse_sources.backend.temporal.data_imports.sources.teachable.teachable import (TeachableResumeConfig,@@ -44,6 +48,15 @@ _COURSE_11_DETAIL = ScriptedResponse(})_NOT_FOUND = ScriptedResponse(status=404, json={"error": "not found"})+_V2_COURSES_PAGE = ScriptedResponse(+ json={"data": [{"id": 11}, {"id": 12}], "meta": {"page": 1, "per_page": 100, "total_pages": 1, "total_count": 2}}+)+++def _v2_page(rows: list[dict[str, Any]]) -> ScriptedResponse:+ return ScriptedResponse(+ json={"data": rows, "meta": {"page": 1, "per_page": 100, "total_pages": 1, "total_count": len(rows)}}+ )def _users_response(meta: dict[str, Any]) -> Mock:@@ -126,15 +139,49 @@ class TestTeachableTransport:@parameterized.expand([- (200, True, None),- (401, False, "Invalid Teachable API key"),- (403, False, "Growth plan"),- (500, False, "unexpected status code"),+ ("v1_ok", V1, None, {}, True, None, ["/v1/courses"]),+ ("v1_bad_key", V1, None, {"/v1/courses": 401}, False, "Invalid Teachable API key", ["/v1/courses"]),+ ("v1_forbidden", V1, None, {"/v1/courses": 403}, False, "Growth plan", ["/v1/courses"]),+ ("v1_server_error", V1, None, {"/v1/courses": 500}, False, "unexpected status code", ["/v1/courses"]),+ ("v2_ok", V2, None, {}, True, None, ["/v2/products/courses"]),+ # v2 keys carry per-resource scopes, so a key that can only read users still connects.+ (+ "v2_users_only_key",+ V2,+ None,+ {"/v2/products/courses": 403},+ True,+ None,+ ["/v2/products/courses", "/v2/users"],+ ),+ ("v2_users_table", V2, "users", {"/v2/products/courses": 403}, True, None, ["/v2/users"]),+ (+ "v2_course_table",+ V2,+ "lectures",+ {"/v2/products/courses": 403},+ False,+ "turn on API v2",+ ["/v2/products/courses"],+ ),+ ("v2_beta_table_on_v1", V2, "transactions", {}, True, None, ["/v1/courses"]),+ # v2 is in beta and granted per school, so a 403 points at the access request.+ (+ "v2_no_access",+ V2,+ None,+ {"/v2/products/courses": 403, "/v2/users": 403},+ False,+ "turn on API v2",+ ["/v2/products/courses", "/v2/users"],+ ),])- def test_validate_credentials_status_mapping(self, status, expected_valid, message_fragment) -> None:- with scripted_network(always(ScriptedResponse(status=status))) as network:- is_valid, message = validate_credentials("teachable-key")+ def test_validate_credentials_status_mapping(+ self, _name, api_version, schema_name, statuses, expected_valid, message_fragment, expected_paths+ ) -> None:+ with scripted_network(lambda request: ScriptedResponse(status=statuses.get(request.path, 200))) as network:+ is_valid, message = validate_credentials("teachable-key", api_version, schema_name)assert is_valid is expected_validif message_fragment is None:@@ -142,9 +189,11 @@ class TestTeachableTransport:else:assert message is not None and message_fragment in message- request = network.requests_log[0]- assert request.origin + request.path == "https://developers.teachable.com/v1/courses"- assert request.headers["apikey"] == "teachable-key"+ # A 5xx is retried, so compare the distinct paths in request order.+ assert list(dict.fromkeys(r.origin + r.path for r in network.requests_log)) == [+ f"https://developers.teachable.com{path}" for path in expected_paths+ ]+ assert network.requests_log[0].headers["apikey"] == "teachable-key"# The validation session must refuse redirects so a 3xx can't replay the apiKey off-host.assert network.session_options[0]["allow_redirects"] is False@@ -170,6 +219,7 @@ class TestTeachableTransport:],}),+ api_version=V1,)assert result.rows == [{"user_id": 7, "enrolled_at": "2026-01-01T00:00:00Z", "course_id": 11}]@@ -181,15 +231,20 @@ class TestTeachableTransport:@parameterized.expand([- ("page_endpoint", "courses", TeachableResumeConfig(page=5), "page", "5"),- ("users_endpoint", "users", TeachableResumeConfig(search_after=99), "search_after", "99"),+ ("page_endpoint", V1, "courses", TeachableResumeConfig(page=5), "page", "5"),+ ("users_endpoint", V1, "users", TeachableResumeConfig(search_after=99), "search_after", "99"),+ ("v2_page_endpoint", V2, "users", TeachableResumeConfig(page=5), "page", "5"),+ # A v1 /users cursor has no meaning on v2 paging, so the walk starts over.+ ("v2_ignores_v1_users_cursor", V2, "users", TeachableResumeConfig(search_after=99), "search_after", None),])- def test_resume_state_seeds_paginator(self, _name, endpoint, saved_state, param, expected) -> None:+ def test_resume_state_seeds_paginator(self, _name, api_version, endpoint, saved_state, param, expected) -> None:+ body: dict[str, Any] = {+ "data" if api_version == V2 else endpoint: [{"id": 100}],+ "meta": {"number_of_pages": 5, "total_pages": 5, "per_page": 100},+ }result = _driver().run(- endpoint,- [ScriptedResponse(json={endpoint: [{"id": 100}], "meta": {"number_of_pages": 5, "per_page": 100}})],- resume_state=saved_state,+ endpoint, [ScriptedResponse(json=body)], resume_state=saved_state, api_version=api_version)assert result.requests[0].param(param) == expected@@ -201,6 +256,7 @@ class TestTeachableTransport:ScriptedResponse(json={"courses": [{"id": 1}], "meta": {"number_of_pages": 2}}),ScriptedResponse(json={"courses": [{"id": 2}], "meta": {"number_of_pages": 2}}),],+ api_version=V1,)# The cursor is saved while another page remains, and not after the last page.@@ -359,7 +415,7 @@ class TestTeachableNestedFanout:])def test_nested_fanout_walk(self, endpoint, routes, expected_rows) -> None:- result = _driver().run(endpoint, route(routes))+ result = _driver().run(endpoint, route(routes), api_version=V1)assert result.raised is Noneassert result.rows == expected_rows@@ -386,6 +442,7 @@ class TestTeachableNestedFanout:"/v1/courses/11/progress": [progress, progress],}),+ api_version=V1,)assert [row["user_id"] for row in result.rows] == [7, 8]@@ -408,7 +465,136 @@ class TestTeachableNestedFanout:}routes[broken_path] = [ScriptedResponse(json=broken_body)]- result = _driver().run(endpoint, route(routes))+ result = _driver().run(endpoint, route(routes), api_version=V1)assert isinstance(result.raised, ValueError)assert key in str(result.raised)+++class TestTeachableV2:+ @parameterized.expand(+ [+ ("users", "/v2/users", "id"),+ ("courses", "/v2/products/courses", "created_at"),+ ]+ )+ def test_list_walk_pages_to_total_pages(self, endpoint, path, sort_by) -> None:+ meta = {"per_page": 1, "total_pages": 2, "total_count": 2}+ result = _driver().run(+ endpoint,+ [+ ScriptedResponse(json={"data": [{"id": 1}], "meta": {**meta, "page": 1}}),+ ScriptedResponse(json={"data": [{"id": 2}], "meta": {**meta, "page": 2}}),+ ],+ api_version=V2,+ )++ assert result.raised is None+ assert result.rows == [{"id": 1}, {"id": 2}]+ assert result.paths == [path, path]+ assert result.params("page") == ["1", "2"]+ assert {(r.param("per_page"), r.param("sort_by"), r.param("sort_direction")) for r in result.requests} == {+ ("100", sort_by, "asc")+ }++ @parameterized.expand(+ [+ ("v1_pin_keeps_v1_path", "courses", V1, "/v1/courses", {"courses": [], "meta": {"number_of_pages": 1}}),+ # v2 marks /v2/transactions beta, so a v2 pin keeps reading the v1 endpoint.+ (+ "beta_table_stays_on_v1_under_v2",+ "transactions",+ V2,+ "/v1/transactions",+ {"transactions": [], "meta": {"number_of_pages": 1}},+ ),+ (+ "unpinned_resolves_to_v2",+ "courses",+ None,+ "/v2/products/courses",+ {"data": [], "meta": {"total_pages": 1}},+ ),+ ]+ )+ def test_request_path_follows_pin(self, _name, endpoint, api_version, expected_path, body) -> None:+ result = _driver().run(endpoint, always(ScriptedResponse(json=body)), api_version=api_version)++ assert result.raised is None+ assert result.paths == [expected_path]++ @parameterized.expand(+ [+ (+ # Course 12 was deleted after the course listing. User 7 holds one enrollment from a+ # sale and one granted without a sale.+ "course_enrollments",+ {+ "/v2/products/courses": [_V2_COURSES_PAGE],+ "/v2/products/courses/11/enrollments": [+ _v2_page(+ [+ {"user": {"id": 7, "href": "u7"}, "sale": {"id": 3, "href": "s3"}, "status": "active"},+ {"user": {"id": 7, "href": "u7"}, "sale": None, "status": "expired"},+ ]+ )+ ],+ "/v2/products/courses/12/enrollments": [_NOT_FOUND],+ },+ [+ {+ "user": {"id": 7, "href": "u7"},+ "sale": {"id": 3, "href": "s3"},+ "status": "active",+ "course_id": 11,+ "user_id": 7,+ "sale_id": 3,+ },+ {+ "user": {"id": 7, "href": "u7"},+ "sale": None,+ "status": "expired",+ "course_id": 11,+ "user_id": 7,+ "sale_id": None,+ },+ ],+ ["course_id", "user_id", "sale_id"],+ ),+ (+ # User 7 appears twice but is fetched once; user 8 was unenrolled after the listing.+ "course_progress",+ {+ "/v2/products/courses": [_V2_COURSES_PAGE],+ "/v2/products/courses/11/enrollments": [+ _v2_page([{"user": {"id": 7}}, {"user": {"id": 8}}, {"user": {"id": 7}}])+ ],+ "/v2/products/courses/12/enrollments": [_v2_page([])],+ "/v2/products/courses/11/users/7/progress": [+ ScriptedResponse(json={"data": {"id": 70, "percent_complete": 40, "sections": []}})+ ],+ "/v2/products/courses/11/users/8/progress": [_NOT_FOUND],+ },+ [{"id": 70, "percent_complete": 40, "sections": [], "course_id": 11, "user_id": 7}],+ ["course_id", "user_id"],+ ),+ (+ "lectures",+ {+ "/v2/products/courses": [_V2_COURSES_PAGE],+ "/v2/products/courses/11/lectures": [_v2_page([{"id": 5, "name": "Intro", "section": {"id": 1}}])],+ "/v2/products/courses/12/lectures": [_NOT_FOUND],+ },+ [{"id": 5, "name": "Intro", "section": {"id": 1}, "course_id": 11}],+ ["course_id", "id"],+ ),+ ]+ )+ def test_nested_walk(self, endpoint, routes, expected_rows, expected_primary_keys) -> None:+ result = _driver().run(endpoint, route(routes), api_version=V2)++ assert result.raised is None+ assert result.rows == expected_rows+ assert result.response is not None+ assert result.response.primary_keys == expected_primary_keys+ assert len(result.requests) == sum(len(answers) for answers in routes.values())