:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(data-warehouse): add 4 missing tables to the Teachable source (#120037)
Tom Owers74e8ce2ddab8
diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.mdindex 75690baa1a73..70f7f5f7967a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/COVERAGE_GAPS_APPENDIX.md@@ -12037,10 +12037,10 @@ Today (5): `course_enrollments`, `courses`, `pricing_plans`, `transactions`, `usDiffed against: <https://docs.teachable.com/reference/listcourses>-- [ ] `courses/{course_id}/progress` — per-student completion percentage — the headline learning metric, and the only way to measure course engagement (high)-- [ ] `courses/{course_id}/lectures/{lecture_id}` — lecture lookup table resolving the lecture IDs referenced by progress and quiz data (high)-- [ ] `courses/{course_id}/lectures/{lecture_id}/quizzes/{quiz_id}/responses` — individual learner answers and scores — the assessment fact table (medium)-- [ ] `courses/{course_id}/lectures/{lecture_id}/quizzes` — quiz lookup needed to interpret quiz responses (medium)+- [x] `courses/{course_id}/progress` — per-student completion percentage — the headline learning metric, and the only way to measure course engagement (high)+- [x] `courses/{course_id}/lectures/{lecture_id}` — lecture lookup table resolving the lecture IDs referenced by progress and quiz data (high)+- [x] `courses/{course_id}/lectures/{lecture_id}/quizzes/{quiz_id}/responses` — individual learner answers and scores — the assessment fact table (medium)+- [x] `courses/{course_id}/lectures/{lecture_id}/quizzes` — quiz lookup needed to interpret quiz responses (medium)Note: Full v1 path list read out of the embedded ReadMe OAS on the reference page: courses, enrollments, lectures, quizzes, quiz responses, progress, pricing_plans, transactions, users, webhooks. Lectures/quizzes/progress are per-course sub-resources that must be fanned out over the courses we already sync.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.pyindex c0f7572a5d96..3231385dfc41 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/canonical_descriptions.py@@ -76,4 +76,59 @@ CANONICAL_DESCRIPTIONS: CanonicalDescriptions = {"course_id": "Identifier of the course this pricing plan belongs to.",},},+ "course_progress": {+ "description": "A student's progress through a course, with completion state for each lecture section and lecture.",+ "docs_url": "https://docs.teachable.com/reference/courseprogress",+ "columns": {+ "course_id": "Unique identifier of the course.",+ "user_id": "Unique identifier of the student.",+ "id": "Unique identifier of the student's course progress.",+ "certificate": "The certificate issued to the student for the course: issued_at, page_id and serial_number.",+ "completed_at": "When the student completed the course, or null if not yet finished.",+ "enrolled_at": "When the student enrolled in the course, in ISO8601 format.",+ "lecture_sections": "Progress for each lecture section, with the completion state of each lecture in it.",+ "percent_complete": "Percentage of the course the student has marked complete.",+ },+ },+ "lectures": {+ "description": "A lecture in a course curriculum, with its attachments.",+ "docs_url": "https://docs.teachable.com/reference/showlecture",+ "columns": {+ "course_id": "Unique identifier of the course that contains the lecture.",+ "id": "Unique identifier of the lecture.",+ "name": "The lecture name.",+ "is_published": "Whether the lecture is published.",+ "position": "Position of the lecture within its section (1 is first).",+ "lecture_section_id": "Unique identifier of the section that contains the lecture.",+ "attachments": "Content attached to the lecture: text, video, image, PDF, quiz, code and other blocks.",+ },+ },+ "quizzes": {+ "description": "A quiz in a course lecture, with its questions and answers.",+ "docs_url": "https://docs.teachable.com/reference/showquiz",+ "columns": {+ "course_id": "Unique identifier of the course that contains the lecture.",+ "lecture_id": "Unique identifier of the lecture that contains the quiz.",+ "id": "Unique identifier of the quiz attachment.",+ "name": "The quiz name.",+ "kind": "The attachment kind. Always 'quiz'.",+ "position": "Position of the quiz within the lecture (1 is first).",+ "quiz": "The quiz content: its id and the list of questions, answers and correct answers.",+ },+ },+ "quiz_responses": {+ "description": "A student's submission of a quiz, with the score.",+ "docs_url": "https://docs.teachable.com/reference/showquizresponses",+ "columns": {+ "course_id": "Unique identifier of the course.",+ "lecture_id": "Unique identifier of the lecture that contains the quiz.",+ "quiz_id": "Unique identifier of the quiz attachment.",+ "graded": "Whether the quiz is graded.",+ "student_id": "Unique identifier of the student.",+ "student_name": "The student's name.",+ "student_email": "The student's email address.",+ "submitted_at": "When the student submitted the quiz.",+ "percent_correct": "Percentage of correct answers.",+ },+ },}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.pyindex 2bc6d1639a50..edae05e75db0 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/settings.py@@ -87,6 +87,40 @@ TEACHABLE_ENDPOINTS: dict[str, TeachableEndpointConfig] = {partition_key="created_at",# No server-side updated-since filter, so full refresh (the table is small).),+ # The four endpoints below sit two or more levels under /v1/courses, so `teachable.py` walks+ # them with its own iterators. None has a server-side time filter: all are full refresh.+ "course_progress": TeachableEndpointConfig(+ name="course_progress",+ # Needs a `user_id` query param, so this fans out over each course's enrollments.+ path="/v1/courses/{course_id}/progress",+ data_selector="course_progress",+ primary_key=["course_id", "user_id"],+ partition_key="enrolled_at",+ ),+ "lectures": TeachableEndpointConfig(+ name="lectures",+ # There is no lecture list endpoint: lecture ids come from the course detail curriculum.+ path="/v1/courses/{course_id}/lectures/{lecture_id}",+ data_selector="lecture",+ primary_key=["course_id", "id"],+ ),+ "quizzes": TeachableEndpointConfig(+ name="quizzes",+ # The list endpoint returns only `quiz_ids`, so each id is resolved through+ # /quizzes/{quiz_id} to get the quiz name and questions.+ path="/v1/courses/{course_id}/lectures/{lecture_id}/quizzes",+ data_selector="quiz_ids",+ primary_key=["course_id", "lecture_id", "id"],+ ),+ "quiz_responses": TeachableEndpointConfig(+ name="quiz_responses",+ path="/v1/courses/{course_id}/lectures/{lecture_id}/quizzes/{quiz_id}/responses",+ data_selector="responses",+ # The docs don't say whether a retake replaces the earlier submission, so the+ # submission time is part of the key.+ primary_key=["course_id", "lecture_id", "quiz_id", "student_id", "submitted_at"],+ partition_key="submitted_at",+ ),}ENDPOINTS = tuple(TEACHABLE_ENDPOINTS)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.pyindex eb2eae19b490..d2e87fe4bbbb 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/source.py@@ -116,8 +116,8 @@ class TeachableSource(ResumableSource[TeachableSourceConfig, TeachableResumeConfcategory=DataWarehouseSourceCategory.E_COMMERCE,label="Teachable",caption=(- "Import users, courses, enrollments, sales transactions, and pricing plans from "- "your Teachable school.\n\n"+ "Import users, courses, enrollments, course progress, lectures, quizzes, quiz responses, "+ "sales transactions, and pricing plans from your Teachable school.\n\n""Create an API key in your Teachable admin under Settings > API. ""The Teachable API requires a school on the Growth plan or higher."),diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.pyindex 635569d335d2..8e218cec64ca 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/teachable.py@@ -1,9 +1,11 @@import dataclasses-from collections.abc import Callable, Iterable+from collections.abc import Callable, Iterable, Iteratorfrom typing import Any, Optional, castfrom requests import Request, Response+from posthog.dataclasses import frozen+from products.warehouse_sources.backend.temporal.data_imports.sources.common.datetime_utils import (coerce_datetime_to_utc,)@@ -12,13 +14,19 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.resRESTAPIConfig,rest_api_resource,)+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.auth import APIKeyAuth+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.config_setup import (+ create_response_hooks,+)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.fanout import (build_dependent_resource,)from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.paginators import (BasePaginator,PageNumberPaginator,+ SinglePagePaginator,)+from products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.rest_client import RESTClientfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.rest_source.typing import (ClientConfig,Endpoint,@@ -28,6 +36,7 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.resfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManagerfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SortMode, SourceResponsefrom products.warehouse_sources.backend.temporal.data_imports.sources.teachable.settings import (+ DEFAULT_PAGE_SIZE,TEACHABLE_BASE_URL,TEACHABLE_ENDPOINTS,TeachableEndpointConfig,@@ -190,7 +199,7 @@ def get_resource(incremental_field_name: str | None = None,) -> EndpointResource:config = TEACHABLE_ENDPOINTS[endpoint]- if config.fanout:+ if config.fanout or endpoint in NESTED_FANOUT_ROWS:raise ValueError(f"Fan-out endpoint '{endpoint}' must use the fan-out path")endpoint_config: Endpoint = {@@ -218,14 +227,227 @@ def get_resource(}+SafePoint = Callable[[], None]+NestedRows = Iterator[list[dict[str, Any]]]++# A course, lecture, quiz or enrollment can be deleted between the parent listing and the+# child request; skip it rather than fail the whole sync.+_IGNORE_NOT_FOUND = create_response_hooks([{"status_code": 404, "action": "ignore"}], resource_name="teachable")+++def _rest_client(api_key: str) -> RESTClient:+ return RESTClient(+ base_url=TEACHABLE_BASE_URL,+ headers={"Accept": "application/json"},+ auth=APIKeyAuth(api_key=api_key, name="apiKey", location="header"),+ allowed_hosts=[],+ allow_redirects=False,+ )+++def _get_object(client: RESTClient, path: str, params: Optional[dict[str, Any]] = None) -> Optional[dict[str, Any]]:+ """GET one JSON object, or None when the API answers 404."""+ for page in client.paginate(path=path, params=params, paginator=SinglePagePaginator(), hooks=_IGNORE_NOT_FOUND):+ if page and isinstance(page[0], dict):+ return page[0]+ return None+++def _required(body: dict[str, Any], key: str, path: str) -> Any:+ if key not in body:+ raise ValueError(f"Teachable response for {path} has no {key!r} key. The API response shape may have changed.")+ return body[key]+++@frozen+class LectureKey:+ course_id: int+ lecture_id: int+++@frozen+class QuizKey:+ course_id: int+ lecture_id: int+ quiz_id: int+++def _iter_course_ids(client: RESTClient, safe_point: SafePoint) -> Iterator[int]:+ for courses in client.paginate(+ path="/v1/courses",+ params={"per": DEFAULT_PAGE_SIZE},+ paginator=_page_paginator(),+ data_selector="courses",+ data_selector_required=True,+ ):+ safe_point()+ for course in courses:+ if isinstance(course, dict) and isinstance(course.get("id"), int):+ yield course["id"]+++def _lecture_ids(client: RESTClient, course_id: int) -> list[int]:+ path = f"/v1/courses/{course_id}"+ body = _get_object(client, path)+ if body is None:+ return []+ course = _required(body, "course", path) or {}+ return [+ lecture["id"]+ for section in course.get("lecture_sections") or []+ if isinstance(section, dict)+ for lecture in section.get("lectures") or []+ if isinstance(lecture, dict) and isinstance(lecture.get("id"), int)+ ]+++def _quiz_ids(client: RESTClient, course_id: int, lecture_id: int) -> list[int]:+ path = f"/v1/courses/{course_id}/lectures/{lecture_id}/quizzes"+ body = _get_object(client, path)+ if body is None:+ return []+ return [quiz_id for quiz_id in _required(body, "quiz_ids", path) or [] if isinstance(quiz_id, int)]+++def _iter_lecture_keys(client: RESTClient, safe_point: SafePoint) -> Iterator[LectureKey]:+ for course_id in _iter_course_ids(client, safe_point):+ lecture_ids = _lecture_ids(client, course_id)+ safe_point()+ for lecture_id in lecture_ids:+ yield LectureKey(course_id=course_id, lecture_id=lecture_id)+++def _iter_quiz_keys(client: RESTClient, safe_point: SafePoint) -> Iterator[QuizKey]:+ for lecture in _iter_lecture_keys(client, safe_point):+ quiz_ids = _quiz_ids(client, lecture.course_id, lecture.lecture_id)+ safe_point()+ for quiz_id in quiz_ids:+ yield QuizKey(course_id=lecture.course_id, lecture_id=lecture.lecture_id, quiz_id=quiz_id)+++def _lecture_rows(client: RESTClient, safe_point: SafePoint) -> NestedRows:+ for key in _iter_lecture_keys(client, safe_point):+ course_id = key.course_id+ path = f"/v1/courses/{course_id}/lectures/{key.lecture_id}"+ body = _get_object(client, path)+ if body is None:+ safe_point()+ continue+ yield [{**_required(body, "lecture", path), "course_id": course_id}]+++def _quiz_rows(client: RESTClient, safe_point: SafePoint) -> NestedRows:+ for key in _iter_quiz_keys(client, safe_point):+ course_id, lecture_id, quiz_id = key.course_id, key.lecture_id, key.quiz_id+ quiz = _get_object(client, f"/v1/courses/{course_id}/lectures/{lecture_id}/quizzes/{quiz_id}")+ if quiz is None:+ safe_point()+ continue+ yield [{**quiz, "course_id": course_id, "lecture_id": lecture_id}]+++def _quiz_response_rows(client: RESTClient, safe_point: SafePoint) -> NestedRows:+ for key in _iter_quiz_keys(client, safe_point):+ course_id, lecture_id, quiz_id = key.course_id, key.lecture_id, key.quiz_id+ path = f"/v1/courses/{course_id}/lectures/{lecture_id}/quizzes/{quiz_id}/responses"+ body = _get_object(client, path)+ responses = _required(body, "responses", path) if body is not None else None+ if body is None or not responses:+ safe_point()+ continue+ yield [+ {+ **response,+ "course_id": course_id,+ "lecture_id": lecture_id,+ "quiz_id": quiz_id,+ "graded": body.get("graded"),+ }+ for response in responses+ if isinstance(response, dict)+ ]+++def _course_progress(+ client: RESTClient, course_id: int, user_id: int, safe_point: SafePoint+) -> Optional[dict[str, Any]]:+ """Fetch one student's progress, joining `lecture_sections` across pages.++ The endpoint documents `page`/`per` and pagination meta but not which list they page, so+ every page is read and its sections appended to the first page's object.+ """+ path = f"/v1/courses/{course_id}/progress"+ progress: Optional[dict[str, Any]] = None+ page = 1+ while True:+ body = _get_object(client, path, params={"user_id": user_id, "page": page, "per": DEFAULT_PAGE_SIZE})+ if body is None:+ return progress+ page_progress = _required(body, "course_progress", path) or {}+ if progress is None:+ progress = dict(page_progress)+ else:+ progress["lecture_sections"] = [+ *(progress.get("lecture_sections") or []),+ *(page_progress.get("lecture_sections") or []),+ ]+ meta = body.get("meta")+ number_of_pages = meta.get("number_of_pages") if isinstance(meta, dict) else None+ if not isinstance(number_of_pages, int) or page >= number_of_pages or not page_progress.get("lecture_sections"):+ return progress+ safe_point()+ page += 1+++def _course_progress_rows(client: RESTClient, safe_point: SafePoint) -> NestedRows:+ for course_id in _iter_course_ids(client, safe_point):+ seen_user_ids: set[int] = set()+ for enrollments in client.paginate(+ path=f"/v1/courses/{course_id}/enrollments",+ params={"per": DEFAULT_PAGE_SIZE, "sort_direction": "asc"},+ paginator=_page_paginator(),+ data_selector="enrollments",+ data_selector_required=True,+ hooks=_IGNORE_NOT_FOUND,+ ):+ safe_point()+ user_ids = [+ enrollment["user_id"]+ for enrollment in enrollments+ if isinstance(enrollment, dict) and isinstance(enrollment.get("user_id"), int)+ ]+ for user_id in user_ids:+ if user_id in seen_user_ids:+ continue+ seen_user_ids.add(user_id)+ progress = _course_progress(client, course_id, user_id, safe_point)+ if progress is None:+ safe_point()+ continue+ yield [{**progress, "course_id": course_id, "user_id": user_id}]+++# These tables sit two or more levels under /v1/courses, which the dependent-resource helper+# can't express. Nothing is resumable: a hand-off restarts the walk, and every request is+# followed by a yield or a safe point.+NESTED_FANOUT_ROWS: dict[str, Callable[[RESTClient, SafePoint], NestedRows]] = {+ "course_progress": _course_progress_rows,+ "lectures": _lecture_rows,+ "quizzes": _quiz_rows,+ "quiz_responses": _quiz_response_rows,+}++def _make_source_response(endpoint_config: TeachableEndpointConfig,items_fn: Callable[[], Iterable[Any]],sort_mode: SortMode = "asc",+ supports_resume: bool = True,) -> SourceResponse:return SourceResponse(name=endpoint_config.name,items=items_fn,+ supports_resume=supports_resume,primary_keys=endpoint_config.primary_key,sort_mode=sort_mode,partition_count=1,@@ -248,6 +470,14 @@ def teachable_source() -> SourceResponse:endpoint_config = TEACHABLE_ENDPOINTS[endpoint]+ nested_rows = NESTED_FANOUT_ROWS.get(endpoint)+ if nested_rows is not None:+ client = _rest_client(api_key)+ safe_point: SafePoint = (+ resumable_source_manager.safe_point if resumable_source_manager is not None else lambda: None+ )+ return _make_source_response(endpoint_config, lambda: nested_rows(client, safe_point), supports_resume=False)+if endpoint_config.fanout:# Dependent resources don't currently support resume in the rest_source framework;# the manager is intentionally not threaded into this path.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.pyindex 12b3afda515d..c37bb75742cb 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable.py@@ -31,6 +31,21 @@ def _driver() -> SourceDriver:return SourceDriver(TeachableSource(), TeachableSourceConfig(api_key="key"))+_COURSES_PAGE = ScriptedResponse(json={"courses": [{"id": 11}, {"id": 12}], "meta": {"number_of_pages": 1}})+_COURSE_11_DETAIL = ScriptedResponse(+ json={+ "course": {+ "id": 11,+ "lecture_sections": [+ {"id": 1, "lectures": [{"id": 5, "position": 1}, {"id": 6, "position": 2}]},+ {"id": 2, "lectures": []},+ ],+ }+ }+)+_NOT_FOUND = ScriptedResponse(status=404, json={"error": "not found"})++def _users_response(meta: dict[str, Any]) -> Mock:response = Mock()response.json.return_value = {"users": [], "meta": meta}@@ -139,9 +154,10 @@ class TestTeachableTransport:assert config["allowed_hosts"] == []assert config["allow_redirects"] is False- def test_get_resource_rejects_fanout_endpoint(self) -> None:+ @parameterized.expand([("dependent_resource", "course_enrollments"), ("nested_fanout", "lectures")])+ def test_get_resource_rejects_fanout_endpoint(self, _name, endpoint) -> None:with pytest.raises(ValueError, match="Fan-out endpoint"):- get_resource("course_enrollments", should_use_incremental_field=False)+ get_resource(endpoint, should_use_incremental_field=False)def test_enrollments_fanout_row_format(self) -> None:result = _driver().run(@@ -207,3 +223,192 @@ class TestTeachableTransport:assert response.sort_mode == expectedassert response.primary_keys == ["id"]assert response.partition_keys == ["created_at"]+++class TestTeachableNestedFanout:+ @parameterized.expand(+ [+ (+ # Course 12 and lecture 6 were deleted after the parent listing, so their 404s are skipped.+ "lectures",+ {+ "/v1/courses": [_COURSES_PAGE],+ "/v1/courses/11": [_COURSE_11_DETAIL],+ "/v1/courses/12": [_NOT_FOUND],+ "/v1/courses/11/lectures/5": [+ ScriptedResponse(json={"lecture": {"id": 5, "name": "Intro", "lecture_section_id": 1}})+ ],+ "/v1/courses/11/lectures/6": [_NOT_FOUND],+ },+ [{"id": 5, "name": "Intro", "lecture_section_id": 1, "course_id": 11}],+ ),+ (+ # Lecture 6 has no quizzes; quiz 10 was deleted after the id listing.+ "quizzes",+ {+ "/v1/courses": [_COURSES_PAGE],+ "/v1/courses/11": [_COURSE_11_DETAIL],+ "/v1/courses/12": [ScriptedResponse(json={"course": {"id": 12, "lecture_sections": []}})],+ "/v1/courses/11/lectures/5/quizzes": [ScriptedResponse(json={"quiz_ids": [9, 10]})],+ "/v1/courses/11/lectures/6/quizzes": [ScriptedResponse(json={"quiz_ids": []})],+ "/v1/courses/11/lectures/5/quizzes/9": [+ ScriptedResponse(json={"id": 9, "name": "Check", "kind": "quiz", "quiz": {"id": 3}})+ ],+ "/v1/courses/11/lectures/5/quizzes/10": [_NOT_FOUND],+ },+ [{"id": 9, "name": "Check", "kind": "quiz", "quiz": {"id": 3}, "course_id": 11, "lecture_id": 5}],+ ),+ (+ # Quiz 10 has no submissions, so it adds no rows.+ "quiz_responses",+ {+ "/v1/courses": [_COURSES_PAGE],+ "/v1/courses/11": [_COURSE_11_DETAIL],+ "/v1/courses/12": [_NOT_FOUND],+ "/v1/courses/11/lectures/5/quizzes": [ScriptedResponse(json={"quiz_ids": [9, 10]})],+ "/v1/courses/11/lectures/6/quizzes": [ScriptedResponse(json={"quiz_ids": []})],+ "/v1/courses/11/lectures/5/quizzes/9/responses": [+ ScriptedResponse(+ json={+ "course_id": 11,+ "graded": True,+ "responses": [+ {"student_id": 7, "submitted_at": "2026-01-02T00:00:00Z", "percent_correct": 80},+ {"student_id": 8, "submitted_at": "2026-01-03T00:00:00Z", "percent_correct": 50},+ ],+ }+ )+ ],+ "/v1/courses/11/lectures/5/quizzes/10/responses": [+ ScriptedResponse(json={"graded": False, "responses": []})+ ],+ },+ [+ {+ "student_id": 7,+ "submitted_at": "2026-01-02T00:00:00Z",+ "percent_correct": 80,+ "course_id": 11,+ "lecture_id": 5,+ "quiz_id": 9,+ "graded": True,+ },+ {+ "student_id": 8,+ "submitted_at": "2026-01-03T00:00:00Z",+ "percent_correct": 50,+ "course_id": 11,+ "lecture_id": 5,+ "quiz_id": 9,+ "graded": True,+ },+ ],+ ),+ (+ # User 7's progress spans two pages and is joined into one row; user 8 was unenrolled+ # after the enrollment listing.+ "course_progress",+ {+ "/v1/courses": [_COURSES_PAGE],+ "/v1/courses/11/enrollments": [+ ScriptedResponse(+ json={"enrollments": [{"user_id": 7}, {"user_id": 8}], "meta": {"number_of_pages": 1}}+ )+ ],+ "/v1/courses/12/enrollments": [_NOT_FOUND],+ "/v1/courses/11/progress": [+ ScriptedResponse(+ json={+ "course_progress": {+ "id": 70,+ "enrolled_at": "2026-01-01T00:00:00Z",+ "percent_complete": 40,+ "lecture_sections": [{"lecture_section_progress": {"id": 1}}],+ },+ "meta": {"page": 1, "number_of_pages": 2},+ }+ ),+ ScriptedResponse(+ json={+ "course_progress": {+ "id": 70,+ "enrolled_at": "2026-01-01T00:00:00Z",+ "percent_complete": 40,+ "lecture_sections": [{"lecture_section_progress": {"id": 2}}],+ },+ "meta": {"page": 2, "number_of_pages": 2},+ }+ ),+ _NOT_FOUND,+ ],+ },+ [+ {+ "id": 70,+ "enrolled_at": "2026-01-01T00:00:00Z",+ "percent_complete": 40,+ "lecture_sections": [+ {"lecture_section_progress": {"id": 1}},+ {"lecture_section_progress": {"id": 2}},+ ],+ "course_id": 11,+ "user_id": 7,+ }+ ],+ ),+ ]+ )+ def test_nested_fanout_walk(self, endpoint, routes, expected_rows) -> None:+ result = _driver().run(endpoint, route(routes))++ assert result.raised is None+ assert result.rows == expected_rows+ # Every scripted answer was used, so no branch of the walk was skipped.+ assert len(result.requests) == sum(len(answers) for answers in routes.values())++ def test_course_progress_requests_each_enrolled_user_once(self) -> None:+ progress = ScriptedResponse(+ json={"course_progress": {"id": 1, "lecture_sections": []}, "meta": {"number_of_pages": 1}}+ )+ result = _driver().run(+ "course_progress",+ route(+ {+ "/v1/courses": [ScriptedResponse(json={"courses": [{"id": 11}], "meta": {"number_of_pages": 1}})],+ "/v1/courses/11/enrollments": [+ ScriptedResponse(+ json={+ "enrollments": [{"user_id": 7}, {"user_id": 8}, {"user_id": 7}],+ "meta": {"number_of_pages": 1},+ }+ )+ ],+ "/v1/courses/11/progress": [progress, progress],+ }+ ),+ )++ assert [row["user_id"] for row in result.rows] == [7, 8]+ progress_requests = [r for r in result.requests if r.path.endswith("/progress")]+ assert [r.param("user_id") for r in progress_requests] == ["7", "8"]++ @parameterized.expand(+ [+ ("course_detail", "lectures", "/v1/courses/11", {"id": 11}, "'course'"),+ ("lecture_detail", "lectures", "/v1/courses/11/lectures/5", {"id": 5}, "'lecture'"),+ ("quiz_ids", "quizzes", "/v1/courses/11/lectures/5/quizzes", {"ids": []}, "'quiz_ids'"),+ ]+ )+ def test_missing_wrapper_key_fails_loud(self, _name, endpoint, broken_path, broken_body, key) -> None:+ routes = {+ "/v1/courses": [ScriptedResponse(json={"courses": [{"id": 11}], "meta": {"number_of_pages": 1}})],+ "/v1/courses/11": [_COURSE_11_DETAIL],+ "/v1/courses/11/lectures/5": [ScriptedResponse(json={"lecture": {"id": 5}})],+ "/v1/courses/11/lectures/5/quizzes": [ScriptedResponse(json={"quiz_ids": []})],+ }+ routes[broken_path] = [ScriptedResponse(json=broken_body)]++ result = _driver().run(endpoint, route(routes))++ assert isinstance(result.raised, ValueError)+ assert key in str(result.raised)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable_source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable_source.pyindex 5a9a2b157848..235731b5853e 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable_source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/teachable/tests/test_teachable_source.py@@ -40,7 +40,16 @@ class TestTeachableSource:assert [f["field"] for f in schemas["transactions"].incremental_fields] == ["created_at"]assert schemas["transactions"].default_incremental_lookback_seconds == TRANSACTIONS_INCREMENTAL_LOOKBACK_SECONDS- for name in ("users", "courses", "course_enrollments", "pricing_plans"):+ for name in (+ "users",+ "courses",+ "course_enrollments",+ "pricing_plans",+ "course_progress",+ "lectures",+ "quizzes",+ "quiz_responses",+ ):assert schemas[name].supports_incremental is False, nameassert schemas[name].incremental_fields == [], name