:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
fix(data-warehouse): continue an import only when the source proves its row order (#114359)
Tom Owers75f44dc3c971
diff --git a/.agents/skills/implementing-warehouse-sources/SKILL.md b/.agents/skills/implementing-warehouse-sources/SKILL.mdindex e9c7a0d24bb9..0849c311f2e7 100644--- a/.agents/skills/implementing-warehouse-sources/SKILL.md+++ b/.agents/skills/implementing-warehouse-sources/SKILL.md@@ -472,6 +472,30 @@ python posthog/test/repo_invariants/test_warehouse_source_static_contract.pypython posthog/test/repo_invariants/test_resume_state_staged_before_yield.py```+#### Row order and `sort_mode`++`SourceResponse.sort_mode` says whether the rows of an incremental or append read are ordered:++- `"asc"`: the rows arrive in ascending order of the incremental field, from the first row of the run to the last.+- `"desc"`: the rows arrive in descending order. Use it only when the API cannot return ascending order.+- `None` (the default): not ordered, or not known.++It has no meaning for a full refresh. A full refresh can read in any order, and nothing requires it to sort.++The watermark does not depend on the value: it is the largest incremental value the run read, and the loader saves it when the run completes.++`"asc"` is a promise the pipeline relies on. After a worker shutdown, the next attempt of an `"asc"` run reads only the rows after the last value the earlier attempt queued. A row below that value that arrives later is never read, and the watermark then moves past it. A false `"asc"` loses rows silently. With `None` the interrupted run starts again from the stored watermark, which is always correct.++Before you set `"asc"`, prove all of these:++1. The order comes from the query or the API contract: an `ORDER BY <incremental field> ASC`, or an explicit ascending sort parameter on the incremental field. A default order that the vendor does not document is not proof.+2. The run is one ordered series. Pages by another key (a primary-key keyset, an `_id` cursor), one request series per parent, account or partition, unsorted time windows, and an ordered query whose stored result is read back with no order do not qualify.+3. The value is set per run, where the response is built. Say `"asc"` only on the read path that has the order, not on a keyset, partition, fallback or full refresh path of the same source.+4. No fallback or retry in the generator drops the order, or starts the read again after rows were yielded.+5. A test fails when the incremental query or request loses its sort, and a test asserts `sort_mode` for each read path.++When in doubt, leave `None`. The only cost is that an interrupted run starts again from the stored watermark.+### Webhook source pattern- Implement `webhook_template` returning a `HogFunctionTemplateDC` that transforms incoming webhook payloads.@@ -610,7 +634,7 @@ skill — don't hand-roll version handling.- **Per-endpoint sort enums vary.** Don't hardcode `?sorting=created_at` (or whatever) globally. Verify each list endpoint's allowed sort values against the API spec **and** with a curl smoke-test against the live API — APIs frequently document one set of options and silently reject another, or use a different timestamp column on certain resources.- **Pass `?sorting=` explicitly on a stable monotonic field when paginating.** For incremental sources, the request sort must match `SourceResponse.sort_mode` (`"asc"` typically; `"desc"` only when forced by the API — see `stripe/stripe.py`, `github/settings.py`) so the pipeline's cursor watermark advances correctly. For full-refresh sources, an explicit sort prevents page-boundary skips/duplicates if the API's implicit default is unstable or shifts as rows are inserted during the sync.- If the API only supports cursor pagination, still declare incremental fields if reliable and let merge semantics dedupe.-- **`sort_mode` must match the order rows actually arrive in — verify it, don't assume it.** The pipeline trusts `sort_mode="asc"` to checkpoint the incremental watermark after every batch and to allow safe mid-sync worker shutdowns; declaring `asc` while the API returns newest-first corrupts the watermark and breaks resume semantics. Check the API's _default_ sort (it applies when you can't pass `sort`), and remember cursor pagination often rejects or ignores sort params entirely.+- **`sort_mode` must match the order rows actually arrive in. Verify it, do not assume it.** `sort_mode="asc"` is a promise: after a mid-sync worker shutdown the pipeline continues an `asc` run after the last value it queued, so a row that arrives out of order is lost. Leave the default `None` when the order is not proven: per-parent or per-window paging, an API _default_ sort (it applies when you can't pass `sort`), or cursor pagination that rejects or ignores sort params. See "Row order and `sort_mode`".- `sort_mode="desc"` only if the endpoint truly cannot return ascending. For descending sources, handle `db_incremental_field_earliest_value` to scroll earlier rows before newer ones (see Stripe).- **Incremental pagination must terminate at the watermark.** Some APIs reject mixing their time-window filter with cursor pagination, so only the first page is windowed and later pages walk back through history unbounded. If the server can't keep the filter on every page, the paginator must stop client-side once an entire page predates `db_incremental_field_last_value` (see `typeform/typeform.py:TypeformResponsesPaginator`) — otherwise **every incremental sync re-fetches and re-merges each parent's full history**, which is both an API-cost bug and a per-sync memory amplifier.- Default unknown endpoints to full refresh first; enable incremental only after confirming a stable filter field and API ordering semantics.@@ -907,7 +931,8 @@ After changing source fields, re-run `pnpm run generate:source-configs` and `hog- Generated config class still empty: forgot `generate:source-configs` after updating fields.- Incremental sync misbehaving: wrong field name/type or wrong sort assumptions.- Pod OOMs on a busy table: primary key not actually unique (usually a fan-out child missing the parent id in its key) — duplicate rows accumulate and every merge multi-matches them; often paired with a paginator that re-walks full history each sync because the time filter only applies to page one.-- `sort_mode="asc"` declared on an API that returns newest-first: the watermark checkpoints to ≈now after the first batch and mid-sync shutdowns lose data ordering guarantees.+- `sort_mode="asc"` declared on an API that returns newest-first: a run that continues after a mid-sync shutdown skips the older rows it did not read yet. Declare `"desc"`, or `None` when the order is not known.+- `sort_mode="asc"` on a read that is not one ordered series (per parent, per window, by another key): a run that continues after a worker shutdown skips the rows below its continuation point. See "Row order and `sort_mode`".- Endless retries for bad credentials: missing `get_non_retryable_errors`.- Source won't connect despite a valid token: `validate_credentials(schema_name=None)` probes every resource's scope instead of just the token, so one missing scope — often on a table the user won't sync — blocks the whole source. Probe only the token at create; report per-table scope via `get_endpoint_permissions`.- Resumable state never saved: forgot to call `save_state`; or called `commit()` on a cursor that covers rows the pipeline has not written yet, which skips them on resume.diff --git a/products/warehouse_sources/backend/models/external_data_schema.py b/products/warehouse_sources/backend/models/external_data_schema.pyindex c5de0957e4de..e295586ab8ad 100644--- a/products/warehouse_sources/backend/models/external_data_schema.py+++ b/products/warehouse_sources/backend/models/external_data_schema.py@@ -216,7 +216,7 @@ STAGED_CURSOR_PENDING_LIMIT = MAX_RESUMABLE_SOURCE_RETRIES_PRODUCTION# The key, inside a staged cursor, for the incremental value a later attempt of the same workflow# run can resume after. It differs from the staged `last_value`, which the loader promotes only# when the whole run completes.-STAGED_RESUME_VALUE_KEY = "resume_value"+STAGED_RESUME_VALUE_KEY = "ordered_resume_value"# The key, inside a staged cursor, for the run whose queue rows the resume value actually describes.# An attempt that only inherits the value from an earlier attempt, without queuing a batch of itsdiff --git a/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/extract.py b/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/extract.pyindex 4e394e5f88c1..cf65c8b8edf1 100644--- a/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/extract.py+++ b/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/extract.py@@ -833,7 +833,9 @@ async def update_incremental_field_values(if last_advanced:last_incremental_field_value = last_value- if resource.sort_mode == "asc":+ # The loader promotes the staged value only when the run completes, so the largest value+ # read is the watermark with or without an order. A descending read stages it at the end.+ if resource.sort_mode != "desc":await logger.adebug(f"{log_prefix}Updating incremental_field_last_value with {last_incremental_field_value}")@@ -900,9 +902,8 @@ def should_check_shutdown(reset_pipeline: bool,source_is_resumable: bool,) -> bool:- # Only raise if we're not running in descending order, otherwise we'll often not- # complete the job before the incremental value can be updated. Or if the source is- # resumable+ # A descending run does not raise, because each stop restarts it from the old watermark. An ascending+ # incremental run keeps its progress after a stop only through the handoff checkpoint.# Let a new attempt leave a shutting-down worker before it holds the worker for hours.# Limit handoffs to attempt one so full refresh keeps two attempts for retries.# Raising during a full reset will reset our progress back to 0 rowsdiff --git a/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/test/test_extract.py b/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/test/test_extract.pyindex 0d6d3f3d1878..517912096550 100644--- a/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/test/test_extract.py+++ b/products/warehouse_sources/backend/temporal/data_imports/pipelines/common/test/test_extract.py@@ -1,6 +1,7 @@import uuidfrom contextlib import asynccontextmanagerfrom datetime import UTC, datetime, timedelta+from typing import Anyimport pytestfrom posthog.test.base import BaseTest@@ -27,6 +28,7 @@ from products.warehouse_sources.backend.temporal.data_imports.external_data_jobfrom products.warehouse_sources.backend.temporal.data_imports.pipelines.common.extract import (NON_RETRYABLE_ERROR_RETRY_LIMIT,UNREADABLE_JOB_INPUTS_MESSAGE,+ IncrementalFieldValues,_get_redis,count_worker_deaths,finalize_desc_sort_incremental_value,@@ -51,7 +53,7 @@ from products.warehouse_sources.backend.temporal.data_imports.row_tracking imporget_rows,increment_rows,)-from products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SourceResponse+from products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SortMode, SourceResponsefrom products.warehouse_sources.backend.temporal.data_imports.util import NonRetryableExceptionfrom products.warehouse_sources.backend.types import IncrementalFieldType@@ -67,6 +69,7 @@ class TestShouldCheckShutdown:("outside_activity", False, "asc", True, False, False, 1, 2, False),("resumable_source", False, "asc", True, True, True, 2, 20, True),("ascending_incremental", True, "asc", False, False, True, 2, 20, True),+ ("unordered_incremental", True, None, False, False, True, 2, 20, True),("descending_incremental", True, "desc", False, False, True, 2, 20, False),])@@ -74,7 +77,7 @@ class TestShouldCheckShutdown:self,_name: str,is_incremental: bool,- sort_mode: str,+ sort_mode: str | None,reset_pipeline: bool,source_is_resumable: bool,in_activity: bool,@@ -96,6 +99,63 @@ class TestShouldCheckShutdown:activity_info.assert_not_called()+class TestIncrementalWatermark:+ @parameterized.expand(+ [+ # name, sort_mode, uses_incremental_field, staged after each batch, staged at the end of the run+ ("ascending", "asc", True, [("run-1", 5), ("run-1", 5)], []),+ ("unordered", None, True, [("run-1", 5), ("run-1", 5)], []),+ ("descending", "desc", True, [("run-1", 5, 3), ("run-1", 5, 1)], [("run-1", 5)]),+ ("full_refresh_ascending", "asc", False, [], []),+ ("full_refresh_unordered", None, False, [], []),+ ("full_refresh_descending", "desc", False, [], []),+ ]+ )+ @pytest.mark.asyncio+ async def test_the_staged_watermark_is_the_largest_value_read(+ self,+ _name: str,+ sort_mode: SortMode | None,+ uses_incremental_field: bool,+ expected_per_batch: list[tuple[Any, ...]],+ expected_at_end: list[tuple[Any, ...]],+ ) -> None:+ schema = MagicMock(+ should_use_incremental_field=uses_incremental_field,+ sync_type_config={"incremental_field": "n"},+ incremental_field_type=IncrementalFieldType.Integer,+ )+ resource = SourceResponse(name="orders", items=lambda: iter(()), primary_keys=["id"], sort_mode=sort_mode)+ logger = MagicMock(adebug=AsyncMock())++ async def call(fn: Any, *args: Any) -> Any:+ return fn(*args)++ with patch(+ f"{_EXTRACT_MODULE}.database_sync_to_async_pool", lambda fn: lambda *args, **kwargs: call(fn, *args)+ ):+ values = IncrementalFieldValues(last_value=None, earliest_value=None)+ # The largest value is in the first batch, so the last row of the run does not hold it.+ for batch in ([3, 5, 4], [1, 2]):+ values = await update_incremental_field_values(+ schema,+ pa.table({"n": batch}),+ resource,+ values.last_value,+ values.earliest_value,+ logger,+ staging_run_uuid="run-1",+ )+ staged_per_batch = [staged.args for staged in schema.stage_incremental_field_value.call_args_list]+ schema.stage_incremental_field_value.reset_mock()+ await finalize_desc_sort_incremental_value(+ resource, schema, values.last_value, logger, staging_run_uuid="run-1"+ )++ assert staged_per_batch == expected_per_batch+ assert [staged.args for staged in schema.stage_incremental_field_value.call_args_list] == expected_at_end++class TestResolvePrimaryKeys:@parameterized.expand([diff --git a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/pipeline.py b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/pipeline.pyindex f5721f7f7f81..c9d576f867a8 100644--- a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/pipeline.py+++ b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/pipeline.py@@ -366,17 +366,17 @@ class PipelineV3(Generic[ResumableData]):self._shutdown_monitor = shutdown_monitorself._shutdown_stopwatch = ShutdownStopwatch(shutdown_monitor)self._last_incremental_field_value: Any = None- if is_resume and source_response.sort_mode == "desc":- # A resumed descending attempt reads only rows older than the rows that the earlier attempts- # of this run read. Without their newest value, the run promotes the resume point as its- # watermark, and the next run reads every newer row again. `models.schema` is loaded when- # this attempt starts, so it holds what the earlier attempts staged.+ if is_resume and source_response.sort_mode != "asc":+ # A resumed attempt of a descending or unordered source can skip the rows that held the newest+ # value of the earlier attempts of this run. Without their newest value, the run promotes the+ # largest value of its own rows as the watermark, and the next run reads every newer row again.+ # `models.schema` is loaded when this attempt starts, so it holds what the earlier attempts staged.self._last_incremental_field_value = staged_run_last_value(models.schema.sync_type_config or {}, self._job.workflow_run_id)- if self._last_incremental_field_value is None:+ if self._last_incremental_field_value is None and self._schema.should_use_incremental_field:self._logger.warning(- "V3 Pipeline: Resumed desc extract found no staged last value of this run",+ "V3 Pipeline: Resumed non-ascending extract found no staged last value of this run",workflow_run_id=self._job.workflow_run_id,)self._earliest_incremental_field_value: Any = process_incremental_value(diff --git a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_checkpoint.py b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_checkpoint.pyindex 435d9e52b526..f0e0c903b6d0 100644--- a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_checkpoint.py+++ b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_checkpoint.py@@ -105,27 +105,27 @@ def test_resume_value_covers_only_rows_that_cannot_have_later_siblings([pytest.param({}, "run-1", None, id="nothing_staged"),pytest.param(- {"incremental_staged": {"run_uuid": "run-1-a1", "last_value": 50, "resume_value": 40}},+ {"incremental_staged": {"run_uuid": "run-1-a1", "last_value": 50, "ordered_resume_value": 40}},"run-1",40,id="live_slot_of_this_run",),pytest.param(- {"incremental_staged": {"run_uuid": "run-1-a1", "last_value": 50, "resume_value": 40}},+ {"incremental_staged": {"run_uuid": "run-1-a1", "last_value": 50, "ordered_resume_value": 40}},"run-2",None,id="another_workflow_run_never_reads_it",),pytest.param(- {"incremental_staged": {"run_uuid": "run-1-a1", "last_value": 50, "resume_value": 40}},+ {"incremental_staged": {"run_uuid": "run-1-a1", "last_value": 50, "ordered_resume_value": 40}},None,None,id="no_workflow_run",),pytest.param({- "incremental_staged": {"run_uuid": "run-1-a2", "last_value": 90, "resume_value": 80},- "incremental_staged_pending": [{"run_uuid": "run-1-a1", "last_value": 50, "resume_value": 40}],+ "incremental_staged": {"run_uuid": "run-1-a2", "last_value": 90, "ordered_resume_value": 80},+ "incremental_staged_pending": [{"run_uuid": "run-1-a1", "last_value": 50, "ordered_resume_value": 40}],},"run-1",80,@@ -133,8 +133,8 @@ def test_resume_value_covers_only_rows_that_cannot_have_later_siblings(),pytest.param({- "incremental_staged": {"run_uuid": "run-1-a2", "last_value": 50, "resume_value": 40},- "incremental_staged_pending": [{"run_uuid": "run-1-a10", "last_value": 90, "resume_value": 80}],+ "incremental_staged": {"run_uuid": "run-1-a2", "last_value": 50, "ordered_resume_value": 40},+ "incremental_staged_pending": [{"run_uuid": "run-1-a10", "last_value": 90, "ordered_resume_value": 80}],},"run-1",80,@@ -144,7 +144,7 @@ def test_resume_value_covers_only_rows_that_cannot_have_later_siblings(pytest.param({"incremental_staged": {"run_uuid": "run-1-a2", "last_value": 20},- "incremental_staged_pending": [{"run_uuid": "run-1-a1", "last_value": 50, "resume_value": 40}],+ "incremental_staged_pending": [{"run_uuid": "run-1-a1", "last_value": 50, "ordered_resume_value": 40}],},"run-1",None,@@ -152,15 +152,15 @@ def test_resume_value_covers_only_rows_that_cannot_have_later_siblings(),pytest.param({- "incremental_staged": {"run_uuid": "run-1-a2", "resume_value": None},- "incremental_staged_pending": [{"run_uuid": "run-1-a1", "last_value": 50, "resume_value": 40}],+ "incremental_staged": {"run_uuid": "run-1-a2", "ordered_resume_value": None},+ "incremental_staged_pending": [{"run_uuid": "run-1-a1", "last_value": 50, "ordered_resume_value": 40}],},"run-1",None,id="newest_attempt_recorded_that_it_has_no_value",),pytest.param(- {"incremental_staged": {"run_uuid": "run-10-a1", "last_value": 50, "resume_value": 40}},+ {"incremental_staged": {"run_uuid": "run-10-a1", "last_value": 50, "ordered_resume_value": 40}},"run-1",None,id="run_id_prefix_of_another_run",diff --git a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_continuation_load.py b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_continuation_load.pyindex 11494ee38826..90c2154cadcf 100644--- a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_continuation_load.py+++ b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_handoff_continuation_load.py@@ -85,6 +85,8 @@ class _Import:queue: list[dict[str, Any]] = field(default_factory=list)attempts: int = 0pages_read: int = 0+ reads_in_incremental_order: bool = True+ source_reads: list[int | None] = field(default_factory=list)def __post_init__(self) -> None:self.schema = ExternalDataSchema(@@ -106,8 +108,11 @@ class _Import:def _source(self, read_after: int | None) -> SourceResponse:self.pages_read = 0+ self.source_reads.append(read_after)+ order_column = 1 if self.reads_in_incremental_order else 0rows = sorted(- (row for row in self.source_rows if read_after is None or row[1] > read_after), key=lambda row: row[1]+ (row for row in self.source_rows if read_after is None or row[1] > read_after),+ key=lambda row: row[order_column],)def items() -> Iterator[pa.Table]:@@ -116,7 +121,13 @@ class _Import:self.pages_read += 1yield pa.table({"id": [row[0] for row in page], "n": [row[1] for row in page]})- return SourceResponse(name="orders", items=items, primary_keys=["id"], chunk_size=_PAGE_ROWS)+ return SourceResponse(+ name="orders",+ items=items,+ primary_keys=["id"],+ chunk_size=_PAGE_ROWS,+ sort_mode="asc" if self.reads_in_incremental_order else None,+ )async def run_attempt(self, *, shut_down_after_pages: int | None, reset_pipeline: bool = False) -> bool:"""Run one execution of the import the way the activity builds it. Returns whether it handed off."""@@ -240,6 +251,11 @@ class _Import:primary_keys=["id"],)+ def complete_job(self) -> None:+ """Promote the staged watermark of the last attempt, as the loader does when the job completes."""+ with patch.object(schema_models, "update_sync_type_config_keys", self._update_config):+ self.schema.promote_staged_incremental_values(f"{_WORKFLOW_RUN_ID}-a{self.attempts}")+def loaded_ids(self) -> list[int]:ids = deltalake.DeltaTable(self.table_uri).to_pyarrow_table().column("id").to_pylist()return sorted(cast(list[int], ids))@@ -287,3 +303,47 @@ async def test_a_run_that_continues_after_a_handoff_loads_each_source_row_once(await run.load_queue()assert run.loaded_ids() == [row[0] for row in source_rows]+++# In id order the incremental values go up to 101 and then back to 21. The first two pages are+# ascending, so a run that trusted them would continue after 100 and never read 21 to 28.+_ID_ORDER = _rows([10, 11, 12, 13, 14, 15, 100, 101, 21, 22, 23, 24, 25, 26, 27, 28])+++ "stored_watermark", [pytest.param(None, id="first_sync"), pytest.param(1, id="existing_table")]+)+async def test_an_unordered_source_starts_again_after_a_handoff(tmp_path: Path, stored_watermark: int | None) -> None:+ run = _Import(table_uri=str(tmp_path / "orders"), source_rows=_ID_ORDER, reads_in_incremental_order=False)+ if stored_watermark is not None:+ deltalake.write_deltalake(run.table_uri, pa.table({"id": [0], "n": [stored_watermark]}))+ run.schema.table = DataWarehouseTable(name="orders", team_id=1)+ run.schema.sync_type_config["incremental_field_last_value"] = stored_watermark++ assert await run.run_attempt(shut_down_after_pages=2)+ assert not await run.run_attempt(shut_down_after_pages=None)+ await run.load_queue()++ assert run.source_reads == [stored_watermark, stored_watermark]+ assert set(run.loaded_ids()) >= {row[0] for row in _ID_ORDER}+++async def test_an_unordered_run_moves_the_watermark_to_its_largest_value_only_when_the_job_completes(+ tmp_path: Path,+) -> None:+ run = _Import(table_uri=str(tmp_path / "orders"), source_rows=_ID_ORDER, reads_in_incremental_order=False)+ deltalake.write_deltalake(run.table_uri, pa.table({"id": [0], "n": [1]}))+ run.schema.table = DataWarehouseTable(name="orders", team_id=1)+ run.schema.sync_type_config["incremental_field_last_value"] = 1++ assert await run.run_attempt(shut_down_after_pages=2)+ assert run.schema.sync_type_config["incremental_field_last_value"] == 1++ assert not await run.run_attempt(shut_down_after_pages=None)+ assert run.schema.sync_type_config["incremental_field_last_value"] == 1+ run.complete_job()++ # The last row holds 28. The watermark is the largest value, which arrived in the middle of the read.+ assert run.schema.sync_type_config["incremental_field_last_value"] == 101diff --git a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_pipeline.py b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_pipeline.pyindex 0fc20ee2fb87..c93be7c72b8c 100644--- a/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_pipeline.py+++ b/products/warehouse_sources/backend/temporal/data_imports/pipelines/pipeline_v3/test_pipeline.py@@ -61,6 +61,7 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.resfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManagerfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import (OutputLane,+ SortMode,SourceInputs,SourceResponse,)@@ -1223,7 +1224,7 @@ class TestDescWatermarkAcrossResumedAttempts:},)- def _attempt(self, schema: ExternalDataSchema, attempt: int) -> PipelineV3:+ def _attempt(self, schema: ExternalDataSchema, attempt: int, sort_mode: str | None = "desc") -> PipelineV3:resource = MagicMock(name="events",primary_keys=["id"],@@ -1235,7 +1236,7 @@ class TestDescWatermarkAcrossResumedAttempts:cdc_write_mode=None,lanes=None,webhook_only=False,- sort_mode="desc",+ sort_mode=sort_mode,)manager = MagicMock(can_resume=MagicMock(return_value=attempt > 1))with (@@ -1262,8 +1263,10 @@ class TestDescWatermarkAcrossResumedAttempts:),)- async def _extract(self, schema: ExternalDataSchema, attempt: int, *, final: bool) -> str:- pipeline = self._attempt(schema, attempt)+ async def _extract(+ self, schema: ExternalDataSchema, attempt: int, *, final: bool, sort_mode: str | None = "desc"+ ) -> str:+ pipeline = self._attempt(schema, attempt, sort_mode)run_uuid = cast(str, pipeline._run_uuid)# Each attempt resumes on an older page than the attempt before it read.newest_row = self._NEWEST - timedelta(hours=10 * (attempt - 1))@@ -1314,6 +1317,22 @@ class TestDescWatermarkAcrossResumedAttempts:assert schema.sync_type_config["incremental_field_last_value"] == self._NEWEST.isoformat()assert schema.sync_type_config["incremental_field_earliest_value"] == oldest_row.isoformat()+ @pytest.mark.asyncio+ async def test_an_unordered_resumed_attempt_keeps_the_largest_value_of_the_run(self) -> None:+ # Attempt 2 resumes past the part of the source that held the largest value. It must start from the+ # value attempt 1 staged, or the run promotes the smaller maximum and the next run reads old rows again.+ schema = self._schema()++ with (+ patch.object(schema_models, "update_sync_type_config_keys", side_effect=_in_memory_config_update(schema)),+ patch.object(ExternalDataSchema, "refresh_from_db"),+ ):+ await self._extract(schema, 1, final=False, sort_mode=None)+ run_uuid = await self._extract(schema, 2, final=True, sort_mode=None)+ assert schema.promote_staged_incremental_values(run_uuid)++ assert schema.sync_type_config["incremental_field_last_value"] == self._NEWEST.isoformat()+class TestZeroBatchRunStampsTheFullRunMarker:@pytest.mark.asyncio@@ -2284,6 +2303,36 @@ class TestIncrementalHandoffCheckpoint:def test_an_attempt_that_continues_keeps_the_queue_rows_of_earlier_attempts(self, resumed_incremental_value: int | None, expected_is_resume: bool) -> None:+ pipeline, producer_cls = self._built_pipeline(+ SourceResponse(+ name="orders",+ items=lambda: iter(()),+ primary_keys=["id"],+ sort_mode="asc",+ ),+ resumed_incremental_value,+ )++ # A fresh run replaces the queue rows of earlier attempts and overwrites on batch 0. A run that+ # reads after their rows must not, or the rows it skipped are never loaded.+ assert producer_cls.call_args.kwargs["is_resume"] is expected_is_resume+ assert pipeline._handoff_checkpoint is not None+ assert pipeline._handoff_checkpoint.resume_value == resumed_incremental_value++ @pytest.mark.parametrize("sort_mode", [pytest.param(None, id="unordered"), pytest.param("desc", id="descending")])+ def test_a_source_that_does_not_promise_ascending_order_gets_no_checkpoint(+ self, sort_mode: SortMode | None+ ) -> None:+ pipeline, _ = self._built_pipeline(+ SourceResponse(name="orders", items=lambda: iter(()), primary_keys=["id"], sort_mode=sort_mode), None+ )++ assert pipeline._handoff_checkpoint is None++ @staticmethod+ def _built_pipeline(+ resource: SourceResponse, resumed_incremental_value: int | None+ ) -> tuple[PipelineV3, MagicMock]:schema = MagicMock(id="schema-1",source_id="source-1",@@ -2296,8 +2345,6 @@ class TestIncrementalHandoffCheckpoint:incremental_field_type=IncrementalFieldType.Integer,incremental_field_earliest_value=None,)- resource = SourceResponse(name="orders", items=lambda: iter(()), primary_keys=["id"])-with (patch(f"{_PIPELINE}.current_import_attempt", return_value=2),patch(f"{_PIPELINE}.current_workflow_id", return_value="wf-1"),@@ -2323,12 +2370,7 @@ class TestIncrementalHandoffCheckpoint:incremental_checkpoints_allowed=True,resumed_incremental_value=resumed_incremental_value,)-- # A fresh run replaces the queue rows of earlier attempts and overwrites on batch 0. A run that- # reads after their rows must not, or the rows it skipped are never loaded.- assert producer_cls.call_args.kwargs["is_resume"] is expected_is_resume- assert pipeline._handoff_checkpoint is not None- assert pipeline._handoff_checkpoint.resume_value == resumed_incremental_value+ return pipeline, producer_clsclass _ShutdownSwitch:diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/README.md b/products/warehouse_sources/backend/temporal/data_imports/sources/README.mdindex 612e33ea3e07..88dbaba75334 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/README.md+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/README.md@@ -85,9 +85,32 @@ For a source to support incremental syncing, the `SourceResponse` must have prim#### `sort_mode`-Which direction the source scrolls incremental data, either ascending (the default and preferred sort mode) or descending. Majority of APIs will support ascending sorted responses, but if they only support descending then this will need to be set.+The order in which the rows of an incremental or append read arrive. It has three values:-We store both a `db_incremental_field_last_value` and `db_incremental_field_earliest_value` value - these represent that max/min of the data we've read from the source and processed by the pipeline. When using a `descending` sort mode, we recommend that your source also scrolls for any earlier rows than `db_incremental_field_earliest_value` before trying to scroll for more recent rows. A great example of this is the Stripe source.+- `"asc"`: the rows arrive in ascending order of the incremental field, from the first row of the run to the last.+- `"desc"`: the rows arrive in descending order of the incremental field. Use it only when the API cannot return ascending order.+- `None` (the default): the rows have no order, or the source does not know it.++It applies to incremental and append reads only. A full refresh has no incremental field, so the value has no effect there, and a full refresh can read in any order.++The watermark does not depend on the value. For `"asc"` and `None` the pipeline stages the largest incremental value it read after each batch, and the loader saves it when the run completes. A run that fails does not move the watermark. For `"desc"` the pipeline also keeps the earliest value it read, and it stages the watermark at the end of the run. We store both a `db_incremental_field_last_value` and a `db_incremental_field_earliest_value`. With `"desc"`, we recommend that your source also scrolls for rows earlier than `db_incremental_field_earliest_value` before it scrolls for more recent rows. A great example of this is the Stripe source.++`"asc"` is a promise that the pipeline relies on. When a worker shuts down in the middle of an incremental run, the next attempt of an `"asc"` source does not start again from the stored watermark. It reads the source after the last value the earlier attempt queued. If a row with a lower incremental value can arrive after a row with a higher one, that row is never read, and the watermark then moves past it. A false `"asc"` loses rows silently. A source with `None` starts again from the stored watermark, which is always correct.++So set `"asc"` only when the query or the API contract proves the order:++- one query with `ORDER BY <incremental field> ASC`, read as one result stream (the SQL sources do this for a plain incremental read)+- an API request that sends an explicit ascending sort on the incremental field, in one request series for the whole run++These do not prove it, so leave `None`:++- pages ordered by another key, for example keyset pages by primary key or a cursor sorted by an internal id+- one request series per parent, per account or per partition, because the order starts again for each one+- time windows whose rows are not sorted inside the window+- an ordered query whose result is stored and then read back with no order+- an API that seems to sort by default but does not document it++Set the value per run, where the response is built, not once per source: a source with an ordered incremental read and a keyset full refresh says `"asc"` only for the ordered read. Add a test that fails when the incremental query loses its `ORDER BY`. The pipeline also drops the continuation point when it sees a batch out of order, but that check only sees the rows that arrived before the shutdown, so it does not replace the proof.#### `rows_to_sync`diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/airtable/airtable.py b/products/warehouse_sources/backend/temporal/data_imports/sources/airtable/airtable.pyindex 0a1607e56413..f4a67a071d0a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/airtable/airtable.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/airtable/airtable.py@@ -160,5 +160,4 @@ def airtable_source(primary_keys=config.primary_keys,partition_count=1,partition_size=1,- sort_mode="asc",)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/settings.pyindex 1a713a229542..bbe90c757126 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/settings.py@@ -52,7 +52,7 @@ class AlphaVantageEndpointConfig:# a merge can dedupe them.incremental_fields: list[IncrementalField] = field(default_factory=list)# The order rows actually arrive in, which the pipeline's cursor watermark trusts.- sort_mode: SortMode = "asc"+ sort_mode: SortMode | None = "asc"ALPHA_VANTAGE_ENDPOINTS: dict[str, AlphaVantageEndpointConfig] = {@@ -192,6 +192,8 @@ ALPHA_VANTAGE_ENDPOINTS: dict[str, AlphaVantageEndpointConfig] = {primary_keys=["symbol", "url"],partition_key="time_published",incremental_fields=[incremental_field("time_published")],+ # Each symbol is a separate oldest-first request series, so the rows of a run have no single order.+ sort_mode=None,description="Market news articles mentioning each symbol, with article-level and per-ticker sentiment scores. One row per article per symbol. Supports incremental sync.",should_sync_default=False,),diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/tests/test_alpha_vantage.py b/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/tests/test_alpha_vantage.pyindex d881b8837d09..66a92bfc301c 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/tests/test_alpha_vantage.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/alpha_vantage/tests/test_alpha_vantage.py@@ -619,12 +619,12 @@ class TestAlphaVantage:@parameterized.expand([("insider_is_newest_first", "insider_transactions", "desc"),- ("news_is_oldest_first", "news_sentiment", "asc"),+ ("news_restarts_per_symbol", "news_sentiment", None),("full_refresh_default", "time_series_daily", "asc"),])def test_alpha_vantage_source_reports_the_order_rows_arrive_in(- self, _name: str, endpoint: str, expected: str+ self, _name: str, endpoint: str, expected: str | None) -> None:assert alpha_vantage_source("KEY", ["IBM"], endpoint, MagicMock()).sort_mode == expecteddiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/amazon_ads.py b/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/amazon_ads.pyindex 32f87b15d3af..73d947130916 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/amazon_ads.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/amazon_ads.py@@ -416,9 +416,8 @@ def _report_rows(continuerows = client.download_report(url)- # Amazon does not order the file, and the pipeline advances the `date` cursor from- # each batch it writes, so a batch must never carry a date later than the rows- # still to come for this profile.+ # Amazon does not order the file. The sort gives each report an ascending `date`, but+ # the run has no order: the dates start again for each profile in a window.rows.sort(key=lambda row: str(row.get("date") or ""))yield from _batched_report_rows(rows, profile_id)@@ -517,5 +516,4 @@ def amazon_ads_source(partition_mode="datetime" if is_report else None,partition_format="day" if is_report else None,partition_keys=["date"] if is_report else None,- sort_mode="asc",)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/tests/test_amazon_ads.py b/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/tests/test_amazon_ads.pyindex b151035a4129..e8132bed261b 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/tests/test_amazon_ads.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/amazon_ads/tests/test_amazon_ads.py@@ -376,7 +376,7 @@ class TestAmazonAdsSourceResponse:assert result.response is not Noneassert result.response.name == endpointassert result.response.primary_keys == config.primary_keys- assert result.response.sort_mode == "asc"+ assert result.response.sort_mode is Noneif config.report is None:assert result.response.partition_mode is Noneassert result.response.partition_keys is Nonediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/apple_search_ads.py b/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/apple_search_ads.pyindex e7ba23441154..335b58327dcd 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/apple_search_ads.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/apple_search_ads.py@@ -886,9 +886,7 @@ def apple_search_ads_source(token_provider=token_provider,),primary_keys=list(config.primary_keys),- # Reporting windows are walked oldest-first, so `date` only ever moves forward across- # batches by at most one window — which the schema's trailing lookback re-reads.- sort_mode="asc",+ # No `sort_mode`: each campaign restarts at the start of its window, so `date` has no order.partition_count=1,partition_size=1,partition_mode="datetime" if config.partition_key else None,diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/tests/test_apple_search_ads.py b/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/tests/test_apple_search_ads.pyindex 3e77c379e84d..173876438670 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/tests/test_apple_search_ads.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/apple_search_ads/tests/test_apple_search_ads.py@@ -799,8 +799,7 @@ class TestSourceResponse:assert response is not Noneassert response.name == endpointassert response.primary_keys == config.primary_keys- # Windows are walked oldest-first, so the watermark only ever moves forward.- assert response.sort_mode == "asc"+ assert response.sort_mode is Noneif config.partition_key is None:assert response.partition_mode is Noneassert response.partition_keys is Nonediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/appsflyer/appsflyer.py b/products/warehouse_sources/backend/temporal/data_imports/sources/appsflyer/appsflyer.pyindex 68f6abf81945..075e4d35f082 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/appsflyer/appsflyer.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/appsflyer/appsflyer.py@@ -410,7 +410,6 @@ def appsflyer_source(partition_mode="datetime",partition_format=config.partition_format,partition_keys=[config.partition_key],- sort_mode="asc",# Dimension keys can collide (e.g. blank campaign values) — an expected trait of report# data, not something the user can fix. Don't set has_duplicate_primary_keys: that flag# tells validate_incremental_sync to block incremental syncing altogether. The merge's owndiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/bigquery/bigquery.py b/products/warehouse_sources/backend/temporal/data_imports/sources/bigquery/bigquery.pyindex c8f8829fa3cf..87960dabb175 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/bigquery/bigquery.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/bigquery/bigquery.py@@ -2062,4 +2062,6 @@ class BigQueryImplementation(SQLSourceImplementation[BigQuerySourceConfig, bigqupartition_size=partition_settings.partition_size if partition_settings else None,rows_to_sync=rows_to_sync,has_duplicate_primary_keys=has_duplicate_keys,+ # The ordered query writes a temporary table, and the read of a table has no order.+ sort_mode=None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/bol_retailer/bol_retailer.py b/products/warehouse_sources/backend/temporal/data_imports/sources/bol_retailer/bol_retailer.pyindex ea2e0a2dfe50..1d973f71f9ca 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/bol_retailer/bol_retailer.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/bol_retailer/bol_retailer.py@@ -281,8 +281,7 @@ def bol_retailer_source(window_start=first_day.isoformat(),row_map=flatten_order_items,),- # Days are requested in order, and the next run re-reads the watermark's whole day.- sort_mode="asc",+ # No sort_mode: the days come in order, but the rows inside a day have no documented order.**response_kwargs,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/breezometer.py b/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/breezometer.pyindex ede638999eb4..a105a7251e4b 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/breezometer.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/breezometer.py@@ -338,6 +338,4 @@ def breezometer_source(partition_mode="datetime",partition_format="week",partition_keys=[config.partition_key],- # Forecast/history rows arrive in ascending time order; single-snapshot endpoints have one row.- sort_mode="asc",)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/tests/test_breezometer.py b/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/tests/test_breezometer.pyindex c5f699ba5a13..87c2496924a1 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/tests/test_breezometer.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/breezometer/tests/test_breezometer.py@@ -353,7 +353,7 @@ class TestBreezometerSource:assert response.primary_keys == ["latitude", "longitude", "dt_iso"]assert response.partition_mode == "datetime"assert response.partition_keys == ["dt_iso"]- assert response.sort_mode == "asc"+ assert response.sort_mode is Nonedef test_invalid_locations_raise(self):with pytest.raises(ValueError):diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cast_ai/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cast_ai/settings.pyindex f46ce94bf50f..344dfc1c27c2 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cast_ai/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cast_ai/settings.py@@ -47,7 +47,8 @@ class CastAiEndpointConfig:# Unused: no endpoint documents a page size limit, so callers always pass page_size_param=None# and let the server pick. Kept only to satisfy the shared fan-out helper's protocol.page_size: int = 0- sort_mode: Literal["asc", "desc"] = "asc"+ # The incremental reports fan out per cluster, so the rows of a run have no single order.+ sort_mode: Literal["asc", "desc"] | None = Noneprimary_key: str | list[str] = "id"fanout: DependentEndpointConfig | None = None# Required report window params, as (start, end).diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/cimis.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/cimis.pyindex 665810fb84f3..d22247f059c9 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/cimis.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/cimis.py@@ -194,8 +194,9 @@ def _get_data_rows(session = make_tracked_session(headers=_headers(), redact_values=(app_key,))rpd = _records_per_day(config.scope)- # Keep the date window as the outer loop so rows are emitted in ascending Date order even when the- # target set is large enough to require multiple requests per window (sort_mode="asc" relies on it).+ # Keep the date window as the outer loop so the windows are emitted in ascending Date order even when+ # the target set is large enough to require multiple requests per window. Rows inside a window have+ # no documented order.if len(targets) * rpd <= CIMIS_RECORD_CAP:window_days = max(1, CIMIS_RECORD_CAP // (len(targets) * rpd))target_batches = [targets]@@ -304,5 +305,4 @@ def cimis_source(partition_mode="datetime" if config.partition_key else None,partition_format="month" if config.partition_key else None,partition_keys=[config.partition_key] if config.partition_key else None,- sort_mode="asc",)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/tests/test_cimis.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/tests/test_cimis.pyindex a1eebb180987..ba6b86e8b727 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/tests/test_cimis.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cimis/tests/test_cimis.py@@ -219,7 +219,7 @@ class TestCimisSourceResponse:assert response.name == endpointassert response.primary_keys == primary_keysassert response.partition_mode == partition_mode- assert response.sort_mode == "asc"+ assert response.sort_mode is Noneif partition_mode == "datetime":assert response.partition_keys == [CIMIS_ENDPOINTS[endpoint].partition_key]assert response.partition_format == "month"diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/clickhouse.py b/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/clickhouse.pyindex d17b395adf17..32095f6fdfeb 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/clickhouse.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/clickhouse.py@@ -1303,6 +1303,24 @@ def _needs_to_string_cast(inner: str) -> bool:return any(inner.startswith(prefix) for prefix in _ARROW_UNSUPPORTED_PREFIXES)+def _reads_in_incremental_order(+ columns: list[ClickHouseColumn], should_use_incremental_field: bool, incremental_field: str | None+) -> bool:+ """Whether the read returns its rows in ascending order of the incremental value.++ The query has `ORDER BY <incremental field>`, and a paged read walks ascending ranges of that+ field. A column that the SELECT list casts with `toString()` has an alias with the column's+ name, so the server can order by the text and not by the number.+ """+ if not should_use_incremental_field or incremental_field is None:+ return False+ column = next((column for column in columns if column.name == incremental_field), None)+ if column is None:+ return False+ inner, _ = _strip_type_modifiers(column.data_type)+ return not _needs_to_string_cast(inner)++def _build_select_list(columns: list[ClickHouseColumn]) -> str:"""Build explicit SELECT list, wrapping Arrow-unsupported types in toString()."""parts: list[str] = []@@ -1905,4 +1923,9 @@ def clickhouse_source(partition_size=partition_settings.partition_size if partition_settings else None,rows_to_sync=rows_to_sync,has_duplicate_primary_keys=has_duplicate_primary_keys,+ sort_mode=(+ "asc"+ if _reads_in_incremental_order(list(table.columns), should_use_incremental_field, incremental_field)+ else None+ ),)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/test_clickhouse.py b/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/test_clickhouse.pyindex 72d6360c9053..e82edd5b9fc1 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/test_clickhouse.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/clickhouse/test_clickhouse.py@@ -36,6 +36,7 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.clickhouse_parse_mv_target,_project_columns,_quote_identifier,+ _reads_in_incremental_order,filter_clickhouse_incremental_fields,get_primary_keys_for_schemas,)@@ -176,6 +177,54 @@ class TestBuildQuery:assert "toString(`payload`) AS `payload`" in queryassert "toString(`value`) AS `value`" in query+ @pytest.mark.parametrize(+ "should_use_incremental_field,page_conditions,expected_tail",+ [+ pytest.param(True, (), " ORDER BY `created_at` ASC", id="incremental_single_query"),+ pytest.param(+ True, ("(`created_at`) > (%(page_lower_0)s)",), " ORDER BY `created_at` ASC", id="incremental_paged"+ ),+ pytest.param(False, (), None, id="full_refresh"),+ ],+ )+ def test_orders_only_an_incremental_query_and_ends_with_the_order(+ self, should_use_incremental_field: bool, page_conditions: tuple[str, ...], expected_tail: str | None+ ) -> None:+ query, _ = _build_query(+ database="default",+ table_name="events",+ columns=self._cols(("id", "Int64"), ("created_at", "DateTime64(6)")),+ should_use_incremental_field=should_use_incremental_field,+ incremental_field="created_at" if should_use_incremental_field else None,+ incremental_field_type=IncrementalFieldType.DateTime if should_use_incremental_field else None,+ page_conditions=page_conditions,+ )+ if expected_tail is None:+ assert "ORDER BY" not in query+ else:+ assert query.endswith(expected_tail)++ @pytest.mark.parametrize(+ "data_type,should_use_incremental_field,incremental_field,expected",+ [+ ("DateTime64(6)", True, "created_at", True),+ ("Date", True, "created_at", True),+ ("Nullable(Int64)", True, "created_at", True),+ ("LowCardinality(Nullable(UInt32))", True, "created_at", True),+ # The SELECT list casts these with toString(), and the alias can make the order textual.+ ("Int128", True, "created_at", False),+ ("UInt256", True, "created_at", False),+ ("Nullable(Int256)", True, "created_at", False),+ ("DateTime64(6)", False, "created_at", False),+ ("DateTime64(6)", True, "missing", False),+ ],+ )+ def test_reads_in_incremental_order(+ self, data_type: str, should_use_incremental_field: bool, incremental_field: str, expected: bool+ ) -> None:+ columns = self._cols(("id", "Int64"), ("created_at", data_type))+ assert _reads_in_incremental_order(columns, should_use_incremental_field, incremental_field) is expected+class TestProjectColumns:@staticmethoddiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/cloudflare.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/cloudflare.pyindex 4fb3d5949627..edd0e5f95e2d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/cloudflare.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/cloudflare.py@@ -472,5 +472,4 @@ def cloudflare_source(primary_keys=list(config.primary_keys),partition_count=1,partition_size=1,- sort_mode="asc",)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/settings.pyindex a9dfc9a09977..fa0f5bf8044f 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/settings.py@@ -48,7 +48,7 @@ class CloudflareEndpointConfig:# Most Cloudflare v4 REST lists are small configuration tables with no updated-since# filter, so they sync as full refreshes. `audit_logs` is the exception — it takes a-# server-side `since` filter and can be sorted ascending, so it syncs incrementally.+# server-side `since` filter, so it syncs incrementally.# The high-volume traffic datasets live in the separate GraphQL Analytics API, which# this REST source has no transport for.CLOUDFLARE_ENDPOINTS: dict[str, CloudflareEndpointConfig] = {@@ -235,8 +235,8 @@ CLOUDFLARE_ENDPOINTS: dict[str, CloudflareEndpointConfig] = {path="/accounts/{account_id}/audit_logs",parent=ACCOUNTS_PARENT,parent_key="_account_id",- # `since` filters server-side and `direction=asc` makes the response order- # match SourceResponse's ascending sort mode.+ # `since` filters server-side. `direction=asc` orders the rows of one account only.+ # The read has one request series per account, so the run has no single order.params={"direction": "asc"},incremental_param="since",# This list has no `result_info.total_pages`, so the paginator only learns it hasdiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/tests/test_cloudflare_client.py b/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/tests/test_cloudflare_client.pyindex 0efe8e0c722d..c8069ad7039e 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/tests/test_cloudflare_client.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/cloudflare/tests/test_cloudflare_client.py@@ -214,7 +214,7 @@ class TestCloudflareSourceResponse:assert response is not Noneassert response.name == endpointassert response.primary_keys == list(config.primary_keys)- assert response.sort_mode == "asc"+ assert response.sort_mode is Noneassert response.partition_mode is Noneassert response.partition_keys is Nonediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/common/typings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/common/typings.pyindex ca0806aa1a43..2bc717ee9a34 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/common/typings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/common/typings.py@@ -86,8 +86,18 @@ class SourceResponse:"""Override partition mode at a source level"""partition_format: Optional[PartitionFormat] = None"""Override partition format at a source level"""- sort_mode: Optional[SortMode] = "asc"- """our source typically return data in ascending timestamp order, but some (eg Stripe) do not"""+ sort_mode: Optional[SortMode] = None+ """The order in which the rows of an incremental or append read arrive.++ - `"asc"`: ascending order of the incremental field, from the first row of the run to the last.+ - `"desc"`: descending order of the incremental field.+ - `None`: no order.++ The watermark is the largest value the run read, with or without an order. `"asc"` is a promise+ the pipeline acts on: after a worker shutdown, the next attempt reads only the rows after the+ last value the pipeline queued, so a later row with a lower value is lost. Set it per run, and+ only on a read whose query or API contract proves the order. A full refresh has no incremental+ field, so the value has no effect there."""rows_to_sync: Optional[int] = Nonehas_duplicate_primary_keys: Optional[bool] = None"""Whether incremental tables have non-unique primary keys"""diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/crates_io/crates_io.py b/products/warehouse_sources/backend/temporal/data_imports/sources/crates_io/crates_io.pyindex 89e740b1af37..15c7aa323e31 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/crates_io/crates_io.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/crates_io/crates_io.py@@ -586,6 +586,6 @@ def crates_io_source(items=lambda: get_rows(endpoint=endpoint, crates=crates, logger=logger),primary_keys=config.primary_keys,# No server-side ordering to rely on; rows are grouped per crate as fetched.- sort_mode="asc",+ sort_mode=None,**partition_kwargs,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/custom/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/custom/source.pyindex f97755cf095e..aa3cd535c404 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/custom/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/custom/source.py@@ -228,9 +228,8 @@ class _ManifestResource(BaseModel):name: str = Field(min_length=1)endpoint: _ManifestEndpoint# How the upstream API orders this resource's rows. Incremental syncs commit- # the high-watermark per batch for "asc"; for a source whose order is- # unknown or descending, declaring "desc" here avoids skipping rows on a- # resumed sync. Defaults to "asc" when omitted.+ # the high-watermark per batch for "asc", so declare "asc" only when the API+ # guarantees that order. When omitted, the rows have no declared order.sort_mode: Literal["asc", "desc"] | None = None@@ -1373,9 +1372,9 @@ class CustomSource(SimpleSource[CustomSourceConfig]):else:primary_keys = None- # The manifest declares the upstream's row ordering; default "asc" to- # match PostHog's other REST sources. An incorrect "asc" assumption on a- # non-ascending API can skip rows on a resumed incremental sync.+ # The manifest declares the upstream's row ordering. A manifest with no+ # value gets no order: an incorrect "asc" on a non-ascending API can skip+ # rows on a resumed incremental sync.## "desc" only defers committing the high-watermark until the run finishes# (so a partial run can't advance the cursor past rows it never reached on@@ -1392,7 +1391,14 @@ class CustomSource(SimpleSource[CustomSourceConfig]):# rows arrive grouped per parent, never globally cursor-ascending, so an# "asc" per-batch commit after an interruption would set the watermark past# later parents' older rows and permanently skip them.- sort_mode: SortMode = "desc" if (chain.is_fanout_child or chosen.get("sort_mode") == "desc") else "asc"+ declared_sort_mode = chosen.get("sort_mode")+ sort_mode: SortMode | None+ if chain.is_fanout_child or declared_sort_mode == "desc":+ sort_mode = "desc"+ elif declared_sort_mode == "asc":+ sort_mode = "asc"+ else:+ sort_mode = Nonereturn SourceResponse(name=inputs.schema_name,diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/custom/test_custom_source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/custom/test_custom_source.pyindex 49f58b1297c8..c5e33d601895 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/custom/test_custom_source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/custom/test_custom_source.py@@ -1695,7 +1695,7 @@ class TestCustomSourceSourceForPipeline(SimpleTestCase):assert client_config["base_url"] == manifest["client"]["base_url"]@parameterized.expand(- [("default_asc", None, "asc"), ("explicit_asc", "asc", "asc"), ("explicit_desc", "desc", "desc")]+ [("default_unordered", None, None), ("explicit_asc", "asc", "asc"), ("explicit_desc", "desc", "desc")])@patch("products.warehouse_sources.backend.temporal.data_imports.sources.custom.source.rest_api_resources")def test_sort_mode_threaded_to_source_response(self, _name, declared, expected, mock_resources):@@ -2295,8 +2295,8 @@ class TestCustomSourceFanoutPipeline(SimpleTestCase):# the override that matters: per-batch asc commits on a child would# advance the watermark past later parents' older rows and skip them.("child_overrides_declared_asc", "responses", "asc", "desc"),- # Top-level resource keeps its declaration (default "asc").- ("parent_keeps_declared", "forms", None, "asc"),+ # Top-level resource keeps its declaration (no order when omitted).+ ("parent_keeps_declared", "forms", None, None),])@patch("products.warehouse_sources.backend.temporal.data_imports.sources.custom.source.rest_api_resources")diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/databricks.py b/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/databricks.pyindex 102cf7af6286..bad02e190319 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/databricks.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/databricks.py@@ -563,4 +563,6 @@ class DatabricksImplementation(SQLSourceImplementation[DatabricksSourceConfig, Apartition_count=partition_settings.partition_count if partition_settings else None,partition_size=partition_settings.partition_size if partition_settings else None,rows_to_sync=rows_to_sync,+ # One statement with `ORDER BY <incremental field>`, read as one stream.+ sort_mode="asc" if incremental_field is not None else None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/tests/test_databricks.py b/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/tests/test_databricks.pyindex 91de8f3d4f13..4800dd612a5d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/tests/test_databricks.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/databricks/tests/test_databricks.py@@ -300,6 +300,7 @@ class TestBuildPipeline:# Full refresh rewrites the whole table — the partition probes must be skipped.assert response.partition_count is Noneassert response.partition_size is None+ assert response.sort_mode is None# The stream must terminate on the first empty arrow batch — not loop forever.assert list(response.items()) == [batch]@@ -327,11 +328,12 @@ class TestBuildPipeline:# Table stats feed the shared partition math so incremental merges get md5 partitioning.assert response.partition_count is not Noneassert response.partition_size is not None+ assert response.sort_mode == "asc"list(response.items())sql, params = streaming_cursor.execute.call_args.argsassert "WHERE `updated_at` > :incremental_value" in sql- assert "ORDER BY `updated_at` ASC" in sql+ assert sql.endswith("ORDER BY `updated_at` ASC")assert params == {"incremental_value": "2025-01-01T00:00:00"}diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/fillout.py b/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/fillout.pyindex 685b18d1d622..74b1b6fcb507 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/fillout.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/fillout.py@@ -102,7 +102,7 @@ def _rest_api_client_config(base_api_url: str, api_key: str) -> ClientConfig:class FilloutSubmissionsPaginator(OffsetPaginator):"""Limit/offset paginator for `/forms/{formId}/submissions`.- Pins `sort=asc`, oldest-first, matching the ascending incremental watermark. No `status`+ Pins `sort=asc`, oldest-first within one form. No `status`param: `finished` is already Fillout's default, so sending it only added a way for therequest to be rejected. `totalResponses` reflects the `afterDate`-filtered count, so thewalk stops at the watermark on incremental syncs rather than re-reading each form's fulldiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/settings.pyindex c19072650290..02ba97e0f14e 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/settings.py@@ -31,7 +31,7 @@ class FilloutEndpointConfig:default_incremental_field: str | None = Nonepartition_key: str | None = Nonepage_size: int = SUBMISSIONS_PAGE_SIZE- sort_mode: Literal["asc", "desc"] = "asc"+ sort_mode: Literal["asc", "desc"] | None = "asc"primary_key: str | list[str] = "id"fanout: DependentEndpointConfig | None = None# JSONPath to the rows in the response. Fillout returns bare objects/arrays on every@@ -87,9 +87,9 @@ FILLOUT_ENDPOINTS: dict[str, FilloutEndpointConfig] = {page_size=SUBMISSIONS_PAGE_SIZE,data_selector="responses",paginated=True,- # We request `sort=asc`, so rows arrive oldest-first and the pipeline's ascending- # incremental watermark bookkeeping is correct.- sort_mode="asc",+ # `sort=asc` orders the rows of one form only. The run reads one form after another, so+ # `submissionTime` starts again for each form and the run has no order.+ sort_mode=None,fanout=DependentEndpointConfig(parent_name="forms",resolve_param="form_id",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/tests/test_fillout.py b/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/tests/test_fillout.pyindex c1e3ec4d32c2..1334fea6d1ff 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/tests/test_fillout.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/fillout/tests/test_fillout.py@@ -221,7 +221,7 @@ class TestFilloutTransport:assert result.response.primary_keys == expected_primary_keysassert result.response.partition_mode == expected_partition_modeassert result.response.partition_keys == expected_partition_keys- assert result.response.sort_mode == "asc"+ assert result.response.sort_mode == (None if endpoint == "submissions" else "asc")@parameterized.expand([diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/finnhub.py b/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/finnhub.pyindex a4855e114b12..93d85fe8a4bf 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/finnhub.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/finnhub.py@@ -205,10 +205,9 @@ def _emit(rows: list[dict[str, Any]], symbol: str | None, config: FinnhubEndpoinfor row in rows:row[config.symbol_field] = symbolif config.incremental_fields:- # Guarantee ascending order so the declared `sort_mode="asc"` matches the data the- # incremental watermark is checkpointed against, regardless of the API's response order.- # The watermark field is required, so a row missing it is a hard error rather than a- # silent epoch-0 sort that would corrupt the checkpoint.+ # Sort the rows of one request, regardless of the API's response order. The watermark+ # field is required, so a row missing it is a hard error rather than a silent epoch-0+ # sort.watermark = config.incremental_fields[0]["field"]rows.sort(key=lambda r: r[watermark])return rows@@ -298,9 +297,7 @@ def finnhub_source(partition_mode="datetime" if config.partition_key else None,partition_format="month" if config.partition_key else None,partition_keys=[config.partition_key] if config.partition_key else None,- # Only the incremental endpoints get a sorted data contract — full-refresh endpoints- # emit in whatever order the API returns, so don't promise ascending order there.- sort_mode="asc" if config.incremental_fields else None,+ # No sort_mode: each ticker is one request, so the row order starts again for each ticker.)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/tests/test_finnhub.py b/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/tests/test_finnhub.pyindex 5f727f68a9bd..589630d3ad9e 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/tests/test_finnhub.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/finnhub/tests/test_finnhub.py@@ -218,13 +218,12 @@ class TestFinnhubSource:("country", ["code2"], None, None),("company_profile", ["symbol"], None, None),("quote", ["symbol"], None, None),- # Only the incremental endpoint advertises a sorted data contract.- ("company_news", ["id", "symbol"], None, "asc"),+ ("company_news", ["id", "symbol"], None, None),("recommendation_trends", ["symbol", "period"], "period", None),("earnings_surprises", ["symbol", "period"], "period", None),- ("financials_reported", ["symbol", "accessNumber"], "endDate", "asc"),- ("stock_candles", ["symbol", "t"], "t", "asc"),- ("sec_filings", ["symbol", "accessNumber"], "filedDate", "asc"),+ ("financials_reported", ["symbol", "accessNumber"], "endDate", None),+ ("stock_candles", ["symbol", "t"], "t", None),+ ("sec_filings", ["symbol", "accessNumber"], "filedDate", None),("insider_transactions",[@@ -237,9 +236,9 @@ class TestFinnhubSource:"transactionPrice",],"transactionDate",- "asc",+ None,),- ("dividends", ["symbol", "date", "amount"], "date", "asc"),+ ("dividends", ["symbol", "date", "amount"], "date", None),("peers", ["symbol", "peer"], None, None),("index_constituents", ["index_symbol", "symbol"], None, None),("economic_calendar", ["country", "event", "time"], "time", None),diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/flutterwave/tests/test_flutterwave.py b/products/warehouse_sources/backend/temporal/data_imports/sources/flutterwave/tests/test_flutterwave.pyindex 5acae1737662..52e588b5e7ce 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/flutterwave/tests/test_flutterwave.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/flutterwave/tests/test_flutterwave.py@@ -211,9 +211,8 @@ class TestSourceResponseMetadata:assert result.response is not Noneassert result.response.name == endpointassert result.response.primary_keys == ["id"]- # Every endpoint is full-refresh, so the pipeline never consults sort_mode; it stays the- # framework default.- assert result.response.sort_mode == "asc"+ # Every endpoint is full-refresh, so the pipeline never consults sort_mode; it is None.+ assert result.response.sort_mode is Noneassert result.response.partition_mode == "datetime"assert result.response.partition_keys == ["created_at"]diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/gitguardian/tests/test_gitguardian.py b/products/warehouse_sources/backend/temporal/data_imports/sources/gitguardian/tests/test_gitguardian.pyindex fc1df6b0245a..d11134c70318 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/gitguardian/tests/test_gitguardian.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/gitguardian/tests/test_gitguardian.py@@ -95,6 +95,7 @@ class TestLinkHeaderPagination:)assert result.raised is Noneassert result.params("date_after") == [expected]+ assert result.params("ordering") == ["date"]@parameterized.expand([diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/google_ads/tests/test_google_ads_source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/google_ads/tests/test_google_ads_source.pyindex 266ec8787036..af0f4cbdb05c 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/google_ads/tests/test_google_ads_source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/google_ads/tests/test_google_ads_source.py@@ -1661,7 +1661,7 @@ class TestGoogleAdsQueryConstruction:"ORDER BY segments.date ASC")assert all("2100-01-01" not in q for q in queries)- assert response.sort_mode == "asc"+ assert response.sort_mode is Nonedef test_lookback_overlap_cannot_consume_a_whole_run(self):# Spending the whole budget on lookback overlap leaves the cursor unmoved, so the next rundiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/google_pagespeed_insights.py b/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/google_pagespeed_insights.pyindex dc2ec0655b43..3bcb1f973ac0 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/google_pagespeed_insights.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/google_pagespeed_insights.py@@ -303,8 +303,6 @@ def google_pagespeed_insights_source(partition_mode="datetime",partition_format="week",partition_keys=[config.partition_key],- # One row per URL, each stamped with an analysis timestamp of ~now; rows arrive in config order.- sort_mode="asc",# PageSpeed responses are large, deeply-nested JSON documents (~0.5-1 MiB each, larger for# complex pages). We emit one row per URL, so keep the per-chunk byte budget below a single# report: the batcher then flushes after roughly every report instead of accumulating the wholediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/tests/test_google_pagespeed_insights.py b/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/tests/test_google_pagespeed_insights.pyindex 4b685d383702..0f0b871adf83 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/tests/test_google_pagespeed_insights.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/google_pagespeed_insights/tests/test_google_pagespeed_insights.py@@ -312,7 +312,7 @@ class TestSource:assert response.primary_keys == ["requested_url", "analysis_timestamp"]assert response.partition_mode == "datetime"assert response.partition_keys == ["analysis_timestamp"]- assert response.sort_mode == "asc"+ assert response.sort_mode is Nonedef test_invalid_urls_raise(self):with pytest.raises(ValueError):diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/harvey/tests/test_harvey.py b/products/warehouse_sources/backend/temporal/data_imports/sources/harvey/tests/test_harvey.pyindex d62a70b5e8f2..a0d0b1ea274d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/harvey/tests/test_harvey.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/harvey/tests/test_harvey.py@@ -635,7 +635,7 @@ class TestHarveySourceResponse:assert response.primary_keys == primary_keysassert response.partition_keys == partition_keysassert response.partition_mode == ("datetime" if partition_keys else None)- assert response.sort_mode == "asc"+ assert response.sort_mode is Nonedef test_unknown_endpoint_raises(self) -> None:with mock.patch(f"{HARVEY_MODULE}.make_tracked_session", return_value=mock.MagicMock()):diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/settings.pyindex 72cac1f5d429..6f3c9fa94e69 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/settings.py@@ -28,7 +28,7 @@ class IntercomEndpointConfig:page_size: int = 150# Where page-number endpoints report their page count: top level, or nested under `pages`.total_pages_path: str = "total_pages"- sort_mode: SortMode = "asc"+ sort_mode: SortMode | None = "asc"extra_params: dict[str, str] = field(default_factory=dict)# Query-string param Intercom expects for the cursor filter (e.g. "created_at_after"# on /admins/activity_logs). Only set on endpoints that filter via query params.@@ -183,6 +183,8 @@ INTERCOM_ENDPOINTS: dict[str, IntercomEndpointConfig] = {paginator_kind="substream",parent_endpoint="conversations",partition_key="created_at",+ # The parts of each conversation are a separate series, so the rows of a run have no single order.+ sort_mode=None,),"help_centers": IntercomEndpointConfig(# `GET /help_center/help_centers` — one row per Help Center site. A workspacediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/tests/test_intercom.py b/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/tests/test_intercom.pyindex e6016fd5a641..857d664bd28a 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/tests/test_intercom.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/intercom/tests/test_intercom.py@@ -567,6 +567,7 @@ class TestResumableRestEndpoints:assert result.raised is Noneassert "starting_after" not in result.requests[0].json()["pagination"]assert result.requests[0].json()["query"]["value"] == 1700000000+ assert result.requests[0].json()["sort"] == {"field": "updated_at", "order": "ascending"}@pytest.mark.parametrize("endpoint,selector,state,expected_url,expected_params",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/iterable/iterable.py b/products/warehouse_sources/backend/temporal/data_imports/sources/iterable/iterable.pyindex 02e3a103cd30..50575c2f88d3 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/iterable/iterable.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/iterable/iterable.py@@ -332,7 +332,7 @@ def _export_source_response(partition_mode="datetime" if partition_key else None,partition_format="month" if partition_key else None,partition_keys=[partition_key] if partition_key else None,- sort_mode="asc",+ # No `sort_mode`: the export sends no sort, so the rows in a window have no order.)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/jfrog_artifactory/tests/test_jfrog_artifactory.py b/products/warehouse_sources/backend/temporal/data_imports/sources/jfrog_artifactory/tests/test_jfrog_artifactory.pyindex cb76b44bf607..2dc5d34860f1 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/jfrog_artifactory/tests/test_jfrog_artifactory.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/jfrog_artifactory/tests/test_jfrog_artifactory.py@@ -504,7 +504,7 @@ class TestSourceResponse:)assert response.name == endpointassert response.primary_keys == primary_keys- assert response.sort_mode == "asc"+ assert response.sort_mode is Noneif partition_key is None:assert response.partition_mode is Noneelse:diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/lightdash.py b/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/lightdash.pyindex 53ed74579f56..327cb6d15b86 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/lightdash.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/lightdash.py@@ -176,8 +176,9 @@ def _make_source_response(config: LightdashEndpointConfig, items_fn: Callable[[]items=items_fn,primary_keys=config.primary_key if isinstance(config.primary_key, list) else [config.primary_key],# Lightdash exposes no server-side updated-since filter on any endpoint; rows arrive in the- # endpoint's natural order or the ascending sort set in its params.- sort_mode="asc",+ # endpoint's natural order or the ascending sort set in its params. A fan-out sorts each+ # parent's rows on their own, so the rows of the whole run have no order.+ sort_mode=None if config.fanout is not None else "asc",partition_count=1,partition_size=1,partition_mode="datetime" if config.partition_key else None,diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/tests/test_lightdash.py b/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/tests/test_lightdash.pyindex 0ee9741ec00c..c9b5e38bd6a4 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/tests/test_lightdash.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/lightdash/tests/test_lightdash.py@@ -126,15 +126,15 @@ class TestGetResource:class TestLightdashSourceResponse:@parameterized.expand([- ("projects", ["projectUuid"], "createdAt"),- ("spaces", ["uuid"], None),- ("dashboards", ["uuid"], None),- ("charts", ["uuid"], None),- ("metrics_catalog", ["catalogSearchUuid"], None),- ("org_users", ["userUuid"], "userCreatedAt"),- ("explores", ["projectUuid", "name"], None),- ("schedulers", ["schedulerUuid"], "createdAt"),- ("scheduler_runs", ["runId"], "scheduledTime"),+ ("projects", ["projectUuid"], "createdAt", "asc"),+ ("spaces", ["uuid"], None, None),+ ("dashboards", ["uuid"], None, None),+ ("charts", ["uuid"], None, None),+ ("metrics_catalog", ["catalogSearchUuid"], None, None),+ ("org_users", ["userUuid"], "userCreatedAt", "asc"),+ ("explores", ["projectUuid", "name"], None, None),+ ("schedulers", ["schedulerUuid"], "createdAt", None),+ ("scheduler_runs", ["runId"], "scheduledTime", None),])@mock.patch(@@ -148,6 +148,7 @@ class TestLightdashSourceResponse:endpoint: str,primary_keys: list[str],partition_key: str | None,+ sort_mode: str | None,mock_rest_api_resources: mock.MagicMock,mock_rest_api_resource: mock.MagicMock,) -> None:@@ -163,6 +164,7 @@ class TestLightdashSourceResponse:assert response.name == endpointassert response.primary_keys == primary_keys+ assert response.sort_mode == sort_modeif partition_key:assert response.partition_keys == [partition_key]assert response.partition_mode == "datetime"diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/mongo.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/mongo.pyindex 0aa33ebe2c0d..dca7efba3908 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/mongo.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/mongo.py@@ -1065,4 +1065,7 @@ def mongo_source(chunk_size_bytes=MONGO_CHUNK_SIZE_BYTES,supports_resume=resume_manager is not None,on_complete=resume_manager.clear_state if resume_manager is not None else None,+ # The read is in `_id` order, because the cursor reopens after the last `_id`. A row with a+ # lower incremental value can thus follow a higher one.+ sort_mode=None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/test_mongo.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/test_mongo.pyindex b2a15167d61b..f7ca9676760c 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/test_mongo.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mongodb/test_mongo.py@@ -1125,6 +1125,7 @@ class TestMongoSourceResume(SimpleTestCase):assert len(rows) == 1assert response.supports_resume is False+ assert response.sort_mode is Noneassert "$or" not in collection.find_queries[0]assert (manager.load_calls, manager.staged, manager.clear_calls) == (0, [], 0)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/motherduck.py b/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/motherduck.pyindex f7c2d4bff5ad..d949f5945231 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/motherduck.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/motherduck.py@@ -618,4 +618,6 @@ class MotherDuckImplementation(SQLSourceImplementation[MotherduckSourceConfig, Aitems=get_rows,primary_keys=primary_keys,rows_to_sync=rows_to_sync,+ # One statement with `ORDER BY <incremental field>`, read as one stream.+ sort_mode="asc" if incremental_field is not None else None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/tests/test_motherduck.py b/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/tests/test_motherduck.pyindex e59dce8b73cb..4ca360390886 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/tests/test_motherduck.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/motherduck/tests/test_motherduck.py@@ -260,6 +260,33 @@ class TestMotherDuck:assert metadata_conn.execute.call_args_list[0].args[0] == 'USE "warehouse"'assert 'FROM "sales"."users"' in streaming_conn.execute.call_args.args[0]+ @pytest.mark.parametrize(+ ("incremental", "expected_order_by"),+ [(True, 'ORDER BY "updated_at" ASC'), (False, None)],+ ids=["incremental", "full_refresh"],+ )+ def test_build_pipeline_claims_ascending_order_only_for_the_incremental_read(+ self, impl, incremental, expected_order_by+ ):+ metadata_conn, streaming_conn = self._pipeline_mocks([(["id"],)], 1, [pa.RecordBatch.from_pydict({"id": [1]})])+ inputs = _make_inputs(+ should_use_incremental_field=incremental,+ incremental_field="updated_at" if incremental else None,+ incremental_field_type=IncrementalFieldType.Timestamp if incremental else None,+ db_incremental_field_last_value="2025-01-01T00:00:00" if incremental else None,+ )++ with patch(_CONNECT_PATH, side_effect=[metadata_conn, streaming_conn]):+ response = impl.build_pipeline(_make_config(), inputs)+ list(response.items())++ sql = streaming_conn.execute.call_args.args[0]+ if expected_order_by:+ assert sql.endswith(expected_order_by)+ else:+ assert "ORDER BY" not in sql+ assert response.sort_mode == ("asc" if incremental else None)+def test_build_pipeline_rejects_an_unresolvable_catalog(self, impl):# Account-wide with nothing naming a catalog: `USE` would be a guess, and the sync would# silently read whichever database DuckDB happens to resolve to.diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/mssql.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/mssql.pyindex 8313e181d17d..e912006b8de3 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/mssql.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/mssql.py@@ -1543,4 +1543,6 @@ class MSSQLImplementation(SQLSourceImplementation[MSSQLSourceConfig, pymssql.Con# `supports_resume` defaults to True. A read on the single query has no position to# hand to another worker, so it must say so.supports_resume=takes_keyset_path or resumes_incremental,+ # The single query has `ORDER BY <incremental field>`. The keyset walk orders by its key.+ sort_mode=("asc" if not takes_keyset_path and should_use_incremental_field and incremental_field else None),)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/tests/test_mssql_resume.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/tests/test_mssql_resume.pyindex 30c1b4e72db3..0eecbc7762de 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/tests/test_mssql_resume.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mssql/tests/test_mssql_resume.py@@ -232,6 +232,13 @@ _INCREMENTAL_INPUTS: dict[str, Any] = {"db_incremental_field_last_value": _INCREMENTAL_START,"sync_type": ExternalDataSchemaSyncType.INCREMENTAL,}+# A usable unique index, so only the incremental sync keeps the read off the keyset walk.+_INCREMENTAL_TABLE = (+ _INCREMENTAL_COLUMNS,+ ["id"],+ [_index([("id", "int", False)], clustered=True, primary=True)],+ _INCREMENTAL_ROWS,+)@pytest.fixture@@ -399,6 +406,7 @@ class TestIncrementalResume:def test_interrupted_read_resumes_from_the_last_written_value(self, serve, stop_after, write_last):server = serve(_INCREMENTAL_COLUMNS, ["id"], [], _INCREMENTAL_ROWS)uninterrupted = _read({}, **_INCREMENTAL_INPUTS)+ assert server.queries[-1][0].endswith("ORDER BY [updated_at] ASC")store: dict[str, str] = {}first = _read(store, stop_after=stop_after, write_last=write_last, **_INCREMENTAL_INPUTS)@@ -408,9 +416,45 @@ class TestIncrementalResume:merged = {row["id"]: row for row in first + resumed}assert sorted(merged.values(), key=lambda row: row["id"]) == sorted(uninterrupted, key=lambda row: row["id"])assert store == {}+ assert server.queries[-1][0].endswith("ORDER BY [updated_at] ASC")# Rows at the checkpoint value are read again. Rows below it are not.written_batches = stop_after if write_last else stop_after - 1if written_batches:checkpoint = first[-1]["updated_at"]assert server.queries[-1][1]["incremental_value"] == checkpointassert min(row["updated_at"] for row in resumed) == checkpoint++ @pytest.mark.parametrize(+ "table,input_overrides,expected_ascending,expected_resume",+ [+ pytest.param(_INCREMENTAL_TABLE, _INCREMENTAL_INPUTS, True, True, id="incremental_merge"),+ pytest.param(+ _INCREMENTAL_TABLE,+ {**_INCREMENTAL_INPUTS, "sync_type": ExternalDataSchemaSyncType.APPEND},+ True,+ False,+ id="incremental_append",+ ),+ pytest.param(+ # No `id` column: the source takes a column of that name as the primary key.+ ([("code", "int"), ("updated_at", "datetime2")], None, [], []),+ _INCREMENTAL_INPUTS,+ True,+ False,+ id="incremental_without_primary_key",+ ),+ pytest.param(_TABLES["bigint_key"], {}, False, True, id="full_refresh_keyset"),+ pytest.param(_TABLES["no_key"], {}, False, False, id="full_refresh_single_query"),+ ],+ )+ def test_only_the_incremental_single_query_claims_ascending_order(+ self, serve, table, input_overrides: dict[str, Any], expected_ascending: bool, expected_resume: bool+ ):+ serve(*table)++ source, _ = _build({}, **input_overrides)++ assert (source.sort_mode, source.supports_resume) == (+ "asc" if expected_ascending else None,+ expected_resume,+ )diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/mysql.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/mysql.pyindex 89068d14323f..989813bad268 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/mysql.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/mysql.py@@ -2255,4 +2255,7 @@ class MySQLImplementation(SQLSourceImplementation[MySQLSourceConfig, pymysql.Con# The streaming path can't be resumed across pods (its cursor is bound to one connection),# so this run is not cheaply resumable — treated as non-resumable for shutdown handling.supports_resume=False,+ # One streamed statement with `ORDER BY <incremental field>`. The FORCE INDEX retry+ # keeps the clause and runs only before the first row.+ sort_mode="asc" if should_use_incremental_field and incremental_field is not None else None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/tests/test_mysql.py b/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/tests/test_mysql.pyindex 6f55f1decbdf..c14f6f3b81ba 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/tests/test_mysql.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/mysql/tests/test_mysql.py@@ -1034,6 +1034,33 @@ class TestKeysetReadPath:assert source.supports_resume is False+ @pytest.mark.parametrize(+ "incremental,has_manager,expected_ascending,expected_resume",+ [+ pytest.param(True, False, True, False, id="incremental_streaming"),+ pytest.param(True, True, True, False, id="incremental_streaming_with_manager"),+ pytest.param(False, False, False, False, id="full_refresh_streaming"),+ pytest.param(False, True, False, True, id="full_refresh_keyset"),+ ],+ )+ def test_only_the_incremental_streaming_read_claims_ascending_order(+ self, keyset_mocks, incremental: bool, has_manager: bool, expected_ascending: bool, expected_resume: bool+ ):+ inputs = _make_inputs(+ should_use_incremental_field=incremental,+ incremental_field="id" if incremental else None,+ incremental_field_type=IncrementalFieldType.Integer if incremental else None,+ )++ source = MySQLImplementation().build_pipeline(+ _make_config(), inputs, resumable_source_manager=self._fake_manager() if has_manager else None+ )++ assert (source.sort_mode, source.supports_resume) == (+ "asc" if expected_ascending else None,+ expected_resume,+ )+class TestBuildPipelineSourceLocation:def test_uses_schema_metadata_when_schema_is_blank(self, build_pipeline_mocks):@@ -2265,6 +2292,7 @@ class TestBuildQueryForceIndex:db_incremental_field_last_value="2025-01-01",)assert "FORCE INDEX" not in query+ assert query.endswith("ORDER BY `created_at` ASC")def test_force_index_hint_added_when_provided(self):query, _ = _build_query(@@ -2279,6 +2307,7 @@ class TestBuildQueryForceIndex:assert "FORCE INDEX (`idx_created_at`)" in query# Hint goes between the table and the WHERE clauseassert query.index("FORCE INDEX") < query.index("WHERE")+ assert query.endswith("ORDER BY `created_at` ASC")def test_force_index_hint_applied_for_non_incremental_query_too(self):# Full refresh mode — the hint still attaches so callers can force adiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/openweather.py b/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/openweather.pyindex 4184b88d8c88..166f3fdd2899 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/openweather.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/openweather.py@@ -242,6 +242,4 @@ def openweather_source(partition_mode="datetime",partition_format="week",partition_keys=[config.partition_key],- # Forecast rows arrive in ascending `dt` order; single-snapshot endpoints have one row.- sort_mode="asc",)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/tests/test_openweather.py b/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/tests/test_openweather.pyindex 2a008aa2a5d4..05c05ff6a21f 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/tests/test_openweather.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/openweather/tests/test_openweather.py@@ -263,7 +263,7 @@ class TestOpenWeatherSource:assert response.primary_keys == ["lat", "lon", "dt"]assert response.partition_mode == "datetime"assert response.partition_keys == ["dt_iso"]- assert response.sort_mode == "asc"+ assert response.sort_mode is Nonedef test_invalid_locations_raise(self):with pytest.raises(ValueError):diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/plain/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/plain/settings.pyindex 3141d1a88146..3aa57319f511 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/plain/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/plain/settings.py@@ -44,7 +44,7 @@ class PlainEndpointConfig:partition_mode: PartitionMode | None = "datetime"partition_format: PartitionFormat | None = "week"partition_keys: list[str] | None = field(default_factory=lambda: [CREATED_AT])- sort_mode: SortMode = "asc"+ sort_mode: SortMode | None = NonePLAIN_ENDPOINTS: dict[str, PlainEndpointConfig] = {diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/plain/tests/test_plain.py b/products/warehouse_sources/backend/temporal/data_imports/sources/plain/tests/test_plain.pyindex d46a19b3eb8d..2c79d212420d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/plain/tests/test_plain.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/plain/tests/test_plain.py@@ -452,10 +452,13 @@ class TestLookupEndpoints:@pytest.mark.parametrize("endpoint,expected_sort_mode",[+ ("customers", None),+ ("threads", None),+ ("timeline_entries", None),("companies", "desc"),("tenants", "desc"),- ("users", "asc"),- ("label_types", "asc"),+ ("users", None),+ ("label_types", None),],)def test_source_for_pipeline_sets_sort_mode(self, endpoint, expected_sort_mode):diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/postgres.py b/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/postgres.pyindex 2ce0f5314313..71582df8f681 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/postgres.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/postgres.py@@ -2897,7 +2897,8 @@ def resolve_postgres_keyset() -> PostgresKeyset:"""Decide how far this run can go: no seek, seek only, or seek plus a durable checkpoint."""if should_use_incremental_field:- # Already resumable from its persisted watermark, and seeking would double the work.+ # v3 moves the watermark only when the run's load completes, so a cut-off attempt restarts from the old+ # watermark unless the handoff checkpoint applies. A keyset seek on top of that would double the work.return PostgresKeyset(reason="incremental_sync")if is_xmin:# An xmin read appends deltas, so restarting it from key 0 would duplicate what it wrote.@@ -4781,6 +4782,17 @@ def postgres_source())+ # One statement with `ORDER BY <incremental field>` proves the order for the whole run. A read+ # per partition, per window or per LIMIT/OFFSET page issues several statements, so it does not.+ reads_one_ordered_statement = (+ should_use_incremental_field+ and incremental_field is not None+ and xmin_bounds is None+ and not is_duckdb+ and not use_window_chunking+ and not use_per_partition_chunking+ )+return SourceResponse(name=name,items=lambda: get_rows(chunk_size),@@ -4793,6 +4805,7 @@ def postgres_source(# position to another pod, and one that could checkpoint but reads through a server cursor# has no position to hand over. `supports_resume` defaults to True, so this must be explicit.supports_resume=can_checkpoint and takes_keyset_path,+ sort_mode="asc" if reads_one_ordered_statement else None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/test_postgres.py b/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/test_postgres.pyindex 51157b49cc06..52e571251003 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/test_postgres.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/postgres/test_postgres.py@@ -4433,6 +4433,9 @@ class TestChunkedRereadAfterRecoveryConflict:chunking: _TableChunking | None = None,lock_timeout_before_each_page: bool = False,page_error: BaseException | None = None,+ is_duckdb: bool = False,+ child_partitions: list[ChildPartition] | None = None,+ partition_strategy: PartitionStrategy | None = None,text_ids: list[str] | None = None,worker_deaths: int = 0,) -> list[int | str]:@@ -4470,10 +4473,12 @@ class TestChunkedRereadAfterRecoveryConflict:),patch(f"{module}._get_table", return_value=fake_table),patch(f"{module}._is_read_replica", return_value=True),- patch(f"{module}._is_duckdb_connection", return_value=False),+ patch(f"{module}._is_duckdb_connection", return_value=is_duckdb),patch(f"{module}._get_primary_keys", return_value=primary_keys),patch(f"{module}._has_duplicate_primary_keys", return_value=has_duplicate_pks),- patch(f"{module}._is_partitioned_table", return_value=False),+ patch(f"{module}._is_partitioned_table", return_value=child_partitions is not None),+ patch(f"{module}.list_child_partitions", return_value=child_partitions or []),+ patch(f"{module}.get_partition_strategy", return_value=partition_strategy),patch(f"{module}._get_table_chunk_size",return_value=chunking or _TableChunking(batch_rows=2, fetch_rows=2),@@ -4504,6 +4509,8 @@ class TestChunkedRereadAfterRecoveryConflict:resumable_source_manager=resumable_source_manager,)self.last_response = response+ if pages_to_take == 0:+ return []pages = cast(Iterator[Any], iter(cast(Iterable[Any], response.items())))ids: list[int | str] = []taken = 0@@ -4701,6 +4708,43 @@ class TestChunkedRereadAfterRecoveryConflict:assert self.last_response.supports_resume is False+ @pytest.mark.parametrize(+ "kwargs,expected",+ [+ ({"should_use_incremental_field": True, "primary_keys": ["id"]}, True),+ ({"should_use_incremental_field": True, "primary_keys": ["id"], "is_duckdb": True}, False),+ ({"should_use_incremental_field": True, "primary_keys": ["id"], "child_partitions": []}, False),+ (+ {+ "should_use_incremental_field": True,+ "primary_keys": ["id"],+ "child_partitions": [ChildPartition(oid=1, schema="public", name="orders_p1", partbound="")],+ "partition_strategy": PartitionStrategy(strategy="r", key_columns=("id",)),+ },+ False,+ ),+ ({"should_use_incremental_field": False, "primary_keys": ["id"]}, False),+ ({"should_use_incremental_field": False, "primary_keys": None}, False),+ ({"should_use_incremental_field": False, "is_xmin": True, "primary_keys": ["id"]}, False),+ ],+ ids=[+ "incremental_one_server_cursor",+ "incremental_duckdb_offset_pages",+ "incremental_partitioned_windows",+ "incremental_per_child_partition",+ "full_refresh_keyset",+ "full_refresh_no_seekable_key",+ "xmin",+ ],+ )+ def test_only_one_ordered_statement_claims_ascending_for_the_whole_run(+ self, kwargs: dict[str, Any], expected: bool+ ) -> None:+ # The claim is set when the response is built, so no page is read here.+ self._read_ids(rows_before_conflict=0, pages_to_take=0, **kwargs)++ assert self.last_response.sort_mode == ("asc" if expected else None)+def test_a_text_primary_key_seeks_but_never_checkpoints(self):# The collation decision. Ordering a text key is stable within one process, so the in-process# seek keeps working, but a checkpoint would have that ordering assumption hold across adiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/redshift.py b/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/redshift.pyindex 40c28eddea82..358dc8f6d2fa 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/redshift.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/redshift.py@@ -2176,4 +2176,7 @@ class RedshiftImplementation(SQLSourceImplementation[RedshiftSourceConfig, psycorows_to_sync=rows_to_sync,has_duplicate_primary_keys=duplicate_primary_keys,verified_primary_keys=verified_primary_keys,+ # One statement with `ORDER BY <incremental field>`. Each fallback runs the same+ # statement, and only before the first row.+ sort_mode="asc" if should_use_incremental_field and incremental_field is not None else None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/tests/test_redshift.py b/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/tests/test_redshift.pyindex 1ca232d33992..9e34132272c3 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/tests/test_redshift.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/redshift/tests/test_redshift.py@@ -1243,6 +1243,27 @@ class TestBuildPipeline:impl.build_pipeline(_make_config(), _make_inputs(), chunk_size_override=4242)mocked_chunk_size.assert_not_called()+ @pytest.mark.parametrize(+ "input_overrides,expected",+ [+ pytest.param(+ {+ "should_use_incremental_field": True,+ "incremental_field": "id",+ "incremental_field_type": IncrementalFieldType.Integer,+ },+ True,+ id="incremental",+ ),+ pytest.param({}, False, id="full_refresh"),+ ],+ )+ def test_claims_ascending_order_only_for_an_incremental_run(+ self, build_pipeline_mocks: Any, input_overrides: dict[str, Any], expected: bool+ ) -> None:+ response = RedshiftImplementation().build_pipeline(_make_config(), _make_inputs(**input_overrides))+ assert response.sort_mode == ("asc" if expected else None)+# ---------------------------------------------------------------------------# Connection lifecyclediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/singlestore.py b/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/singlestore.pyindex 965a029e083b..9cfa006c33fa 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/singlestore.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/singlestore.py@@ -227,7 +227,5 @@ def singlestore_source(partition_mode="datetime" if config.partition_key else None,partition_format="month" if config.partition_key else None,partition_keys=[config.partition_key] if config.partition_key else None,- # Full-refresh endpoints have no server-side order guarantee; billing_usage requests- # `startTime` ascending via the incremental window, so asc holds for every endpoint.- sort_mode="asc",+ # No `sort_mode`: billing_usage rows come in one group per metric, so `startTime` has no order.)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/tests/test_singlestore.py b/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/tests/test_singlestore.pyindex b9ee358f1eb1..90ecc7837152 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/tests/test_singlestore.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/singlestore/tests/test_singlestore.py@@ -221,4 +221,4 @@ class TestSourceResponseShape:assert response is not Noneassert response.name == endpointassert response.primary_keys == expected_pk- assert response.sort_mode == "asc"+ assert response.sort_mode is Nonediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/settings.pyindex 43b9067e8639..8ed8a4e9f62f 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/settings.py@@ -57,7 +57,7 @@ class SmartsheetEndpointConfig:fanout: Optional[DependentEndpointConfig] = Nonepagination: Pagination = "page_number"data_selector: str = "data"- sort_mode: SortMode = "asc"+ sort_mode: SortMode | None = None# The parent-report id injected into each report child row (via include_from_parent), used as thediff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/tests/test_smartsheet.py b/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/tests/test_smartsheet.pyindex 938ea525bba3..15ea1d26969f 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/tests/test_smartsheet.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/smartsheet/tests/test_smartsheet.py@@ -238,7 +238,7 @@ class TestSmartsheetSourceResponse:assert response is not Noneassert response.name == endpointassert response.primary_keys == expected_keys- assert response.sort_mode == config.sort_mode+ assert response.sort_mode == ("desc" if config.incremental_fields else None)if config.partition_key:assert response.partition_mode == "datetime"assert response.partition_format == "week"diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/snowflake.py b/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/snowflake.pyindex 08d78d3061f0..bb8eb4c47aeb 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/snowflake.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/snowflake.py@@ -806,4 +806,6 @@ class SnowflakeImplementation(# `supports_resume` defaults to True. A run with no order column saves no checkpoint,# so a hand-off would start it again from the first row, on each attempt.supports_resume=order_column is not None,+ # One statement with `ORDER BY <incremental field>`, read as one stream.+ sort_mode="asc" if should_use_incremental_field and incremental_field is not None else None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/tests/test_snowflake.py b/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/tests/test_snowflake.pyindex cb1d86b8da65..50e449cf45a7 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/tests/test_snowflake.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/snowflake/tests/test_snowflake.py@@ -180,7 +180,7 @@ class TestBuildQuery:def test_incremental_uses_operator_and_orders(self):sql, params = _build_query("DB", "PUBLIC", "t", True, "created_at", IncrementalFieldType.DateTime, "2025-01-01")assert 'WHERE "created_at"' in sql- assert 'ORDER BY "created_at" ASC' in sql+ assert sql.endswith(' ORDER BY "created_at" ASC')assert params == ("DB.PUBLIC.t", "2025-01-01")def test_incremental_seeds_initial_value_when_missing(self):@@ -779,6 +779,13 @@ class TestBuildPipeline:assert stream_param == ("DB.analytics.users",)+_INCREMENTAL_INPUTS: dict[str, Any] = {+ "should_use_incremental_field": True,+ "incremental_field": "created_at",+ "incremental_field_type": IncrementalFieldType.DateTime,+}++class TestResumableStreaming:def _metadata_cursor(self):metadata_cursor = MagicMock()@@ -887,6 +894,35 @@ class TestResumableStreaming:manager.save_state.assert_not_called()assert response.supports_resume is False+ @pytest.mark.parametrize(+ "primary_key_rows,input_overrides,expected_supports_resume,expected_ascending",+ [+ pytest.param([("ID",)], _INCREMENTAL_INPUTS, True, True, id="incremental_with_primary_key"),+ pytest.param([], _INCREMENTAL_INPUTS, False, True, id="incremental_without_primary_key"),+ pytest.param([("ID",)], {"verified_primary_keys": ["ID"]}, True, False, id="full_refresh_by_verified_key"),+ pytest.param([("ID",)], {}, False, False, id="full_refresh_without_order"),+ ],+ )+ def test_claims_ascending_order_only_for_an_incremental_run(+ self,+ impl: SnowflakeImplementation,+ primary_key_rows: list[tuple[str]],+ input_overrides: dict[str, Any],+ expected_supports_resume: bool,+ expected_ascending: bool,+ ) -> None:+ metadata_cursor = self._metadata_cursor()+ metadata_cursor.__iter__.return_value = iter(primary_key_rows)+ manager = MagicMock()+ manager.can_resume.return_value = False+ connection = self._connection(metadata_cursor, self._streaming_cursor([]))+ with patch("snowflake.connector.connect", return_value=connection):+ response = impl.build_pipeline(+ _make_config(), _make_inputs(**input_overrides), resumable_source_manager=manager+ )+ assert response.supports_resume is expected_supports_resume+ assert response.sort_mode == ("asc" if expected_ascending else None)+def test_snowflake_source_is_resumable():# Resumable registration is what routes the import through the resumable retry policy anddiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/todoist/tests/test_todoist.py b/products/warehouse_sources/backend/temporal/data_imports/sources/todoist/tests/test_todoist.pyindex 0a59b4facb23..7cd6cff140bf 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/todoist/tests/test_todoist.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/todoist/tests/test_todoist.py@@ -89,8 +89,7 @@ class TestSourceResponseShape:assert response.primary_keys == expected_keysassert response.partition_mode == expected_partition_modeassert response.partition_keys == ([expected_partition_key] if expected_partition_key else None)- # Ascending order is the safe default; we never declare desc here.- assert response.sort_mode == "asc"+ assert response.sort_mode is Noneclass TestEndpointConfig:diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/trino/tests/test_trino.py b/products/warehouse_sources/backend/temporal/data_imports/sources/trino/tests/test_trino.pyindex 15ba7fd7634c..87b451164dff 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/trino/tests/test_trino.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/trino/tests/test_trino.py@@ -680,9 +680,12 @@ def test_build_pipeline_query("db_incremental_field_last_value": last_value,}- _run_pipeline(_config(schema="s"), _inputs("t", enabled_columns=enabled_columns, **incremental_inputs), cursor)+ response, _ = _run_pipeline(+ _config(schema="s"), _inputs("t", enabled_columns=enabled_columns, **incremental_inputs), cursor+ )assert cursor.data_queries[-1] == (expected_sql, expected_params)+ assert response.sort_mode == ("asc" if incremental is not None else None)def test_build_pipeline_ignores_the_incremental_field_on_a_full_refresh() -> None:diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/trino/trino.py b/products/warehouse_sources/backend/temporal/data_imports/sources/trino/trino.pyindex bc15875cf607..36ef892ac3e6 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/trino/trino.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/trino/trino.py@@ -811,4 +811,6 @@ class TrinoImplementation(SQLSourceImplementation[TrinoSourceConfig, Any, Any],items=get_rows,primary_keys=primary_keys,rows_to_sync=rows_to_sync,+ # One statement with `ORDER BY <incremental field>`, read as one stream.+ sort_mode="asc" if incremental_field is not None else None,)diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/typeform/tests/test_typeform.py b/products/warehouse_sources/backend/temporal/data_imports/sources/typeform/tests/test_typeform.pyindex 56c13c77df5c..fd65fd8ac39d 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/typeform/tests/test_typeform.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/typeform/tests/test_typeform.py@@ -193,6 +193,8 @@ class TestTypeformTransport:assert resource["endpoint"]["path"] == "/forms"assert resource["endpoint"]["data_selector"] == "items"assert resource["endpoint"]["params"]["page_size"] == 200+ assert resource["endpoint"]["params"]["sort_by"] == "last_updated_at"+ assert resource["endpoint"]["params"]["order_by"] == "asc"assert resource["table_format"] == "delta"def test_get_resource_rejects_responses_fanout(self) -> None:diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/ubidots/tests/test_ubidots.py b/products/warehouse_sources/backend/temporal/data_imports/sources/ubidots/tests/test_ubidots.pyindex 465e7ff21f55..1bc816a2defe 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/ubidots/tests/test_ubidots.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/ubidots/tests/test_ubidots.py@@ -375,7 +375,7 @@ class TestUbidotsSourceResponse:assert response.sort_mode == "desc"else:assert response.primary_keys == ["id"]- assert response.sort_mode == "asc"+ assert response.sort_mode is None@pytest.mark.parametrize("api_version,expected_fn",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/vitally/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/vitally/source.pyindex 75fa0ba70cfa..051362e29195 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/vitally/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/vitally/source.py@@ -167,7 +167,8 @@ class VitallySource(SimpleSource[VitallySourceConfig]):partition_mode="datetime",partition_format="week",partition_keys=["created_at"],- sort_mode="desc" if inputs.schema_name == "Messages" else "asc",+ # `sortBy` carries no direction, so only the Messages read claims an order.+ sort_mode="desc" if inputs.schema_name == "Messages" else None,)@propertydiff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/settings.py b/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/settings.pyindex b174b6d3abfc..3a6df4a7de38 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/settings.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/settings.py@@ -134,7 +134,7 @@ INCREMENTAL_TALK_ENDPOINTS = {CURSOR_PAGE_SIZE = 100-@dataclass+@dataclass(frozen=False)class ZendeskEndpointConfig:"""Declarative config for a Zendesk Support API list endpoint.@@ -157,7 +157,7 @@ class ZendeskEndpointConfig:# Query param carrying the server-side time filter, when the endpoint has one.incremental_start_param: str | None = Nonepartition_key: str | None = None- sort_mode: SortMode = "asc"+ sort_mode: SortMode | None = "asc"page_size: int = CURSOR_PAGE_SIZEfanout: DependentEndpointConfig | None = Noneparams: dict[str, Any] = field(default_factory=dict)@@ -233,6 +233,8 @@ ZENDESK_ENDPOINTS: dict[str, ZendeskEndpointConfig] = {# the comments that fall outside the bound.incremental_fields=[_datetime_incremental_field("created_at")],default_incremental_field="created_at",+ # Comments arrive grouped per ticket, so the row order starts again for each ticket.+ sort_mode=None,fanout=DependentEndpointConfig(parent_name=TICKET_COMMENTS_PARENT_NAME,resolve_param="ticket_id",diff --git a/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/source.py b/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/source.pyindex e54306a0b454..5ec488ab4146 100644--- a/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/source.py+++ b/products/warehouse_sources/backend/temporal/data_imports/sources/zendesk/source.py@@ -20,7 +20,11 @@ from products.warehouse_sources.backend.temporal.data_imports.sources.common.res)from products.warehouse_sources.backend.temporal.data_imports.sources.common.resumable import ResumableSourceManagerfrom products.warehouse_sources.backend.temporal.data_imports.sources.common.schema import SourceSchema-from products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import SourceInputs, SourceResponse+from products.warehouse_sources.backend.temporal.data_imports.sources.common.typings import (+ SortMode,+ SourceInputs,+ SourceResponse,+)from products.warehouse_sources.backend.temporal.data_imports.sources.generated_configs.zendesk import (ZendeskSourceConfig,)@@ -303,13 +307,15 @@ class ZendeskSource(ResumableSource[ZendeskSourceConfig, ZendeskResumeConfig]):resumable_source_manager=resumable_source_manager,)# The original nine endpoints aren't in the declarative catalog; they keep the `id`- # primary key and ascending sort they have always used.+ # primary key. The time-ordered incremental exports give them an ascending sort, except+ # `tickets`: its cursor export states no order for `generated_timestamp`.+ legacy_sort_mode: SortMode | None = None if inputs.schema_name == "tickets" else "asc"response = SourceResponse(name=resource.name,items=lambda: resource,primary_keys=endpoint_config.primary_key if endpoint_config else ["id"],column_hints=resource.column_hints,- sort_mode=endpoint_config.sort_mode if endpoint_config else "asc",+ sort_mode=endpoint_config.sort_mode if endpoint_config else legacy_sort_mode,# The fan-out saves no checkpoint (see `zendesk_source`), so a shutdown must not hand it# to another worker expecting one.supports_resume=not is_fanout,diff --git a/products/warehouse_sources/backend/temporal/data_imports/workflow_activities/tests/test_import_data_sync.py b/products/warehouse_sources/backend/temporal/data_imports/workflow_activities/tests/test_import_data_sync.pyindex 4d887cecf048..340070351ed8 100644--- a/products/warehouse_sources/backend/temporal/data_imports/workflow_activities/tests/test_import_data_sync.py+++ b/products/warehouse_sources/backend/temporal/data_imports/workflow_activities/tests/test_import_data_sync.py@@ -1745,7 +1745,11 @@ _QUEUED_UP_TO = datetime(2026, 6, 20, 8, 0, 0)[pytest.param(True,- {"run_uuid": "wfrun-1-a1", "last_value": "2026-06-21T00:00:00", "resume_value": "2026-06-20T08:00:00"},+ {+ "run_uuid": "wfrun-1-a1",+ "last_value": "2026-06-21T00:00:00",+ "ordered_resume_value": "2026-06-20T08:00:00",+ },False,_QUEUED_UP_TO,_QUEUED_UP_TO,@@ -1754,7 +1758,11 @@ _QUEUED_UP_TO = datetime(2026, 6, 20, 8, 0, 0)# The lookback of one hour applies to a run that starts from the stored watermark.pytest.param(False,- {"run_uuid": "wfrun-1-a1", "last_value": "2026-06-21T00:00:00", "resume_value": "2026-06-20T08:00:00"},+ {+ "run_uuid": "wfrun-1-a1",+ "last_value": "2026-06-21T00:00:00",+ "ordered_resume_value": "2026-06-20T08:00:00",+ },False,_STORED_WATERMARK - timedelta(hours=1),None,@@ -1762,7 +1770,11 @@ _QUEUED_UP_TO = datetime(2026, 6, 20, 8, 0, 0)),pytest.param(True,- {"run_uuid": "wfrun-0-a1", "last_value": "2026-06-21T00:00:00", "resume_value": "2026-06-20T08:00:00"},+ {+ "run_uuid": "wfrun-0-a1",+ "last_value": "2026-06-21T00:00:00",+ "ordered_resume_value": "2026-06-20T08:00:00",+ },False,_STORED_WATERMARK - timedelta(hours=1),None,@@ -1778,7 +1790,11 @@ _QUEUED_UP_TO = datetime(2026, 6, 20, 8, 0, 0)),pytest.param(True,- {"run_uuid": "wfrun-1-a1", "last_value": "2026-06-21T00:00:00", "resume_value": "2026-06-20T08:00:00"},+ {+ "run_uuid": "wfrun-1-a1",+ "last_value": "2026-06-21T00:00:00",+ "ordered_resume_value": "2026-06-20T08:00:00",+ },True,None,None,