:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
fix(managed-migrations): correct batch import admin help text (#125951)
Eli Reisman268edeb0f55d
diff --git a/products/managed_migrations/backend/admin/batch_imports.py b/products/managed_migrations/backend/admin/batch_imports.pyindex 630c001db5f4..824b2f291b57 100644--- a/products/managed_migrations/backend/admin/batch_imports.py+++ b/products/managed_migrations/backend/admin/batch_imports.py@@ -157,6 +157,7 @@ class BatchImportAdmin(admin.ModelAdmin):),"description": ("A running job with a future leased_until is actively claimed by a worker. "+ "Changing the status from running stops the worker before it sends another chunk. ""Changing the status to running, or using a resume action, clears the lease, backoff ""and pending commit so a worker can claim the job at once. A pending commit on ""a job no worker holds means a worker stopped mid-commit: the chunk it names "@@ -166,9 +167,10 @@ class BatchImportAdmin(admin.ModelAdmin):"resets the current part to offset 0 first - right when the source bytes ""changed underneath the saved offset (a re-downloaded nondeterministic ""export, or a source file replaced after a data fix), which typically "- "shows up as an 'Invalid JSON syntax' pause. The re-imported overlap "- "dedupes for sources with deterministic event UUIDs (Mixpanel $insert_id, "- "Amplitude uuid)."+ "shows up as an 'Invalid JSON syntax' pause. Re-imported events are "+ "deduplicated only when they carry a source ID (Mixpanel $insert_id, or an "+ "Amplitude or captured event uuid). Other events, including the identify "+ "events created for Amplitude imports, get a new ID and are not deduplicated."),},),@@ -257,8 +259,8 @@ class BatchImportAdmin(admin.ModelAdmin):def resume_with_inflight_part_reset(self, request, queryset):"""Recover a paused job whose committed byte offset no longer matches thesource bytes (re-download of a nondeterministic export, or a source file- replaced after a data-error pause). Safe for sources with deterministic- event UUIDs, which dedupe the re-imported overlap."""+ replaced after a data-error pause). The re-imported overlap dedupes only for+ events with a source ID; see the 'Worker state' section."""for batch_import in queryset:try:reset_key = batch_import.resume_with_inflight_part_reset()diff --git a/products/managed_migrations/backend/api/test/test_batch_imports.py b/products/managed_migrations/backend/api/test/test_batch_imports.pyindex ab6536eee363..6d0764be9a47 100644--- a/products/managed_migrations/backend/api/test/test_batch_imports.py+++ b/products/managed_migrations/backend/api/test/test_batch_imports.py@@ -205,15 +205,6 @@ class TestBatchImportModel(BaseTest):for method in ("resume_after_pause", "resume_with_inflight_part_reset")for case, row, refused in (("marker_and_live_lease", {"leased_until": timedelta(minutes=5)}, True),- (- "legacy_commit_message_and_live_lease",- {- "leased_until": timedelta(minutes=5),- "pending_commit": None,- "status_message": "Starting commit of part day-2 to offset 9, consumed 9 additional bytes",- },- True,- ),("marker_and_expired_lease", {"leased_until": -timedelta(minutes=1)}, False),("marker_and_no_lease", {"lease_id": None, "leased_until": None}, False),)diff --git a/products/managed_migrations/backend/models/batch_imports.py b/products/managed_migrations/backend/models/batch_imports.pyindex 11db6f563624..77b95f2d0be6 100644--- a/products/managed_migrations/backend/models/batch_imports.py+++ b/products/managed_migrations/backend/models/batch_imports.py@@ -17,9 +17,6 @@ if TYPE_CHECKING:DEFAULT_SEND_RATE = 5_000-# The worker before pending_commit wrote this status message while it sent a chunk.-LEGACY_COMMIT_IN_FLIGHT_MESSAGE_PREFIX = "Starting commit of part "-class CommitInFlightError(ValueError):"""A worker is sending a chunk of this import, so a resume now could skip that chunk."""@@ -151,8 +148,9 @@ class BatchImport(ModelActivityMixin, UUIDTModel):temp-bucket staging, or a source file replaced after a data-error pause), thesaved offset can land mid-record in the new stream and the job pauses with aparse error at the resume point. The fix is to re-import that part from- offset 0 - safe for sources with deterministic event UUIDs (Mixpanel- $insert_id, Amplitude uuid), which dedupe the overlap.+ offset 0. The re-imported overlap dedupes only for events with a source ID+ (Mixpanel $insert_id, an Amplitude or captured uuid). Other events, including+ the identify events the worker generates for Amplitude, get a new UUID.Resets the first unfinished part to offset 0, then resumes as`resume_after_pause` does. Returns the key of the reset part. Raises@@ -190,9 +188,7 @@ class BatchImport(ModelActivityMixin, UUIDTModel):sink accepting it, the window in which clearing its lease can lose that chunk."""if self.leased_until is None or self.leased_until <= timezone.now():return False- return self.pending_commit is not None or (self.status_message or "").startswith(- LEGACY_COMMIT_IN_FLIGHT_MESSAGE_PREFIX- )+ return self.pending_commit is not Nonedef raise_if_commit_in_flight(self) -> None:if self.commit_in_flight():