Skip to content

Database governance built for humans and agents — controlling changes and access across every major database.

docs: design retiring the export-data workflow

Danny7f15bfb2492e
diff --git a/docs/design/retire-export-data-workflow.md b/docs/design/retire-export-data-workflow.md
new file mode 100644
index 0000000000..808156ebc1
--- /dev/null
+++ b/docs/design/retire-export-data-workflow.md
@@ -0,0 +1,208 @@
+# Retire the Export-Data Workflow
+
+- Status: design for review — decision: full purge
+- Date: 2026-07-31
+- Target: main (3.22 dev cycle) — not `release/3.21.0`
+
+## Background
+
+The export center was deprecated in #20693. The SQL Editor "Request Export"
+button now opens the JIT access-grant drawer, and JIT-approved exports stream
+synchronously through `SQLService.Export` — they never touch `export_archive`.
+The plan UI blocks export plans with "Data export issue creation is no longer
+supported".
+
+The machinery is still fully wired, reachable only via direct v1 API calls:
+
+- `PlanService.CreatePlan` accepts `export_data_config` specs
+ (`backend/api/v1/plan_service.go:650`); rollout creation turns them into
+ `DATABASE_EXPORT` tasks (`backend/api/v1/rollout_service_task.go:300`).
+- The executor is registered (`backend/server/server.go:222`) and writes
+ `export_archive` rows (`backend/runner/taskrun/data_export_executor.go`).
+- `SQLService.Export` with a rollout name downloads the archive
+ (`doExportFromIssue`, `backend/api/v1/sql_service.go:1038`).
+- The data cleaner purges archives older than 24h, so the table is in practice
+ always near-empty.
+
+## Decision: full purge
+
+Delete legacy `DATABASE_EXPORT` issues and their plans/tasks/runs in a
+migration, drop `export_archive`, and remove every proto value — no
+deprecated read surface, no legacy UI.
+
+Considered alternatives: tombstone read-only (keep ~40 deprecated proto lines
+so history stays legible) and block-creation-only. Rejected in favor of a
+clean end-state: export-center usage was near zero (already deprecated), and
+the durable audit trail lives in `audit_log` — issue/plan rows are redundant
+with it for compliance purposes.
+
+What is irreversibly lost: the issue-level records of past export requests
+(title, approval chain rendering, statement sheet linkage). What remains:
+`audit_log` rows for the create/approve/export RPCs, and `query_history` rows
+for performed exports.
+
+## Migration (`backend/migrator/migration/3.22/xxxx##retire_export_data.sql`)
+
+`issue.type` and `task.type` are text columns. Plan specs are protojson JSONB
+(camelCase keys — the spec key is `exportDataConfig`). All tables involved use
+composite PKs `(project, id)`; every join below carries the project column.
+
+Export-plan set: plans referenced by a `DATABASE_EXPORT` issue, UNION plans
+whose `config->'specs'` contains an `exportDataConfig` key (covers API-created
+plans that never got an issue).
+
+Delete order (children before parents, per the FK graph):
+
+```sql
+-- 1. task_run_log → task_run → task(type DATABASE_EXPORT)
+DELETE FROM task_run_log l USING task_run r, task t
+ WHERE l.project = r.project AND l.task_run_id = r.id
+ AND r.project = t.project AND r.task_id = t.id
+ AND t.type = 'DATABASE_EXPORT';
+-- 2. task_run
+DELETE FROM task_run r USING task t
+ WHERE r.project = t.project AND r.task_id = t.id
+ AND t.type = 'DATABASE_EXPORT';
+-- 3. task
+DELETE FROM task WHERE type = 'DATABASE_EXPORT';
+-- 4-8. with export_plan AS (<set above>):
+-- plan_check_run, issue_comment (via export issues), issue,
+-- plan_webhook_delivery, plan
+-- 9. DROP TABLE export_archive;
+```
+
+Also: update `LATEST.sql` (drop the `export_archive` table, update the
+`issue.type` comment), and bump `TestLatestVersion` in `migrator_test.go`.
+
+Notes:
+
+- The whole file runs in one transaction; DML + `DROP TABLE` is fine in PG.
+- The `jsonb_array_elements` scan over `plan` is a one-time full-table scan;
+ acceptable at migration time.
+- No new store methods → no new `TestCollision_*` obligations.
+- Historical migration files (3.4, 3.17) that mention `export_archive` are
+ immutable and stay untouched.
+
+## Proto removal (all tags `reserved`, number and name)
+
+| File | Change |
+| --- | --- |
+| `store/export_archive.proto` | delete file (+ delete stale generated `export_archive*.pb.go`; `buf generate` does not remove them) |
+| `store/issue.proto` | remove `Issue.Type.DATABASE_EXPORT = 3` |
+| `store/task.proto` | remove `Task.Type.DATABASE_EXPORT = 3` |
+| `store/plan.proto` | remove `Spec.export_data_config = 7` oneof arm + `ExportDataConfig` message |
+| `store/task_run.proto` | remove `export_archive_id = 9` |
+| `v1/issue_service.proto` | remove `Issue.Type.DATABASE_EXPORT = 3` |
+| `v1/plan_service.proto` | remove `Spec.export_data_config = 4` + `ExportDataConfig` message; fix spec_type filter doc (`:139`) |
+| `v1/rollout_service.proto` | remove `Task.Type.DATABASE_EXPORT = 4`, `database_data_export = 9` + `DatabaseDataExport` message, `ExportArchiveStatus` enum + field 10; fix task_type filter doc (`:229`) |
+
+Then `buf format -w proto && buf lint proto && cd proto && buf generate`
+(regenerates backend `generated-go` and frontend `types/proto-es`). If CI runs
+`buf breaking`, this PR is intentionally breaking — handle per the pre-PR
+breaking-change review.
+
+## Backend removal
+
+Compile-driven after proto regen; the enum removals surface every remaining
+reference.
+
+- Delete `runner/taskrun/data_export_executor.go`; unregister in
+ `server.go:222`
+- Delete `store/export_archive.go`; remove `DeleteExpiredExportArchivesAll`
+ (`store/runner_queries.go`) and `cleanupExportArchives` + retention const
+ (`runner/cleaner/data_cleaner.go`)
+- `api/v1/sql_service.go`: remove `doExportFromIssue` and the rollout-name
+ branch of `Export` (RPC then accepts only database targets); remove the zip
+ re-encrypt helpers if unused elsewhere (`doEncrypt` is shared with the
+ synchronous path — verify before removing)
+- `api/v1/rollout_service_task.go`: remove `getTaskCreatesFromExportDataConfig`
+ + its spec-switch case
+- `api/v1/plan_service.go`: `CreatePlan`/`UpdatePlan` reject
+ `export_data_config` specs with an explicit error; remove both converter
+ directions
+- `api/v1/issue_service.go`: `CreateIssue` rejects `DATABASE_EXPORT`;
+ remove converter cases (`issue_service_converter.go`)
+- `api/v1/issue_hook.go`: remove the `Issue_DATABASE_EXPORT` branches from
+ approval finding
+- `api/v1/rollout_service.go:289,1330`: remove creator-only export special
+ cases
+- `api/v1/rollout_service_converter.go`: remove `ExportArchiveStatus`
+ population, `convertToTaskFromDatabaseDataExport`, task-type mapping
+- `component/review/evaluator.go`: remove `Issue_DATABASE_EXPORT` branches
+ (`:335,1053`); KEEP `buildCELVariablesForDataExport` and the JIT
+ access-grant path (`:864` comment documents the split)
+- `runner/taskrun/running_scheduler.go:296`,
+ `runner/taskrun/database_migrate_executor.go:1042`: prune from
+ `exhaustive:enforce` switches
+- Tests: prune export cases in `plan_service_test.go`,
+ `issue_service_test.go`; add rejection tests (below)
+
+## Frontend removal
+
+Type-check-driven after proto regen.
+
+- Delete `IssueDetailDatabaseExportView.tsx` (~800 lines) and its route wiring
+ (`useIssueDetailType.ts`)
+- `actionRegistry.ts`: `exportArchiveReady`, `computeExportArchiveReady`,
+ download action; `IssueDetailActionBar.tsx`: download/expiry UI
+- Export gating in `IssueDetailTaskRolloutActionPanel`,
+ `PlanDetailTaskRolloutActionPanel`, `DeployTaskToolbar`
+ (`data-export-creator-only`)
+- `advanceState.ts` export blocker (backend rejection replaces it)
+- `IssueTable.tsx:537`: remove the `DATABASE_EXPORT` filter option
+- Prune export branches across `utils/v1/issue/*`, `lib/plan/*`,
+ `plan-detail/utils/*` (`createPlan.ts` `targetsForSpec`, `changeReference`,
+ `phaseSummary`, `rolloutPreview`, `diffPlanSpecs`, `check`, `workflow`),
+ `useRedirects.ts`, `useIssueDetailSpecValidation.ts`,
+ `PlanDetailStatementSection.tsx`, `IssueDetailStatementSection.tsx`,
+ `types/v1/issue/issue.ts`, `issueDetailRedirect.test.ts`
+- Locales: remove `issue.data-export.*`, `task.data-export-creator-only`, and
+ any keys that become unused — across all locale files. KEEP `export-data.*`
+ keys used by `DataExportButton` and `sql-editor.request-export` (JIT)
+
+## Explicitly kept (shared with live features)
+
+- JIT access-grant flow: `RequestExportButton`, `AccessGrantRequestDrawer`,
+ `bb.accessGrants.*`
+- Synchronous SQL Editor export: `DataExportButton`, `SQLService.Export` with
+ database targets, `doExport`, export masking/permissions
+- Risk source `DATA_EXPORT` / CEL namespace `request.data_export` and
+ `buildCELVariablesForDataExport` (used by JIT grants and risk rules)
+- `query_history` export rows; `audit_log` — untouched
+- Historical migration files referencing `export_archive`
+
+## Compatibility
+
+- Breaking v1 API change: `CreatePlan`/`CreateIssue` reject export specs;
+ `Issue.Type.DATABASE_EXPORT`, `Task.Type.DATABASE_EXPORT`,
+ `Task.DatabaseDataExport`, `TaskRun.export_archive_status`, and
+ `Plan.ExportDataConfig` leave the wire. Goes through the pre-PR
+ breaking-change review.
+- Terraform provider does not manage plans/issues; `bytebase-action` creates
+ migration rollouts only — both unaffected.
+- In-flight export issues at upgrade time are deleted with the rest; there is
+ no cancel-and-keep state under full purge.
+- Downgrade: none (standard for migrations).
+
+## Testing & verification
+
+1. Migrator: `TestLatestVersion`; migration applies on a DB seeded with a DONE
+ export issue (plan + task + task_run + comment + archive row) and leaves no
+ orphans — verify FK integrity and that unrelated issues/plans survive.
+2. API rejection tests: `CreatePlan` with `export_data_config` →
+ InvalidArgument; `CreateIssue` type `DATABASE_EXPORT` → InvalidArgument;
+ `SQLService.Export` with a rollout name → InvalidArgument.
+3. Full gates: `gofmt`, `golangci-lint` (repeat until clean), backend build,
+ `buf format/lint/generate`, `pnpm fix/check/type-check/test`.
+4. Residual grep: `ExportArchive|DATABASE_EXPORT|exportDataConfig|DatabaseDataExport`
+ returns nothing outside historical migrations and the kept JIT/risk
+ surfaces.
+
+## Implementation order
+
+1. Branch off `origin/main` (currently on `release/3.21.0`)
+2. Migration + `LATEST.sql` + migrator test
+3. Proto edits + regen (backend and frontend generated code)
+4. Backend removal, compile-driven; backend tests
+5. Frontend removal, type-check-driven; frontend tests
+6. Verification gates + residual grep; PR with breaking-change checklist