Database governance built for humans and agents — controlling changes and access across every major database.
Go cicddata-governancedata-maskingdata-securitydatabase-accessdatabase-governancedevopsdevsecopsflywaygitopsliquibasemongodbmysqloraclepostgresqlschema-migrationssecuritysql-clientsqlservertidb
docs: drop the db.project fallback for revision refs
Danny644aa11e21e3
diff --git a/docs/design/sheet-blob-scoping.md b/docs/design/sheet-blob-scoping.mdindex 93e445960a..a668f1cc17 100644--- a/docs/design/sheet-blob-scoping.md+++ b/docs/design/sheet-blob-scoping.md@@ -145,7 +145,7 @@ inside its message (not always top-level). `protojson` camelCases keys.| `task` | `payload` | `sheetSha256` | `task.project` || `release` | `payload` | `files[].sheetSha256` | `release.project` || `plan_check_run` | `result` | `results[].sheetSha256` | `plan_check_run.project` |-| `revision` | `payload` | `sheetSha256` | `release` → else `taskRun` → else `db.project` (see below) |+| `revision` | `payload` | `sheetSha256` | corroborated `release` → else corroborated `taskRun` → else **no ref** (see below) |`plan_check_run` punishes assumption: the column is `result`, not `payload`, and the hash sits on`PlanCheckRunResult.Result` inside the repeated `results` array.@@ -164,46 +164,56 @@ Derive the authoring project from the revision's own provenance instead. `Revisiname. Parse it out, preferring `release` and falling back to `taskRun`; `backend/store/changelog.go:163-168`already does exactly this with `regexp_match(payload->>'taskRun', 'projects/([^/]+)/')`.-**Provenance is usable only if it is corroborated**: the `release` or `task_run` row it names must-still exist under the project it names. Existence of the *project* is not enough. The disposition is-three-way, not a fallback chain, and conflating the middle case with the last one is the trap:+**A revision produces a ref only when its provenance is corroborated**: the `release` or `task_run`+row it names must still exist under the project it names. There is no fallback — not to+`db.project`, not to anything. Revisions with absent or uncorroborated provenance get no ref, and+their content is unreadable until an operator grants it deliberately.| Provenance | Action ||---|---|| Corroborated — the named `release` or `task_run` row still exists under that project | Insert `(that project, sha)` |-| Present but not corroborated | **Insert nothing.** The string stays for attribution |-| Absent — both fields empty | Fall back to `db.project`, and add to the review list |--**Uncorroborated provenance must not fall through to `db.project`.** `revision` is the only source-that survives a purge of its authoring project: `plan`, `task`, `release` and `plan_check_run` rows-are all deleted with the project (`backend/store/project.go:578`, `:641`, `:651`, `:671`), but a+| Present but not corroborated | No ref |+| Absent — both fields empty | No ref |++**There is no `db.project` fallback because shadow mode cannot see an over-grant.** Falling back+would be correct for every database that never moved, which is most of them, and wrong only for+transferred ones — a tempting trade until you notice the asymmetry in how the two errors surface.+Under-granting produces a shadow miss: a log line, during a full release in which content is still+returned, with the requesting project right there in the log. Over-granting produces nothing at all,+because the ref exists and the gate is satisfied. A guess that fails loudly and correctably beats a+guess that fails silently and permanently.++So the operational path for these rows is: no ref, shadow misses during the enforcement-deferred+release, operator reviews the log, and tops up `(requesting project, sha)` for the ones they confirm.+That is an informed grant rather than an inferred one, and the operator has context the migration+does not — whether a database was transferred, and who should see its history.++**Corroboration is checked on the referenced row, not on the project.** `DeleteProject` hard-deletes+the `project` row and `CreateProject` accepts a caller-supplied `ProjectId`+(`backend/api/v1/project_service.go:210`), so a purged ID can be reused; a surviving revision naming+the *old* project would otherwise resolve to an unrelated *new* project of the same name and grant it+the purged project's SQL. Checking the referenced row distinguishes the cases, because ordinary+release deletion is soft (`backend/store/release.go:223`) and leaves the row, while purge+hard-deletes it along with `plan`, `task` and `plan_check_run`+(`backend/store/project.go:578`, `:641`, `:651`, `:671`).++`revision` is the only source that survives a purge of its authoring project at all: aworkspace-instance database is reassigned to the default project and keeps its revisions, whose-`release` and `taskRun` strings still name the deleted project. Inserting that ref would violate-`sheet_blob_ref`'s foreign key and abort the migration; joining only to live projects and then-falling back would hand the default project SQL the purged project authored — the grant this design-refuses.--**Checking that the project exists is not sufficient**, which is why the guard is on the referenced-row. `DeleteProject` hard-deletes the `project` row and `CreateProject` accepts a caller-supplied-`ProjectId` (`backend/api/v1/project_service.go:210`), so a purged ID can be reused. A surviving-revision naming the *old* project would then resolve to an unrelated *new* project of the same name-and grant it the purged project's SQL — a worse outcome than the foreign-key abort, because it is-silent. Corroborating the referenced row distinguishes the cases: ordinary release deletion is soft-(`backend/store/release.go:223`), so the row persists, while purge hard-deletes it.+`release` and `taskRun` strings still name the deleted project.Residual risk: a reused project ID whose new releases happen to collide with the old release or taskIDs would still corroborate. Release IDs are deterministic (`train` plus iteration) and task IDsrestart per project, so this is possible rather than impossible. It is not detectable from the data,-so it goes on the review list alongside the other ambiguous rows rather than being silently trusted.+so it goes on the review list rather than being silently trusted.Their content then becomes unreadable, which is the intended outcome: the authoring project is gone,so there is nobody left to ask. The immutable `release`/`taskRun` string still identifies it, so thewithheld-statement response can name it as purged — see[naming the owner](sheet-history-on-database-transfer.md#naming-the-owner).-A revision with no provenance at all falls back to `db.project`, correct for every database that-never moved, which is the common case. Those rows go on the review list in [Rollout](#rollout)-rather than being trusted silently.+A revision with no provenance at all is treated the same way: no ref, surfaced as a shadow miss, and+granted only if an operator confirms it. Both populations appear together on the review list in+[Rollout](#rollout).Two tables that look like sources are not. `ChangelogPayload` carries only `task_run` and`git_commit`, and the v1 `Changelog` message exposes no statement or sheet field. `issue_comment`@@ -396,32 +406,39 @@ A long list means cross-project references are already widespread and the plan nsince the backfill would make them permanent. Capture it at a known-good point: this design ispublic, including the rule that turns a reference into an ownership edge.-**Review ambiguous revision provenance too.** Two populations, both of which the backfill had to-guess at. Rows with no provenance fell back to `db.project`, so a database transferred before the-migration would have granted its destination project a source project's SQL:+**Review revisions that produced no ref.** These are the rows whose statements are now unreadable —+absent provenance, or provenance the backfill could not corroborate. They will appear as shadow+misses during the deferred-enforcement release; this query is the same population identified up+front, so a top-up can be prepared before the log accumulates.-```sql-SELECT r.instance, r.db_name, d.project-FROM revision r JOIN db d ON d.instance = r.instance AND d.name = r.db_name-WHERE r.payload->>'sheetSha256' IS NOT NULL- AND COALESCE(r.payload->>'release', '') = ''- AND COALESCE(r.payload->>'taskRun', '') = '';-```--Rows whose provenance named a project that no longer corroborates it produced no ref, so their-content is now unreadable. A large count here means either widespread project purges or reused-project IDs, and is worth understanding before enforcing:+The test must repeat the corroboration check rather than asking whether the hash has any ref at all.+The design expects hashes to be shared, so a revision whose own provenance failed may still have a+ref from an unrelated project that authored identical SQL — and a ref-existence proxy would silently+drop exactly the rows this list exists to surface.```sqlSELECT r.instance, r.db_name,- (regexp_match(r.payload->>'release', 'projects/([^/]+)/'))[1] AS named_project+ (regexp_match(COALESCE(NULLIF(r.payload->>'release', ''),+ r.payload->>'taskRun'), 'projects/([^/]+)/'))[1] AS named_projectFROM revision rWHERE r.payload->>'sheetSha256' IS NOT NULL- AND COALESCE(r.payload->>'release', '') <> ''- AND NOT EXISTS (SELECT 1 FROM sheet_blob_ref b- WHERE b.sha256 = decode(r.payload->>'sheetSha256', 'hex'));+ AND NOT EXISTS (+ -- the corroboration test from the backfill, repeated verbatim+ SELECT 1 FROM release rel+ WHERE rel.project = (regexp_match(r.payload->>'release', 'projects/([^/]+)/'))[1]+ AND rel.release_id = (regexp_match(r.payload->>'release', 'releases/([^/]+)'))[1]+ )+ AND NOT EXISTS (+ SELECT 1 FROM task t+ WHERE t.project = (regexp_match(r.payload->>'taskRun', 'projects/([^/]+)/'))[1]+ AND t.id = (regexp_match(r.payload->>'taskRun', 'tasks/(\d+)/'))[1]::int+ );```+A large count means either widespread project purges, reused project IDs, or a population of+revisions created without provenance — each worth understanding before enforcing, since all of them+end in unreadable statements.+## Transaction lock ordering`sheet_blob_ref` is a project-owned branch and takes a position in the canonical sibling order in