Database governance built for humans and agents — controlling changes and access across every major database.
Go cicddata-governancedata-maskingdata-securitydatabase-accessdatabase-governancedevopsdevsecopsflywaygitopsliquibasemongodbmysqloraclepostgresqlschema-migrationssecuritysql-clientsqlservertidb
docs: expand releases when revalidating, drop the ref-table owner fallback
Dannya03abb279b82
diff --git a/docs/design/sheet-blob-scoping.md b/docs/design/sheet-blob-scoping.mdindex f30b86d798..f9deb81767 100644--- a/docs/design/sheet-blob-scoping.md+++ b/docs/design/sheet-blob-scoping.md@@ -117,17 +117,16 @@ CREATE INDEX idx_sheet_blob_ref_sha256 ON sheet_blob_ref(sha256);so project IDs are one global namespace across every workspace, and a project resolves to exactlyone workspace through `project.workspace`.-The gate and `HasSheets` both query `WHERE project = ? AND sha256 IN (…)`, which the-project-leading primary key serves directly. The secondary index exists for the two paths that-start from a hash with no project in hand: the owner lookup for a withheld statement-([naming the owner](sheet-history-on-database-transfer.md#naming-the-owner)) and the zero-ref-verification query in [Rollout](#rollout). PostgreSQL can only full-scan a `(project, sha256)` btree-for a `sha256`-only predicate, so both would degrade as the table grows.--Those hash-first queries are a deliberate exception to the composite-PK predicate rule in-`AGENTS.md`: their whole purpose is to discover *which* project holds a hash, so the scope column-cannot be supplied. They are made safe by joining through `project` and constraining to the caller's-workspace instead — never by omitting scope entirely.+The gate, `HasSheets` and the source-level missing-ref audit all query+`WHERE project = ? AND sha256 …`, which the project-leading primary key serves directly. The+secondary index exists for the one query that starts from a hash with no project in hand: the+zero-ref verification in [Rollout](#rollout), which PostgreSQL can only satisfy by full-scanning a+`(project, sha256)` btree. That is an operator query rather than a hot path, but it runs over the+whole table and a future `sheet_blob` GC would need the same access shape.++That query is a deliberate exception to the composite-PK predicate rule in `AGENTS.md`: it asks+which blobs have no ref at all, so there is no scope column to supply. It is an offline audit, not a+request-path read, which is what makes the exception acceptable — no request-path query omits scope.Migration slot `backend/migrator/migration/3.22/0005##scope_sheet_blob.sql`; `TestLatestVersion` in`backend/migrator/migrator_test.go` needs updating, and `LATEST.sql` mirrored.@@ -413,8 +412,22 @@ after enforcement, still have A's SQL parsed and summarized by plan checks or exdatabase of their own — through the front door, with the gate untouched. Plan-check advices quoteobject names and statement fragments, so this is a disclosure path as well as an execution one.-Each of the three revalidates the plan's sheets against the plan's project, using the same scoped-`HasSheets` as `CreatePlan`, and is shadow-gated on the same switch as everything else. Shadow mode+Each of the three revalidates the plan's sheets against the plan's project, and is shadow-gated on+the same switch as everything else.++**Revalidation must expand releases; reusing `CreatePlan`'s check is not enough.** `validateSpecs`+collects only the direct `ChangeDatabaseConfig.Sheet` hashes into `HasSheets`, and for a release it+verifies the project matches and the row exists — it never touches `release.Payload.Files[]`+(`backend/api/v1/plan_service.go:755-774`). The release runners then read every file's hash by bare+`GetSheetFull` (`backend/runner/taskrun/database_migrate_executor.go:543`, `:654`). So a release in+project B whose files reference project A's hash — creatable today, since+`validateAndSanitizeReleaseFiles` discards the project and `HasSheets` is deployment-wide — passes a+`CreatePlan`-shaped check with nothing to test, and the runner still executes A's SQL.++Revalidation therefore expands each referenced release and checks every `files[].sheetSha256`+against the plan's project, not just the spec's direct sheet. `validateSpecs` gains the same+expansion so the two agree: new releases cannot carry foreign hashes once `CreateRelease` uses+scoped `HasSheets`, but plans may reference releases created before that. Shadow modematters here more than elsewhere: these are *stored* objects, so a miss identifies a plan that willstop working at enforcement, and the operator needs that list while content is still being served.@@ -650,7 +663,9 @@ No schema needed; worth landing separately and first.gate.- **Stored-plan revalidation** — create a plan in project B referencing project A's hash while`HasSheets` is unscoped, then enforce, then assert each of `CreateRollout`, `RunPlanChecks` and- `BatchRunTasks` refuses it. Without this the runner exemption is a bypass: the runners read by bare+ `BatchRunTasks` refuses it. Run it twice: once with the hash on the spec directly, and once with+ the hash inside a referenced release's `files[]`, which is the shape a `CreatePlan`-style check+ does not look at. Without this the runner exemption is a bypass: the runners read by barehash, so anything these three admit is executed or summarized outside the gate.- **Query count** — a release with many files must issue a bounded number of queries. A correctnesstest passes just as happily against an N+1.diff --git a/docs/design/sheet-history-on-database-transfer.md b/docs/design/sheet-history-on-database-transfer.mdindex 1bd2a0fef0..8244645c4b 100644--- a/docs/design/sheet-history-on-database-transfer.md+++ b/docs/design/sheet-history-on-database-transfer.md@@ -127,10 +127,7 @@ resource name; `backend/store/changelog.go:163-168` already parses a project out1. The revision's `release`, else its `taskRun` — **corroborated by exactly the rule the backfilluses**, and resolving to a project in the caller's workspace.-2. `sheet_blob_ref` — which projects currently hold a ref for that hash, **joined through `project`- and constrained to the caller's workspace**, and only when that leaves exactly one candidate.- Useful when the revision carries no provenance, both fields being optional.-3. Anything else — report the owner as unknown, and never echo a project ID that failed either test.+2. Anything else — report the owner as unknown, and never echo a project ID that failed the test.**Provenance is a hint, not an assertion.** Both fields are stored verbatim from the caller:`convertRevision` writes `Release: revision.Release` and `TaskRun: revision.TaskRun` straight through@@ -151,7 +148,7 @@ provenance in it, and the weaker one would be the user-visible one.| Corroborated, project in the caller's workspace | Name it || Corroborated, project in another workspace | Owner unknown || Present but not corroborated — stale, purged, or fabricated | Owner unknown |-| Absent | Fall through to step 2 |+| Absent | Owner unknown |The response may still distinguish *why* it is unavailable — no provenance, provenance that cannotbe verified, an owner outside the caller's workspace — since a reason is not an identifier. What it@@ -164,31 +161,29 @@ that genuinely authored a sheet is correct regardless of which workspace the refsits in — the backfill's corroboration check is about attributing ownership, while this is about whata caller is allowed to be told.-Step 2 must not read the ref table unqualified. `sheet_blob_ref` has no workspace column, and-content-addressed dedup means one blob is shared by every project that authored identical SQL —-across tenants. Generated boilerplate makes that routine rather than rare: `getCreateDatabaseStatement`-emits `CREATE DATABASE %s;` for several engines, so a common statement will carry refs from many-workspaces. Naming an arbitrary one would leak a foreign tenant's project name from a design whose-purpose is closing cross-tenant leakage.+**`sheet_blob_ref` is not a fallback for authorship.** An earlier draft resolved a provenance-less+revision by finding which projects hold a ref for its hash and naming the one candidate in the+caller's workspace. That is not evidence about *this revision*: a ref proves only that some project+authored identical content. Content-addressed dedup makes identical content the normal case rather+than a coincidence — `getCreateDatabaseStatement` emits `CREATE DATABASE %s;` for several engines,+so exactly-one collisions on boilerplate are plausible rather than contrived, and the answer would+be a project that had nothing to do with the revision.-```sql-SELECT r.project-FROM sheet_blob_ref r-JOIN project p ON p.resource_id = r.project-WHERE r.sha256 = ? AND p.workspace = ?-```+Naming it would send the user to request access from a project that never authored the statement,+which is the misattribution this section already rejects for uncorroborated provenance. Applying the+corroboration standard to provenance and a weaker inference to the ref table would be the same+inconsistency in a different place.-Ambiguity resolves to unknown rather than to a guess. Two projects in the same workspace holding-refs for one hash is honest dedup, and neither is more the author than the other.+So there is no ref-table fallback. A revision names an owner when its own provenance corroborates+one, and reports unknown otherwise.A purged authoring project resolves to unknown, not to a name. `DeleteProject` hard-deletes the`release`, `task` and `plan` rows along with the project, so provenance naming it can no longer-corroborate, and `sheet_blob_ref WHERE project = A` is deleted too, so step 2 finds nothing either.-That is the correct outcome rather than a gap: the statement is permanently unavailable, there is-nobody left to ask, and the only thing withheld beyond the content is an ID that can no longer be-verified against any workspace.+corroborate. That is the correct outcome rather than a gap: the statement is permanently+unavailable, there is nobody left to ask, and the only thing withheld beyond the content is an ID+that can no longer be verified against any workspace.-Step 1 is attribution only and never implies access. A purged project holds no refs and cannot be+Provenance resolution is attribution only and never implies access. A purged project holds no refs and cannot begranted any, since `sheet_blob_ref.project` has a live foreign key, and the backfill skipsuncorroborated provenance for the same reason.@@ -236,7 +231,7 @@ corruption outweighs the modelling improvement.## Implementation- A field on the revision response carrying the withheld-content reason and the owning project,- populated by the resolution order above — provenance first, `sheet_blob_ref` second, unknown last.+ populated by the resolution above — corroborated provenance, else unknown.- `convertToRevision` formats `Revision.sheet` from the resolved owner rather than from`database.ProjectID` (`backend/api/v1/revision_service.go:304`), and emits no sheet name when theowner does not resolve. Leaving it built from the current project would keep exactly the false@@ -266,8 +261,9 @@ Tests:is complete under B while the statement is withheld with A named. Archiving a *project* instance isthe wrong path — `backend/store/instance.go:434` rejects it — so a test written that way would passvacuously without exercising any transfer.-- A revision carrying neither `release` nor `taskRun` falls through to `sheet_blob_ref`, and to- unknown when that is empty too.+- A revision carrying neither `release` nor `taskRun` reports unknown, even when exactly one project+ in the caller's workspace holds a ref for that hash — that ref proves identical content, not+ authorship.- Cross-tenant leak guard, both paths: (a) seed identical SQL in two workspaces so one blob carriesrefs from both, then request a withheld statement from one and assert the response never names theother workspace's project; (b) seed a revision whose `release` provenance names a project in@@ -275,8 +271,6 @@ Tests:echoing it; (c) seed a revision naming a real project in the caller's *own* workspace that neverauthored the sheet, and assert the response reports unknown rather than misattributing it. Generated `CREATE DATABASE` boilerplate makes this collision realistic, so itis a regression test rather than a contrived one.-- Ambiguity: two projects in the caller's own workspace holding refs for one hash resolves to- unknown, not to whichever row comes back first.- Sheet name: after a transfer, `Revision.sheet` names the resolved owner and resolves to thatproject's readable sheet for a caller who holds rights there; when the owner is unknown, no sheetname is emitted rather than one pointing at the current project.