The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.
TypeScript aialternativeauthdatabasedenoembeddingsexamplefirebasenextjsoauth2pgvectorpostgispostgrespostgresqlpostgrestrealtimesupabasevectorswebsockets
ci(api): verify generated types against production (#49993)
Jordi Enric84db103ebb6d
diff --git a/.agents/skills/api-types/SKILL.md b/.agents/skills/api-types/SKILL.mdnew file mode 100644index 00000000000..3d206497a2c--- /dev/null+++ b/.agents/skills/api-types/SKILL.md@@ -0,0 +1,27 @@+---+name: api-types+description: Maintain Supabase API types. Use when changing generated API type declarations, OpenAPI schemas, or investigating API type deployment drift.+---++# API types++The generated API contract has three specs: API v1, API v2, and Platform. Their committed outputs are `packages/api-types/types/api-v1.d.ts`, `packages/api-types/types/api-v2.d.ts`, and `packages/api-types/types/platform.d.ts`.++## Update types++1. Make the API/schema change and ensure it is deployed to production before relying on a type PR. Production is the merge-gate source of truth.+2. Run `pnpm api:codegen` against a running local API environment. It fetches all three local OpenAPI specs and updates the committed files.+3. Inspect and commit only the intended generated type changes.+4. Run `pnpm api:verify-types`. It fetches the three production OpenAPI specs, regenerates types with the repository tooling, and compares them with the committed files.++Complete the update only when `pnpm api:verify-types` passes after the production deployment is available.++## Interpret verification++- A pass means the committed generated declarations match all three production specs at the time of the check.+- A mismatch means production and the committed files differ. If the API is not deployed, deploy it and rerun the check. If production is correct, regenerate and review the changed files.+- A fetch failure means the production schema endpoint could not be read; fix or retry the endpoint before treating the result as a type mismatch.++## Pull requests++The `Verify production API types` CI job runs when `packages/api-types/types/**` changes and performs the same production comparison. It is currently observational, not a required merge check. The `api-deploy-required` label is informational only. Still run the local verifier before requesting review and treat a failed CI verification as production drift that must be resolved.diff --git a/.github/workflows/label_prs.yml b/.github/workflows/label_prs.ymlindex 9e9ac01756e..50af84c9457 100644--- a/.github/workflows/label_prs.yml+++ b/.github/workflows/label_prs.yml@@ -22,5 +22,5 @@ jobs:owner: context.repo.owner,repo: context.repo.repo,issue_number: context.issue.number,- body: 'The `api-deploy-required` label was auto-applied to this PR because it updates the API types. Ensure that the new or updated API, if any, is deployed on production before **removing the label** and merging this PR.',+ body: 'The `api-deploy-required` label was auto-applied to this PR because it updates the API types. The `Verify production API types` check reports whether the committed types match production; it is currently observational and does not block merging.',})diff --git a/.github/workflows/validate-pr.yml b/.github/workflows/validate-pr.ymlindex 2ee49d84c8e..1b10f526421 100644--- a/.github/workflows/validate-pr.yml+++ b/.github/workflows/validate-pr.yml@@ -17,12 +17,6 @@ jobs:echo "PR blocked: [tag: do not merge]"exit 1- - name: Tagged with 'api-deploy-required'- if: contains( github.event.pull_request.labels.*.name, 'api-deploy-required')- run: |- echo "PR blocked: [tag: api-deploy-required] — confirm the API is deployed in production, then remove the label."- exit 1-- name: All goodif: ${{ success() }}run: |diff --git a/.github/workflows/verify-production-api-types.yml b/.github/workflows/verify-production-api-types.ymlnew file mode 100644index 00000000000..bc99921ca89--- /dev/null+++ b/.github/workflows/verify-production-api-types.yml@@ -0,0 +1,50 @@+name: Verify production API types++on:+ pull_request:+ types: [opened, reopened, synchronize]++permissions:+ contents: read+ pull-requests: read++jobs:+ verify-production-api-types:+ runs-on: ubuntu-latest+ steps:+ - id: changes+ env:+ GH_TOKEN: ${{ github.token }}+ run: |+ if gh api "repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files" --paginate --jq '.[].filename' | grep -q '^packages/api-types/types/'; then+ echo "api_types_changed=true" >> "$GITHUB_OUTPUT"+ else+ echo "api_types_changed=false" >> "$GITHUB_OUTPUT"+ fi++ - name: Check out pull request+ if: steps.changes.outputs.api_types_changed == 'true'+ uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0+ with:+ persist-credentials: false++ - name: Install pnpm+ if: steps.changes.outputs.api_types_changed == 'true'+ uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271+ with:+ run_install: false++ - name: Set up Node.js+ if: steps.changes.outputs.api_types_changed == 'true'+ uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020+ with:+ node-version-file: '.nvmrc'+ cache: pnpm++ - name: Install API types dependencies+ if: steps.changes.outputs.api_types_changed == 'true'+ run: pnpm install --frozen-lockfile --filter=api-types...++ - name: Verify production API types+ if: steps.changes.outputs.api_types_changed == 'true'+ run: pnpm --filter=api-types run verify-production-typesdiff --git a/package.json b/package.jsonindex aa441729e00..e1f7d90f0d2 100644--- a/package.json+++ b/package.json@@ -50,6 +50,7 @@"setup:cli": "supabase start -x studio && supabase status --output json > keys.json && node scripts/generateLocalEnv.js","generate:types": "supabase gen types typescript --local > ./supabase/functions/common/database-types.ts","api:codegen": "cd packages/api-types && pnpm run codegen",+ "api:verify-types": "pnpm --filter=api-types run verify-production-types","knip": "knip","authorize-vercel-deploys": "tsx scripts/authorizeVercelDeploys.ts"},diff --git a/packages/api-types/package.json b/packages/api-types/package.jsonindex d975cab4de8..f230553959b 100644--- a/packages/api-types/package.json+++ b/packages/api-types/package.json@@ -7,7 +7,9 @@"scripts": {"preinstall": "npx only-allow pnpm","clean": "rimraf .turbo tsconfig.tsbuildinfo",- "codegen": "openapi-typescript --redocly ./redocly.yaml --alphabetize --default-non-nullable=false && prettier --cache --write types/*.d.ts"+ "codegen": "openapi-typescript --redocly ./redocly.yaml --alphabetize --default-non-nullable=false && prettier --cache --write types/*.d.ts",+ "verify-production-types": "node ./scripts/verify-production-types.mjs",+ "test": "node --test ./scripts/*.test.mjs"},"author": "","license": "MIT",diff --git a/packages/api-types/scripts/verify-production-types.mjs b/packages/api-types/scripts/verify-production-types.mjsnew file mode 100644index 00000000000..308de036e7c--- /dev/null+++ b/packages/api-types/scripts/verify-production-types.mjs@@ -0,0 +1,121 @@+import { execFile } from 'node:child_process'+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'+import { tmpdir } from 'node:os'+import { dirname, join } from 'node:path'+import { fileURLToPath } from 'node:url'+import { promisify } from 'node:util'++const run = promisify(execFile)+const packageDirectory = dirname(dirname(fileURLToPath(import.meta.url)))+const typesDirectory = process.env.API_TYPES_DIRECTORY ?? join(packageDirectory, 'types')+const platformApiUrl =+ process.env.PLATFORM_API_OPENAPI_URL ?? 'https://api.supabase.com/api/platform-json'+const fetchTimeout = 30_000++const specifications = [+ { name: 'api-v1', url: 'https://api.supabase.com/api/v1-json' },+ { name: 'api-v2', url: 'https://api.supabase.com/api/v2-json' },+ { name: 'platform', url: platformApiUrl },+]++export async function fetchOpenApiSpecifications(+ specifications,+ { fetchImpl = fetch, writeFileImpl = writeFile, temporaryDirectory, generatedTypesDirectory }+) {+ return Promise.all(+ specifications.map(async ({ name, url }) => {+ let response++ try {+ response = await fetchImpl(url, {+ signal: AbortSignal.timeout(fetchTimeout),+ })+ } catch {+ throw new Error(`Could not fetch ${name} OpenAPI specification from ${url}.`)+ }++ if (!response.ok) {+ throw new Error(+ `Could not fetch ${name} OpenAPI specification from ${url}: ${response.status}.`+ )+ }++ await writeFileImpl(join(temporaryDirectory, `${name}.json`), await response.text())++ return ` ${name}:\n root: ${join(temporaryDirectory, `${name}.json`)}\n x-openapi-ts:\n output: ${join(generatedTypesDirectory, `${name}.d.ts`)}`+ })+ )+}++export async function findMismatchedTypes(+ specifications,+ { readFileImpl = readFile, generatedTypesDirectory, typesDirectory }+) {+ const mismatches = await Promise.all(+ specifications.map(async ({ name }) => {+ const filename = `${name}.d.ts`+ const [generated, committed] = await Promise.all([+ readFileImpl(join(generatedTypesDirectory, filename), 'utf8'),+ readFileImpl(join(typesDirectory, filename), 'utf8'),+ ])++ return generated === committed ? undefined : filename+ })+ )++ return mismatches.filter((filename) => filename !== undefined)+}++export async function verifyProductionTypes() {+ const temporaryDirectory = await mkdtemp(join(tmpdir(), 'api-types-'))+ const generatedTypesDirectory = join(temporaryDirectory, 'types')++ try {+ await mkdir(generatedTypesDirectory)++ const config = await fetchOpenApiSpecifications(specifications, {+ temporaryDirectory,+ generatedTypesDirectory,+ })++ await writeFile(join(temporaryDirectory, 'redocly.yaml'), `apis:\n${config.join('\n')}`)+ await run(+ 'pnpm',+ [+ 'exec',+ 'openapi-typescript',+ '--redocly',+ join(temporaryDirectory, 'redocly.yaml'),+ '--alphabetize',+ '--default-non-nullable=false',+ ],+ { cwd: packageDirectory }+ )++ await run(+ 'pnpm',+ [+ 'exec',+ 'prettier',+ '--write',+ ...specifications.map(({ name }) => join(generatedTypesDirectory, `${name}.d.ts`)),+ ],+ { cwd: packageDirectory }+ )++ const changedTypes = await findMismatchedTypes(specifications, {+ generatedTypesDirectory,+ typesDirectory,+ })++ if (changedTypes.length > 0) {+ throw new Error(`Committed API types do not match production: ${changedTypes.join(', ')}`)+ }+ } finally {+ await rm(temporaryDirectory, { force: true, recursive: true })+ }+}++if (process.argv[1] === fileURLToPath(import.meta.url)) {+ await verifyProductionTypes()+}diff --git a/packages/api-types/scripts/verify-production-types.test.mjs b/packages/api-types/scripts/verify-production-types.test.mjsnew file mode 100644index 00000000000..f2c67b2cecc--- /dev/null+++ b/packages/api-types/scripts/verify-production-types.test.mjs@@ -0,0 +1,87 @@+import assert from 'node:assert/strict'+import test from 'node:test'++import { fetchOpenApiSpecifications, findMismatchedTypes } from './verify-production-types.mjs'++const specifications = [+ { name: 'api-v1', url: 'https://example.com/api/v1-json' },+ { name: 'platform', url: 'https://example.com/api/platform-json' },+]++test('writes fetched specifications and returns their Redocly configuration', async () => {+ const writes = []+ const config = await fetchOpenApiSpecifications(specifications, {+ temporaryDirectory: '/tmp/api-types',+ generatedTypesDirectory: '/tmp/api-types/types',+ fetchImpl: async (url) => ({+ ok: true,+ text: async () => `OpenAPI specification from ${url}`,+ }),+ writeFileImpl: async (path, content) => writes.push({ path, content }),+ })++ assert.deepEqual(writes, [+ {+ path: '/tmp/api-types/api-v1.json',+ content: 'OpenAPI specification from https://example.com/api/v1-json',+ },+ {+ path: '/tmp/api-types/platform.json',+ content: 'OpenAPI specification from https://example.com/api/platform-json',+ },+ ])+ assert.deepEqual(config, [+ ' api-v1:\n root: /tmp/api-types/api-v1.json\n x-openapi-ts:\n output: /tmp/api-types/types/api-v1.d.ts',+ ' platform:\n root: /tmp/api-types/platform.json\n x-openapi-ts:\n output: /tmp/api-types/types/platform.d.ts',+ ])+})++test('returns no mismatches when generated types match committed types', async () => {+ const mismatches = await findMismatchedTypes(specifications, {+ generatedTypesDirectory: '/generated',+ typesDirectory: '/committed',+ readFileImpl: async (path) =>+ path.includes('/generated/') ? 'generated type' : 'generated type',+ })++ assert.deepEqual(mismatches, [])+})++test('reports every generated type that differs from its committed counterpart', async () => {+ const mismatches = await findMismatchedTypes(specifications, {+ generatedTypesDirectory: '/generated',+ typesDirectory: '/committed',+ readFileImpl: async (path) => {+ if (path.endsWith('api-v1.d.ts')) return 'matching type'+ return path.includes('/generated/') ? 'new platform type' : 'committed platform type'+ },+ })++ assert.deepEqual(mismatches, ['platform.d.ts'])+})++test('rejects unsuccessful OpenAPI responses', async () => {+ await assert.rejects(+ fetchOpenApiSpecifications([specifications[0]], {+ temporaryDirectory: '/tmp/api-types',+ generatedTypesDirectory: '/tmp/api-types/types',+ fetchImpl: async () => ({ ok: false, status: 503 }),+ writeFileImpl: async () => assert.fail('does not write an unsuccessful response'),+ }),+ /Could not fetch api-v1 OpenAPI specification from https:\/\/example\.com\/api\/v1-json: 503\./+ )+})++test('rejects failed OpenAPI requests', async () => {+ await assert.rejects(+ fetchOpenApiSpecifications([specifications[0]], {+ temporaryDirectory: '/tmp/api-types',+ generatedTypesDirectory: '/tmp/api-types/types',+ fetchImpl: async () => {+ throw new Error('network unavailable')+ },+ writeFileImpl: async () => assert.fail('does not write a failed response'),+ }),+ /Could not fetch api-v1 OpenAPI specification from https:\/\/example\.com\/api\/v1-json\./+ )+})