🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
ci: fail PRs when served OpenAPI specs drift from Fern (#16749)
Niklas Semmler6d96b32f15d0
diff --git a/.agents/AGENTS.md b/.agents/AGENTS.mdindex 1fa9333cb3..da853d6c75 100644--- a/.agents/AGENTS.md+++ b/.agents/AGENTS.md@@ -157,7 +157,8 @@ langfuse/regressions.- Public API contracts in `web/src/pages/api/public/**`,`web/src/features/public-api/types/**`, or `fern/apis/**`: `pnpm run lint`,- targeted server API tests, and Fern update/regeneration.+ targeted server API tests, Fern update/regeneration, and+ `pnpm run openapi:check`.- Cross-package refactors: `pnpm run lint`, `pnpm run typecheck`, and targetedtests for impacted packages.- Client-bundle soundness: CI scans every prod web builddiff --git a/.agents/skills/backend-dev-guidelines/references/routing-and-controllers.md b/.agents/skills/backend-dev-guidelines/references/routing-and-controllers.mdindex 60c7c290df..33fc975378 100644--- a/.agents/skills/backend-dev-guidelines/references/routing-and-controllers.md+++ b/.agents/skills/backend-dev-guidelines/references/routing-and-controllers.md@@ -389,7 +389,10 @@ packages/shared/src/features/<domain>/interfaces/api/When modifying public API types in `web/src/features/public-api/types/` or under`packages/shared/src/features/<domain>/interfaces/api/`, update the matching-Fern API definitions in `fern/apis/server/definition/`.+Fern API definitions in `fern/apis/server/definition/`. After editing Fern+sources, run `pnpm run openapi:export` and commit `web/public/generated/**`.+PRs that touch `fern/**` fail CI when those served specs drift+(`pnpm run openapi:check`).**Zod to Fern Type Mapping:**diff --git a/.github/workflows/openapi-export-check.yml b/.github/workflows/openapi-export-check.ymlnew file mode 100644index 0000000000..2ac55c8052--- /dev/null+++ b/.github/workflows/openapi-export-check.yml@@ -0,0 +1,61 @@+# Re-export the served OpenAPI specs and fail when web/public/generated/**+# drifts from fern/. `fern-api export` is local (no FERN_TOKEN), so this can+# run on fork PRs. `sdk-api-spec.yml` still owns `fern-api generate` + SDK+# sync and stays post-merge because that path needs Fern credentials.+name: OpenAPI Export Check++on:+ pull_request:+ paths:+ - "fern/**"+ - "web/public/generated/**"+ - "web/scripts/openapi/**"+ - "scripts/openapi/**"+ - ".github/workflows/openapi-export-check.yml"+ push:+ branches:+ - main+ paths:+ - "fern/**"+ - "web/public/generated/**"+ - "web/scripts/openapi/**"+ - "scripts/openapi/**"+ - ".github/workflows/openapi-export-check.yml"+ workflow_dispatch:++concurrency:+ group: openapi-export-check-${{ github.ref }}+ cancel-in-progress: true++permissions:+ contents: read++jobs:+ check-openapi-export:+ runs-on: blacksmith-4vcpu-ubuntu-2404+ timeout-minutes: 15+ steps:+ - name: Checkout repository+ uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ with:+ persist-credentials: false++ - name: Assert helper rejects drifted specs+ run: bash scripts/openapi/assert-generated-current.test.sh++ - name: Install pnpm+ uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9+ with:+ version: 11.22.0++ - name: Setup Node.js+ uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0+ with:+ node-version: "24"+ package-manager-cache: false++ - name: Install dependencies+ run: pnpm install --frozen-lockfile++ - name: Export OpenAPI specs and fail on drift+ run: pnpm run openapi:checkdiff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 59d5b0d17a..2cb0d3e664 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -551,6 +551,8 @@ To export the respective `openapi.yml` files which power the online API referencpnpm run openapi:export```+Commit the updated files under `web/public/generated/`. CI re-runs this export on PRs that touch `fern/**` or the served specs and fails if they drift (`pnpm run openapi:check`).+This command also syncs standard OpenAPI `deprecated` flags and `**Deprecated:** …` description notices from the endpoint `availability` metadata in the Fern definitions.To generate the server SDKs, run:diff --git a/package.json b/package.jsonindex 7db9875d2c..20cecd87a8 100644--- a/package.json+++ b/package.json@@ -39,6 +39,7 @@"format:check": "prettier --check \"**/*.{js,jsx,ts,tsx,css}\" --experimental-cli","playwright:install": "pnpm --filter web exec playwright install chromium","openapi:export": "npx fern-api export --api server web/public/generated/api/openapi.yml && pnpm --filter web exec tsx scripts/openapi/postprocess-openapi.ts && npx fern-api export --api client web/public/generated/api-client/openapi.yml && npx fern-api export --api organizations web/public/generated/organizations-api/openapi.yml",+ "openapi:check": "pnpm run openapi:export && bash scripts/openapi/assert-generated-current.sh","storybook": "pnpm --filter web run storybook","test": "turbo run test","release": "bash ./scripts/release-preflight.sh && dotenv -e ../.env -- release-it",diff --git a/scripts/openapi/assert-generated-current.sh b/scripts/openapi/assert-generated-current.shnew file mode 100755index 0000000000..c4cc1ab9c6--- /dev/null+++ b/scripts/openapi/assert-generated-current.sh@@ -0,0 +1,63 @@+#!/usr/bin/env bash+# Fail when the served OpenAPI specs under web/public/generated/ differ from+# the git tree. CI runs this after `pnpm run openapi:export` so a forgotten+# re-export becomes a blocking, self-explaining failure.++set -euo pipefail++if ! git rev-parse --is-inside-work-tree >/dev/null 2>&1; then+ echo "Must run inside a git work tree." >&2+ exit 1+fi++root="$(git rev-parse --show-toplevel)"+cd "$root"++generated_root="web/public/generated"+spec_paths=(+ "$generated_root/api/openapi.yml"+ "$generated_root/api-client/openapi.yml"+ "$generated_root/organizations-api/openapi.yml"+)++missing=0+for spec_path in "${spec_paths[@]}"; do+ if [ ! -f "$spec_path" ]; then+ echo "Missing served OpenAPI spec: $spec_path" >&2+ missing=1+ fi+done+if [ "$missing" -ne 0 ]; then+ echo "Re-export with: pnpm run openapi:export" >&2+ if [ -n "${GITHUB_ACTIONS:-}" ]; then+ echo "::error::Served OpenAPI spec files are missing under ${generated_root}/. Run: pnpm run openapi:export"+ fi+ exit 1+fi++status="$(git status --porcelain -- "$generated_root")"+if [ -z "$status" ]; then+ echo "Served OpenAPI specs match their Fern sources."+ exit 0+fi++{+ echo+ echo "Served OpenAPI specs are out of date with their Fern sources."+ echo "These files are what customers fetch at /generated/.../openapi.yml."+ echo+ echo "Re-export and commit the result:"+ echo+ echo " pnpm run openapi:export"+ echo+ echo "Changed files:"+ git status --porcelain -- "$generated_root"+ echo+ git diff --stat -- "$generated_root"+} >&2++if [ -n "${GITHUB_ACTIONS:-}" ]; then+ echo "::error::Served OpenAPI specs drifted from fern/. Run: pnpm run openapi:export"+fi++exit 1diff --git a/scripts/openapi/assert-generated-current.test.sh b/scripts/openapi/assert-generated-current.test.shnew file mode 100755index 0000000000..40b5cb0fae--- /dev/null+++ b/scripts/openapi/assert-generated-current.test.sh@@ -0,0 +1,47 @@+#!/usr/bin/env bash+# Proves the served-spec drift assertion fails on a dirty generated OpenAPI+# file and passes when the three tracked artifacts match HEAD.++set -euo pipefail++repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"+script="$repo_root/scripts/openapi/assert-generated-current.sh"+tmpdir="$(mktemp -d)"++cleanup() {+ rm -rf "$tmpdir"+}+trap cleanup EXIT++cd "$tmpdir"+git init -q+git config user.name "openapi-check-test"+git config user.email "[email protected]"++mkdir -p \+ web/public/generated/api \+ web/public/generated/api-client \+ web/public/generated/organizations-api++printf 'server: spec\n' > web/public/generated/api/openapi.yml+printf 'client: spec\n' > web/public/generated/api-client/openapi.yml+printf 'orgs: spec\n' > web/public/generated/organizations-api/openapi.yml++git add web/public/generated+git commit -q -m "seed specs"++"$script"++printf 'stale\n' >> web/public/generated/api/openapi.yml+if "$script"; then+ echo "expected assert-generated-current.sh to fail on a dirty spec" >&2+ exit 1+fi++rm web/public/generated/api-client/openapi.yml+if "$script"; then+ echo "expected assert-generated-current.sh to fail when a spec is missing" >&2+ exit 1+fi++echo "assert-generated-current.sh: ok"diff --git a/web/src/features/README.md b/web/src/features/README.mdindex 9cdc152648..170aa4185f 100644--- a/web/src/features/README.md+++ b/web/src/features/README.md@@ -20,7 +20,7 @@ TestingAPI Reference- Add to `fern` including `docs` attributes-- Export with `pnpm run openapi:export`, then commit the changes to the API reference+- Export with `pnpm run openapi:export`, then commit the changes to the API reference. CI fails if the served specs under `web/public/generated/` drift from `fern/`.SDKs