Skip to content

Open-source manufacturing ERP, MES and QMS. Quoting, MRP, inventory, shop floor, quality and lot/serial traceability on one Postgres schema, with a REST API and MCP server. Self-host or use Carbon Cloud.

fix(ci): set the Inngest event key with set_inngest_event_config, with a retry (#1829)

Sid9e8170c2ae46
diff --git a/.claude/rules/event-system.md b/.claude/rules/event-system.md
index dd8257b99a..48549c90c5 100644
--- a/.claude/rules/event-system.md
+++ b/.claude/rules/event-system.md
@@ -31,7 +31,7 @@ pg_cron 'event-queue-sweeper' (* * * * *, in-DB): if visible messages exist →
The wake path (`20260721184852_event-queue-wake.sql`, rewired by `20261002170250_send-inngest-events-from-postgres.sql` and the managed files in `packages/database/src/event-system/functions/`) — the helpers live in the internal `util` schema, NOT `public`:
- `util.send_inngest_event(name, data)` — SECURITY DEFINER; `net.http_post`s `{ name, data }` to the Vault secret `inngest_event_url` (`<base>/e/<eventKey>`). Error-swallowed and no-ops when the secret is unset — OLTP writes never fail on push failure. pg_net queues the request transactionally, so the event fires only after commit. Postgres calls no edge function.
- `util.wake_event_queue()` — sends `carbon/event-queue.process` through it.
-- `public.set_inngest_event_url(url)` writes the secret (refuses `anon`/`authenticated`). The URL carries the event key, so it lives in Vault, not in `config` (which has a SELECT policy). A no-op when the URL is unchanged (`20261003134512`). The ERP keeps the KEY in it current on boot in production (`apps/erp/app/utils/inngest-event-url.server.ts` → `public.set_inngest_event_config(key, base)`, `20261003142107`): it creates the URL from its own `INNGEST_BASE_URL` when none is stored, and otherwise rewrites only the `<eventKey>` segment — a stored address is never changed, since the database may reach Inngest somewhere the app does not. So a deployment needs no separate step. Set in full also by `ci/src/migrations.ts` on every deploy (workspace `inngest_event_key` / `inngest_base_url`), by `packages/database/src/seed.ts` (`resolveInngestEventUrl`), locally by `ensureConfigRow` in `packages/dev/src/services/migrations.ts` (`http://inngest:8288/e/NO_EVENT_KEY_SET`) and by `scripts/restore-database.sh` after it clears Vault on a local restore, and on self-host by `contrib/deploying/simple-docker-caddy/deploy.sh migrate` (`set_inngest_event_url`, reading the key from the erp task). In CI a workspace missing `inngest_event_key` or the service role key logs an error and fails the run, after seeding and scripts, which do not depend on it.
+- `public.set_inngest_event_url(url)` writes the secret (refuses `anon`/`authenticated`). The URL carries the event key, so it lives in Vault, not in `config` (which has a SELECT policy). A no-op when the URL is unchanged (`20261003134512`). The ERP keeps the KEY in it current on boot in production (`apps/erp/app/utils/inngest-event-url.server.ts` → `public.set_inngest_event_config(key, base)`, `20261003142107`): it creates the URL from its own `INNGEST_BASE_URL` when none is stored, and otherwise rewrites only the `<eventKey>` segment — a stored address is never changed, since the database may reach Inngest somewhere the app does not. So a deployment needs no separate step. `ci/src/migrations.ts` calls the same function on every deploy for a workspace with an `inngest_event_key` (retried while PostgREST reloads its schema), so a changed `inngest_base_url` there needs `set_inngest_event_url` by hand. Set in full by `packages/database/src/seed.ts` (`resolveInngestEventUrl`), locally by `ensureConfigRow` in `packages/dev/src/services/migrations.ts` (`http://inngest:8288/e/NO_EVENT_KEY_SET`) and by `scripts/restore-database.sh` after it clears Vault on a local restore, and on self-host by `contrib/deploying/simple-docker-caddy/deploy.sh migrate` (`set_inngest_event_url`, reading the key from the erp task). In CI a workspace missing `inngest_event_key` or the service role key logs an error and fails the run, after seeding and scripts, which do not depend on it.
- `dispatch_event_batch()` calls `util.wake_event_queue()` at most **once per transaction** via the txn-local GUC `carbon.event_wake_sent` (`set_config(..., true)`).
- `util.sweep_event_queue()` — pg_cron job `event-queue-sweeper` re-wakes every minute while *visible* messages (`vt <= clock_timestamp()`) sit in `pgmq.q_event_system`.
- **Why `util`, not `public`:** a public function is auto-exposed as a PostgREST RPC, and a non-superuser reference to a function that transitively calls `net.http_post` segfaults the backend (pg_net 0.20 / PG15) — a remote-DoS surface a `REVOKE` can't close because the crash precedes the privilege check. anon/authenticated have no `USAGE` on `util`, so the API can't reach it. The trigger and pg_cron call it as the owner (superuser), where the pg_net path is safe.
diff --git a/ci/src/migrations.ts b/ci/src/migrations.ts
index cbd01a4b8d..12e82248f5 100644
--- a/ci/src/migrations.ts
+++ b/ci/src/migrations.ts
@@ -221,22 +221,32 @@ async function migrate(): Promise<void> {
await $$`supabase functions deploy`;
}
- // Postgres posts its Inngest events (util.send_inngest_event) to this
- // URL. The app also writes it on boot from its own INNGEST_EVENT_KEY, so
- // a workspace with no key here is wired by its first instance instead.
+ // Postgres posts its Inngest events (util.send_inngest_event) to a URL in
+ // its Vault. This sets the event key in it, and the address only when
+ // none is stored yet: a changed `inngest_base_url` here needs
+ // set_inngest_event_url run by hand. The app does the same on boot from
+ // its own INNGEST_EVENT_KEY, so a workspace with no key here is wired by
+ // its first instance instead.
if (!workspace.inngest_event_key || !service_role_key) {
console.log(
`⏭️ 📨 ${workspace.id} has no Inngest event key here: the app sets the database's event URL on boot`
);
} else {
- const eventUrl = new URL(
- `e/${workspace.inngest_event_key}`,
- workspace.inngest_base_url ?? "https://inn.gs/"
- ).href;
- const { error: eventUrlError } = await createClient(
- database_url,
- service_role_key
- ).rpc("set_inngest_event_url", { p_url: eventUrl });
+ // PostgREST reloads its schema a few seconds after a migration, so the
+ // function this run just created is not callable at once.
+ const eventUrlClient = createClient(database_url, service_role_key);
+ let eventUrlError: { message: string } | null = null;
+ for (let attempt = 1; attempt <= 6; attempt++) {
+ ({ error: eventUrlError } = await eventUrlClient.rpc(
+ "set_inngest_event_config",
+ {
+ p_key: workspace.inngest_event_key,
+ p_base_url: workspace.inngest_base_url ?? "https://inn.gs/",
+ }
+ ));
+ if (!eventUrlError) break;
+ await new Promise((resolve) => setTimeout(resolve, 5_000));
+ }
if (eventUrlError) {
console.error(
`🔴 📨 Failed to set the Inngest event URL for ${workspace.id}: ${eventUrlError.message}`
diff --git a/packages/utils/src/async.test.ts b/packages/utils/src/async.test.ts
index 8c2e056d24..3be805b38b 100644
--- a/packages/utils/src/async.test.ts
+++ b/packages/utils/src/async.test.ts
@@ -35,20 +35,25 @@ describe("async.map", () => {
it("rejects with the first failure and starts nothing after it", async () => {
const started: number[] = [];
+ // Item 1 is held open until item 2 has failed: no timers to race.
+ let finishFirst!: () => void;
+ const firstHeld = new Promise<void>((resolve) => {
+ finishFirst = resolve;
+ });
await expect(
async.map(
[1, 2, 3, 4, 5],
async (n) => {
started.push(n);
- // Item 1 is still running when item 2 fails.
- await tick(n === 1 ? 20 : 5);
+ if (n === 1) await firstHeld;
if (n === 2) throw new Error("two failed");
return n;
},
{ concurrency: 2 }
)
).rejects.toThrow("two failed");
- await tick(30);
+ finishFirst();
+ await tick(5);
expect(started).toEqual([1, 2]);
});