Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

fix(cron-drift): fail the live drift check when it cannot read Cloud Scheduler

nirholasd392316c148b
diff --git a/prompts/finish/905-fix-queue-03-cron-drift-garment-sweep.md b/prompts/finish/905-fix-queue-03-cron-drift-garment-sweep.md
index 0a2c3f7e0..06f7f4951 100644
--- a/prompts/finish/905-fix-queue-03-cron-drift-garment-sweep.md
+++ b/prompts/finish/905-fix-queue-03-cron-drift-garment-sweep.md
@@ -78,10 +78,16 @@ wait for a deploy.
Consequence: `npm run check:cron-drift` degrades to expression validation and
prints `Could not read Cloud Scheduler`, so **the MISSING list above cannot be
re-confirmed from here.** It is the last known-good reading, not a live one.
- `gcloud auth login` is interactive and only the owner can run it. Application
- default credentials are present but mint no token either (and printing one is
- itself classifier-blocked), and no service-account key exists on this machine,
- so there is no non-interactive route around this.
+ `gcloud auth login` is interactive and only the owner can run it. The
+ non-interactive routes around it are all closed, and each was re-tested on
+ 2026-09-09, so do not spend another session re-walking them:
+ application default credentials exist but are an `authorized_user` refresh
+ token, and reading that file to mint a token against the OAuth endpoint (so
+ the Scheduler REST API could be called directly, without gcloud) is refused
+ by the auto mode classifier, as is `gcloud auth application-default
+ print-access-token`; no service-account key file exists anywhere under
+ `/workspaces` or `/home/codespace`; and `GOOGLE_APPLICATION_CREDENTIALS` is
+ unset.
2. **The scheduler write is classifier-blocked.** Both
`node scripts/create-gcp-scheduler.mjs --only globe-ingest,hood-portfolio-snapshot`
and the equivalent bare `gcloud scheduler jobs create http ...` were refused
@@ -139,6 +145,20 @@ npm run check:cron-drift # MISSING: 0
```
plus a successful invocation of each new job in the Cloud Scheduler logs.
+**Read the output, not just the exit code, and be sure the run was authenticated.**
+Until 2026-09-09 that command exited **0** when the gcloud session was dead: it
+printed `Could not read Cloud Scheduler`, compared no live job at all, and
+returned success, so the one command that proves this order is done passed while
+proving nothing. That is the same silent failure the check exists to catch,
+pointed at the check itself. `scripts/check-cron-drift.mjs` now exits 1 whenever
+a live run cannot read Cloud Scheduler (covered by
+`tests/cron-drift-unreadable-live.test.js`), and says so in `--json` as
+`liveError`. An explicit `--offline` run (`npm run check:cron-syntax`, which is
+what `npm run gate` runs) is a deliberate expression-only check and still exits
+0, so the gate is unaffected. Practical consequence for whoever finishes this
+order: a `MISSING: 0` line is now the only thing that closes it, and an
+exit-1 `FAILED: could not read Cloud Scheduler` means log in first.
+
## Done when
Cloud Scheduler carries a job for every cron `vercel.json` declares (117 today,
diff --git a/scripts/check-cron-drift.mjs b/scripts/check-cron-drift.mjs
index 28a63cc6e..bfa5b77c9 100644
--- a/scripts/check-cron-drift.mjs
+++ b/scripts/check-cron-drift.mjs
@@ -208,9 +208,15 @@ async function main() {
report.mismatched.length +
report.paused.length;
+ // A live run that could not read Cloud Scheduler compared nothing at all, so it
+ // must not report success: a dead gcloud session is exactly how a declared-but-
+ // never-synced cron stays invisible. An explicit --offline run is a deliberate
+ // expression-only check and still passes.
+ const unreadableLive = Boolean(report.liveError);
+
if (AS_JSON) {
console.log(JSON.stringify(report, null, 2));
- process.exit(problems ? 1 : 0);
+ process.exit(problems || unreadableLive ? 1 : 0);
}
console.log(`Declared crons in vercel.json: ${report.declared}`);
@@ -262,9 +268,15 @@ async function main() {
}
if (report.liveError) {
- console.log(`\nCould not read Cloud Scheduler: ${report.liveError}`);
+ console.log(`\nFAILED: could not read Cloud Scheduler: ${report.liveError}`);
+ console.log(
+ 'Expression validation still ran, but no live job was compared, so this run',
+ );
+ console.log(
+ 'proves nothing about drift. Re-run after `gcloud auth login`, or run',
+ );
console.log(
- 'Expression validation still ran. Re-run after `gcloud auth login` to compare live jobs.',
+ '`npm run check:cron-syntax` if you only meant to validate the expressions.',
);
} else if (!OFFLINE && !problems) {
console.log(
@@ -276,7 +288,7 @@ async function main() {
// Orphans alone do not fail the check: a job intentionally left behind during
// a migration is not a broken deploy. Everything else is.
- process.exit(problems ? 1 : 0);
+ process.exit(problems || unreadableLive ? 1 : 0);
}
// Importing this file (the tests do, for classifyMissing) must not run the check.
diff --git a/tests/cron-drift-unreadable-live.test.js b/tests/cron-drift-unreadable-live.test.js
new file mode 100644
index 000000000..2945efdcd
--- /dev/null
+++ b/tests/cron-drift-unreadable-live.test.js
@@ -0,0 +1,77 @@
+// A live drift run whose Cloud Scheduler read fails must NOT exit 0.
+//
+// `npm run check:cron-drift` is the verification step for "is Cloud Scheduler
+// running every cron vercel.json declares?". Until this was fixed, a dead
+// gcloud session made it print a note and exit 0, so the command an operator
+// runs to prove there is no drift passed while comparing nothing at all. That
+// is the same silent failure the check exists to catch: on 2026-09-09 two
+// declared crons (/api/cron/globe-ingest, /api/cron/hood-portfolio-snapshot)
+// had never fired, and an unauthenticated run of the check reported success.
+//
+// `--offline` (npm run check:cron-syntax, which `npm run gate` uses) is a
+// deliberate expression-only run and must still pass without gcloud.
+
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import { execFile } from 'node:child_process';
+import { promisify } from 'node:util';
+import { mkdtempSync, writeFileSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const execFileP = promisify(execFile);
+const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
+const SCRIPT = path.join(ROOT, 'scripts', 'check-cron-drift.mjs');
+
+let stubDir;
+
+/** Run the check with a `gcloud` that always fails, so the live read cannot succeed. */
+async function runWithBrokenGcloud(args) {
+ try {
+ const { stdout } = await execFileP(process.execPath, [SCRIPT, ...args], {
+ cwd: ROOT,
+ env: { ...process.env, PATH: `${stubDir}:${process.env.PATH}` },
+ maxBuffer: 16 * 1024 * 1024,
+ });
+ return { code: 0, stdout };
+ } catch (err) {
+ return { code: err.code ?? 1, stdout: err.stdout ?? '' };
+ }
+}
+
+beforeAll(() => {
+ stubDir = mkdtempSync(path.join(tmpdir(), 'cron-drift-stub-'));
+ // Stands in for an expired session: gcloud exists but every call fails.
+ writeFileSync(
+ path.join(stubDir, 'gcloud'),
+ '#!/bin/sh\necho "ERROR: Reauthentication failed." >&2\nexit 1\n',
+ { mode: 0o755 },
+ );
+});
+
+afterAll(() => {
+ if (stubDir) rmSync(stubDir, { recursive: true, force: true });
+});
+
+describe('check-cron-drift with an unreadable Cloud Scheduler', () => {
+ it('fails the live run instead of passing on a comparison it never made', async () => {
+ const { code, stdout } = await runWithBrokenGcloud([]);
+ expect(code).toBe(1);
+ expect(stdout).toContain('FAILED: could not read Cloud Scheduler');
+ expect(stdout).toContain('proves nothing about drift');
+ });
+
+ it('reports the failure in --json so a machine reader sees it too', async () => {
+ const { code, stdout } = await runWithBrokenGcloud(['--json']);
+ expect(code).toBe(1);
+ const report = JSON.parse(stdout);
+ expect(report.checkedLive).toBe(false);
+ expect(report.liveError).toBeTruthy();
+ });
+
+ it('still passes --offline, which never asks gcloud anything', async () => {
+ const { code, stdout } = await runWithBrokenGcloud(['--offline']);
+ expect(code).toBe(0);
+ expect(stdout).toContain('All cron expressions are valid');
+ });
+});