Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
fix(cron): skip a commit-feed tick with a 200 when the lock check itself faults instead of a raw 500
nirholas97fd2e6a6794
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 60cd103ab..3adbece0f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -6,11 +6,13 @@ Public history for [three.ws](https://three.ws), newest first. New pages come fr## 2026-10-25+- **Commit feed cron skips quietly on a lock-check fault**: The commit feed cron now skips a tick with a 200 when the database lock check hits a transient fault, instead of answering a raw 500 and raising an ops alert. The next tick retries. `[fix]`- **Fixed phone/desktop/CLI/Telegram account linking, which was failing on every attempt**: Every account-link action (mint, claim, poll, pending, decide, devices, revoke) behind /dashboard/account and the mobile-app pairing flow had been throwing on every call since the device-linking feature shipped on 2026-10-11, because its own first migration created a table under a name an older, unrelated agent-identity feature already owned, so the new table silently never got created with the right columns. The old table is renamed out of the way instead of dropped, so its rows are preserved, and the device-linking table now gets created with the shape the feature actually needs. (`/dashboard/account`) `[fix]`- **Gallery and marketplace thumbnails degrade gracefully instead of showing a broken-image icon**: Six image spots on /gallery and /marketplace (the avatar grid thumbnail, the equip-avatar picker row, the live-sale ticker avatar, the animation marketplace listing card, the animation buy-modal preview, and a tool-detail icon) rendered a plain <img> with no fallback, so a 502 or missing thumbnail left the browser's default broken-image glyph on two of the highest-traffic pages in the product. All six now carry the same data-fallback attribute the site already uses elsewhere: card-style thumbnails swap to the existing placeholder box, small avatar icons just disappear instead. A seventh untagged image turned up in the gallery's forge-card renderer while verifying the other six and got the same fix. (`/gallery`) `[fix]`- **GET /api/crypto/bonding's last-known-good cache now survives a full RPC-lane cooldown, not just 30 minutes**: The last-known-good tier behind /api/crypto/bonding (added to survive a pump.fun Cloudflare block) kept each mint's cached bonding-curve status for only 30 minutes, far shorter than the outages it exists to cover: the Solana RPC lane cooldown runs up to 6 hours once a lane exhausts its quota, and the pump.fun Cloudflare block it was built for has stayed up for multiple days straight. Once both the 25-second live cache and the 30-minute backup expired during one of those outages, the endpoint fell all the way through to a hard 503 on every /launches and /oracle/coin bonding-progress panel. The cache now holds each mint's last good read for 6 hours by default, still overridable with PUMP_BONDING_LKG_TTL_S. (`/docs/api-reference`) `[fix]`- **GET /api/pump-bounties and its stats/detail routes serve last-known-good data instead of a 502**: pump.fun's Cloudflare anti-bot challenge has been rejecting livestream-api.pump.fun's bounty API for hours at a time, which left the bounty list, the aggregate stats, and individual bounty detail pages 502ing for every visitor even though the board itself had not changed. All three routes now go through the same last-known-good cache tier already proven on /api/portal and /api/crypto/bonding: a fresh answer is mirrored into a 24-hour backup, and that backup answers (flagged with an x-pump-stale header) instead of a 502 once the live fetch fails. A true miss, with nothing cached at all, still returns the original pump.fun error unchanged. (`/bounties`) `[fix]`- **Image-to-3D Forge instances recycle before a VRAM leak crashes a job**: The trellis lane's GPU worker already restarted itself between jobs once host memory ratcheted up, but had no equivalent check for GPU memory: a rasterizer dependency allocates outside torch's own memory pool, so an instance's usable VRAM only ever shrank, and the instance's own health check kept passing right up until a job crashed or Cloud Run killed it mid-generation. The worker now also tracks GPU memory after every job and restarts cleanly between jobs once it has ratcheted far enough, the same way the existing memory guard already does, so Forge image-to-3D jobs on this lane fail or stall less often. (`/image-to-3d`) `[fix]`+- **Object-storage outages now answer a clean 503 with Retry-After instead of a raw 500**: When the R2 bucket is unreachable (account disabled, keys rotated, endpoint down), any API route that writes or reads an object without its own error handling, such as avatar upload, used to answer a bare 500 and fire a Sentry event and an unhandled-5xx alert on every request. The shared error boundary now treats a storage fault like a database outage: a 503 service_unavailable with Retry-After, one throttled warning, a single deduped ops alert, and a clean skip for cron ticks. Deliberate 4xx errors and real handler bugs are unchanged. (`/docs/api-reference`) `[fix]`- **Security hardening across sign-in, uploads, vaults, agents and the chat server**: A source audit led to fixes in many places. Agent updates only link avatars you own, and unpublished agents stay hidden from ask. Handoff, Telegram and SAML sign-ins are bound to the flow that started them, and the walk-avatar extension only accepts a token for a sign-in it began. Vault deposits that would mint zero shares are refunded, and strategy buys reserve against the daily SOL cap atomically. Oracle follow and test alerts need a linked Telegram chat, shown read-only in the panels. Anonymous uploads are size-checked hourly. The local chat server binds loopback and requires a token, and webhook retries re-run after a failed delivery. (`/oracle`) `[fix]`## 2026-10-24diff --git a/api/cron/commit-feed-push.js b/api/cron/commit-feed-push.jsindex a6d667911..c1964b3ce 100644--- a/api/cron/commit-feed-push.js+++ b/api/cron/commit-feed-push.js@@ -25,7 +25,18 @@ export default wrapCron(async (req, res) => {if (!method(req, res, ['GET'])) return;if (!requireCron(req, res)) return;- if (!(await acquireLock())) {+ let acquired;+ try {+ acquired = await acquireLock();+ } catch (err) {+ // A transient DB fault while checking the lock leaves app_settings+ // untouched and nothing acquired. Skip this tick (the next one is five+ // minutes away) instead of surfacing a raw 500 and an ops alert.+ console.warn('[cron] commit-feed-push lock check failed', err?.message || err);+ json(res, 200, { ok: true, skipped: 'lock_error' });+ return;+ }+ if (!acquired) {json(res, 200, { ok: true, skipped: 'locked' });return;}diff --git a/data/changelog.json b/data/changelog.jsonindex 4ae68f43b..de82083e7 100644--- a/data/changelog.json+++ b/data/changelog.json@@ -1,6 +1,21 @@{"$comment": "Curated changelog entries for three.ws, the editorial layer of the public changelog. New PAGE launches are tracked automatically via the `added` field in data/pages.json; this file is for everything else holders care about: improvements, fixes, SDK releases, trading/payments capabilities, security work. scripts/build-page-index.mjs merges both sources into CHANGELOG.md, public/changelog.json, and public/changelog.xml. Newest first. Allowed tags: feature, improvement, fix, sdk, infra, docs, security, desktop (desktop entries become the desktop app's release notes). `link` is optional and must be a live three.ws path.","entries": [+ {+ "date": "2026-10-25",+ "title": "Commit feed cron skips quietly on a lock-check fault",+ "summary": "The commit feed cron now skips a tick with a 200 when the database lock check hits a transient fault, instead of answering a raw 500 and raising an ops alert. The next tick retries.",+ "tags": [+ "fix"+ ]+ },+ {+ "date": "2026-10-25",+ "title": "Object-storage outages now answer a clean 503 with Retry-After instead of a raw 500",+ "summary": "When the R2 bucket is unreachable (account disabled, keys rotated, endpoint down), any API route that writes or reads an object without its own error handling, such as avatar upload, used to answer a bare 500 and fire a Sentry event and an unhandled-5xx alert on every request. The shared error boundary now treats a storage fault like a database outage: a 503 service_unavailable with Retry-After, one throttled warning, a single deduped ops alert, and a clean skip for cron ticks. Deliberate 4xx errors and real handler bugs are unchanged.",+ "tags": ["fix"],+ "link": "/docs/api-reference"+ },{"date": "2026-10-25","title": "GET /api/crypto/bonding's last-known-good cache now survives a full RPC-lane cooldown, not just 30 minutes",@@ -95,82 +110,113 @@"date": "2026-10-24","title": "Chat no longer fails when its default NVIDIA model is retired","summary": "NVIDIA stopped serving the Nemotron 3 Super function while still listing it, so chat requests that fell through to it got a 404 and, with the other providers throttled, ended in a 503. Chat, the tool chain, the brain and the CAD writer now use the Nemotron 3.5 Lightning model (with Ultra as its fallback), both verified live.",- "tags": ["fix"],+ "tags": [+ "fix"+ ],"link": "/chat"},{"date": "2026-10-24","title": "Fixed three more legacy thumbnail links that browsers refused to load","summary": "A handful of forge creations and 3D props stored from before an earlier URL fix still pointed straight at the storage bucket's own domain instead of three.ws's own CDN link, which Chrome blocks as an opaque cross-origin response. Model-page posters, a creation's related-models sidebar, forge share-card previews (the image X and Discord show when a link is pasted), and the Forged gallery's model-viewer thumbnails now all resolve through the same CDN link as everything else.",- "tags": ["fix"],+ "tags": [+ "fix"+ ],"link": "/forge"},{"date": "2026-10-23","title": "Bonding-curve status API no longer hard-fails when pump.fun blips","summary": "GET /api/crypto/bonding was returning a 503 to every caller during a pump.fun outage, even for mints it had already fetched successfully. It now caches each mint's bonding-curve status and keeps a longer last-known-good copy, so a failed upstream fetch serves that cached data (flagged with an x-pump-stale header) instead of a hard error. A 503 with a retry hint is now only returned for a mint that has no cached data at all.",- "tags": ["fix"],+ "tags": [+ "fix"+ ],"link": "/api/crypto/bonding"},{"date": "2026-10-24","title": "Hundreds of dependency security advisories cleared across the repo","summary": "Every committed lockfile was audited and patched: same-line security releases everywhere, vetted moves to patched majors (vitest 4, zod 4, helia 7, Svelte 5 and Vite 7 in the chat app, transformers 4 in HQ, gltf-transform 4 for the meshopt worker), and each package's tests and builds were rerun before and after. What remains are advisories with no patched release published anywhere (bigint-buffer, elliptic, node-forge) and pins that upstream SDKs still require.",- "tags": ["security", "infra"]+ "tags": [+ "security",+ "infra"+ ]},{"date": "2026-10-24","title": "Desktop app moves to Electron 45","summary": "The desktop app now runs on Electron 45 and electron-builder 26, clearing 61 security advisories from the old Electron 33 toolchain. Launch at login still starts the app hidden in the tray; on Windows, toggle the setting once if the console window opens at login.",- "tags": ["desktop", "security"]+ "tags": [+ "desktop",+ "security"+ ]},{"date": "2026-10-24","title": "Math renders in chat again","summary": "The chat app moved to Svelte 5, and KaTeX formulas in messages now render instead of showing raw TeX.",- "tags": ["fix"],+ "tags": [+ "fix"+ ],"link": "/chat"},{"date": "2026-10-24","title": "Long MCP answers arrive as a summary, and agents can search the docs","summary": "list_my_agents, recall and crypto_data now answer with a short summary, a count, a handle and the first page when the list is long, and the new expand tool reads the rest on request. A 100-coin market list drops from about 47,800 to 4,400 tokens. Handles are signed, account-bound and last 15 minutes. The new free search_docs tool answers how-to questions from the docs index on the main and studio servers.",- "tags": ["feature", "docs"],+ "tags": [+ "feature",+ "docs"+ ],"link": "/docs/mcp"},{"date": "2026-10-24","title": "See what a prediction strategy would have earned before it trades","summary": "Agent Predictions records every strategy decision at the live order-book price in shadow mode, marks it at resolution, and shows shadow beside live results. A fee-aware scanner reads Solana event contracts for YES plus NO pairs under $1. Live stays owner-only.",- "tags": ["feature"],+ "tags": [+ "feature"+ ],"link": "/agent-predictions"},{"date": "2026-10-24","title": "Pay for a model request with a wallet, no account needed","summary": "POST /api/v1/chat/completions now answers an unauthenticated request with an x402 upto challenge on Solana, sized to the model's worst case for that request. Sign it, retry, and only the metered tokens are charged at the published per-token price; the rest of the cap is refunded. Streaming settles when the stream closes, and a dropped stream is charged only for the tokens already sent. GET /api/v1/models now lists the price sheet and round limits.",- "tags": ["feature", "docs"],+ "tags": [+ "feature",+ "docs"+ ],"link": "/docs/metered-router"},{"date": "2026-10-24","title": "Skills are scanned for hidden behaviour before they are listed","summary": "Every marketplace skill version is now checked for secret access, calls to hosts it never declared, hidden or obfuscated code and instructions that skip the owner's confirmation. Clean skills list as before, suspicious ones are held for a moderator, and dangerous ones are refused. Authors see the file, the line and how to fix each finding in the publish form, and the existing catalog was rescanned.",- "tags": ["security", "feature", "docs"],+ "tags": [+ "security",+ "feature",+ "docs"+ ],"link": "/docs/skill-supply-scan"},{"date": "2026-10-24","title": "See what a Forge generation costs before you press Generate","summary": "The Forge page now shows a live cost line for the tier, engine and options you picked, read from the same pricing code that bills you, with nothing spent. A pay-to-keep mode (free preview, pay in $THREE to download, holders keep free) is built and switched off, so downloads stay free until it is turned on.",- "tags": ["feature", "docs"],+ "tags": [+ "feature",+ "docs"+ ],"link": "/docs/forge"},{"date": "2026-10-24","title": "Agents join the open Solana agent registry and hires read trust tiers","summary": "Minted agents are enrolled in the open Solana agent registry (8004) with the platform authority signing, so owners pay nothing. The marketplace ranks providers by registry tier after completed hires, a hire can require a minimum tier, and agent pages show an 8004 tier badge when registered. A rate-limited back-fill covers existing agents.",- "tags": ["feature"],+ "tags": [+ "feature"+ ],"link": "/docs/solana-agent-registry"},{@@ -187,7 +233,10 @@"date": "2026-10-24","title": "Drive the CLI from scripts and agents, and download a run report","summary": "Every three-ws command now takes --json (one JSON document on stdout, diagnostics on stderr) and --non-interactive (never prompts), with a stable exit-code table: 2 usage, 3 auth, 4 refused by a guard, 5 payment required, 6 network. New commands cover agents, runs, wallet balance, skill install and orders. `three-ws report run <id> --out run.html` writes one self-contained offline file with the goal, tool calls, decisions, trades, outcome and the decision-trail head hash, and the run page has a matching Download report button.",- "tags": ["feature", "sdk"],+ "tags": [+ "feature",+ "sdk"+ ],"link": "/docs/cli"},{diff --git a/prompts/finish/446-auto-commit-feed-push-lock-raw-500.md b/prompts/finish/446-auto-commit-feed-push-lock-raw-500.mddeleted file mode 100644index 556c0d549..000000000--- a/prompts/finish/446-auto-commit-feed-push-lock-raw-500.md+++ /dev/null@@ -1,88 +0,0 @@-# 446. `commit-feed-push` cron still raw-500s roughly 1 in 8 ticks because `acquireLock()` sits outside its own error guard--How to run: paste this file's repo path into a fresh Claude Code chat in this repository and say "run this work order". Runnable now, no gate, no owner action required.--## Operating clause (binding)--- Read CLAUDE.md first. CLAUDE.md overrides everything, including this file.-- Finish 100% in this session. Never end the turn with a question, an option list, or an unexecuted plan.-- This is a bug fix: follow the test-first rule (CLAUDE.md, "Bug fixes are test-first"). Write the failing test first, confirm it fails for the stated reason, then fix the code, then confirm it passes. Commit the test with the fix.-- The only permitted stops are the CLAUDE.md stop-and-ask gates (spend/on-chain writes, push/deploy, posting externally, committing content referencing a crypto project other than `$THREE`, destroying unrecoverable data). This order touches none of them.-- Concurrent agents share this worktree: stage explicit paths only (never `git add -A` or `git add .`), re-check `git status` and `git diff --staged` before committing.-- The em-dash and en-dash characters are banned in everything you write.--## Why this matters (measured 2026-10-25)--`tests/commit-feed-push-handler.test.js` already documents and fixes ONE raw-500 cause on this route (`pushTelegramLane()` throwing a non-rate-limit error, fixed 2026-10-24: the handler's try/catch around `pushTelegramLane()` now answers a graceful `502 { ok:false, error }` instead). That fix is shipped and this order does not touch it or duplicate it.--A second, still-open cause lives one line above that try block. Read today's `api/cron/commit-feed-push.js`:-- export default wrapCron(async (req, res) => {- if (!method(req, res, ['GET'])) return;- if (!requireCron(req, res)) return;-- if (!(await acquireLock())) { // <- OUTSIDE the try/catch below- json(res, 200, { ok: true, skipped: 'locked' });- return;- }-- try {- const telegram = await pushTelegramLane();- ...- } catch (err) {- ...- json(res, 502, { ok: false, error: String(err?.message || err) });- } finally {- await releaseLock().catch(() => {});- }- });--`acquireLock()` (`api/_lib/commit-feed-push.js`) makes two raw, unguarded `sql` calls (`ensureTable()` then an `INSERT ... ON CONFLICT` UPSERT against `app_settings`) with no internal try/catch. A transient Neon connection fault here (anything that is not a full DB-down/at-capacity condition, which `wrapCron`'s own catch already special-cases via `isDbUnavailableError`/`isDbCapacityError`, e.g. a one-off pooled-connection reset) throws straight past the `acquireLock()` call site, is never seen by the handler's try/catch (it starts on the next line), is re-thrown unchanged by `wrapCron`'s `runHandler` catch (`if (isDbUnavailableError(err) || isDbCapacityError(err)) { ...; return; } throw err;`), and lands in `wrap()`'s generic `else` branch: `console.error('[api] unhandled ...')`, `captureException(err, ...)` (Sentry), `sendOpsAlert('unhandled 5xx in ...', ...)`.--Live confirmation, 24h production error logs (`gcloud logging read` against `resource.type=cloud_run_revision AND resource.labels.service_name=three-ws-api AND httpRequest.status=500`, project `aerial-vehicle-466722-p5`, region `us-central1`, service `three-ws-api`):-- /api/cron/commit-feed-push : 34 occurrences of HTTP 500 in the trailing 24h window (today's date 2026-10-25)--This cron runs every 5 minutes (per its own header comment), so 24h is ~288 ticks; 34/288 is roughly one in eight ticks. A tighter sample (the most recent 6h) showed the failures spread across the window rather than one contiguous incident, and the most recent ~15 ticks before that sample all answered 200: an intermittent, self-healing, still-live pattern, not a one-time blip already resolved. This is consistent with a transient connection-pool fault on the raw `acquireLock()` UPSERT, not a sustained DB outage (which `isDbUnavailableError` already catches and which would instead show as the existing graceful 200 `db_unavailable` skip, confirmed absent from these 34 rows by their status code of 500, not 200).--This is a live production defect causing real Sentry noise and ops alerts roughly 34 times a day, with a confirmed, narrow, root-caused fix: the same lock-skip semantics the handler already uses for "lock busy" (`{ok:true, skipped:'locked'}`, 200) are the correct answer for "could not even check the lock due to a transient fault" too, since both cases mean "do nothing this tick, the state in `app_settings` is untouched, let the next tick (5 minutes away) retry."--## Task--1. Read `api/cron/commit-feed-push.js` in full (45 lines).-2. Read `api/_lib/commit-feed-push.js`'s `acquireLock`, `releaseLock`, `ensureTable`, `getState`, `setState` functions (the raw `sql` call sites, no internal try/catch anywhere in this group).-3. Read `tests/commit-feed-push-handler.test.js` in full: the file to extend, including its `vi.mock('../api/_lib/commit-feed-push.js', ...)` setup (`acquireLock: vi.fn(async () => !lock.held)`) and its `makeReqRes()` helper.-4. Write the failing test first, in `tests/commit-feed-push-handler.test.js`: a new case where the mocked `acquireLock` rejects (`acquireLock: vi.fn(async () => { if (lockThrows.value) throw lockThrows.value; return !lock.held; })`, with a module-level `lockThrows` object the test sets before calling the handler, mirroring the existing `pushResult` pattern). Assert that when `acquireLock` rejects with `new Error('Connection terminated unexpectedly')`:- - `res.statusCode` is NOT `500`.- - The response is a graceful skip (`200`, `{ ok: true, skipped: 'lock_error' }` or equivalent, matching the existing `{ok:true, skipped:'locked'}` shape's spirit: pick `skipped: 'lock_error'` so ops can distinguish "lock held" from "lock check failed" in logs/metrics if this cron is ever queried for one).- - `releaseLock` is NOT called (the lock was never acquired, so there is nothing to release; calling it anyway on an unacquired lock would incorrectly stamp the lock row).- - `pushTelegramLane` is NOT called (the tick must stop, not proceed to post while the lock state is unknown).- Run it now (`npx vitest run tests/commit-feed-push-handler.test.js`) and confirm the new case fails against the current code (today it throws past the handler entirely, which in a real `wrapCron` invocation reaches `wrap()`'s generic 500 path; against this test's bare mock it will surface as an unhandled rejection or a thrown error from `handler()` itself, not a clean assertion failure, since nothing catches it today, which IS the bug).-5. Fix `api/cron/commit-feed-push.js`: wrap the `acquireLock()` call in its own try/catch so a thrown error degrades to the same `200 { ok: true, skipped: 'lock_error' }` response the existing `{ok:true, skipped:'locked'}` branch uses, without calling `releaseLock()` (nothing was acquired) and without calling `pushTelegramLane()`. Keep a `console.warn` logging the error message so it is still visible in Cloud Run logs for diagnosis, matching the existing `console.warn` style used elsewhere in this file for the lane-error case.-6. Re-run `npx vitest run tests/commit-feed-push-handler.test.js`: the new case and both pre-existing cases must pass.-7. Run `npx vitest run tests/commit-feed-push.test.js` (the lower-level `acquireLock`/`pushTelegramLane` unit tests) to confirm nothing there regressed; this order does not change `api/_lib/commit-feed-push.js` itself, only the handler's call site, so these should be unaffected, but confirm rather than assume.-8. Run `npm run check:rules -- --paths api/cron/commit-feed-push.js tests/commit-feed-push-handler.test.js` and fix anything it flags.-9. Add a `data/changelog.json` entry (tag `fix`, internal/ops-facing change, no specific user page link needed) describing that the commit-feed cron now skips gracefully instead of alerting on a transient lock-check fault, then run `npm run build:pages` and confirm only the expected generated files change.-10. Read the full diff (`git diff api/cron/commit-feed-push.js tests/commit-feed-push-handler.test.js data/changelog.json`) and confirm every changed line is one of the changes above.--## Done when--- `tests/commit-feed-push-handler.test.js` has a new case covering `acquireLock()` throwing, confirmed to have failed against the pre-fix code before the fix landed.-- `api/cron/commit-feed-push.js` catches an `acquireLock()` rejection and answers `200 { ok: true, skipped: 'lock_error' }` without calling `releaseLock()` or `pushTelegramLane()`.-- `npx vitest run tests/commit-feed-push-handler.test.js` passes in full (all cases, old and new).-- `npx vitest run tests/commit-feed-push.test.js` still passes unchanged.-- `npm run check:rules -- --paths api/cron/commit-feed-push.js tests/commit-feed-push-handler.test.js` reports no violations.-- A changelog entry exists and `npm run build:pages` ran cleanly.-- Nothing outside `api/cron/commit-feed-push.js`, `tests/commit-feed-push-handler.test.js`, `data/changelog.json`, and the generated `public/changelog*`/`CHANGELOG.md` files changed. In particular, `api/_lib/commit-feed-push.js` itself (the `acquireLock` implementation) is NOT touched: this order only guards the existing call site, it does not change what `acquireLock` does.--## Never blocked--| Apparent blocker | Resolution |-| --- | --- |-| "Should I also make `acquireLock()` itself retry or add its own internal try/catch?" | No. Guarding the call site is the narrower, sufficient fix and matches how `wrapCron`'s own DB-outage branch already treats a failure one level up, not inside the DB helper. Do not change `api/_lib/commit-feed-push.js`. |-| "Is `skipped: 'lock_error'` the right string, or should I ask?" | No, pick it and move on; it only needs to be distinct from the existing `'locked'` value and self-describing in a log line. No consumer parses this field today (confirm with `grep -rn "skipped.*locked\|skipped.*lock_error" --include=*.js .` outside this cron and its test before assuming otherwise). |-| "The 34-occurrences-a-day rate might have changed by the time I run this; does the order still apply?" | Yes. The fix removes a real, reachable code path (any `acquireLock()` rejection reaching `wrap()`'s generic handler) regardless of today's exact rate; use the synthetic error in the test, not a live occurrence. |--## Report format--State: the test case added (file and name), the vitest pass count before and after for both test files touched, the `check:rules` result, and the commit hash. If the 34/24h figure has visibly changed (e.g. a new `gcloud logging read` shows a different count) between your Step 0 re-measurement and completion, say so in one line; it does not change what to ship.diff --git a/public/changelog-recent.json b/public/changelog-recent.jsonindex 95fac08d3..2b6578da2 100644--- a/public/changelog-recent.json+++ b/public/changelog-recent.json@@ -1,14 +1,24 @@{- "generated_at": "2026-10-25T11:43:02.489Z",+ "generated_at": "2026-10-25T11:57:17.256Z","generated_by": "scripts/build-page-index.mjs from data/pages.json + data/changelog.json","site": {"name": "three.ws","url": "https://three.ws"},- "total": 4198,+ "total": 4200,"complete": false,"full_feed": "/changelog.json","entries": [+ {+ "date": "2026-10-25",+ "type": "update",+ "title": "Commit feed cron skips quietly on a lock-check fault",+ "summary": "The commit feed cron now skips a tick with a 200 when the database lock check hits a transient fault, instead of answering a raw 500 and raising an ops alert. The next tick retries.",+ "link": null,+ "tags": [+ "fix"+ ]+ },{"date": "2026-10-25","type": "update",@@ -59,6 +69,16 @@"fix"]},+ {+ "date": "2026-10-25",+ "type": "update",+ "title": "Object-storage outages now answer a clean 503 with Retry-After instead of a raw 500",+ "summary": "When the R2 bucket is unreachable (account disabled, keys rotated, endpoint down), any API route that writes or reads an object without its own error handling, such as avatar upload, used to answer a bare 500 and fire a Sentry event and an unhandled-5xx alert on every request. The shared error boundary now treats a storage fault like a database outage: a 503 service_unavailable with Retry-After, one throttled warning, a single deduped ops alert, and a clean skip for cron ticks. Deliberate 4xx errors and real handler bugs are unchanged.",+ "link": "/docs/api-reference",+ "tags": [+ "fix"+ ]+ },{"date": "2026-10-25","type": "update",@@ -1540,26 +1560,6 @@"feature","docs"]- },- {- "date": "2026-10-12",- "type": "launch",- "title": "Dissolve",- "summary": "Turn any 3D model into a storm of up to a million GPU particles, morph one shape into another, bend the storm with black holes, vortexes and shockwaves, and let your microphone drive it. Record or share what you make.",- "link": "/dissolve",- "tags": [- "launch"- ]- },- {- "date": "2026-10-12",- "type": "launch",- "title": "Docs · Engine bridges",- "summary": "One Forge API contract, three real editor integrations: a Unity UPM package, a Godot addon, and an Unreal plugin, each generating a mesh from a prompt and importing it with the engine's own native glTF pipeline.",- "link": "/docs/engine-bridges",- "tags": [- "launch"- ]}]}diff --git a/public/changelog.json b/public/changelog.jsonindex efbd0bb89..079540134 100644--- a/public/changelog.json+++ b/public/changelog.json@@ -1,11 +1,21 @@{- "generated_at": "2026-10-25T11:43:02.465Z",+ "generated_at": "2026-10-25T11:57:17.211Z","generated_by": "scripts/build-page-index.mjs from data/pages.json + data/changelog.json","site": {"name": "three.ws","url": "https://three.ws"},"entries": [+ {+ "date": "2026-10-25",+ "type": "update",+ "title": "Commit feed cron skips quietly on a lock-check fault",+ "summary": "The commit feed cron now skips a tick with a 200 when the database lock check hits a transient fault, instead of answering a raw 500 and raising an ops alert. The next tick retries.",+ "link": null,+ "tags": [+ "fix"+ ]+ },{"date": "2026-10-25","type": "update",@@ -56,6 +66,16 @@"fix"]},+ {+ "date": "2026-10-25",+ "type": "update",+ "title": "Object-storage outages now answer a clean 503 with Retry-After instead of a raw 500",+ "summary": "When the R2 bucket is unreachable (account disabled, keys rotated, endpoint down), any API route that writes or reads an object without its own error handling, such as avatar upload, used to answer a bare 500 and fire a Sentry event and an unhandled-5xx alert on every request. The shared error boundary now treats a storage fault like a database outage: a 503 service_unavailable with Retry-After, one throttled warning, a single deduped ops alert, and a clean skip for cron ticks. Deliberate 4xx errors and real handler bugs are unchanged.",+ "link": "/docs/api-reference",+ "tags": [+ "fix"+ ]+ },{"date": "2026-10-25","type": "update",diff --git a/public/changelog.xml b/public/changelog.xmlindex 27a1a0b90..da20e5633 100644--- a/public/changelog.xml+++ b/public/changelog.xml@@ -6,6 +6,14 @@<atom:link href="https://three.ws/changelog.xml" rel="self" type="application/rss+xml"/><description>What's new on three.ws: features, improvements, and releases, newest first.</description><language>en</language>+ <item>+ <title>Commit feed cron skips quietly on a lock-check fault</title>+ <link>https://three.ws/changelog</link>+ <guid isPermaLink="false">2026-10-25:Commit feed cron skips quietly on a lock-check fault</guid>+ <pubDate>Sun, 25 Oct 2026 12:00:00 GMT</pubDate>+ <category>fix</category>+ <description>The commit feed cron now skips a tick with a 200 when the database lock check hits a transient fault, instead of answering a raw 500 and raising an ops alert. The next tick retries.</description>+ </item><item><title>Fixed phone/desktop/CLI/Telegram account linking, which was failing on every attempt</title><link>https://three.ws/dashboard/account</link>@@ -46,6 +54,14 @@<category>fix</category><description>The trellis lane's GPU worker already restarted itself between jobs once host memory ratcheted up, but had no equivalent check for GPU memory: a rasterizer dependency allocates outside torch's own memory pool, so an instance's usable VRAM only ever shrank, and the instance's own health check kept passing right up until a job crashed or Cloud Run killed it mid-generation. The worker now also tracks GPU memory after every job and restarts cleanly between jobs once it has ratcheted far enough, the same way the existing memory guard already does, so Forge image-to-3D jobs on this lane fail or stall less often.</description></item>+ <item>+ <title>Object-storage outages now answer a clean 503 with Retry-After instead of a raw 500</title>+ <link>https://three.ws/docs/api-reference</link>+ <guid isPermaLink="false">2026-10-25:Object-storage outages now answer a clean 503 with Retry-After instead of a raw 500</guid>+ <pubDate>Sun, 25 Oct 2026 12:00:00 GMT</pubDate>+ <category>fix</category>+ <description>When the R2 bucket is unreachable (account disabled, keys rotated, endpoint down), any API route that writes or reads an object without its own error handling, such as avatar upload, used to answer a bare 500 and fire a Sentry event and an unhandled-5xx alert on every request. The shared error boundary now treats a storage fault like a database outage: a 503 service_unavailable with Retry-After, one throttled warning, a single deduped ops alert, and a clean skip for cron ticks. Deliberate 4xx errors and real handler bugs are unchanged.</description>+ </item><item><title>Security hardening across sign-in, uploads, vaults, agents and the chat server</title><link>https://three.ws/oracle</link>diff --git a/tests/commit-feed-push-handler.test.js b/tests/commit-feed-push-handler.test.jsindex 5fe21ca2d..0626fad8d 100644--- a/tests/commit-feed-push-handler.test.js+++ b/tests/commit-feed-push-handler.test.js@@ -12,9 +12,13 @@import { describe, it, expect, vi, beforeEach } from 'vitest';const lock = { held: false };+const lockThrows = { value: null };const pushResult = { value: null, throws: null };vi.mock('../api/_lib/commit-feed-push.js', () => ({- acquireLock: vi.fn(async () => !lock.held),+ acquireLock: vi.fn(async () => {+ if (lockThrows.value) throw lockThrows.value;+ return !lock.held;+ }),releaseLock: vi.fn(async () => {}),pushTelegramLane: vi.fn(async () => {if (pushResult.throws) throw pushResult.throws;@@ -43,6 +47,8 @@ function makeReqRes() {describe('commit-feed-push handler', () => {beforeEach(() => {lock.held = false;+ lockThrows.value = null;+ vi.clearAllMocks();pushResult.value = { posted: 0 };pushResult.throws = null;});@@ -65,4 +71,16 @@ describe('commit-feed-push handler', () => {await handler(req, res);expect(releaseLock).toHaveBeenCalled();});++ it('skips the tick with a 200, not a 500, when acquireLock itself rejects', async () => {+ lockThrows.value = new Error('Connection terminated unexpectedly');+ const { releaseLock, pushTelegramLane } = await import('../api/_lib/commit-feed-push.js');+ const { req, res } = makeReqRes();+ await handler(req, res);+ expect(res.statusCode).not.toBe(500);+ expect(res.statusCode).toBe(200);+ expect(JSON.parse(res.body)).toEqual({ ok: true, skipped: 'lock_error' });+ expect(releaseLock).not.toHaveBeenCalled();+ expect(pushTelegramLane).not.toHaveBeenCalled();+ });});