Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

test(sitemap): pin the sub-sitemap contract so the prototype-key 500 and the seed-cron profile flood cannot come back

nirholas40ed17e1957c
diff --git a/data/changelog.json b/data/changelog.json
index 39746ec26..30cdf9f6e 100644
--- a/data/changelog.json
+++ b/data/changelog.json
@@ -1,6 +1,15 @@
{
"$comment": "Curated changelog entries for three.ws, the editorial layer of the public changelog. New PAGE launches are tracked automatically via the `added` field in data/pages.json; this file is for everything else holders care about: improvements, fixes, SDK releases, trading/payments capabilities, security work. scripts/build-page-index.mjs merges both sources into CHANGELOG.md, public/changelog.json, and public/changelog.xml. Newest first. Allowed tags: feature, improvement, fix, sdk, infra, docs, security. `link` is optional and must be a live three.ws path.",
"entries": [
+ {
+ "date": "2026-08-16",
+ "title": "The public profile sitemap now lists people, not the seed crons",
+ "summary": "The sitemap Google and Bing crawl is split into one file per kind of page, and the profiles file was handing search engines every account row with a username. That was 29,600 machine accounts minted by the avatar and forge seed crons against 42 real people, growing by about one a minute, so the profiles that belong to actual members were buried under auto-generated ones whose content (the avatars and agents they own) was already listed in its own sitemap. Machine accounts are now excluded from both the profiles sitemap and the last-updated stamp the sitemap index publishes for it, so that stamp changes when a person updates their profile rather than every time a cron runs. Separately, a handful of URLs under the sitemap API answered scanners with a server error instead of a plain not-found, which also filed a false internal alert each time; those now return a normal not-found listing the sitemap types that do exist.",
+ "tags": [
+ "fix",
+ "infra"
+ ]
+ },
{
"date": "2026-08-16",
"title": "Trending: muted text on the board is readable, and the Tip button that only navigated is gone",
diff --git a/prompts/swarm-100/api-sitemap-01.md b/prompts/swarm-100/api-sitemap-01.md
deleted file mode 100644
index 52a5dd88b..000000000
--- a/prompts/swarm-100/api-sitemap-01.md
+++ /dev/null
@@ -1,101 +0,0 @@
-# API audit: api/sitemap batch 01 (1 files)
-
-How to run: paste this file's repo path into a fresh Claude Code chat in this
-repository and say "run this work order". This file is fully self-contained:
-it depends on no other prompt file anywhere. If neighboring files in
-prompts/swarm-100/ are gone, that work is done; if present, ignore them.
-Every claim below rots; step 0 re-measures, and what you measure wins.
-
-## Operating clause (binding)
-
-- Read CLAUDE.md first. Its rules override everything, including this file.
-- Finish 100% in this session. Never end the turn with a question, an option
- list, or an unexecuted plan. A judgment call goes in one line of the final
- report; it never becomes a question that halts work.
-- The only permitted stops are the CLAUDE.md stop-and-ask gates: spending real
- funds or any irreversible on-chain write, git push or a production deploy,
- committing content that references a crypto project other than $THREE, and
- destroying unrecoverable data.
-- No mocks, no fake data, no placeholder stubs, no unfinished-work markers, no
- commented-out code. Real APIs and real integrations only.
-- The em-dash and en-dash characters are banned in everything you write.
-- Concurrent agents share this worktree: stage explicit paths only (never a
- bare add-everything), and commit finished work promptly.
-- Before committing, run: npm run check:rules -- --paths <files you touched>.
- It must exit 0.
-
-## Step 0: re-derive the current state
-
-1. Confirm each file below still exists; deleted ones are done, skip them.
-2. For each, decide routability: grep its path in vercel.json and read how
- server/index.mjs maps filesystem handlers. Routed files get an endpoint
- audit; internal modules get a code and test audit.
-3. Start a local server for endpoint checks: node server/index.mjs with .env
- loaded, on a free PORT if 3000 is busy.
-
-## Task: audit these api files, then fix everything found
-
-Files in this batch:
-
-- api/sitemap/[type].js
-
-For every routed handler:
-
-1. Exercise it with curl against the local server: the success path returns
- real data with the correct status; malformed input returns a 4xx JSON
- error, never a stack trace or an HTML error page; auth is enforced where
- the code intends it; CORS matches neighboring handlers.
-2. Hard-rule scan: no mock fallbacks, no fake-data arrays, no stubbed
- responses, no unfinished markers. Errors are handled at the boundary and
- internal code trusts itself.
-3. Do not spend funds or sign anything on-chain while probing. Payment and
- trading handlers get their validation and error paths exercised, never a
- live settlement.
-
-For every internal (non-routed) module:
-
-4. Something must import it; if nothing does, delete it as dead code (verify
- with a repo-wide grep first).
-5. Its main path and one failure path must be covered in tests/; add the
- missing test if not.
-
-## Definition of done
-
-- [ ] Every listed file has recorded evidence: a curl transcript for routed
- handlers, a passing test or a deletion for internal modules.
-- [ ] Zero hard-rule violations remain in these files.
-- [ ] vitest run passes; npm run check:rules -- --paths <touched files>
- exits 0.
-
-## Never blocked
-
-| Blocker | Resolution (act, do not ask) |
-|---|---|
-| Missing env var or credential | Check .env and .env.local, then the Cloud Run service env, then Secret Manager (the CLAUDE.md self-unblock playbook has the exact commands). If it truly exists nowhere, wire the code fully behind the env var, prove the wiring with a dry run, and name the single missing var in the report. |
-| gcloud not on PATH | Run: export PATH="$HOME/google-cloud-sdk/bin:$PATH" |
-| Dev server port 3000 busy | Another agent may be serving this repo; probe it and reuse if so. Otherwise start your own on a free port: npx vite --port 3101 |
-| Playwright browser missing | npx playwright install chromium |
-| A surface needs a signed-in user | Register a fresh account through the real /register flow against the real API and use it. Never mock the session. |
-| A defect sits in code you did not touch | Fix it if it blocks a Definition of done line (root cause it, never mask it). Otherwise note it in the report and continue. |
-| An unrelated test is red | Same rule. Never pipe npm test through tail; it masks exit codes. |
-
-## Close out (required)
-
-1. Verify every Definition of done line with the actual command output in
- front of you. Never claim a line you did not verify.
-2. Commit with explicit paths and a subject that describes the diff (house
- style: type(scope): what changed and why a reader cares).
-3. Delete this prompt file in that same commit:
-
- git rm prompts/swarm-100/api-sitemap-01.md
-
- The shrinking directory is this campaign's only progress ledger; there is
- no progress log to update.
-4. Final report: what step 0 measured, what changed (file list), evidence per
- Definition of done line, and any one-line judgment calls. No trailing
- questions, no unexecuted plans.
-
-If a line genuinely cannot pass inside this session (an external party must
-respond, or an owner-gated action is the final step), finish everything else,
-leave this file in place, and state exactly which line remains and who owns
-it. Never delete this file on a partial.
diff --git a/tests/sitemap-type.test.js b/tests/sitemap-type.test.js
new file mode 100644
index 000000000..0809e4984
--- /dev/null
+++ b/tests/sitemap-type.test.js
@@ -0,0 +1,213 @@
+// Coverage for the per-entity sub-sitemaps at /sitemap/<type>.xml
+// (api/sitemap/[type].js), the files Google and Bing fetch from the
+// sitemap index that api/sitemap.js emits.
+//
+// Two regressions are pinned here because both were live on 2026-08-16:
+//
+// - `constructor`, `toString`, `valueOf` and `hasOwnProperty` resolved
+// through the prototype chain of the builder lookup object, so they passed
+// the "is this a known type?" guard, got called as if they were builders,
+// and answered any scanner walking /api/sitemap/<word> with a 500 plus a
+// fabricated sitemap_failed report in error tracking. The lookup is a Map
+// now, and every one of them has to stay a 404.
+//
+// - profiles.xml listed every users row with a username, which on that date
+// was 29,600 seed-cron machine accounts against 42 humans. The
+// service_account predicate has to stay in the query, and no machine
+// account may reach the XML.
+//
+// Plus the handler contract: the six real types, XML well-formedness and
+// escaping, the .xml suffix the vercel.json rewrite leaves on, and the 500
+// boundary when a builder throws.
+
+import { describe, it, expect, vi, beforeEach } from 'vitest';
+import { XMLValidator } from 'fast-xml-parser';
+
+const db = vi.hoisted(() => ({ queries: [], rows: new Map(), fail: null }));
+
+vi.mock('../api/_lib/env.js', () => ({
+ env: { APP_ORIGIN: 'https://three.ws' },
+}));
+
+vi.mock('../api/_lib/db.js', () => ({
+ sql: (strings, ...values) => {
+ const text = strings.join('?');
+ db.queries.push({ text, values });
+ if (db.fail) return Promise.reject(db.fail);
+ for (const [table, rows] of db.rows) {
+ if (text.includes(`from ${table}`)) return Promise.resolve(rows);
+ }
+ return Promise.resolve([]);
+ },
+ isDbUnavailableError: () => false,
+ isDbCapacityError: () => false,
+}));
+
+vi.mock('../api/_lib/http.js', () => ({
+ reportServerError: () => 'ref00000deadbeef',
+ redactUrl: (u) => String(u ?? ''),
+}));
+
+const news = vi.hoisted(() => ({ months: [], byMonth: new Map() }));
+
+vi.mock('../api/_lib/news-archive-store.js', () => ({
+ getMonths: async () => news.months,
+ loadMonth: async (m) => news.byMonth.get(m) || [],
+}));
+
+const { default: handler } = await import('../api/sitemap/[type].js');
+
+function makeRes() {
+ return {
+ statusCode: 0,
+ headers: {},
+ body: '',
+ setHeader(k, v) { this.headers[k.toLowerCase()] = v; },
+ end(chunk) { this.body = chunk == null ? this.body : String(chunk); return this; },
+ };
+}
+
+async function call(type) {
+ const res = makeRes();
+ await handler({ query: { type }, url: `/api/sitemap/${type}` }, res);
+ return res;
+}
+
+function locs(xml) {
+ return [...xml.matchAll(/<loc>([^<]*)<\/loc>/g)].map((m) => m[1]);
+}
+
+function expectWellFormed(xml) {
+ const verdict = XMLValidator.validate(xml);
+ expect(verdict, typeof verdict === 'object' ? JSON.stringify(verdict.err) : '').toBe(true);
+}
+
+const AT = new Date('2026-03-04T05:06:07Z');
+
+beforeEach(() => {
+ db.queries = [];
+ db.fail = null;
+ db.rows = new Map([
+ ['agent_identities', [{ id: 'agent-1', updated_at: AT, created_at: AT }]],
+ ['avatars', [{ id: 'avatar-1', updated_at: null, created_at: AT }]],
+ ['widgets', [{ id: 'widget-1', updated_at: AT, created_at: AT }]],
+ ['users u', [{ username: 'realhuman', updated_at: AT, created_at: AT }]],
+ ]);
+ news.months = ['2026-01', '2026-02', '2026-03'];
+ // storyPath() only builds a URL for a 16-hex id plus a datable pub_date.
+ news.byMonth = new Map([
+ ['2026-03', [{ id: 'a'.repeat(16), title: 'A story', pub_date: '2026-03-02T00:00:00Z' }]],
+ ['2026-02', [{ id: 'b'.repeat(16), title: 'B story', pub_date: '2026-02-02T00:00:00Z' }]],
+ ['2026-01', [{ id: 'c'.repeat(16), title: 'C story', pub_date: '2026-01-02T00:00:00Z' }]],
+ ]);
+});
+
+describe('GET /sitemap/<type>.xml: the six real types', () => {
+ it.each(['core', 'agents', 'avatars', 'widgets', 'profiles', 'news'])(
+ 'serves %s as cacheable, well-formed XML with at least one absolute https URL',
+ async (type) => {
+ const res = await call(type);
+ expect(res.statusCode).toBe(200);
+ expect(res.headers['content-type']).toBe('application/xml; charset=utf-8');
+ expect(res.headers['cache-control']).toContain('s-maxage=600');
+ expectWellFormed(res.body);
+ expect(res.body.startsWith('<?xml version="1.0" encoding="UTF-8"?>\n')).toBe(true);
+ expect(res.body).toContain('xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"');
+ const found = locs(res.body);
+ expect(found.length).toBeGreaterThan(0);
+ for (const loc of found) expect(loc.startsWith('https://three.ws/')).toBe(true);
+ // A crawler that sees the same URL twice wastes budget on it twice.
+ expect(new Set(found).size).toBe(found.length);
+ },
+ );
+
+ it('strips the .xml the vercel.json rewrite leaves on the type', async () => {
+ const bare = await call('widgets');
+ const suffixed = await call('widgets.xml');
+ expect(suffixed.statusCode).toBe(200);
+ expect(suffixed.body).toBe(bare.body);
+ });
+
+ it('advertises the xhtml namespace only when entries carry hreflang alternates', async () => {
+ // core localizes (public/locales/localized-pages.json); widgets never do.
+ expect((await call('core')).body).toContain('xmlns:xhtml="http://www.w3.org/1999/xhtml"');
+ expect((await call('widgets')).body).not.toContain('xmlns:xhtml');
+ });
+
+ it('escapes XML metacharacters that reach a loc through user data', async () => {
+ db.rows.set('users u', [{ username: `a&b<c>d"e'f`, updated_at: AT, created_at: AT }]);
+ const res = await call('profiles');
+ expectWellFormed(res.body);
+ expect(res.body).toContain('<loc>https://three.ws/u/a&amp;b&lt;c&gt;d&quot;e&apos;f</loc>');
+ });
+
+ it('drains the news archive newest month first', async () => {
+ const found = locs((await call('news')).body);
+ expect(found).toEqual([
+ `https://three.ws/markets/news/2026-03/${'a'.repeat(16)}-a-story`,
+ `https://three.ws/markets/news/2026-02/${'b'.repeat(16)}-b-story`,
+ `https://three.ws/markets/news/2026-01/${'c'.repeat(16)}-c-story`,
+ ]);
+ });
+});
+
+describe('profiles.xml excludes seed-cron machine accounts', () => {
+ it('filters on service_account in SQL, not after the fact', async () => {
+ await call('profiles');
+ const q = db.queries.find((x) => x.text.includes('from users u'));
+ expect(q, 'profiles must query users').toBeTruthy();
+ expect(q.text).toContain('service_account = false');
+ expect(q.text).toContain('deleted_at is null');
+ expect(q.text).toContain('username is not null');
+ });
+
+ it('emits nothing when every profile row is a machine account', async () => {
+ // The predicate lives in SQL, so a filtered query returns no rows at all.
+ db.rows.set('users u', []);
+ const res = await call('profiles');
+ expect(res.statusCode).toBe(200);
+ expectWellFormed(res.body);
+ expect(locs(res.body)).toEqual([]);
+ expect(res.body).toContain('</urlset>');
+ });
+});
+
+describe('GET /sitemap/<type>.xml: failure paths', () => {
+ it.each(['bogus', '', 'CORE', '../../etc/passwd'])('404s on the unknown type %j', async (type) => {
+ const res = await call(type);
+ expect(res.statusCode).toBe(404);
+ expect(res.headers['content-type']).toBe('text/plain; charset=utf-8');
+ expect(res.headers['cache-control']).toBe('no-store');
+ expect(res.body).toContain('unknown sitemap');
+ expect(res.body).not.toContain('<');
+ });
+
+ it.each(['constructor', 'toString', 'valueOf', 'hasOwnProperty', 'isPrototypeOf', '__proto__'])(
+ '404s on the prototype-chain key %j instead of 500ing',
+ async (type) => {
+ const res = await call(type);
+ expect(res.statusCode).toBe(404);
+ expect(res.body).toContain('unknown sitemap');
+ },
+ );
+
+ it('returns a plain-text 500 with a support ref, never a stack trace, when the DB fails', async () => {
+ db.fail = Object.assign(new Error('connection terminated at 10.0.0.1:5432'), { stack: 'Error: boom\n at agentsSitemap' });
+ const res = await call('agents');
+ expect(res.statusCode).toBe(500);
+ expect(res.headers['content-type']).toBe('text/plain; charset=utf-8');
+ expect(res.headers['cache-control']).toBe('no-store');
+ expect(res.body).toContain('ref00000deadbeef');
+ expect(res.body).not.toContain('at agentsSitemap');
+ expect(res.body).not.toContain('10.0.0.1');
+ expect(res.body).not.toContain('<');
+ });
+
+ it('falls back to the home page rather than 500ing when the news store is unreachable', async () => {
+ news.months = [];
+ const res = await call('news');
+ expect(res.statusCode).toBe(200);
+ expectWellFormed(res.body);
+ expect(locs(res.body)).toEqual([]);
+ });
+});