Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

28.2 KB
Raw Blame

Internationalization (i18n)

three.ws localizes its UI the way LobeHub does: a single source-of-truth catalog is translated incrementally by an LLM, the output is committed as static JSON, and the browser swaps copy at runtime from those static files. There is no per-request translation call, no runtime cost, and full SEO/caching of each locale's JSON.

The one difference from LobeHub: their copy was already keyed (greenfield i18next). Ours lives inline in 600+ static HTML pages, so we add an extraction step that derives the catalog from annotated markup.

i18n:annotate ──▶ i18n:extract ──▶ public/locales/en.json ──▶ i18n:translate ──▶ public/locales/<lang>.json
 (auto data-i18n)     (build)        (source of truth)        (LLM, glossary-locked)    (committed, static)
        │                                                                                      │
   or hand-annotate                                                              runtime: /i18n.js swaps the DOM

1. Annotate copy

You almost never do this by hand. npm run i18n:annotate scans static HTML and injects the right data-i18n* attribute on every user-visible copy element, so a whole page becomes translatable in one command.

npm run i18n:annotate -- --pages="pages/pricing.html"          # dry-run, preview keys
npm run i18n:annotate -- --pages="pages/pricing.html" --apply  # write attrs + wire /i18n.js
npm run i18n:annotate -- --apply                               # every htmlExtract entry

The annotator (scripts/i18n-annotate.mjs) is built to be safe on hand-authored HTML:

  • Byte-surgical. It parses only to locate elements (via node-html-parser source offsets) and splices data-i18n="…" into the original string. Untouched bytes are preserved exactly (no whole-file reserialization), so diffs are minimal and reviewable, and stripping the inserted attributes reproduces the original file byte-for-byte.
  • Idempotent. An element that already carries an annotation is left alone; re-runs only pick up newly-added copy.
  • Conservative. Only leaf copy elements with real letters are touched. script/style/svg/pre/code/template/textarea, dynamic {{…}}/${…} copy, translate="no" / data-no-i18n opt-outs, and anything nested inside an element it already chose are skipped. <title>, meta description, and og/twitter tags are annotated too.
  • Self-wiring. With --apply it also injects <script type="module" src="/i18n.js"> if the page lacks it, and the runtime then auto-mounts a floating <lang-switcher> on any page that doesn't place its own, so a freshly-annotated page gets a working language picker with zero layout edits.

Keys are namespaced by page (pricing.plans_h, what_is.hero). To annotate by hand, or to understand what the tool emits, the attributes are:

Attribute Applies to Example
data-i18n="key" textContent <span data-i18n="common.take_tour">Take the tour</span>
data-i18n-html="key" innerHTML (copy with inline markup) <li data-i18n-html="home.bullet_earn"><strong>Earn</strong> USDC…</li>
data-i18n-attr="attr:key;…" element attributes <meta data-i18n-attr="content:home.meta_desc" content="…">

Keys are dot-namespaced (home.hero_h, common.language). Group by surface.

2. Extract the source catalog

npm run i18n:extract          # scan annotated HTML → public/locales/en.json
npm run i18n:extract -- --force   # overwrite values from current HTML (after copy edits)
npm run i18n:extract -- --prune   # drop keys no element references anymore

Existing values are preserved by default, so re-running never clobbers a reviewed string. Collisions (same key, different English in two files) fail the command.

Copy that JS builds, not markup

The extractor reads data-i18n attributes, so it sees every static page and none of the copy a module assembles at runtime. That gap is real: the connected-home lane renders its rooms, its devices, its refusals, its confirmation and every spoken announcement from live data, and none of it has markup to annotate. Those call sites go through a tiny bridge instead (src/home/i18n-home.js), where the English argument IS the source value, exactly as an element's own text is for an annotated tag:

import { plural, t } from './i18n-home.js';

t('home_floorplan.saved_title', 'Floorplan saved');
plural('home_floorplan.loose_count', n, '{{count}} device not in a room', '{{count}} devices not in a room');

npm run i18n:home (scripts/i18n-home-keys.mjs) reads those call sites and reports the difference against en.json; --write applies it. Run it after adding a string, before i18n:translate, or the key never reaches a translator:

npm run i18n:home             # report drift, exit 1 if any
npm run i18n:home -- --write  # add and update keys in en.json

User data is never part of a source string. A room name, an area name, a scene name, a device name, an address somebody pasted and a message their own Home Assistant returned are their words, not ours. They are passed as interpolated values, which the masker already protects byte for byte:

t('home_floorplan.filed_title', '{{device}} is in {{room}} now', { device: entityId, room: nameOf(areaId) });

and never as t('...', `${entityId} is in ${nameOf(areaId)} now`), which would put somebody's "Nursery" into the source catalog and hand it back to them machine-translated. i18n:home refuses any source value that interpolates a template expression, so the rule is mechanical rather than remembered.

Two things do not belong in the catalog at all. Relative times (relativeAge in the same bridge) go through Intl.RelativeTimeFormat, and numbers through Intl.NumberFormat, because an English s appended in code is the wrong plural in most of these languages and several need forms English has no word for. The browser already ships every rule.

3. Translate

npm run i18n:translate                     # translate only missing keys, every locale (incremental)
npm run i18n:translate -- --locale=es      # one locale
npm run i18n:translate -- --force          # retranslate everything
npm run i18n:translate -- --dry-run        # show what would translate (also refreshes manifest)
npm run i18n:translate -- --concurrency=8  # widen the chunk pool for a bulk run
npm run i18n:translate -- --split-token=3600 # bigger chunks, fewer requests
npm run i18n                               # extract + translate

Configuration lives in .i18nrc.json (schema: scripts/i18n-rc.schema.json). Backend is selected by provider, overridable per run with --provider= / --model=. Env-var names match api/_lib/chat-models.js, so a key that already powers /chat works here too. .env and .env.local are read automatically - an exported variable still wins, so a one-off GEMINI_API_KEY=… npm run i18n:translate overrides the file.

--split-token= overrides the chunk budget for one run. It is the only lever that changes the request count for a fixed amount of text, which matters because the free lanes meter requests per day, not tokens: doubling the budget halves the requests and leaves the token spend unchanged. A 668-key backlog across 84 locales is ~840 requests at the committed budget and ~570 at --split-token=3600, against a free-tier ceiling of 1000 requests/day. Raise it on a lane whose model holds a long JSON reply together; leave it alone on the weak free models, where small chunks truncate less.

provider free? credentials notes
vertex (default) credits GOOGLE_CLOUD_PROJECT + GCP auth billed to the platform's GCP credits; no free-tier quota wall
gemini ✅ free GEMINI_API_KEY / GOOGLE_API_KEY https://aistudio.google.com/apikey - free tier, best CJK
groq ✅ free GROQ_API_KEY https://console.groq.com/keys - fast; the script defaults to qwen/qwen3.8-27b (the model /chat uses on this lane since Groq retired llama-3.3-70b-versatile in August 2026); pass --model= to override
openrouter ✅ free OPENROUTER_API_KEY https://openrouter.ai/keys - use a :free model
nvidia ✅ free NVIDIA_API_KEY https://build.nvidia.com - free NIM credits
openai paid OPENAI_API_KEY (+ OPENAI_BASE_URL) any OpenAI-compatible endpoint
anthropic paid ANTHROPIC_API_KEY Claude Messages API

Vertex takes GCP credentials three ways, in order: GCP_SERVICE_ACCOUNT_JSON, the Cloud Run / GCE metadata server, then the gcloud CLI. On a developer machine the first two are absent, so an authenticated gcloud auth login is all it needs. If none of the three resolve, the run aborts rather than degrading (see Never silently English below).

An OPENROUTER_API_KEY in the environment is automatically added as a failover rung behind whichever provider is primary, so a single 429 wall does not end a run.

Leave modelName unset to use the provider's default model, or pin one (e.g. "modelName": "meta-llama/llama-3.3-70b-instruct:free" for OpenRouter). Free tiers rate-limit, so concurrency defaults to 2 and the translator backs off and retries on HTTP 429 (honoring Retry-After). A Vertex run billed to our own project has far more headroom - pass --concurrency=8 for a bulk job like generating a locale from scratch (~550 chunks) instead of editing the config every other lane shares.

Never silently English

When a translation call fails, the pipeline bakes the English source into the key so the catalog stays complete and the runtime degrades gracefully. That is right for the occasional value a model can't render as valid JSON, and very wrong for a missing credential, which fails identically on every key: the run would march through the whole catalog rewriting it into English and exit 0, with lint green and the language still offered in the picker. So a credential/config error is classified separately - it is never retried, never baked, and aborts the run with the one line that says what to fix. Everything already translated is kept (the pipeline persists after each chunk) and a re-run resumes.

A spent balance counts as one of these. HTTP 402 joins 401 and 403 in that classifier because a funded key with no credits left answers 402 Insufficient credits to every single call: not a config error, not an auth rejection, and so for a while not fatal either. The run treated it as a value the model could not render, halved the chunk, halved it again, and baked English onto each lone key. It cost 198 Spanish keys before anyone noticed, because the run exited 0. A payment wall is a property of the account, never of the string, so splitting and retrying cannot change the answer.

When a run does abort on credentials, read the status before assuming the key is wrong. vertex 403: Lightning dunning decision is deny for project is Google billing enforcement against the whole GCP project, so every Vertex model in every region returns it and no amount of re-authenticating helps; the fix is on the billing account. 402 is the balance. 401 really is the credential.

Two more failures abort rather than degrade, and both cost a catalog before they did. A GCP credential that exists but no longer refreshes (this workspace's Application Default Credentials, once its refresh token was revoked) used to escape the Vertex token path as an ordinary error and route every key into the English fallback; it is now a config error that names both the credential's reason and gcloud's. And a 5xx that outlives its whole backoff budget is treated exactly like a 429 that does: a gateway that is down is a property of the backend, not of the string being translated, and no amount of halving the chunk brings it back. On 2026-09-09 the threews proxy answered 502 {"error":"upstream_error","status":403} and then 503 {"error":"provider_unavailable"} on every request because its own free-tier chain was exhausted, and the run walked an entire Arabic catalog baking English one key at a time, printing a warning per key, heading for exit 0.

i18n:backfill (below) exists to clean up catalogs damaged by this before the guard existed.

Quality note: the Groq lane is fast and solid for Latin-script languages. For CJK/Arabic/Hindi, Gemini (free tier or Vertex) is noticeably better - switch per run with no code change. Whatever you pick, npm run i18n:lint catches any model that drops a placeholder, a glossary term, or a tag.

Glossary lock (the important part)

Terms in doNotTranslate ($THREE, the contract address, x402, USDC, Solana, IBM watsonx.ai, the official handles and channels such as @trythreews, t.me/three_ws, [email protected], and so on) plus every {{placeholder}}, HTML tag, and HTML entity are masked to opaque sentinels before the text reaches the model and restored after. The model literally never sees them, so they come back byte-for-byte and can never be translated, reordered, or hallucinated around. $THREE is the only coin this platform references; this guarantees no machine translation can change that.

Entities are masked for a reason that is easy to miss. A value harvested from a data-i18n-html element is raw markup, and the runtime puts it back through innerHTML. So &lt; is source syntax, not typography: a model that helpfully decodes

<code>&lt;agent-3d&gt;</code>     <!-- a printed code sample -->

into

<code><agent-3d></code>          <!-- a live custom element -->

has just instantiated a 3D avatar in the middle of the SDK docs and swallowed the copy that followed. Masking makes the round trip byte-exact; the lint rule below catches any that predate it.

4. Lint

npm run i18n:lint

Validates every committed locale against the source: no missing/empty keys, no {{placeholder}} drift, no dropped glossary terms, no stale keys, and no markup drift. Locales not yet generated are skipped (not failures). Needs no API key, so it is safe to run anywhere, but nothing runs it for you: it is not wired into npm run gate or the build, so run it yourself after touching a catalog.

Markup drift compares the set of HTML tags in the translation against the source. It fires when a translation grows a tag the English never had (an escaped tag that got decoded into a live element, or one the model invented) or loses one it should have kept. It deliberately stays quiet on cosmetic entities: &amp;, &mdash; and &#39; decode to the same glyph the reader sees either way, so flagging them would bury the failures that actually break a page.

Prune (drop keys the source retired)

npm run i18n:prune -- --dry-run   # report what would go, write nothing
npm run i18n:prune                # every locale
npm run i18n:prune -- --locale=de # one locale

Renaming or deleting a source key leaves its old translation behind in all 84 catalogs, and lint counts every copy, so one renamed key reads as 84 failures (stale key (not in source)). A full i18n:translate run clears them as a side effect, because persist() rebuilds each catalog from the source. That is the wrong dependency for a purely subtractive fix: it puts a deletion behind a working backend, so an exhausted quota or an expired credential leaves lint red with no way forward. i18n:prune is that deletion on its own. No API key, no network, and it only ever removes: a key the source no longer defines goes, a namespace left empty goes with it, and everything else stays byte-identical.

It deliberately does not reuse mergeOrdered. That helper rebuilds the catalog from the source, so it would materialize an empty string for every key the locale has not been translated for yet. Those keys would stop reporting as missing and start reporting as empty, and because writeManifest counts empty values, every affected language would drop straight out of the picker.

Repair (fix lint failures in place)

npm run i18n:repair                 # every locale
npm run i18n:repair -- --locale=he  # one locale

Glossary terms, {{placeholders}}, tags, and entities are masked to sentinels the model must reproduce verbatim. A small model occasionally omits a sentinel mid-sentence, so the term never gets restored and the key fails lint. --repair re-translates only the lint-failing keys (dropped glossary term, placeholder drift, or markup drift): first all of them together in normal-sized chunks, then one at a time for whatever the batch got wrong, retrying each until it passes its own integrity check. A key that never converges after four attempts is baked to English (lint-clean, and the runtime already falls back to English for anything missing). Idempotent: a clean catalog reports nothing to repair. Takes the same --provider= / --model= overrides as i18n:translate.

The batched first pass is what makes repair finishable. One request per key is fine for the stray sentinel drop repair was written for, but the failure that actually fills the list is an English string edited after the locales were translated: i18n:translate only fills keys that are missing, never keys whose source changed under them, so the stale translation sits there until lint notices the glossary term or the markup no longer matches. That fails the same key in every locale at once, and 11 such keys is 924 single-key requests, more than a free lane's entire daily budget.

Backfill (find copy that silently stayed in English)

npm run i18n:backfill                    # report only
npm run i18n:backfill -- --apply         # re-translate every locale
npm run i18n:backfill -- --apply --locale=de

i18n:translate only fills keys that are missing. That interacts badly with the English bake above: when the backend is unreachable (an expired credential, a quota wall), whole runs bake English into the locale, and because those keys now exist, every later run skips them. Nothing reports it. Lint stays green, the key count is complete, and the language picker still offers the language while the page reads as English. This is how Punjabi ended up 99.5% English while looking fully generated.

i18n:backfill compares each locale against the source, clears the values that are byte-identical to English, and re-runs the translator so they come back as real copy. Matching English is not on its own proof of a failure, so it only touches strings carrying two or more real words once glossary terms, markup, URLs and numbers are discounted. A single word (Avatar, 3D), a brand string (three.ws 3D API), a version (OpenAPI 3.1), a copyright line, or a fragment that is mostly markup stays identical in every language, correctly; clearing those would burn a translation call per run to get the same bytes back. Locales are processed worst-first, and it refuses to start while another translation run is live, since two runs rewriting the same catalogs drop each other's keys. Only languages listed in manifest.json are repaired, so a locale nobody can select yet never takes capacity from one visitors are reading; pass --all-locales to include the rest.

Clearing a key is safe at every instant: the runtime resolves a missing or empty value against the English catalog, so a locale mid-backfill renders exactly what it rendered before. Progress is saved per chunk, so an interrupted run keeps what it finished and re-running resumes.

Run it after any incident where the translation backend was down, and any time a locale's word count looks right but the page still reads in English.

5. Runtime

src/i18n.js (shipped at the stable URL /i18n.js) auto-runs on any page that includes it:

<script type="module" src="/i18n.js"></script>
<lang-switcher></lang-switcher>
<!-- accessible language picker -->

It detects the locale (?lang= → localStorage → navigator.languages → default), fetches /locales/<code>.json once, rewrites the annotated DOM, sets <html lang>/dir (RTL-aware), and persists the choice. Missing translations fall back to English, so partial locales degrade gracefully. window.threewsI18n exposes { t, setLocale, getLocale, initI18n }.

An annotated element whose content becomes live data (a balance, a status, a count) sets data-i18n-owned="1" the moment its script writes a real value into it. The annotation still ships so the pre-render placeholder is translated, but from then on the catalog pass skips the element: without that, the pass (which lands after the async catalog fetch) reverted a freshly rendered value back to its translated "Loading" placeholder.

Most writers never set that attribute, so the runtime also infers ownership on its own (primeOwnership in src/i18n.js, run synchronously at boot). It records what each annotated element held when it first saw it and what the catalog pass last wrote there. If the element now holds anything else, and that content is not its English catalog value either, the page wrote it, and the pass leaves it alone. When the page puts the source text back (a Pause button returning to "Play"), the next pass translates it again. Pinned by tests/i18n-script-ownership.test.js.

Inference covers text a script writes into the annotated element itself. It does not make a live value inside a data-i18n-html block safe: that pass rewrites the block's innerHTML and detaches every child a script cached. Keep the label in its own data-i18n element and the value as a sibling:

<span class="stat"><span data-i18n="monitor.stat_agents_label">agents</span> <b id="t-agents">·</b></span>

The <lang-switcher> reads public/locales/manifest.json, which lists only locales whose catalog is complete, so it never offers a language that would fall back to English. Presence of the file is not enough: writeManifest() counts untranslated (empty) values via untranslatedCount() and holds back any catalog that still has them, reporting what it skipped:

• manifest: 48 locale(s) listed, 1 held back until complete: as (3 untranslated keys)

So a locale that is mid-translation is simply absent from the switcher until the run finishes, instead of shipping as a half-translated page. i18n:translate writes the manifest twice, before the run and again after it, so the locales a run just completed are listed by the time it exits rather than one run later.

The switcher shrinks where space is tight. Below the 640px breakpoint, or on any page that sets a compact attribute on <lang-switcher> (a narrow HUD rail), the control shows the locale code (shortLocaleLabel(): EN, PT, ZH) in place of the full language name, which otherwise makes a ~170px control. The native <select> still owns the interaction and keeps the full names in its options, so the picker, keyboard and screen readers behave exactly as before; the host also accepts a max-width and follows it instead of pushing its siblings off screen.

Adding a language

Add the code (and display name) to outputLocales / localeNames in .i18nrc.json (plus rtlLocales for a right-to-left script), run npm run i18n:translate -- --locale=<code>, review the output, and commit public/locales/<code>.json. The manifest and switcher pick it up automatically.

Only ship a locale once its catalog is complete. A run that dies partway (expired credentials, an exhausted provider) still leaves a catalog file with the full key skeleton and empty string values for everything it never reached. That file looks finished by key count, and because the runtime falls back to English per key, the page renders as a half-translated mix rather than failing loudly. Verify before committing:

npm run i18n:translate -- --lint --locale=<code>   # flags empty values + glossary drops
node -e "const f=require('./public/locales/<code>.json'),e=require('./public/locales/en.json');
const n=o=>Object.values(o).reduce((a,s)=>a+Object.keys(s).length,0);
let m=0;for(const s of Object.values(f))for(const v of Object.values(s))if(!String(v).trim())m++;
console.log('keys',n(f),'of',n(e),'| empty',m)"

Re-running i18n:translate is the fix: missingKeys() treats an empty string as untranslated, so a resumed run tops up exactly the gaps and costs nothing for the keys already done. For a catalog that is complete but trips the glossary/placeholder/markup lint, npm run i18n:repair -- --locale=<code> re-translates only the failing keys.

Shipping is gated on completeness rather than on your memory: every run rewrites public/locales/manifest.json and omits any locale with untranslated keys, so a half-built catalog is never offered in the switcher. That is what makes a from-scratch locale safe to build incrementally across several sessions.

Building the locales that have never been generated

outputLocales is deliberately longer than the shipped set. To work through the backlog:

scripts/i18n-generate-queue.sh          # every incomplete locale, 5 at a time
scripts/i18n-generate-queue.sh 3        # narrower pool
scripts/i18n-generate-queue.sh 5 "ca sr my"   # a specific list, in order

Each locale is ~18k keys / ~550 chunks, so this is an hours-long job, not a minutes-long one. It picks locales most-complete-first (closest to shipping ships soonest), skips any locale another process is already translating (two writers on one catalog would lose one set of writes), and is resumable: the translator persists after every chunk, so a killed queue loses at most one chunk.

Concurrent agents share this worktree. Before starting a bulk run, check ps -eo args | grep i18n- - a locale being written by someone else's backfill must not also be written by you.

SEO: hreflang alternates

Copy swaps client-side (the LobeHub model), and each locale is reachable at a distinct URL via ?lang=<code>, which detectLocale() honors. Search engines are told those translated URLs exist two ways, both driven off the manifest so they track annotation automatically:

  1. Sitemap alternates. npm run build:pages writes public/locales/localized-pages.json (every registered page whose HTML carries a data-i18n annotation). The dynamic core sitemap (api/sitemap/[type].js) reads it and emits an <xhtml:link rel="alternate" hreflang> per locale, plus x-default, for exactly those routes.
  2. On-page alternates. The runtime injects <link rel="alternate" hreflang> tags into <head> for the current path (default locale and x-default at the bare URL, others at ?lang=xx), so a page crawled directly advertises its translations even without the sitemap.

<html lang>/dir are set to the active locale on every swap. A build-time prerender that bakes each locale into static HTML (eliminating the first-paint English flash) is the natural next step; the catalogs are static and ready.

Tests

tests/i18n.test.js covers extraction, glossary round-trips, lint rules, runtime resolution/fallback, and catalog merge/prune. tests/i18n-markup.test.js pins the markup guarantees: entities survive a round trip byte-for-byte, an escaped tag that got decoded into a live element fails lint, and cosmetic entities do not. tests/i18n-backfill.test.js pins the rules for clearing silently-English values out of a shipped catalog.