The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
i18n info
Metabase's internationalization pipeline has two halves that the scripts in this directory orchestrate:
- Extraction — scan Clojure/TypeScript source for
trs/tru/ ttagttagged-template call sites, emit a gettext source template. - Artifact generation — read back the pinned translated
*.pofiles, apply autofixes for unambiguously repairable translator mistakes, validate the rest, and emit the runtime artifacts atresources/i18n/*.edn(backend) andresources/frontend_client/app/locales/*.json(frontend). Translations the scanner still flags as fatal are dropped; the full triage queue lands in the violations report.
Between the two, Crowdin is where translators fill in
the msgstr values. The .github/workflows/translation-update.yml
workflow runs weekly to pull translations down and open a PR.
Pinned catalogs
locales/translations.pin pins one snapshot of the catalogs, as <version>/<commit>. For
example, v66/516af13bc7e…. Master uses latest/<commit>, and each release branch uses
vNN/<commit>. <commit> is the Metabase commit that published the snapshot.
Snapshots are .tar.gz files in s3://metabase-ci-infra/translations/<version>/<commit>.tar.gz.
Builds download them through CloudFront at
https://d3rmfk6uwjyg5s.cloudfront.net/translations/<version>/<commit>.tar.gz, which needs no
credentials. A snapshot never changes after upload, so a commit always builds with the same
translations.
The catalogs are downloaded into locales/metabase-i18n/<version>-<commit>/, which Git ignores.
Only the current pin is kept: a new pin replaces the old download.
When a release branch is cut, cut-release-branch.yml copies the master snapshot to
vNN/<commit>.tar.gz and pins it on the new branch.
Build
Run ./bin/i18n/build-translation-resources to download the pinned catalogs and build the
frontend JSON and backend EDN resources. ./bin/build.sh runs it. Frontend builds do not:
without the resources, the app shows English. The build also generates the en-ZZ
pseudo-locale into target/i18n/en-ZZ.po.
Publish
The Crowdin workflow publishes updates. To publish by hand, you need AWS credentials that can
write to s3://metabase-ci-infra/translations/. Run from the project root:
checkout=$(./bin/i18n/prepare-translation-update)
./bin/i18n/update-translation-template "$checkout/locales"
./bin/i18n/build-translation-resources "$checkout/locales"
./bin/i18n/publish-translations "$checkout"
rm -rf "$checkout"
publish-translations uploads a new snapshot named after the current HEAD and updates the pin.
It does not overwrite a snapshot that exists, so publish from a new commit each time.
Commit the pin and resources/locales.clj.
To merge the catalogs of another branch, run ./bin/i18n/merge-translations <branch> [-o].
To test the scripts, run ./bin/i18n/test-translations.
flowchart LR
subgraph Extract ["Phase 1: Extraction"]
direction LR
Source["source<br/>(.clj/.cljc/.ts/.tsx)"]
Pot[(External metabase.po)]
Source -->|update-translation-template| Pot
end
subgraph Sync ["Phase 2: Crowdin sync"]
direction LR
Crowdin{{Crowdin}}
LocalePo[(Pinned S3 snapshot)]
Crowdin -->|download| LocalePo
end
subgraph Build ["Phase 3: Artifact generation"]
direction TB
Autofix[["autofix pass<br/>(e.g. escape bare apostrophes)"]]
Scanner[["validation scanner"]]
Edn["resources/i18n/*.edn<br/>(backend)"]
Json["resources/frontend_client/<br/>app/locales/*.json<br/>(frontend)"]
Csv["target/i18n-violations.csv"]
Autofix --> Scanner
Scanner ==> Edn
Scanner ==> Json
Scanner ==> Csv
end
Pot -->|upload| Crowdin
LocalePo ==>|build-translation-resources| Autofix
Extraction
Source code → external metabase.po
Building the backend pot file
Building the backend pot file can be done from the command line (from the project root directory):
❯ clojure -X:build:build/i18n
Created pot file at cli.pot
Found 1393 forms for translations
Grouped into 1313 distinct pot entries
This is called from update-translation-template which builds frontend, backend, and automagic dashboard pot files and then merges them into one artifact with msgcat.
Developer information
We use a custom script to build up our backend pot (po template) file. Start a repl from this folder and the work is done in the src/i18n/enumerate.clj file.
It uses the very helpful library Grasp to use a spec and a list of sources to find all uses of trs, tru, etc. It returns the forms found from the source, along with metadata that includes the filenames and line numbers.
enumerate=> (def single-file (str u/project-root-directory "/src/metabase/util.clj"))
#'i18n.enumerate/single-file
enumerate=> (map (juxt meta identity)
(g/grasp single-file ::translate))
([{:line 81,
:column 13,
:uri "file:/Users/dan/projects/work/metabase/src/metabase/util.clj"}
(trs
"Maximum memory available to JVM: {0}"
(format-bytes (.maxMemory (Runtime/getRuntime))))]
[{:line 313,
:column 33,
:uri "file:/Users/dan/projects/work/metabase/src/metabase/util.clj"}
(tru "Timed out after {0}" (format-milliseconds timeout-ms))]
[{:line 504,
:column 26,
:uri "file:/Users/dan/projects/work/metabase/src/metabase/util.clj"}
(tru "Not something with an ID: {0}" (pr-str object-or-id))])
Pot files can only have a single entry for each string, so these are grouped by the message and then we use the org.fedorahosted.tennera.jgettext library to emit the pot file.
You can build quick pot files with the helpful function create-pot-file!
enumerate=> (create-pot-file! single-file "pot.pot")
Created pot file at pot.pot
{:valid-usages 3, :entry-count 4, :bad-forms ()}
which will output
# Copyright (C) 2022 Metabase <[email protected]>
# This file is distributed under the same license as the Metabase package
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: 1.0\n"
"Report-Msgid-Bugs-To: [email protected]\n"
"POT-Creation-Date: 2022-07-22 14:03-0500\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
#: /Users/dan/projects/work/metabase/src/metabase/sync/analyze/fingerprint/fingerprinters.clj
msgid "Error generating fingerprint for {0}"
msgstr ""
#: metabase/util.clj:81
msgid "Maximum memory available to JVM: {0}"
msgstr ""
#: metabase/util.clj:313
msgid "Timed out after {0}"
msgstr ""
#: metabase/util.clj:504
msgid "Not something with an ID: {0}"
msgstr ""
Overrides
You'll note that we defined a single file "/src/metabase/util.clj" but ended up with an entry from fingerprinters.clj. This is because that usage of trs is inside of a macro which emits a defrecord. The quoting means that the form doesn't actually match our spec since it has more sequence stuff:
user=> '`(trs "foobar")
(clojure.core/seq (clojure.core/concat (clojure.core/list (quote user/trs)) (clojure.core/list "foobar")))
More information in the grasp issue. A quick workaround is just including this manual override.
Artifact generation
Pinned *.po files → resources/i18n/*.edn + resources/frontend_client/app/locales/*.json
Once translators have filled in the external <locale>.po files through Crowdin, the build pipeline turns
those into the runtime artifacts that Metabase actually ships:
- Backend:
resources/i18n/<locale>.edn— consumed bymetabase.util.i18n.impl/translateviajava.text.MessageFormat. - Frontend:
resources/frontend_client/app/locales/<locale>.json— consumed by ttag in the browser.
Run the whole pipeline locally with:
❯ ./bin/i18n/build-translation-resources
This calls clojure -X:build:build/i18n, which invokes
i18n.create-artifacts/create-all-artifacts!. For each locale:
- Parse the
.pofile viai18n.common/po-contents(jgettext). - Run the autofix pass via
i18n.autofix/autofix-po-contentsto silently correct translator mistakes that can be unambiguously repaired (currently: unescaped apostrophes for backend strings). - Run the scanner
i18n.validation/invalid-messages-in-poon the post-autofix contents to produce a seq of violation maps for everymsgstrthat still fails validation — see violations report below. - Derive a drop-set of msgids using
i18n.validation/drop-from-build?(see drop policy) and pass it, along with the post-autofixpo-contents, to both backend and frontend artifact writers. The writers skip any message in the drop-set — the runtime falls back to English for those msgids. - After all locales finish, aggregate violations across every locale and write
target/i18n-violations.csvandtarget/i18n-violations.edn.
The source of truth for this pipeline lives in bin/build/src/i18n/:
autofix.clj— pre-validation fixes (e.g. apostrophe escaping).validation.clj— the scanner + drop policy.create_artifacts.clj— orchestrator + report writer.create_artifacts/backend.clj— backend.ednwriter.create_artifacts/frontend.clj— frontend.jsonwriter.common.clj—.poparser and shared helpers (backend-message?,po-contents).
Autofix pass
Runs once per locale, before validation, on the parsed po-contents. Its job is to silently
correct translator mistakes that can be unambiguously repaired — making translations that would
otherwise fail the scanner ship correctly. Both the scanner and the artifact writers consume the
post-autofix contents, so they agree on what's being shipped.
Current autofixes (all backend-only — frontend msgstr values pass through unchanged because
ttag has different escaping rules):
- Apostrophe escaping. Word-adjacent single apostrophes (e.g. Catalan
d'aquí, EnglishIt's) are doubled to''sojava.text.MessageFormattreats them as literal characters instead of escape delimiters. Apostrophes that are already doubled, or that sit next to{/}(i.e. intentional MessageFormat escapes like'{0}'), pass through untouched.
This is handled before the scanner, so translations that only had apostrophe typos never reach
the violations report. If you ever see d'aquí-style entries in the report, it means autofix
couldn't reach them (likely a non-backend context) and you'd need to fix them in Crowdin.
Extending the autofix layer: add additional transformations to i18n.autofix/autofix-po-contents
only when the fix is (a) deterministic, (b) semantically neutral (rendered output unchanged), and
(c) covers a common translator mistake. Everything else should surface in the violations report
instead.
Drop policy
drop-from-build? only returns true for violations whose translations would throw at runtime
and trigger English fallback anyway — i.e. :invalid-message-format. :skipped-arg-index and
:arg-count-mismatch render imperfectly but don't throw; we keep them so users see mostly-
localized text rather than regress to full English fallback.
Violations report
target/i18n-violations.csv
Every artifact-generation run emits a report of violations found, in two forms:
target/i18n-violations.csv— one row per violation, RFC 4180-compliant, openable in any spreadsheet app. This is the file that gets uploaded as thei18n-violationsartifact in CI.target/i18n-violations.edn— same data as a vector of Clojure maps, for programmatic use.
In CI, .github/workflows/i18n.yml and
.github/workflows/translation-update.yml
upload the artifact. The auto-generated translation-update PR gets a summary comment with
per-type counts and a direct link to the artifact.
Columns
| Column | Meaning |
|---|---|
locale |
The locale the violation was found in (e.g. fr, de, pt-BR, ar-SA). |
types |
The set of violation types for this row, joined with + (e.g. skipped-arg-index+arg-count-mismatch). See violation types below. |
dropped |
true if this translation was excluded from the built .edn/.json artifact (runtime will fall back to English). false if it was kept in the artifact despite imperfection (runtime renders with the issue visible). |
backend |
true if the msgid originates from Clojure source (.clj / .cljc), false if from frontend source (.ts / .tsx / .js / .jsx). This determines which format-system rules were applied. |
plural_index |
For singular (non-plural) strings this is empty. For plural strings it's the index of the affected form (0, 1, …) within the locale's msgstr[N] array. |
msgid |
The English singular source string — the key translators see in Crowdin. |
msgid_plural |
The English plural form, if the string has one. Empty for non-plural strings. |
msgstr |
The translated string that failed validation — the actual text the translator produced. |
error |
Populated only for invalid-message-format rows — the java.text.MessageFormat exception message (e.g. Unmatched braces in the pattern.). Empty for other types. |
expected_args |
For arg-count-mismatch rows: the acceptable placeholder count(s), derived from msgid and msgid_plural. Comma-separated (e.g. 0,1 for a plural where singular has 0 placeholders and plural has 1). |
actual_args |
For arg-count-mismatch rows: the number of distinct placeholder indices actually used in msgstr. |
source_refs |
Space-separated path:line pointers back to the source code where this msgid is used. |
Violation types
Three types can appear in the types column. A single row may list multiple (separated by +)
when they fire together.
invalid-message-format
The msgstr isn't a valid java.text.MessageFormat pattern — the constructor throws
IllegalArgumentException. Typical causes:
- Unmatched braces (
{0or{{...}}without proper escaping). - Named placeholders like
{login}that Java MessageFormat can't parse (it expects numeric indices).
These rows always have dropped=true. At runtime, MessageFormat's constructor would throw,
triggering the runtime's try/catch and falling back to English anyway — so the build pre-empts
it and guarantees a clean .edn artifact.
Action for translators: Fix the msgstr in Crowdin. Usually means matching the English
string's escaping conventions (double-apostrophes for literal ', matched braces, etc.).
skipped-arg-index
The translated string's {N} placeholders reference non-contiguous indices (e.g. {0} and
{2} but no {1}), or use the same index multiple times in a way that produces a gap. At
runtime, missing indices render literally ({2} appears as the four characters {2} in the
output).
These rows have dropped=false — the translation is kept in the artifact. The user sees mostly-
localized text with a literal {N} glitch, rather than full English fallback.
Action for translators: Fix the msgstr so all placeholders are contiguous ({0}, {1},
{2}, …).
arg-count-mismatch
The translated string's placeholder count doesn't match any acceptable count from the English
source forms. expected_args is the set of acceptable counts (singular's count and plural's
count, if applicable); actual_args is what the translation has.
These rows have dropped=false. At runtime, missing placeholders render as literal text ({0}
shows up verbatim), or extra caller-supplied args get silently dropped.
Common cause: the translator added an extra {0} (e.g. for pluralization) when English didn't
have one, or omitted a {0} that English required.
Note: Romance-language apostrophe mistakes (e.g. Catalan
d'aquíswallowing a trailing{0}in MessageFormat's escape semantics) would have triggered this violation. The autofix pass handles them upstream, so they shouldn't appear in the report. The mistakes would still be present in Crowdin though.
Action for translators: Match the English source's placeholder count. If the source has
{0} and {1}, the translation must too — or the plural form of the source, which might have
a different count.
How to use the report
- Sort by
localeto see per-language issues — often one translator has a systematic problem (e.g. Catalan translators consistently forget to double apostrophes). - Filter
dropped=trueto see what's silently falling back to English — these are the most user-visible regressions for locale users. - Filter
types=invalid-message-formatif you're triaging Crowdin bugs — these are always the highest-priority fixes. - Use
source_refsto understand which UI the violation affects, and whether it's a high- traffic string or rare edge case.
Each row has enough info to find the string in Crowdin: the msgid is the source key, locale
tells you which language to open.
Runtime behavior summary
| Violation type | Build-time action | Runtime behavior |
|---|---|---|
invalid-message-format |
Dropped from artifact | English fallback |
skipped-arg-index |
Kept in artifact | Localized text with literal {N} for missing indices |
arg-count-mismatch |
Kept in artifact | Localized text with literal {N} for missing placeholders, or extra args silently dropped |