Skip to content

The open alternative to Salesforce, designed for AI.

[Apps 2-way sync] Pull write translations (#25567)

Weiko550e4e6fddd0
diff --git a/packages/twenty-docs/developers/extend/apps/getting-started/project-structure.mdx b/packages/twenty-docs/developers/extend/apps/getting-started/project-structure.mdx
index 2baa77e4405..681169670cd 100644
--- a/packages/twenty-docs/developers/extend/apps/getting-started/project-structure.mdx
+++ b/packages/twenty-docs/developers/extend/apps/getting-started/project-structure.mdx
@@ -49,7 +49,7 @@ my-twenty-app/
| `src/front-components/`, `src/navigation-menu-items/`, `src/page-layouts/` | A starter welcome page: a front component rendered by a standalone page layout, reachable from the sidebar. |
| `src/__tests__/` | A unit test plus an integration test (with its global setup) that syncs the app against a real server. |
| `public/` | Static assets (images, fonts) served with your app. |
-| `locales/` | Translation catalogs. `en.json` is the source catalog `dev:translations-extract` fills in; the set of files here is what a sync publishes, so an empty folder declares no translations. |
+| `locales/` | Translation catalogs. `en.json` is the source catalog `dev:translations-extract` fills in; the set of files here is what a sync publishes, so an empty folder declares no translations. `locales/compiled/` is written by `pull` for entries whose source strings are not in the tree yet, is published by a sync too, and is not meant to be edited by hand. |
| `AGENTS.md` / `CLAUDE.md` | Guidance for AI coding agents working on the app. |
| `CHANGELOG.md` / `SETUP.md` | Changelog of notable changes and setup instructions for local development. |
diff --git a/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdx b/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdx
index ce11e4ba1e2..3bb2ca93d51 100644
--- a/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdx
+++ b/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdx
@@ -98,7 +98,7 @@ A plan only previews **metadata** changes. It also works for an app that was nev
## Pulling an installed app back to source
<Warning>
-`yarn twenty pull` is **experimental**. It covers only part of an application today, its output shape may change between releases, and it overwrites the files that define the entities it pulls. It does not restore translations yet: the report names the locales the app has published, and pushing a pulled tree replaces them with whatever your `locales/` folder holds. Commit your work before running it, and review the diff.
+`yarn twenty pull` is **experimental**. It covers only part of an application today, its output shape may change between releases, and it overwrites the files that define the entities it pulls. It restores published translations into `locales/`, keeping the entries whose source strings are not in your tree yet in `locales/compiled/` so that a push does not lose them. Commit your work before running it, and review the diff.
</Warning>
`yarn twenty pull` is the inverse of `apply`: it reads an application out of the workspace and writes it back as define files.
@@ -114,7 +114,8 @@ Pull needs an existing project — scaffold one with `npx create-twenty-app@late
- the application config;
- objects, with their fields inline;
- fields your app added to objects it does not own;
-- authored indexes.
+- authored indexes;
+- published translations, as `locales/<locale>.json` for the strings pull can read back from the pulled metadata and source, and `locales/compiled/<locale>.json` for the rest.
Each entity goes to the file that already defines it, so a second pull rewrites only what changed on the server. New entities are placed beside existing files of their kind, and a generated name that would land on an unrelated file is qualified rather than overwriting it.
diff --git a/packages/twenty-docs/developers/extend/apps/translations/overview.mdx b/packages/twenty-docs/developers/extend/apps/translations/overview.mdx
index 2dfde4d5c7c..33a171eeee7 100644
--- a/packages/twenty-docs/developers/extend/apps/translations/overview.mdx
+++ b/packages/twenty-docs/developers/extend/apps/translations/overview.mdx
@@ -132,6 +132,19 @@ syncing is enough to see the new text. Front component catalogs are baked into
each component bundle at build time, so changing one of those still means
re-running `twenty dev:build` (and redeploying).
+## Pulling translations
+
+`twenty pull` writes an application's published translations back into
+`locales/`. A published catalog is keyed by message id, so pull recovers the
+readable form only for strings whose source it can see: the labels of the
+metadata it pulled and the `t()`/`msg()`/`Trans` strings of front components
+present in the project. Everything else is kept as is in
+`locales/compiled/<locale>.json`, keyed by message id. The build merges those
+files into the catalog (an entry in `locales/<locale>.json` wins over a
+compiled one with the same id), `dev:translations-extract` leaves them alone,
+and every pull rewrites them, so they shrink as more of the app's source lands
+in the tree. Do not edit them by hand; translate in `locales/<locale>.json`.
+
A project without a `locales/` folder says nothing about translations, so a
sync from it leaves whatever the app already published untouched. To remove
published translations on purpose, keep the folder and delete the locale files
diff --git a/packages/twenty-sdk/src/cli/commands/dev/pull.ts b/packages/twenty-sdk/src/cli/commands/dev/pull.ts
index 75f14d68ed8..da92c206f94 100644
--- a/packages/twenty-sdk/src/cli/commands/dev/pull.ts
+++ b/packages/twenty-sdk/src/cli/commands/dev/pull.ts
@@ -49,7 +49,8 @@ export class AppPullCommand {
coverage: result.data.coverage,
localOnlyRelativePaths: result.data.localOnlyRelativePaths,
unreadableRelativePaths: result.data.unreadableRelativePaths,
- translations: result.data.translations,
+ compiledTranslationEntryCountByLocale:
+ result.data.compiledTranslationEntryCountByLocale,
verbose: options.verbose,
});
diff --git a/packages/twenty-sdk/src/cli/operations/pull.ts b/packages/twenty-sdk/src/cli/operations/pull.ts
index bc148638616..0d6b2b37da3 100644
--- a/packages/twenty-sdk/src/cli/operations/pull.ts
+++ b/packages/twenty-sdk/src/cli/operations/pull.ts
@@ -16,6 +16,7 @@ import {
type PullDeletion,
type PullWrite,
} from '@/cli/utilities/pull/plan-pull-writes';
+import { planTranslationWrites } from '@/cli/utilities/pull/plan-translation-writes';
import {
readPullBaseManifest,
writePullBaseManifest,
@@ -23,7 +24,6 @@ import {
import { scanProjectDefineFiles } from '@/cli/utilities/pull/scan-project-define-files';
import { runSafe } from '@/cli/utilities/run-safe';
import { join } from 'node:path';
-import { type TranslationsManifest } from 'twenty-shared/application';
import { isDefined } from 'twenty-shared/utils';
export type AppPullOptions = {
@@ -42,7 +42,7 @@ export type AppPullResult = {
skipped: SkippedPullEntity[];
coverage: ApplicationExportCoverageEntry[];
unreadableRelativePaths: string[];
- translations: TranslationsManifest | undefined;
+ compiledTranslationEntryCountByLocale: Record<string, number>;
hadBase: boolean;
};
@@ -162,14 +162,23 @@ const innerAppPull = async (
});
const plan = planPullWrites({ manifest, baseManifest, scannedFiles });
+ const translationPlan = await planTranslationWrites({
+ appPath,
+ manifest,
+ baseManifest,
+ frontComponentSourcePaths: scannedFiles
+ .filter(
+ (scannedFile) =>
+ scannedFile.entityKey === ManifestEntityKey.FrontComponents,
+ )
+ .map((scannedFile) => join(appPath, scannedFile.relativePath)),
+ });
+ const writes = [...plan.writes, ...translationPlan.writes];
+ const deletions = [...plan.deletions, ...translationPlan.deletions];
onProgress?.('Writing source files...');
- await applyPullWrites({
- appPath,
- writes: plan.writes,
- deletions: plan.deletions,
- });
+ await applyPullWrites({ appPath, writes, deletions });
await writePullBaseManifest({ appPath, manifest });
@@ -179,8 +188,8 @@ const innerAppPull = async (
applicationDisplayName: applicationExport.application.displayName,
applicationUniversalIdentifier:
applicationExport.application.universalIdentifier,
- writes: plan.writes,
- deletions: plan.deletions,
+ writes,
+ deletions,
unchangedCount: plan.unchanged.length,
localOnlyRelativePaths: plan.localOnlyRelativePaths,
skipped: plan.skipped,
@@ -188,7 +197,8 @@ const innerAppPull = async (
unreadableRelativePaths: scannedFiles
.filter((scannedFile) => !scannedFile.isReadable)
.map((scannedFile) => scannedFile.relativePath),
- translations: manifest.translations,
+ compiledTranslationEntryCountByLocale:
+ translationPlan.compiledEntryCountByLocale,
hadBase: isDefined(baseManifest),
},
};
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__integration__/pull-round-trip.integration.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__integration__/pull-round-trip.integration.spec.ts
index 5d1d4c0a2a9..1dce8fafe88 100644
--- a/packages/twenty-sdk/src/cli/utilities/pull/__integration__/pull-round-trip.integration.spec.ts
+++ b/packages/twenty-sdk/src/cli/utilities/pull/__integration__/pull-round-trip.integration.spec.ts
@@ -1,12 +1,15 @@
import { buildManifest } from '@/cli/utilities/build/manifest/manifest-build';
import { buildPullEntities } from '@/cli/utilities/pull/build-pull-entities';
+import { planTranslationWrites } from '@/cli/utilities/pull/plan-translation-writes';
+import { compileApplicationTranslations } from '@/cli/utilities/translations/compile-application-translations';
import { writeDefineFile } from '@/cli/utilities/pull/write-define-file';
-import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';
+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
import { dirname, join, resolve } from 'node:path';
import {
getFieldUniversalIdentifier,
type Manifest,
} from 'twenty-shared/application';
+import { generateMessageId } from 'twenty-shared/i18n';
import { STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS } from 'twenty-shared/metadata';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
@@ -26,7 +29,15 @@ const JUNCTION_UID = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb';
const JUNCTION_PET_FIELD_UID = 'cccccccc-cccc-4ccc-8ccc-cccccccccccc';
const PET_AGREEMENTS_FIELD_UID = 'dddddddd-dddd-4ddd-8ddd-dddddddddddd';
+const EXPORTED_TRANSLATIONS = {
+ 'fr-FR': {
+ [generateMessageId('Pet', 'objectMetadata.labelSingular')]: 'Animal',
+ zzzzzz: 'orphan',
+ },
+};
+
const EXPORTED_MANIFEST = {
+ translations: EXPORTED_TRANSLATIONS,
application: {
universalIdentifier: APP_UID,
displayName: 'Pet Care',
@@ -267,6 +278,20 @@ describe('pull round trip', () => {
);
}
+ const translationPlan = await planTranslationWrites({
+ appPath,
+ manifest: EXPORTED_MANIFEST,
+ baseManifest: null,
+ frontComponentSourcePaths: [],
+ });
+
+ for (const write of translationPlan.writes) {
+ const filePath = join(appPath, write.relativePath);
+
+ await mkdir(dirname(filePath), { recursive: true });
+ await writeFile(filePath, write.content);
+ }
+
const buildResult = await buildManifest(appPath);
builtManifest = buildResult.manifest;
@@ -277,6 +302,20 @@ describe('pull round trip', () => {
await rm(appPath, { recursive: true, force: true });
});
+ it('should rebuild the exported translations from the written locale files', async () => {
+ expect(
+ JSON.parse(await readFile(join(appPath, 'locales/fr-FR.json'), 'utf8')),
+ ).toEqual({ 'objectMetadata.labelSingular': { Pet: 'Animal' } });
+ expect(
+ JSON.parse(
+ await readFile(join(appPath, 'locales/compiled/fr-FR.json'), 'utf8'),
+ ),
+ ).toEqual({ zzzzzz: 'orphan' });
+ expect(await compileApplicationTranslations(appPath)).toEqual(
+ EXPORTED_TRANSLATIONS,
+ );
+ });
+
it('should build the written source without errors', () => {
expect(buildErrors).toEqual([]);
expect(builtManifest).not.toBeNull();
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/apply-pull-writes.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/apply-pull-writes.spec.ts
index eb1afbe1281..d3380fb0f7c 100644
--- a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/apply-pull-writes.spec.ts
+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/apply-pull-writes.spec.ts
@@ -145,6 +145,37 @@ describe('applyPullWrites', () => {
).rejects.toThrow('same file path');
});
+ it('should refuse a plan that leaves the application directory', async () => {
+ await expect(
+ applyPullWrites({
+ appPath,
+ writes: [
+ buildWrite('my-dir/../../.../../../whatever.ts', 'export default 1;'),
+ ],
+ deletions: [],
+ }),
+ ).rejects.toThrow('leaves the application directory');
+ await expect(
+ applyPullWrites({
+ appPath,
+ writes: [],
+ deletions: [
+ { universalIdentifier: 'x', relativePath: 'src/../../escape.ts' },
+ ],
+ }),
+ ).rejects.toThrow('leaves the application directory');
+ });
+
+ it('should accept a folder whose name is only dots', async () => {
+ await applyPullWrites({
+ appPath,
+ writes: [buildWrite('.../kept.ts', 'export default 1;')],
+ deletions: [],
+ });
+
+ expect(await readAppFile('.../kept.ts')).toBe('export default 1;');
+ });
+
it('should refuse to delete a folder standing where a planned file goes', async () => {
await mkdir(join(appPath, 'src/pet.object.ts'), { recursive: true });
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/format-pull-report.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/format-pull-report.spec.ts
index bd654a86893..8792f59bcf9 100644
--- a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/format-pull-report.spec.ts
+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/format-pull-report.spec.ts
@@ -144,21 +144,20 @@ describe('formatPullReport', () => {
expect(report).toContain('src/objects/unpushed.object.ts');
});
- it('should warn that published translations were not written and point at the locales folder', () => {
+ it('should list the locales whose entries stayed in compiled form', () => {
const report = buildReport({
- translations: {
- 'fr-FR': { greeting: 'Bonjour' },
- en: { greeting: 'Hello' },
- },
+ compiledTranslationEntryCountByLocale: { 'fr-FR': 3, en: 1 },
});
- expect(report).toMatch(/^Published translations \(en, fr-FR\).*locales\//m);
+ expect(report).toContain('locales/compiled/');
+ expect(report).toMatch(/^ {2}en {11}1 entry$/m);
+ expect(report).toMatch(/^ {2}fr-FR {8}3 entries$/m);
});
- it('should stay silent about translations when the workspace has none', () => {
- expect(buildReport()).not.toContain('Published translations');
- expect(buildReport({ translations: { en: {} } })).not.toContain(
- 'Published translations',
- );
+ it('should stay silent about compiled translations when every entry was decoded', () => {
+ expect(buildReport()).not.toContain('compiled form');
+ expect(
+ buildReport({ compiledTranslationEntryCountByLocale: { 'fr-FR': 0 } }),
+ ).not.toContain('compiled form');
});
});
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/plan-translation-writes.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/plan-translation-writes.spec.ts
new file mode 100644
index 00000000000..e1f05d5d63e
--- /dev/null
+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/plan-translation-writes.spec.ts
@@ -0,0 +1,378 @@
+import {
+ planTranslationWrites,
+ type TranslationWritePlan,
+} from '@/cli/utilities/pull/plan-translation-writes';
+import { compileApplicationTranslations } from '@/cli/utilities/translations/compile-application-translations';
+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { type Manifest } from 'twenty-shared/application';
+import { generateMessageId } from 'twenty-shared/i18n';
+import { afterEach, describe, expect, it } from 'vitest';
+
+const PET_LABEL_ID = generateMessageId('Pet', 'objectMetadata.labelSingular');
+
+const buildManifest = (translations?: Record<string, unknown>): Manifest =>
+ ({
+ application: {},
+ objects: [
+ {
+ universalIdentifier: 'pet',
+ nameSingular: 'pet',
+ namePlural: 'pets',
+ labelSingular: 'Pet',
+ labelPlural: 'Pets',
+ fields: [],
+ },
+ ],
+ fields: [],
+ indexes: [],
+ views: [],
+ viewFields: [],
+ ...(translations === undefined ? {} : { translations }),
+ }) as unknown as Manifest;
+
+const EXPORTED_TRANSLATIONS = {
+ 'fr-FR': { [PET_LABEL_ID]: 'Animal', zzzzzz: 'orphan' },
+ 'de-DE': { [PET_LABEL_ID]: 'Haustier' },
+};
+
+const createdAppPaths: string[] = [];
+
+const createAppPath = async () => {
+ const appPath = await mkdtemp(join(tmpdir(), 'plan-translation-writes-'));
+
+ createdAppPaths.push(appPath);
+
+ return appPath;
+};
+
+const materialize = async (appPath: string, plan: TranslationWritePlan) => {
+ for (const write of plan.writes) {
+ await mkdir(dirname(join(appPath, write.relativePath)), {
+ recursive: true,
+ });
+ await writeFile(join(appPath, write.relativePath), write.content);
+ }
+
+ for (const deletion of plan.deletions) {
+ await rm(join(appPath, deletion.relativePath), { force: true });
+ }
+};
+
+const describePlan = (plan: TranslationWritePlan) => ({
+ writes: plan.writes.map(
+ ({ relativePath, isRegeneration }) =>
+ `${isRegeneration ? 'regenerate' : 'write'} ${relativePath}`,
+ ),
+ deletions: plan.deletions.map(({ relativePath }) => relativePath),
+});
+
+const pullFresh = async (appPath: string) => {
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest(EXPORTED_TRANSLATIONS),
+ baseManifest: null,
+ frontComponentSourcePaths: [],
+ });
+
+ await materialize(appPath, plan);
+
+ return plan;
+};
+
+describe('planTranslationWrites', () => {
+ afterEach(async () => {
+ await Promise.all(
+ createdAppPaths
+ .splice(0)
+ .map((appPath) => rm(appPath, { recursive: true, force: true })),
+ );
+ });
+
+ it('should write the readable file and the compiled remainder of every exported locale on a fresh pull', async () => {
+ const appPath = await createAppPath();
+
+ const plan = await pullFresh(appPath);
+
+ expect(describePlan(plan)).toEqual({
+ writes: [
+ 'write locales/de-DE.json',
+ 'write locales/fr-FR.json',
+ 'write locales/compiled/fr-FR.json',
+ ],
+ deletions: [],
+ });
+ expect(plan.compiledEntryCountByLocale).toEqual({ 'fr-FR': 1 });
+ expect(await compileApplicationTranslations(appPath)).toEqual(
+ EXPORTED_TRANSLATIONS,
+ );
+ });
+
+ it('should leave a locale alone when its catalog did not change since the base', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+ await writeFile(
+ join(appPath, 'locales/de-DE.json'),
+ JSON.stringify({ 'objectMetadata.labelSingular': { Pet: 'Tier' } }),
+ );
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest(EXPORTED_TRANSLATIONS),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({ writes: [], deletions: [] });
+ expect(plan.compiledEntryCountByLocale).toEqual({ 'fr-FR': 1 });
+ });
+
+ it('should restore a wanted file that went missing even when the catalog did not change', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+ await rm(join(appPath, 'locales/compiled/fr-FR.json'));
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest(EXPORTED_TRANSLATIONS),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({
+ writes: ['write locales/compiled/fr-FR.json'],
+ deletions: [],
+ });
+ });
+
+ it('should regenerate the locale that changed on the server and delete the one it dropped', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest({
+ 'fr-FR': { [PET_LABEL_ID]: 'Bête', zzzzzz: 'orphan' },
+ }),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({
+ writes: ['regenerate locales/fr-FR.json'],
+ deletions: ['locales/de-DE.json'],
+ });
+ expect(plan.writes[0].content).toContain('Bête');
+ });
+
+ it('should delete the compiled file once every entry of a locale can be decoded', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest({
+ 'fr-FR': { [PET_LABEL_ID]: 'Animal' },
+ 'de-DE': { [PET_LABEL_ID]: 'Haustier' },
+ }),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({
+ writes: [],
+ deletions: ['locales/compiled/fr-FR.json'],
+ });
+ expect(plan.compiledEntryCountByLocale).toEqual({});
+ });
+
+ it('should promote entries whose source landed since the last pull without touching other local edits', async () => {
+ const appPath = await createAppPath();
+ const noContentId = generateMessageId('No content yet');
+ const catalog = buildManifest({
+ 'fr-FR': {
+ [PET_LABEL_ID]: 'Animal',
+ [noContentId]: 'Rien pour le moment',
+ },
+ });
+
+ const firstPlan = await planTranslationWrites({
+ appPath,
+ manifest: catalog,
+ baseManifest: null,
+ frontComponentSourcePaths: [],
+ });
+
+ await materialize(appPath, firstPlan);
+ await writeFile(
+ join(appPath, 'locales/fr-FR.json'),
+ JSON.stringify({ 'objectMetadata.labelSingular': { Pet: 'Bête' } }),
+ );
+
+ const componentPath = join(appPath, 'card.front-component.tsx');
+
+ await writeFile(
+ componentPath,
+ `
+ import { t } from 'twenty-sdk/front-component';
+
+ const Component = () => t('No content yet');
+
+ export default defineFrontComponent({ component: Component });
+ `,
+ );
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: catalog,
+ baseManifest: catalog,
+ frontComponentSourcePaths: [componentPath],
+ });
+
+ expect(describePlan(plan)).toEqual({
+ writes: ['regenerate locales/fr-FR.json'],
+ deletions: ['locales/compiled/fr-FR.json'],
+ });
+ expect(JSON.parse(plan.writes[0].content)).toEqual({
+ 'No content yet': 'Rien pour le moment',
+ 'objectMetadata.labelSingular': { Pet: 'Bête' },
+ });
+ expect(plan.compiledEntryCountByLocale).toEqual({});
+
+ await materialize(appPath, plan);
+
+ expect(
+ JSON.parse(await readFile(join(appPath, 'locales/fr-FR.json'), 'utf8')),
+ ).toEqual({
+ 'No content yet': 'Rien pour le moment',
+ 'objectMetadata.labelSingular': { Pet: 'Bête' },
+ });
+ });
+
+ it('should leave a locally added compiled entry alone when nothing became decodable', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+
+ const compiledPath = join(appPath, 'locales/compiled/fr-FR.json');
+
+ await writeFile(
+ compiledPath,
+ JSON.stringify({ zzzzzz: 'orphan', yyyyyy: 'added by hand' }),
+ );
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest(EXPORTED_TRANSLATIONS),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({ writes: [], deletions: [] });
+ expect(JSON.parse(await readFile(compiledPath, 'utf8'))).toEqual({
+ zzzzzz: 'orphan',
+ yyyyyy: 'added by hand',
+ });
+ });
+
+ it('should treat a base or an export whose translations are not a record as saying nothing', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+
+ const brokenBasePlan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest(EXPORTED_TRANSLATIONS),
+ baseManifest: buildManifest(
+ 'not a record' as unknown as Record<string, unknown>,
+ ),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(brokenBasePlan)).toEqual({ writes: [], deletions: [] });
+
+ const brokenExportPlan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest(['fr-FR'] as unknown as Record<string, unknown>),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(brokenExportPlan)).toEqual({
+ writes: [],
+ deletions: [],
+ });
+ });
+
+ it('should never delete the files of a locale the export still lists, even with a malformed catalog', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest({
+ 'fr-FR': 'not a catalog',
+ 'de-DE': { [PET_LABEL_ID]: 'Haustier' },
+ }),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({ writes: [], deletions: [] });
+ });
+
+ it('should only delete the files of a locale the base recorded with a valid catalog', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest({ 'de-DE': { [PET_LABEL_ID]: 'Haustier' } }),
+ baseManifest: buildManifest({
+ 'fr-FR': ['not a catalog'],
+ 'de-DE': { [PET_LABEL_ID]: 'Haustier' },
+ }),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({ writes: [], deletions: [] });
+ });
+
+ it('should only write the compiled file for a locale with no decodable entry', async () => {
+ const appPath = await createAppPath();
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest({ 'es-ES': { zzzzzz: 'huérfano' } }),
+ baseManifest: null,
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({
+ writes: ['write locales/compiled/es-ES.json'],
+ deletions: [],
+ });
+ });
+
+ it('should touch nothing when the export says nothing about translations', async () => {
+ const appPath = await createAppPath();
+
+ await pullFresh(appPath);
+
+ const plan = await planTranslationWrites({
+ appPath,
+ manifest: buildManifest(),
+ baseManifest: buildManifest(EXPORTED_TRANSLATIONS),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(describePlan(plan)).toEqual({ writes: [], deletions: [] });
+ });
+});
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/split-pulled-translations.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/split-pulled-translations.spec.ts
new file mode 100644
index 00000000000..707df3a3925
--- /dev/null
+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/split-pulled-translations.spec.ts
@@ -0,0 +1,208 @@
+import { splitPulledTranslations } from '@/cli/utilities/pull/split-pulled-translations';
+import { mkdtemp, rm, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { type Manifest } from 'twenty-shared/application';
+import { generateMessageId } from 'twenty-shared/i18n';
+import { afterEach, describe, expect, it } from 'vitest';
+
+const PET_LABEL_ID = generateMessageId('Pet', 'objectMetadata.labelSingular');
+const AGE_LABEL_ID = generateMessageId('Age', 'fieldMetadata.label');
+const VIEW_NAME_ID = generateMessageId('All pets', 'view.name');
+
+const buildManifest = (overrides: Record<string, unknown>): Manifest =>
+ ({
+ application: {},
+ objects: [
+ {
+ universalIdentifier: 'pet',
+ nameSingular: 'pet',
+ namePlural: 'pets',
+ labelSingular: 'Pet',
+ labelPlural: 'Pets',
+ description: 'A pet',
+ fields: [
+ {
+ universalIdentifier: 'age',
+ name: 'age',
+ label: 'Age',
+ description: '',
+ type: 'NUMBER',
+ },
+ ],
+ },
+ ],
+ fields: [],
+ indexes: [],
+ views: [
+ {
+ universalIdentifier: 'all-pets',
+ name: 'All pets',
+ objectUniversalIdentifier: 'pet',
+ },
+ ],
+ viewFields: [],
+ ...overrides,
+ }) as unknown as Manifest;
+
+const createdDirectories: string[] = [];
+
+const createDirectory = async () => {
+ const directory = await mkdtemp(join(tmpdir(), 'split-pulled-translations-'));
+
+ createdDirectories.push(directory);
+
+ return directory;
+};
+
+describe('splitPulledTranslations', () => {
+ afterEach(async () => {
+ await Promise.all(
+ createdDirectories
+ .splice(0)
+ .map((directory) => rm(directory, { recursive: true, force: true })),
+ );
+ });
+
+ it('should write entries whose source is a pulled label in the authoring format and keep the rest compiled', async () => {
+ const catalogs = await splitPulledTranslations({
+ manifest: buildManifest({
+ translations: {
+ 'fr-FR': {
+ [PET_LABEL_ID]: 'Animal',
+ [AGE_LABEL_ID]: 'Âge',
+ [VIEW_NAME_ID]: 'Tous les animaux',
+ zzzzzz: 'orphan',
+ },
+ },
+ }),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(catalogs).toEqual([
+ {
+ locale: 'fr-FR',
+ authored: {
+ 'fieldMetadata.label': { Age: 'Âge' },
+ 'objectMetadata.labelSingular': { Pet: 'Animal' },
+ 'view.name': { 'All pets': 'Tous les animaux' },
+ },
+ compiled: { zzzzzz: 'orphan' },
+ },
+ ]);
+ });
+
+ it('should decode the strings of the front components it is given', async () => {
+ const componentPath = join(
+ await createDirectory(),
+ 'card.front-component.tsx',
+ );
+
+ await writeFile(
+ componentPath,
+ `
+ import { t, Trans } from 'twenty-sdk/front-component';
+
+ const Component = () => {
+ const label = t('No content yet');
+ const verb = t({ message: 'Open', context: 'door' });
+
+ return <Trans context="card">Untitled</Trans>;
+ };
+
+ export default defineFrontComponent({ component: Component });
+ `,
+ );
+
+ const catalogs = await splitPulledTranslations({
+ manifest: buildManifest({
+ translations: {
+ 'de-DE': {
+ [generateMessageId('No content yet')]: 'Noch kein Inhalt',
+ [generateMessageId('Open', 'door')]: 'Öffnen',
+ [generateMessageId('Untitled', 'card')]: 'Ohne Titel',
+ },
+ },
+ }),
+ frontComponentSourcePaths: [componentPath],
+ });
+
+ expect(catalogs).toEqual([
+ {
+ locale: 'de-DE',
+ authored: {
+ 'No content yet': 'Noch kein Inhalt',
+ card: { Untitled: 'Ohne Titel' },
+ door: { Open: 'Öffnen' },
+ },
+ compiled: {},
+ },
+ ]);
+ });
+
+ it('should keep an entry compiled when the label it was translated from has changed', async () => {
+ const staleId = generateMessageId('Kitten', 'objectMetadata.labelSingular');
+
+ const catalogs = await splitPulledTranslations({
+ manifest: buildManifest({
+ translations: { 'fr-FR': { [staleId]: 'Chaton' } },
+ }),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(catalogs).toEqual([
+ { locale: 'fr-FR', authored: {}, compiled: { [staleId]: 'Chaton' } },
+ ]);
+ });
+
+ it('should ignore locales the platform does not support, including path-shaped keys', async () => {
+ const catalogs = await splitPulledTranslations({
+ manifest: buildManifest({
+ translations: {
+ 'fr-FR': { [PET_LABEL_ID]: 'Animal' },
+ klingon: { [PET_LABEL_ID]: 'targh' },
+ '../../escape': { [PET_LABEL_ID]: 'nope' },
+ },
+ }),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(catalogs.map(({ locale }) => locale)).toEqual(['fr-FR']);
+ });
+
+ it('should ignore a locale whose catalog is not a map of strings', async () => {
+ const catalogs = await splitPulledTranslations({
+ manifest: buildManifest({
+ translations: {
+ 'fr-FR': 'Animal',
+ 'de-DE': ['Haustier'],
+ 'es-ES': { [PET_LABEL_ID]: 7 },
+ 'it-IT': { [PET_LABEL_ID]: 'Animale' },
+ },
+ }),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(catalogs.map(({ locale }) => locale)).toEqual(['it-IT']);
+ });
+
+ it('should order locales and return nothing for an export without translations', async () => {
+ const catalogs = await splitPulledTranslations({
+ manifest: buildManifest({
+ translations: {
+ 'fr-FR': { [PET_LABEL_ID]: 'Animal' },
+ 'de-DE': { [PET_LABEL_ID]: 'Haustier' },
+ },
+ }),
+ frontComponentSourcePaths: [],
+ });
+
+ expect(catalogs.map(({ locale }) => locale)).toEqual(['de-DE', 'fr-FR']);
+ expect(
+ await splitPulledTranslations({
+ manifest: buildManifest({}),
+ frontComponentSourcePaths: [],
+ }),
+ ).toEqual([]);
+ });
+});
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.ts b/packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.ts
index fa5e580f3cd..724d04b49f6 100644
--- a/packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.ts
+++ b/packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.ts
@@ -9,7 +9,7 @@ import {
type PullWrite,
} from '@/cli/utilities/pull/plan-pull-writes';
import { lstat, mkdtemp, rm, writeFile } from 'node:fs/promises';
-import { dirname, join } from 'node:path';
+import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path';
const PULL_WORK_DIRECTORY = '.twenty';
@@ -30,10 +30,28 @@ const assertPlanIsApplicable = async ({
);
}
+ const applicationRoot = resolve(appPath);
+
for (const relativePath of [
...relativePaths,
...deletions.map((deletion) => deletion.relativePath),
]) {
+ const containedPath = relative(
+ applicationRoot,
+ resolve(applicationRoot, relativePath),
+ );
+
+ if (
+ containedPath.length === 0 ||
+ containedPath === '..' ||
+ containedPath.startsWith(`..${sep}`) ||
+ isAbsolute(containedPath)
+ ) {
+ throw new Error(
+ `Refusing to write: ${relativePath} leaves the application directory`,
+ );
+ }
+
const destinationPath = join(appPath, relativePath);
if (!(await pathExists(destinationPath))) {
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/format-pull-report.ts b/packages/twenty-sdk/src/cli/utilities/pull/format-pull-report.ts
index 3083501694d..cbe3e9f95b5 100644
--- a/packages/twenty-sdk/src/cli/utilities/pull/format-pull-report.ts
+++ b/packages/twenty-sdk/src/cli/utilities/pull/format-pull-report.ts
@@ -4,8 +4,6 @@ import {
type PullDeletion,
type PullWrite,
} from '@/cli/utilities/pull/plan-pull-writes';
-import { type TranslationsManifest } from 'twenty-shared/application';
-import { isDefined } from 'twenty-shared/utils';
const MAX_LISTED_IDENTIFIERS = 20;
@@ -67,7 +65,7 @@ export const formatPullReport = ({
coverage,
localOnlyRelativePaths,
unreadableRelativePaths = [],
- translations = {},
+ compiledTranslationEntryCountByLocale = {},
verbose = false,
}: {
writes: PullWrite[];
@@ -77,7 +75,7 @@ export const formatPullReport = ({
coverage: ApplicationExportCoverageEntry[];
localOnlyRelativePaths: string[];
unreadableRelativePaths?: string[];
- translations?: TranslationsManifest;
+ compiledTranslationEntryCountByLocale?: Record<string, number>;
verbose?: boolean;
}): string => {
const lines: string[] = [
@@ -166,17 +164,18 @@ export const formatPullReport = ({
}
}
- const publishedTranslationLocales = Object.entries(translations)
- .filter(
- ([, messages]) => isDefined(messages) && Object.keys(messages).length > 0,
- )
- .map(([locale]) => locale)
- .sort();
+ const compiledLocales = Object.entries(compiledTranslationEntryCountByLocale)
+ .filter(([, count]) => count > 0)
+ .sort(([left], [right]) => left.localeCompare(right));
- if (publishedTranslationLocales.length > 0) {
+ if (compiledLocales.length > 0) {
lines.push(
'',
- `Published translations (${publishedTranslationLocales.join(', ')}) not written: pull cannot restore them yet, and pushing this tree replaces them with the contents of locales/.`,
+ 'Translations kept in compiled form until their source strings are in this tree (see locales/compiled/):',
+ ...compiledLocales.map(
+ ([locale, count]) =>
+ ` ${locale.padEnd(13)}${count} ${count === 1 ? 'entry' : 'entries'}`,
+ ),
);
}
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.ts b/packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.ts
index d9848a320eb..c41d0fee789 100644
--- a/packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.ts
+++ b/packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.ts
@@ -10,8 +10,10 @@ import { dirname, posix } from 'node:path';
import { type Manifest } from 'twenty-shared/application';
import { isDefined } from 'twenty-shared/utils';
+export type PullWriteKind = PullEntityKind | 'translation';
+
export type PullWrite = {
- kind: PullEntityKind;
+ kind: PullWriteKind;
universalIdentifier: string;
relativePath: string;
content: string;
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/plan-translation-writes.ts b/packages/twenty-sdk/src/cli/utilities/pull/plan-translation-writes.ts
new file mode 100644
index 00000000000..bff1fa17bff
--- /dev/null
+++ b/packages/twenty-sdk/src/cli/utilities/pull/plan-translation-writes.ts
@@ -0,0 +1,255 @@
+import { pathExists } from '@/cli/utilities/file/fs-utils';
+import {
+ type PullDeletion,
+ type PullWrite,
+} from '@/cli/utilities/pull/plan-pull-writes';
+import {
+ type PulledLocaleCatalog,
+ splitPulledTranslations,
+} from '@/cli/utilities/pull/split-pulled-translations';
+import {
+ COMPILED_LOCALES_DIR,
+ LOCALES_DIR,
+} from '@/cli/utilities/translations/constants';
+import { isLocaleCatalog } from '@/cli/utilities/translations/is-locale-catalog';
+import { isSupportedLocale } from '@/cli/utilities/translations/is-supported-locale';
+import {
+ buildLocaleCatalog,
+ flattenLocaleCatalog,
+ type LocaleCatalogEntry,
+} from '@/cli/utilities/translations/locale-catalog-format';
+import { readFile } from 'node:fs/promises';
+import { join } from 'node:path';
+import { type Manifest } from 'twenty-shared/application';
+import { generateMessageId } from 'twenty-shared/i18n';
+import { isDefined } from 'twenty-shared/utils';
+
+export type TranslationWritePlan = {
+ writes: PullWrite[];
+ deletions: PullDeletion[];
+ compiledEntryCountByLocale: Record<string, number>;
+};
+
+type LocaleFile = {
+ relativePath: string;
+ content: string;
+ isWanted: boolean;
+ existingContent: string | null;
+};
+
+type LocaleCatalogs = Partial<Record<string, Record<string, string>>>;
+
+const isLocaleCatalogs = (value: unknown): value is LocaleCatalogs =>
+ isDefined(value) && typeof value === 'object' && !Array.isArray(value);
+
+const serializeLocaleFile = (value: unknown): string =>
+ `${JSON.stringify(value, null, 2)}\n`;
+
+const parseLocaleFile = (content: string | null): Record<string, unknown> => {
+ if (!isDefined(content)) {
+ return {};
+ }
+
+ try {
+ const parsed: unknown = JSON.parse(content);
+
+ return isDefined(parsed) && typeof parsed === 'object'
+ ? (parsed as Record<string, unknown>)
+ : {};
+ } catch {
+ return {};
+ }
+};
+
+const getEntryKey = ({ message, context }: LocaleCatalogEntry): string =>
+ JSON.stringify([context ?? null, message]);
+
+const promoteDecodedEntries = ({
+ catalog,
+ existingAuthoredContent,
+ existingCompiledContent,
+}: {
+ catalog: PulledLocaleCatalog;
+ existingAuthoredContent: string | null;
+ existingCompiledContent: string | null;
+}): Record<string, string | Record<string, string>> | null => {
+ const authoredMessageIds = new Set(
+ flattenLocaleCatalog(catalog.authored).map((entry) =>
+ generateMessageId(entry.message, entry.context),
+ ),
+ );
+ const promotedMessageIds = new Set(
+ Object.keys(parseLocaleFile(existingCompiledContent)).filter(
+ (messageId) =>
+ !(messageId in catalog.compiled) && authoredMessageIds.has(messageId),
+ ),
+ );
+
+ if (promotedMessageIds.size === 0) {
+ return null;
+ }
+
+ const existingEntries = flattenLocaleCatalog(
+ parseLocaleFile(existingAuthoredContent),
+ );
+ const existingKeys = new Set(existingEntries.map(getEntryKey));
+ const promotedEntries = flattenLocaleCatalog(catalog.authored).filter(
+ (entry) =>
+ promotedMessageIds.has(generateMessageId(entry.message, entry.context)) &&
+ !existingKeys.has(getEntryKey(entry)),
+ );
+
+ return buildLocaleCatalog([...existingEntries, ...promotedEntries]);
+};
+
+const getAuthoredRelativePath = (locale: string): string =>
+ `${LOCALES_DIR}/${locale}.json`;
+
+const getCompiledRelativePath = (locale: string): string =>
+ `${COMPILED_LOCALES_DIR}/${locale}.json`;
+
+const readExistingContent = async (
+ absolutePath: string,
+): Promise<string | null> =>
+ (await pathExists(absolutePath))
+ ? await readFile(absolutePath, 'utf8')
+ : null;
+
+export const planTranslationWrites = async ({
+ appPath,
+ manifest,
+ baseManifest,
+ frontComponentSourcePaths,
+}: {
+ appPath: string;
+ manifest: Manifest;
+ baseManifest: Manifest | null;
+ frontComponentSourcePaths: string[];
+}): Promise<TranslationWritePlan> => {
+ const writes: PullWrite[] = [];
+ const deletions: PullDeletion[] = [];
+ const compiledEntryCountByLocale: Record<string, number> = {};
+ const exportedTranslations = manifest.translations;
+
+ if (!isLocaleCatalogs(exportedTranslations)) {
+ return { writes, deletions, compiledEntryCountByLocale };
+ }
+
+ const catalogs = await splitPulledTranslations({
+ manifest,
+ frontComponentSourcePaths,
+ });
+ const exportedLocales = new Set(Object.keys(exportedTranslations));
+ const baseTranslations: LocaleCatalogs = isLocaleCatalogs(
+ baseManifest?.translations,
+ )
+ ? baseManifest.translations
+ : {};
+ const currentTranslations: LocaleCatalogs = exportedTranslations;
+
+ for (const catalog of catalogs) {
+ const { locale, authored, compiled } = catalog;
+ const compiledEntryCount = Object.keys(compiled).length;
+
+ if (compiledEntryCount > 0) {
+ compiledEntryCountByLocale[locale] = compiledEntryCount;
+ }
+
+ const authoredRelativePath = getAuthoredRelativePath(locale);
+ const compiledRelativePath = getCompiledRelativePath(locale);
+ const existingAuthoredContent = await readExistingContent(
+ join(appPath, authoredRelativePath),
+ );
+ const existingCompiledContent = await readExistingContent(
+ join(appPath, compiledRelativePath),
+ );
+ const catalogUnchangedSinceBase =
+ locale in baseTranslations &&
+ JSON.stringify(baseTranslations[locale]) ===
+ JSON.stringify(currentTranslations[locale]);
+ const everyWantedFileExists =
+ (Object.keys(authored).length === 0 ||
+ isDefined(existingAuthoredContent)) &&
+ (compiledEntryCount === 0 || isDefined(existingCompiledContent));
+ const promotedAuthored =
+ catalogUnchangedSinceBase && everyWantedFileExists
+ ? promoteDecodedEntries({
+ catalog,
+ existingAuthoredContent,
+ existingCompiledContent,
+ })
+ : null;
+
+ if (
+ catalogUnchangedSinceBase &&
+ everyWantedFileExists &&
+ !isDefined(promotedAuthored)
+ ) {
+ continue;
+ }
+
+ const localeFiles: LocaleFile[] = [
+ {
+ relativePath: authoredRelativePath,
+ content: serializeLocaleFile(promotedAuthored ?? authored),
+ isWanted: Object.keys(promotedAuthored ?? authored).length > 0,
+ existingContent: existingAuthoredContent,
+ },
+ {
+ relativePath: compiledRelativePath,
+ content: serializeLocaleFile(compiled),
+ isWanted: compiledEntryCount > 0,
+ existingContent: existingCompiledContent,
+ },
+ ];
+
+ for (const {
+ relativePath,
+ content,
+ isWanted,
+ existingContent,
+ } of localeFiles) {
+ const exists = isDefined(existingContent);
+
+ if (!isWanted) {
+ if (exists && relativePath === compiledRelativePath) {
+ deletions.push({ universalIdentifier: locale, relativePath });
+ }
+ continue;
+ }
+
+ if (existingContent === content) {
+ continue;
+ }
+
+ writes.push({
+ kind: 'translation',
+ universalIdentifier: locale,
+ relativePath,
+ content,
+ isRegeneration: exists,
+ });
+ }
+ }
+
+ for (const [locale, baseCatalog] of Object.entries(baseTranslations)) {
+ if (
+ exportedLocales.has(locale) ||
+ !isSupportedLocale(locale) ||
+ !isLocaleCatalog(baseCatalog)
+ ) {
+ continue;
+ }
+
+ for (const relativePath of [
+ getAuthoredRelativePath(locale),
+ getCompiledRelativePath(locale),
+ ]) {
+ if (await pathExists(join(appPath, relativePath))) {
+ deletions.push({ universalIdentifier: locale, relativePath });
+ }
+ }
+ }
+
+ return { writes, deletions, compiledEntryCountByLocale };
+};
diff --git a/packages/twenty-sdk/src/cli/utilities/pull/split-pulled-translations.ts b/packages/twenty-sdk/src/cli/utilities/pull/split-pulled-translations.ts
new file mode 100644
index 00000000000..cc269e90729
--- /dev/null
+++ b/packages/twenty-sdk/src/cli/utilities/pull/split-pulled-translations.ts
@@ -0,0 +1,68 @@
+import { collectFrontComponentStrings } from '@/cli/utilities/translations/collect-front-component-strings';
+import { collectTranslatableStrings } from '@/cli/utilities/translations/collect-translatable-strings';
+import {
+ buildLocaleCatalog,
+ type LocaleCatalogEntry,
+} from '@/cli/utilities/translations/locale-catalog-format';
+import { isLocaleCatalog } from '@/cli/utilities/translations/is-locale-catalog';
+import { isSupportedLocale } from '@/cli/utilities/translations/is-supported-locale';
+import { type MessageDescriptor } from '@/sdk/front-component/translations/message';
+import { type Manifest } from 'twenty-shared/application';
+import { generateMessageId } from 'twenty-shared/i18n';
+import { type AppLocale } from 'twenty-shared/translations';
+import { isDefined } from 'twenty-shared/utils';
+
+export type PulledLocaleCatalog = {
+ locale: AppLocale;
+ authored: Record<string, string | Record<string, string>>;
+ compiled: Record<string, string>;
+};
+
+export const splitPulledTranslations = async ({
+ manifest,
+ frontComponentSourcePaths,
+}: {
+ manifest: Manifest;
+ frontComponentSourcePaths: string[];
+}): Promise<PulledLocaleCatalog[]> => {
+ const descriptors: MessageDescriptor[] = [
+ ...collectTranslatableStrings(manifest),
+ ...(await collectFrontComponentStrings(frontComponentSourcePaths)),
+ ];
+ const descriptorByMessageId = new Map(
+ descriptors.map(
+ (descriptor) =>
+ [
+ generateMessageId(descriptor.message, descriptor.context),
+ descriptor,
+ ] as const,
+ ),
+ );
+
+ return Object.entries(manifest.translations ?? {})
+ .filter(
+ (entry): entry is [AppLocale, Record<string, string>] =>
+ isSupportedLocale(entry[0]) && isLocaleCatalog(entry[1]),
+ )
+ .sort(([left], [right]) => left.localeCompare(right))
+ .map(([locale, messages]) => {
+ const entries: LocaleCatalogEntry[] = [];
+ const compiled: Record<string, string> = {};
+
+ for (const [messageId, translation] of Object.entries(messages)) {
+ const descriptor = descriptorByMessageId.get(messageId);
+
+ if (isDefined(descriptor)) {
+ entries.push({
+ message: descriptor.message,
+ context: descriptor.context,
+ translation,
+ });
+ } else {
+ compiled[messageId] = translation;
+ }
+ }
+
+ return { locale, authored: buildLocaleCatalog(entries), compiled };
+ });
+};
diff --git a/packages/twenty-sdk/src/cli/utilities/translations/__tests__/application-translations.spec.ts b/packages/twenty-sdk/src/cli/utilities/translations/__tests__/application-translations.spec.ts
index ec6b470c874..6d8c944d76d 100644
--- a/packages/twenty-sdk/src/cli/utilities/translations/__tests__/application-translations.spec.ts
+++ b/packages/twenty-sdk/src/cli/utilities/translations/__tests__/application-translations.spec.ts
@@ -69,6 +69,25 @@ describe('collectTranslatableStrings', () => {
// Pins every manifest collection the shared registry maps, so a change to
// TRANSLATABLE_PROPERTIES_BY_METADATA_NAME that silently drops a collection
// shows up here rather than as an app shipping untranslatable strings.
+ it('collects the fields declared inline on an object', () => {
+ const manifest = buildManifest({
+ objects: [
+ {
+ labelSingular: 'Pet',
+ labelPlural: 'Pets',
+ fields: [{ label: 'Age', description: 'In years' }],
+ },
+ ],
+ });
+
+ expect(collectTranslatableStrings(manifest)).toEqual(
+ expect.arrayContaining([
+ { message: 'Age', context: 'fieldMetadata.label' },
+ { message: 'In years', context: 'fieldMetadata.description' },
+ ]),
+ );
+ });
+
it('collects every translatable property of every mapped manifest collection', () => {
const manifest = buildManifest({
objects: [
@@ -189,4 +208,98 @@ describe('compileApplicationTranslations', () => {
expect(await compileApplicationTranslations(appPath)).toEqual({});
});
+
+ it('merges compiled catalogs and lets an authored entry win on the same id', async () => {
+ const appPath = await mkdtemp(join(tmpdir(), 'twenty-translations-merge-'));
+ const localesDir = join(appPath, 'locales');
+ await mkdir(join(localesDir, 'compiled'), { recursive: true });
+ await writeFile(
+ join(localesDir, 'fr-FR.json'),
+ JSON.stringify({ Company: 'Entreprise' }),
+ );
+ await writeFile(
+ join(localesDir, 'compiled', 'fr-FR.json'),
+ JSON.stringify({
+ [generateMessageId('Company')]: 'Société',
+ zzzzzz: 'orphan',
+ }),
+ );
+
+ expect(await compileApplicationTranslations(appPath)).toEqual({
+ 'fr-FR': {
+ [generateMessageId('Company')]: 'Entreprise',
+ zzzzzz: 'orphan',
+ },
+ });
+ });
+
+ it('declares a locale that only has a compiled catalog', async () => {
+ const appPath = await mkdtemp(
+ join(tmpdir(), 'twenty-translations-compiled-only-'),
+ );
+ await mkdir(join(appPath, 'locales', 'compiled'), { recursive: true });
+ await writeFile(
+ join(appPath, 'locales', 'compiled', 'de-DE.json'),
+ JSON.stringify({ zzzzzz: 'Waise' }),
+ );
+
+ expect(await compileApplicationTranslations(appPath)).toEqual({
+ 'de-DE': { zzzzzz: 'Waise' },
+ });
+ });
+
+ it('skips empty compiled entries and compiled files of unsupported locales', async () => {
+ const appPath = await mkdtemp(
+ join(tmpdir(), 'twenty-translations-compiled-skip-'),
+ );
+ await mkdir(join(appPath, 'locales', 'compiled'), { recursive: true });
+ await writeFile(
+ join(appPath, 'locales', 'compiled', 'fr-FR.json'),
+ JSON.stringify({ aaaaaa: '', bbbbbb: 'kept' }),
+ );
+ await writeFile(
+ join(appPath, 'locales', 'compiled', 'klingon.json'),
+ JSON.stringify({ cccccc: 'nuqneH' }),
+ );
+
+ expect(await compileApplicationTranslations(appPath)).toEqual({
+ 'fr-FR': { bbbbbb: 'kept' },
+ });
+ });
+
+ it('skips a locale file whose JSON is not an object', async () => {
+ const appPath = await mkdtemp(
+ join(tmpdir(), 'twenty-translations-not-object-'),
+ );
+ const localesDir = join(appPath, 'locales');
+ await mkdir(localesDir, { recursive: true });
+ await writeFile(join(localesDir, 'fr-FR.json'), JSON.stringify(''));
+ await writeFile(join(localesDir, 'de-DE.json'), JSON.stringify(['x']));
+ await writeFile(
+ join(localesDir, 'it-IT.json'),
+ JSON.stringify({ Company: 'Azienda' }),
+ );
+
+ expect(await compileApplicationTranslations(appPath)).toEqual({
+ 'it-IT': { [generateMessageId('Company')]: 'Azienda' },
+ });
+ });
+
+ it('skips a locale file that cannot be parsed', async () => {
+ const appPath = await mkdtemp(
+ join(tmpdir(), 'twenty-translations-unparsable-'),
+ );
+ const localesDir = join(appPath, 'locales');
+ await mkdir(join(localesDir, 'compiled'), { recursive: true });
+ await writeFile(join(localesDir, 'fr-FR.json'), '');
+ await writeFile(join(localesDir, 'compiled', 'de-DE.json'), '{');
+ await writeFile(
+ join(localesDir, 'it-IT.json'),
+ JSON.stringify({ Company: 'Azienda' }),
+ );
+
+ expect(await compileApplicationTranslations(appPath)).toEqual({
+ 'it-IT': { [generateMessageId('Company')]: 'Azienda' },
+ });
+ });
});
diff --git a/packages/twenty-sdk/src/cli/utilities/translations/collect-translatable-strings.ts b/packages/twenty-sdk/src/cli/utilities/translations/collect-translatable-strings.ts
index 3b1043108ff..efebf8814c1 100644
--- a/packages/twenty-sdk/src/cli/utilities/translations/collect-translatable-strings.ts
+++ b/packages/twenty-sdk/src/cli/utilities/translations/collect-translatable-strings.ts
@@ -70,6 +70,12 @@ export const collectTranslatableStrings = (
}
}
+ for (const objectManifest of manifest.objects ?? []) {
+ for (const field of objectManifest.fields ?? []) {
+ addEntityStrings(field, 'fieldMetadata');
+ }
+ }
+
// Tab and widget titles live nested under pageLayouts[].tabs[], not in the
// flat pageLayoutTabs array, so walk the tree to reach them.
for (const pageLayout of manifest.pageLayouts ?? []) {
diff --git a/packages/twenty-sdk/src/cli/utilities/translations/compile-application-translations.ts b/packages/twenty-sdk/src/cli/utilities/translations/compile-application-translations.ts
index 1b4aa999d56..983cc1d0f49 100644
--- a/packages/twenty-sdk/src/cli/utilities/translations/compile-application-translations.ts
+++ b/packages/twenty-sdk/src/cli/utilities/translations/compile-application-translations.ts
@@ -3,16 +3,94 @@ import path from 'path';
import { compileCatalogToMessageIds } from '@/cli/utilities/translations/compile-catalog-to-message-ids';
import { pathExists, readJson } from '@/cli/utilities/file/fs-utils';
-import { LOCALES_DIR } from '@/cli/utilities/translations/constants';
-import { type TranslationsManifest } from 'twenty-shared/application';
import {
- APP_LOCALES,
- SOURCE_LOCALE,
- type AppLocale,
-} from 'twenty-shared/translations';
+ COMPILED_LOCALES_DIR,
+ LOCALES_DIR,
+} from '@/cli/utilities/translations/constants';
+import { isSupportedLocale } from '@/cli/utilities/translations/is-supported-locale';
+import { type TranslationsManifest } from 'twenty-shared/application';
+import { SOURCE_LOCALE } from 'twenty-shared/translations';
+import { isDefined } from 'twenty-shared/utils';
+
+const readLocaleCatalogFile = async (
+ filePath: string,
+): Promise<Record<string, unknown> | null> => {
+ let parsed: unknown;
+
+ try {
+ parsed = await readJson<unknown>(filePath);
+ } catch {
+ console.warn(
+ `Skipping translation file "${path.basename(filePath)}": it is not valid JSON.`,
+ );
+
+ return null;
+ }
+
+ if (
+ !isDefined(parsed) ||
+ typeof parsed !== 'object' ||
+ Array.isArray(parsed)
+ ) {
+ console.warn(
+ `Skipping translation file "${path.basename(filePath)}": expected a JSON object.`,
+ );
+
+ return null;
+ }
+
+ return parsed as Record<string, unknown>;
+};
+
+const readCompiledCatalogs = async (
+ appPath: string,
+): Promise<Record<string, Record<string, string>>> => {
+ const compiledDir = path.join(appPath, COMPILED_LOCALES_DIR);
+
+ if (!(await pathExists(compiledDir))) {
+ return {};
+ }
+
+ const catalogs: Record<string, Record<string, string>> = {};
+
+ for (const compiledFile of (await readdir(compiledDir)).filter((entry) =>
+ entry.endsWith('.json'),
+ )) {
+ const locale = path.basename(compiledFile, '.json');
+
+ if (locale === SOURCE_LOCALE) {
+ continue;
+ }
+
+ if (!isSupportedLocale(locale)) {
+ console.warn(
+ `Skipping compiled translation file "${compiledFile}": "${locale}" is not a supported locale.`,
+ );
+ continue;
+ }
+
+ const compiledCatalog = await readLocaleCatalogFile(
+ path.join(compiledDir, compiledFile),
+ );
+
+ if (compiledCatalog === null) {
+ continue;
+ }
-const isSupportedLocale = (locale: string): locale is AppLocale =>
- Object.prototype.hasOwnProperty.call(APP_LOCALES, locale);
+ const messages = Object.fromEntries(
+ Object.entries(compiledCatalog).filter(
+ (entry): entry is [string, string] =>
+ typeof entry[1] === 'string' && entry[1].length > 0,
+ ),
+ );
+
+ if (Object.keys(messages).length > 0) {
+ catalogs[locale] = messages;
+ }
+ }
+
+ return catalogs;
+};
export const compileApplicationTranslations = async (
appPath: string,
@@ -43,10 +121,13 @@ export const compileApplicationTranslations = async (
continue;
}
- const sourceToTranslation =
- (await readJson<Record<string, unknown>>(
- path.join(localesDir, localeFile),
- )) ?? {};
+ const sourceToTranslation = await readLocaleCatalogFile(
+ path.join(localesDir, localeFile),
+ );
+
+ if (sourceToTranslation === null) {
+ continue;
+ }
const compiled = compileCatalogToMessageIds({
catalog: sourceToTranslation,
@@ -61,5 +142,11 @@ export const compileApplicationTranslations = async (
}
}
+ for (const [locale, messages] of Object.entries(
+ await readCompiledCatalogs(appPath),
+ )) {
+ translations[locale] = { ...messages, ...(translations[locale] ?? {}) };
+ }
+
return translations as TranslationsManifest;
};
diff --git a/packages/twenty-sdk/src/cli/utilities/translations/constants.ts b/packages/twenty-sdk/src/cli/utilities/translations/constants.ts
index fa6a338bd28..b2c3031b9fc 100644
--- a/packages/twenty-sdk/src/cli/utilities/translations/constants.ts
+++ b/packages/twenty-sdk/src/cli/utilities/translations/constants.ts
@@ -1 +1,5 @@
+import { posix } from 'node:path';
+
export const LOCALES_DIR = 'locales';
+
+export const COMPILED_LOCALES_DIR = posix.join(LOCALES_DIR, 'compiled');
diff --git a/packages/twenty-sdk/src/cli/utilities/translations/is-locale-catalog.ts b/packages/twenty-sdk/src/cli/utilities/translations/is-locale-catalog.ts
new file mode 100644
index 00000000000..840e638c6ba
--- /dev/null
+++ b/packages/twenty-sdk/src/cli/utilities/translations/is-locale-catalog.ts
@@ -0,0 +1,9 @@
+import { isDefined } from 'twenty-shared/utils';
+
+export const isLocaleCatalog = (
+ value: unknown,
+): value is Record<string, string> =>
+ isDefined(value) &&
+ typeof value === 'object' &&
+ !Array.isArray(value) &&
+ Object.values(value).every((translation) => typeof translation === 'string');
diff --git a/packages/twenty-sdk/src/cli/utilities/translations/is-supported-locale.ts b/packages/twenty-sdk/src/cli/utilities/translations/is-supported-locale.ts
new file mode 100644
index 00000000000..b9e2840a5c0
--- /dev/null
+++ b/packages/twenty-sdk/src/cli/utilities/translations/is-supported-locale.ts
@@ -0,0 +1,4 @@
+import { APP_LOCALES, type AppLocale } from 'twenty-shared/translations';
+
+export const isSupportedLocale = (locale: string): locale is AppLocale =>
+ Object.prototype.hasOwnProperty.call(APP_LOCALES, locale);