Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

fix(pieces): self-hosted instances get piece translations from the cloud registry again (#16968)

Amr Elmohamadye18245a17937
diff --git a/brain/knowledge/pieces-engine/pieces.md b/brain/knowledge/pieces-engine/pieces.md
index 32070742730..702b97837c4 100644
--- a/brain/knowledge/pieces-engine/pieces.md
+++ b/brain/knowledge/pieces-engine/pieces.md
@@ -24,10 +24,12 @@ The metadata catalog of automation integrations ("pieces") — each a named inte
- Install and sync also enqueue a tool-search reindex, but only when `isToolSearchEnabled()`; no-op otherwise.
- `delete` removes all versions sharing the name on that platform, and only for `CUSTOM` pieces the caller owns.
- **A piece silently vanishes from the list when its `minimumSupportedRelease` is ahead of the root `package.json` version.** `fetchLatestPieces` filters every piece through `isSupportedRelease(apVersionUtil.getCurrentRelease(), piece)`. Pieces are routinely merged targeting the *next* release, so on `main` a couple dozen are invisible locally until the version bump lands. No warning is logged — it just isn't there. The same gate hides the version from exact-version lookups (`findExactVersion`), so a flow step saved on it fails to load on an older server, and `AP_DEV_PIECES` does not bypass it: the database copies are dropped by name and the dev copy by release, so the piece vanishes entirely. To test a piece that targets the next release, bump the root `package.json` version locally and do not commit it. A server that already reports that release but lacks the server code the piece needs is the dangerous case; check `origin/main`'s root version at merge time. Cloud runs the same gate on its root version: on 2026-10-20 cloud reported 0.92.2 and served piece-ai 0.11.0 while main had 0.14.1, so every version gated on the next release (0.12 to 0.15) switches on for everyone at once with the bump. See what cloud serves with `curl https://cloud.activepieces.com/api/v1/pieces/@activepieces/piece-<name>` (`version`, `minimumSupportedRelease`). Merging publishes the piece to the registry at once (`release-pieces.yml` on push to main), before any deploy. That is safe because the gate also covers self-hosted: sync asks the registry with `release=<own version>` (`listCloudPieces`), so an older backend never downloads it. Self-hosted releases are made by hand, so moving the root version on main changes cloud only until someone cuts that self-hosted release; bump it in its own `chore: release X` PR with only `package.json` (leave `docker-compose.yml` and the Helm chart on the last published image), and never revert it: a server that drops back below a gated piece version can no longer load steps saved on it.
-- **`/v1/pieces` is uncached, so a new `piece_metadata` column must be added to `PIECE_COLUMNS_WITHOUT_TRANSLATIONS` or it silently will not appear in the list.** `pieceCache` caches only the *registry* (name/version/platformId tuples) for `get`/`findExactVersion`/`registry`; the list path never touches it, and the `dedupe()` helper in `piece-metadata-service.ts` is in-flight-only (it deletes the key in `finally`), so it collapses concurrent callers but caches nothing between requests. Every request therefore re-runs the whole pipeline against the DB. Because of that, `fetchLatestCompatiblePiecesFromDB` selects an explicit column list that deliberately omits `i18n` — that column holds every locale for the piece (~50 MB across the catalogue, written by `pieceInstallService`) and the list path discards it, so loading it was pure waste. Non-English requests now pull only the one locale in a second query (`pm."i18n" -> :locale`) and hand it to `pieceTranslation.translatePiece`. The single-piece path (`get`/`getOrThrow`) selects the same columns and goes through the same `translatePieces`, so no response ever carries `i18n`. The cost is that the explicit select is a list to keep in sync with the entity.
+- **`/v1/pieces` is uncached, so a new `piece_metadata` column must be added to `PIECE_COLUMNS_WITHOUT_TRANSLATIONS` or it silently will not appear in the list.** `pieceCache` caches only the *registry* (name/version/platformId tuples) for `get`/`findExactVersion`/`registry`; the list path never touches it, and the `dedupe()` helper in `piece-metadata-service.ts` is in-flight-only (it deletes the key in `finally`), so it collapses concurrent callers but caches nothing between requests. Every request therefore re-runs the whole pipeline against the DB. Because of that, `fetchLatestCompatiblePiecesFromDB` selects an explicit column list that deliberately omits `i18n` — that column holds every locale for the piece (~50 MB across the catalogue, written by `pieceInstallService`) and the list path discards it, so loading it was pure waste. Non-English requests now pull only the one locale in a second query (`pm."i18n" -> :locale`) and hand it to `pieceTranslation.translatePiece`. The single-piece path (`get`/`getOrThrow`) selects the same columns and goes through the same `translatePieces`, so internal callers never get `i18n`. The cost is that the explicit select is a list to keep in sync with the entity.
- **Piece sorting mutates its input, which is why the list result cannot simply be cached.** `pieceSorting`'s comparators and `reverseIfDesc` sort in place (`pieces.sort(...)`, `pieces.reverse()`). That is safe only because the array is rebuilt per request; making them non-mutating is a prerequisite for ever caching the built list, or requests would reorder shared state under each other.
- **`translatePiece` is copy-on-write: never deep-clone a piece to translate it, and never cache translated lists.** It copies only the objects on a translated path and returns the input's other subtrees by reference, in one walk over a tree built from `pathsToValuesToTranslate`. The input is shared through `dedupe()` across concurrent requests in different locales, so mutating it leaks one locale into another. The old `JSON.parse(JSON.stringify(piece))` made every non-English list ~3x slower than English (GIT-1891), and caching the translated catalogue per locale was tried twice (#15340, #15639) and dropped over memory. `packages/pieces/framework/test/i18n.test.ts` fails if a deep clone comes back. Translations are read from `piece_translation` (one row per piece version and locale, `pieceTranslationService`), not from the `json` `i18n` column: extracting one locale from that column re-parsed every locale (~160 ms on prod, paid even for a locale with no rows) and was ~5x slower than the table (GIT-2013).
- **A piece's `displayName` is translated only for `PieceCategory.CORE` pieces.** Never add it to `pathsToValuesToTranslate`: PR 9124 removed it after brand names got translated (Salesforce became "Force de ventes"). `pieceTranslation.pathsToTranslate(categories)` is the one rule, used by `translatePiece` and the CLI generator. Search runs on the translated catalogue, so `fetchLatestPieces` also returns `untranslatedNames` (English name of every piece whose name was translated) and search matches it in both the Fuse piece match and `searchForSuggestion`. Miss the second and the picker gets the piece with no actions or triggers, which hides it (GIT-1995).
+- **The single-piece routes return every locale's `i18n` when called without `locale`: that is the cloud registry contract self-hosted sync depends on.** `pieceSyncService` fetches `GET /v1/pieces/:name?version=X` with no locale and stores whatever `i18n` comes back. #16620 stripped it, so every piece version synced from Oct 20 2026 until the fix landed got stored English-only (GIT-2050). The controller maps a missing locale to `locale: 'all'`, which makes `get()` attach every locale via `pieceTranslationService.getAllLocales`. Internal callers pass no locale, which still means English with `i18n` stripped, so they don't pay for the extra query. Never strip `i18n` from a locale-less single-piece response.
+
- **`piece_metadata.i18n` is still written but only read as a fallback, until it is dropped.** `pieceMetadataService.create` writes both the column and `piece_translation` in one transaction; `BackfillPieceTranslation` (background) copies existing rows. While the backfill runs, a piece with no `piece_translation` rows falls back to `i18n -> :locale`, so nobody sees English mid-backfill. The follow-up drops the column behind `assertBackgroundMigrationComplete` and deletes that fallback in `pieceTranslationService.getForLocale`. Synced pieces keep the cloud's `id`, so translation rows must use the id that was actually saved, never a freshly generated one.
- **DynamicProperties clears its value before it knows the new schema, so the merge source must be a snapshot.** `DynamicPropertiesImplementation` re-fetches the child schema on every refresher change, clearing the form value synchronously and re-populating it in the mutation callback. The merge source for `getDefaultValueForProperties` has to be a `lastKnownValue` ref captured *before* the clear — reading `form.getValues()` in the callback sees the cleared `null` and defaults every child (GIT-1514). The snapshot must be spread-cloned: RHF `getValues(name)` hands back the live object and the clear's `setValue(...child, null)` mutates it in place. Guard the ref with `isNil` so it survives rapid successive changes, where later effect runs already observe `null`.
- `DynamicPropertiesContext` tracks loading by property name only, so two in-flight requests for the same property let the first completion clear the flag for both — briefly re-enabling Test Step while the value is still cleared.
diff --git a/packages/server/api/src/app/database/migration/postgres/background/1879000000000-RefetchMissingPieceTranslations.ts b/packages/server/api/src/app/database/migration/postgres/background/1879000000000-RefetchMissingPieceTranslations.ts
new file mode 100644
index 00000000000..9d7f3147ea5
--- /dev/null
+++ b/packages/server/api/src/app/database/migration/postgres/background/1879000000000-RefetchMissingPieceTranslations.ts
@@ -0,0 +1,182 @@
+import { chunk, isNil, isObject, sanitizeObjectForPostgresql, tryCatch } from '@activepieces/core-utils'
+import { safeHttp } from '@activepieces/server-utils'
+import { ApEdition, PieceAudienceFilter, PieceSyncMode } from '@activepieces/shared'
+import { isAxiosError } from 'axios'
+import semver from 'semver'
+import { QueryRunner } from 'typeorm'
+import { system } from '../../../../helper/system/system'
+import { AppSystemProp } from '../../../../helper/system/system-props'
+import { BackgroundMigration } from '../../../background-migration'
+
+export class RefetchMissingPieceTranslations1879000000000 implements BackgroundMigration {
+ name = 'RefetchMissingPieceTranslations1879000000000'
+ release = '0.93.0'
+ transaction = false as const
+
+ public async up(queryRunner: QueryRunner): Promise<void> {
+ if (system.getEdition() === ApEdition.CLOUD || system.get(AppSystemProp.PIECES_SYNC_MODE) !== PieceSyncMode.OFFICIAL_AUTO) {
+ return
+ }
+ const versions = await listOfficialVersions({ queryRunner })
+ const candidates = versions.filter((version) => !version.translated)
+ if (candidates.length === 0) {
+ return
+ }
+ await assertRegistryServesTranslations({ versions })
+ const outcomes = await chunk(candidates, BATCH_SIZE).reduce<Promise<RestoreOutcome[]>>(
+ async (previous, batch) => [
+ ...await previous,
+ ...await Promise.all(batch.map((piece) => restoreTranslations({ queryRunner, piece }))),
+ ],
+ Promise.resolve([]),
+ )
+ const counts = countOutcomes({ outcomes })
+ system.globalLogger().info({
+ migration: { name: this.name, candidates: candidates.length, ...counts },
+ }, '[RefetchMissingPieceTranslations] Restored translations for piece versions synced without them')
+ if (counts.transient > Math.max(MAX_SKIPPED_TRANSIENT_FAILURES, candidates.length * MAX_SKIPPED_TRANSIENT_RATIO)) {
+ throw new Error(`Registry failed transiently for ${counts.transient} of ${candidates.length} piece versions; retrying the remaining ones later`)
+ }
+ }
+
+ public async down(): Promise<void> {
+ return
+ }
+}
+
+async function listOfficialVersions({ queryRunner }: { queryRunner: QueryRunner }): Promise<OfficialVersion[]> {
+ const rows: unknown = await queryRunner.query(`
+ SELECT pm."id", pm."name", pm."version",
+ (pm."i18n" IS NOT NULL OR EXISTS (SELECT 1 FROM "piece_translation" t WHERE t."pieceMetadataId" = pm."id")) AS "translated"
+ FROM "piece_metadata" pm
+ WHERE pm."pieceType" = 'OFFICIAL'
+ `)
+ return Array.isArray(rows) ? rows.filter(isOfficialVersion) : []
+}
+
+async function assertRegistryServesTranslations({ versions }: { versions: OfficialVersion[] }): Promise<void> {
+ const canaries = sortBySemver({ versions: versions.filter((version) => version.translated) }).reverse().slice(0, MAX_CANARIES)
+ for (const canary of canaries) {
+ const { data: body, error } = await tryCatch(async () => fetchPieceVersion({ piece: canary }))
+ if (error && classifyFailure({ error }) === 'transient') {
+ throw new Error(`Registry canary ${versionKey(canary)} failed (${error.message}); retrying later`)
+ }
+ if (error || !isVersionRef(body) || versionKey(body) !== versionKey(canary)) {
+ continue
+ }
+ if (toTranslationRows({ pieceMetadataId: canary.id, body }).length === 0) {
+ throw new Error(`Registry returned ${versionKey(canary)} without translations although it is translated here; the registry may be serving a build without them, retrying later`)
+ }
+ return
+ }
+ system.globalLogger().warn({ triedCanaries: canaries.map(versionKey) }, '[RefetchMissingPieceTranslations] No translated piece version installed that the registry still serves, skipping the registry canary check')
+}
+
+async function restoreTranslations({ queryRunner, piece }: RestoreTranslationsParams): Promise<RestoreOutcome> {
+ const { data: body, error } = await tryCatch(async () => fetchPieceVersion({ piece }))
+ if (error) {
+ const outcome = classifyFailure({ error })
+ system.globalLogger().warn({ piece: { name: piece.name, version: piece.version }, outcome, error: error.message }, outcome === 'transient'
+ ? '[RefetchMissingPieceTranslations] Registry request failed, the run retries if too many fail'
+ : '[RefetchMissingPieceTranslations] Registry will not serve this version, leaving it untranslated')
+ return outcome
+ }
+ if (!isVersionRef(body) || versionKey(body) !== versionKey(piece)) {
+ system.globalLogger().warn({ piece: { name: piece.name, version: piece.version } }, '[RefetchMissingPieceTranslations] Registry returned a different piece version, leaving it untranslated')
+ return 'mismatched'
+ }
+ const rows = toTranslationRows({ pieceMetadataId: piece.id, body })
+ if (rows.length === 0) {
+ return 'untranslated'
+ }
+ await queryRunner.query(`
+ INSERT INTO "piece_translation" ("pieceMetadataId", "locale", "translations")
+ SELECT row."pieceMetadataId", row."locale", row."translations"
+ FROM jsonb_to_recordset($1::jsonb) AS row("pieceMetadataId" varchar, "locale" varchar, "translations" jsonb)
+ ON CONFLICT DO NOTHING
+ `, [JSON.stringify(rows)])
+ return 'restored'
+}
+
+async function fetchPieceVersion({ piece }: { piece: OfficialVersion }): Promise<unknown> {
+ const params = new URLSearchParams({ audience: PieceAudienceFilter.ALL, version: piece.version })
+ const response = await safeHttp.axios.get<unknown>(`${CLOUD_PIECES_URL}/${piece.name}?${params.toString()}`, { timeout: REGISTRY_TIMEOUT_MS })
+ return response.data
+}
+
+function classifyFailure({ error }: { error: Error }): 'notFound' | 'rejected' | 'transient' {
+ const status = isAxiosError(error) ? error.response?.status : undefined
+ if (isNil(status) || status === 408 || status === 429 || status >= 500) {
+ return 'transient'
+ }
+ return status === 404 ? 'notFound' : 'rejected'
+}
+
+function toTranslationRows({ pieceMetadataId, body }: { pieceMetadataId: string, body: unknown }): TranslationRow[] {
+ if (!isObject(body) || !('i18n' in body)) {
+ return []
+ }
+ const sanitized = sanitizeObjectForPostgresql(body.i18n)
+ if (!isObject(sanitized)) {
+ return []
+ }
+ return Object.entries(sanitized).flatMap(([locale, translations]) =>
+ isObject(translations) ? [{ pieceMetadataId, locale, translations }] : [],
+ )
+}
+
+function sortBySemver({ versions }: { versions: OfficialVersion[] }): OfficialVersion[] {
+ return versions
+ .filter((version) => !isNil(semver.valid(version.version)))
+ .sort((left, right) => semver.compare(left.version, right.version))
+}
+
+function countOutcomes({ outcomes }: { outcomes: RestoreOutcome[] }): Record<RestoreOutcome, number> {
+ return {
+ restored: outcomes.filter((outcome) => outcome === 'restored').length,
+ untranslated: outcomes.filter((outcome) => outcome === 'untranslated').length,
+ mismatched: outcomes.filter((outcome) => outcome === 'mismatched').length,
+ notFound: outcomes.filter((outcome) => outcome === 'notFound').length,
+ rejected: outcomes.filter((outcome) => outcome === 'rejected').length,
+ transient: outcomes.filter((outcome) => outcome === 'transient').length,
+ }
+}
+
+function versionKey({ name, version }: { name: string, version: string }): string {
+ return `${name}@${version}`
+}
+
+function isVersionRef(value: unknown): value is { name: string, version: string } {
+ return isObject(value) && 'name' in value && typeof value.name === 'string' && 'version' in value && typeof value.version === 'string'
+}
+
+function isOfficialVersion(value: unknown): value is OfficialVersion {
+ return isVersionRef(value) && 'id' in value && typeof value.id === 'string' && 'translated' in value && typeof value.translated === 'boolean'
+}
+
+const CLOUD_PIECES_URL = 'https://cloud.activepieces.com/api/v1/pieces'
+const BATCH_SIZE = 5
+const MAX_CANARIES = 5
+const REGISTRY_TIMEOUT_MS = 30_000
+const MAX_SKIPPED_TRANSIENT_FAILURES = 5
+const MAX_SKIPPED_TRANSIENT_RATIO = 0.05
+
+type OfficialVersion = {
+ id: string
+ name: string
+ version: string
+ translated: boolean
+}
+
+type RestoreOutcome = 'restored' | 'untranslated' | 'mismatched' | 'notFound' | 'rejected' | 'transient'
+
+type RestoreTranslationsParams = {
+ queryRunner: QueryRunner
+ piece: OfficialVersion
+}
+
+type TranslationRow = {
+ pieceMetadataId: string
+ locale: string
+ translations: object
+}
diff --git a/packages/server/api/src/app/database/postgres-connection.ts b/packages/server/api/src/app/database/postgres-connection.ts
index a87c13142fb..1f0d2ea8f75 100644
--- a/packages/server/api/src/app/database/postgres-connection.ts
+++ b/packages/server/api/src/app/database/postgres-connection.ts
@@ -468,6 +468,7 @@ import { AddAiUsageTables1878000000000 } from './migration/postgres/187800000000
import { AddAgentConversationProviderConfigIdIndex1869000000000 } from './migration/postgres/background/1869000000000-AddAgentConversationProviderConfigIdIndex'
import { ReshapeFlowRunListIndexesForBarrierChildren1871000000000 } from './migration/postgres/background/1871000000000-ReshapeFlowRunListIndexesForBarrierChildren'
import { BackfillPieceTranslation1874000000000 } from './migration/postgres/background/1874000000000-BackfillPieceTranslation'
+import { RefetchMissingPieceTranslations1879000000000 } from './migration/postgres/background/1879000000000-RefetchMissingPieceTranslations'
const getSslConfig = (): boolean | TlsOptions => {
const useSsl = system.get(AppSystemProp.POSTGRES_USE_SSL)
@@ -483,6 +484,7 @@ export const getBackgroundMigrations = (): (new () => BackgroundMigration)[] =>
return [
AddAgentConversationProviderConfigIdIndex1869000000000,
BackfillPieceTranslation1874000000000,
+ RefetchMissingPieceTranslations1879000000000,
ReshapeFlowRunListIndexesForBarrierChildren1871000000000,
]
}
diff --git a/packages/server/api/src/app/pieces/metadata/piece-metadata-controller.ts b/packages/server/api/src/app/pieces/metadata/piece-metadata-controller.ts
index d301b11c70c..cc07dee26e6 100644
--- a/packages/server/api/src/app/pieces/metadata/piece-metadata-controller.ts
+++ b/packages/server/api/src/app/pieces/metadata/piece-metadata-controller.ts
@@ -80,7 +80,7 @@ const basePiecesController: FastifyPluginAsyncZod = async (app) => {
platformId,
name: `${decodeScope}/${decodedName}`,
version,
- locale: req.query.locale as LocalesEnum | undefined,
+ locale: (req.query.locale as LocalesEnum | undefined) ?? 'all',
})
const policy = await resolveVisibility({ platformId, projectId: req.query.projectId, log: req.log })
const visiblePiece = applyVisibilityPolicy({ policy, piece })
@@ -101,7 +101,7 @@ const basePiecesController: FastifyPluginAsyncZod = async (app) => {
platformId,
name: decodedName,
version,
- locale: req.query.locale as LocalesEnum | undefined,
+ locale: (req.query.locale as LocalesEnum | undefined) ?? 'all',
})
const policy = await resolveVisibility({ platformId, projectId: req.query.projectId, log: req.log })
const visiblePiece = applyVisibilityPolicy({ policy, piece })
diff --git a/packages/server/api/src/app/pieces/metadata/piece-metadata-service.ts b/packages/server/api/src/app/pieces/metadata/piece-metadata-service.ts
index faf0fe1a203..53404c6f63e 100644
--- a/packages/server/api/src/app/pieces/metadata/piece-metadata-service.ts
+++ b/packages/server/api/src/app/pieces/metadata/piece-metadata-service.ts
@@ -70,16 +70,16 @@ export const pieceMetadataService = (log: FastifyBaseLogger) => {
if (isNil(piece)) {
return undefined
}
- const [translatedPiece] = await translatePieces({ pieces: [piece], locale: locale ?? LocalesEnum.ENGLISH })
+ const resolvedPiece = await resolveTranslations({ piece, locale: locale ?? LocalesEnum.ENGLISH })
const policy = await resolveVisibility({ platformId, projectId, log })
if (isNil(policy)) {
- return translatedPiece
+ return resolvedPiece
}
- if (!policy.isPieceVisible(translatedPiece.name)) {
+ if (!policy.isPieceVisible(resolvedPiece.name)) {
return undefined
}
- return policy.filterPieceComponents(translatedPiece)
+ return policy.filterPieceComponents(resolvedPiece)
},
async checkActionsExist({ actions, platformId }: CheckActionsExistParams): Promise<ActionExistence> {
const latestVersions = await Promise.all(Object.keys(actions).map((name) => findExactVersion(log, { name, version: undefined, platformId })))
@@ -522,6 +522,15 @@ function pickLatestVersionIds(pieces: PieceKey[]): string[] {
return Array.from(latest.values()).map((p) => p.id)
}
+async function resolveTranslations({ piece, locale }: ResolveTranslationsParams): Promise<PieceMetadataSchema> {
+ if (locale === 'all') {
+ const i18n = await pieceTranslationService.getAllLocales({ pieceMetadataId: piece.id })
+ return { ...piece, i18n: i18n ?? piece.i18n }
+ }
+ const [translatedPiece] = await translatePieces({ pieces: [piece], locale })
+ return translatedPiece
+}
+
async function translatePieces({ pieces, locale }: TranslatePiecesParams): Promise<PieceMetadataSchema[]> {
if (locale === LocalesEnum.ENGLISH) {
return pieces.map((piece) => ({ ...piece, i18n: undefined }))
@@ -593,7 +602,7 @@ type GetOrThrowParams = {
entityManager?: EntityManager
projectId?: string
platformId?: string
- locale?: LocalesEnum
+ locale?: LocalesEnum | 'all'
}
type DeleteParams = {
@@ -662,6 +671,11 @@ type UntranslatedNamesOfParams = {
translatedPieces: PieceMetadataSchema[]
}
+type ResolveTranslationsParams = {
+ piece: PieceMetadataSchema
+ locale: LocalesEnum | 'all'
+}
+
type TranslatePiecesParams = {
pieces: PieceMetadataSchema[]
locale: LocalesEnum
diff --git a/packages/server/api/src/app/pieces/metadata/piece-translation-service.ts b/packages/server/api/src/app/pieces/metadata/piece-translation-service.ts
index d5ed04fb2a7..31648ceda22 100644
--- a/packages/server/api/src/app/pieces/metadata/piece-translation-service.ts
+++ b/packages/server/api/src/app/pieces/metadata/piece-translation-service.ts
@@ -29,6 +29,17 @@ export const pieceTranslationService = {
.getRawMany<{ id: string, translations: Record<string, string> | null }>()
return new Map(rows.flatMap((row) => isNil(row.translations) ? [] : [[row.id, row.translations]]))
},
+
+ async getAllLocales({ pieceMetadataId }: GetAllLocalesParams): Promise<I18nForPiece | undefined> {
+ const row = await pieceTranslationRepos().manager
+ .createQueryBuilder(PieceMetadataEntity, 'pm')
+ .leftJoin(TABLE, 'pt', 'pt."pieceMetadataId" = pm."id"')
+ .select(ALL_TRANSLATIONS_WITH_LEGACY_FALLBACK, 'i18n')
+ .where('pm."id" = :pieceMetadataId', { pieceMetadataId })
+ .groupBy('pm."id"')
+ .getRawOne<{ i18n: I18nForPiece | null }>()
+ return row?.i18n ?? undefined
+ },
}
function toRows({ pieceMetadataId, i18n }: Pick<SaveParams, 'pieceMetadataId' | 'i18n'>): PieceTranslationSchema[] {
@@ -43,6 +54,11 @@ const TRANSLATIONS_WITH_LEGACY_FALLBACK = `COALESCE(
CASE WHEN NOT EXISTS (SELECT 1 FROM "${TABLE}" stored WHERE stored."pieceMetadataId" = pm."id") THEN pm."i18n" -> :locale END
)`
+const ALL_TRANSLATIONS_WITH_LEGACY_FALLBACK = `COALESCE(
+ json_object_agg(pt."locale", pt."translations") FILTER (WHERE pt."locale" IS NOT NULL),
+ CASE WHEN NOT EXISTS (SELECT 1 FROM "${TABLE}" stored WHERE stored."pieceMetadataId" = pm."id") THEN pm."i18n" END
+)`
+
type SaveParams = {
pieceMetadataId: string
i18n: I18nForPiece
@@ -53,3 +69,7 @@ type GetForLocaleParams = {
pieceIds: string[]
locale: LocalesEnum
}
+
+type GetAllLocalesParams = {
+ pieceMetadataId: string
+}
diff --git a/packages/server/api/test/integration/ce/database/background-migration-runner.test.ts b/packages/server/api/test/integration/ce/database/background-migration-runner.test.ts
index 94e139b8908..d3a7e89a9f1 100644
--- a/packages/server/api/test/integration/ce/database/background-migration-runner.test.ts
+++ b/packages/server/api/test/integration/ce/database/background-migration-runner.test.ts
@@ -10,6 +10,7 @@ import { databaseConnection } from '../../../../src/app/database/database-connec
import { BackgroundMigrationNotCompleteError, migrationHelpers } from '../../../../src/app/database/migration-helpers'
import { BackfillPieceTranslation1874000000000 } from '../../../../src/app/database/migration/postgres/background/1874000000000-BackfillPieceTranslation'
import { ReshapeFlowRunListIndexesForBarrierChildren1871000000000 } from '../../../../src/app/database/migration/postgres/background/1871000000000-ReshapeFlowRunListIndexesForBarrierChildren'
+import { RefetchMissingPieceTranslations1879000000000 } from '../../../../src/app/database/migration/postgres/background/1879000000000-RefetchMissingPieceTranslations'
import { getBackgroundMigrations } from '../../../../src/app/database/postgres-connection'
import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'
@@ -167,6 +168,13 @@ describe('getBackgroundMigrations', () => {
expect(order.indexOf(BackfillPieceTranslation1874000000000)).toBeLessThan(order.indexOf(ReshapeFlowRunListIndexesForBarrierChildren1871000000000))
expect(order.at(-1)).toBe(ReshapeFlowRunListIndexesForBarrierChildren1871000000000)
})
+
+ it('runs the translation repair after the backfill and before the flow_run index reshape', () => {
+ const order = getBackgroundMigrations()
+
+ expect(order.indexOf(RefetchMissingPieceTranslations1879000000000)).toBeGreaterThan(order.indexOf(BackfillPieceTranslation1874000000000))
+ expect(order.indexOf(RefetchMissingPieceTranslations1879000000000)).toBeLessThan(order.indexOf(ReshapeFlowRunListIndexesForBarrierChildren1871000000000))
+ })
})
describe('runMigrationsWithCatchup', () => {
diff --git a/packages/server/api/test/integration/ce/database/refetch-missing-piece-translations.test.ts b/packages/server/api/test/integration/ce/database/refetch-missing-piece-translations.test.ts
new file mode 100644
index 00000000000..4029424299f
--- /dev/null
+++ b/packages/server/api/test/integration/ce/database/refetch-missing-piece-translations.test.ts
@@ -0,0 +1,412 @@
+import { apId, LocalesEnum } from '@activepieces/core-utils'
+import { ApEdition, PackageType, PieceSyncMode, PieceType } from '@activepieces/shared'
+import { AxiosError } from 'axios'
+import { FastifyInstance } from 'fastify'
+import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
+import { backgroundMigrationRunner } from '../../../../src/app/database/background-migration-runner'
+import { databaseConnection } from '../../../../src/app/database/database-connection'
+import { RefetchMissingPieceTranslations1879000000000 } from '../../../../src/app/database/migration/postgres/background/1879000000000-RefetchMissingPieceTranslations'
+import { pieceMetadataService } from '../../../../src/app/pieces/metadata/piece-metadata-service'
+import { createMockPieceMetadata } from '../../../helpers/mocks'
+import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'
+
+const { getMock } = vi.hoisted(() => ({ getMock: vi.fn() }))
+
+vi.mock('@activepieces/server-utils', async (importOriginal) => {
+ const original = await importOriginal<typeof import('@activepieces/server-utils')>()
+ return {
+ ...original,
+ safeHttp: { ...original.safeHttp, axios: { get: getMock } },
+ }
+})
+
+const REGISTRY_I18N = {
+ [LocalesEnum.GERMAN]: { 'Send a message': 'Eine Nachricht senden' },
+ [LocalesEnum.JAPANESE]: { 'Send a message': 'メッセージを送信' },
+}
+
+const lastGoodSync = officialPiece({ name: '@activepieces/piece-refetch-slack', version: '0.23.0', i18n: REGISTRY_I18N })
+const brokenAfterLastGoodSync = officialPiece({ name: '@activepieces/piece-refetch-slack', version: '0.23.1', created: '2023-06-05T00:00:00.000Z' })
+const untranslatedBeforeLastGoodSync = officialPiece({ name: '@activepieces/piece-refetch-slack', version: '0.1.0' })
+const brokenGoneFromCloud = officialPiece({ name: '@activepieces/piece-refetch-gone', version: '0.1.0' })
+const brokenMismatched = officialPiece({ name: '@activepieces/piece-refetch-mismatched', version: '5.0.0' })
+const brokenFetchFails = officialPiece({ name: '@activepieces/piece-refetch-unreachable', version: '1.0.0' })
+const untranslatedUpstream = officialPiece({ name: '@activepieces/piece-refetch-untranslated', version: '2.0.0' })
+const canary = officialPiece({ name: '@activepieces/piece-refetch-canary', version: '9.0.0', i18n: REGISTRY_I18N })
+const customPiece = createMockPieceMetadata({ name: '@activepieces/piece-refetch-custom', version: '4.0.0', pieceType: PieceType.CUSTOM, packageType: PackageType.REGISTRY, platformId: apId(), i18n: undefined })
+const seeded = [lastGoodSync, brokenAfterLastGoodSync, untranslatedBeforeLastGoodSync, brokenGoneFromCloud, brokenMismatched, brokenFetchFails, untranslatedUpstream, canary, customPiece]
+
+let app: FastifyInstance | null = null
+
+beforeAll(async () => {
+ app = await setupTestEnvironment()
+ await backgroundMigrationRunner.ensureTable()
+})
+
+afterAll(async () => {
+ await teardownTestEnvironment()
+})
+
+beforeEach(async () => {
+ process.env.AP_PIECES_SYNC_MODE = PieceSyncMode.OFFICIAL_AUTO
+ await databaseConnection().getRepository('piece_metadata').createQueryBuilder().delete().execute()
+ await databaseConnection().query('DELETE FROM "background_migrations" WHERE "name" = $1', [new RefetchMissingPieceTranslations1879000000000().name])
+ await seed({ pieces: seeded })
+ getMock.mockImplementation(async (url: string) => stubRegistry({ url }))
+})
+
+afterEach(() => {
+ process.env.AP_PIECES_SYNC_MODE = PieceSyncMode.NONE
+ getMock.mockReset()
+})
+
+describe('RefetchMissingPieceTranslations', () => {
+ it('restores every locale for a version newer than the last good sync, whatever its registry created date', async () => {
+ await runThroughRunner()
+
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ expect(getMock.mock.calls.every(([, config]) => typeof config?.timeout === 'number' && config.timeout > 0)).toBe(true)
+ })
+
+ it('fetches every untranslated version, including ones older than a translated version of the same piece', async () => {
+ await runThroughRunner()
+
+ expect(fetchedVersions()).toContain(key(untranslatedBeforeLastGoodSync))
+ expect(await translationsOf(untranslatedBeforeLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ })
+
+ it('does not fetch versions that are already translated, except as a canary', async () => {
+ const translatedOlder = officialPiece({ name: '@activepieces/piece-refetch-slack', version: '0.20.0', i18n: REGISTRY_I18N })
+ await seed({ pieces: [translatedOlder] })
+
+ await runThroughRunner()
+
+ expect(fetchedVersions()).not.toContain(key(translatedOlder))
+ expect(fetchedVersions().filter((fetched) => fetched === key(lastGoodSync)).length).toBeLessThanOrEqual(1)
+ })
+
+ it('repairs a broken version even after sync installed a newer translated version once the cloud fix was live', async () => {
+ const syncedAfterFix = officialPiece({ name: '@activepieces/piece-refetch-slack', version: '0.24.0', i18n: REGISTRY_I18N })
+ await seed({ pieces: [syncedAfterFix] })
+
+ await runThroughRunner()
+
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ })
+
+ it('repairs on the retry a broken version that failed while a newer one was restored', async () => {
+ const firstGood = officialPiece({ name: '@activepieces/piece-refetch-retry', version: '1.0.0', i18n: REGISTRY_I18N })
+ const brokenFailsFirst = officialPiece({ name: '@activepieces/piece-refetch-retry', version: '1.0.1' })
+ const brokenRestoredFirst = officialPiece({ name: '@activepieces/piece-refetch-retry', version: '1.0.2' })
+ const outage = Array.from({ length: 6 }, (_, index) => officialPiece({ name: `@activepieces/piece-refetch-retry-outage-${index}`, version: '1.0.0' }))
+ await seed({ pieces: [firstGood, brokenFailsFirst, brokenRestoredFirst, ...outage] })
+ const serve = ({ url }: { url: string }): { data: unknown } => {
+ const version = new URL(url).searchParams.get('version')
+ const piece = [brokenFailsFirst, brokenRestoredFirst, ...outage].find((candidate) => url.includes(`${candidate.name}?`) && candidate.version === version)
+ return piece ? { data: { name: piece.name, version, i18n: REGISTRY_I18N } } : stubRegistry({ url })
+ }
+ getMock.mockImplementation(async (url: string) => {
+ const failing = outage.some((piece) => url.includes(`${piece.name}?`)) || (url.includes(`${brokenFailsFirst.name}?`) && url.includes(`version=${brokenFailsFirst.version}`))
+ if (failing) {
+ throw httpError({ status: 503 })
+ }
+ return serve({ url })
+ })
+
+ await expect(runThroughRunner()).rejects.toThrow(/failed transiently/)
+ expect(await translationsOf(brokenRestoredFirst.id)).toEqual(REGISTRY_I18N)
+ expect(await translationsOf(brokenFailsFirst.id)).toEqual({})
+
+ getMock.mockImplementation(async (url: string) => serve({ url }))
+ await runThroughRunner()
+
+ expect(await translationsOf(brokenFailsFirst.id)).toEqual(REGISTRY_I18N)
+ expect(await markedComplete()).toBe(true)
+ })
+
+ it('fetches versions directly, without the instance\'s registry list', async () => {
+ await runThroughRunner()
+
+ expect(getMock.mock.calls.some(([url]) => String(url).includes('/registry'))).toBe(false)
+ expect(fetchedVersions()).toEqual(expect.arrayContaining([key(brokenAfterLastGoodSync), key(brokenGoneFromCloud)]))
+ })
+
+ it('leaves a version cloud no longer serves (404) installed and untranslated, and completes', async () => {
+ await runThroughRunner()
+
+ expect(await rowExists(brokenGoneFromCloud.id)).toBe(true)
+ expect(await translationsOf(brokenGoneFromCloud.id)).toEqual({})
+ expect(await markedComplete()).toBe(true)
+ })
+
+ it('does not attach translations when the registry answers with a different version', async () => {
+ await runThroughRunner()
+
+ expect(await translationsOf(brokenMismatched.id)).toEqual({})
+ })
+
+ it('skips a few transient failures and still completes', async () => {
+ await runThroughRunner()
+
+ expect(await translationsOf(brokenFetchFails.id)).toEqual({})
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ expect(await markedComplete()).toBe(true)
+ })
+
+ it('fails the run when many requests fail transiently, and restores them on the retry', async () => {
+ const outage = Array.from({ length: 6 }, (_, index) => officialPiece({ name: `@activepieces/piece-refetch-outage-${index}`, version: '1.0.0' }))
+ await seed({ pieces: outage })
+ getMock.mockImplementation(async (url: string) => {
+ if (outage.some((piece) => url.includes(piece.name))) {
+ throw httpError({ status: 503 })
+ }
+ return stubRegistry({ url })
+ })
+
+ await expect(runThroughRunner()).rejects.toThrow(/failed transiently/)
+ expect(await markedComplete()).toBe(false)
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual(REGISTRY_I18N)
+
+ getMock.mockImplementation(async (url: string) => {
+ const piece = outage.find((candidate) => url.includes(candidate.name))
+ return piece ? { data: { name: piece.name, version: piece.version, i18n: REGISTRY_I18N } } : stubRegistry({ url })
+ })
+ await runThroughRunner()
+
+ expect(await translationsOf(outage[0].id)).toEqual(REGISTRY_I18N)
+ expect(await markedComplete()).toBe(true)
+ })
+
+ it('completes when many versions are gone from cloud, since a 404 is not worth retrying', async () => {
+ const removed = Array.from({ length: 6 }, (_, index) => officialPiece({ name: `@activepieces/piece-refetch-removed-${index}`, version: '1.0.0' }))
+ await seed({ pieces: removed })
+ getMock.mockImplementation(async (url: string) => removed.some((piece) => url.includes(piece.name))
+ ? Promise.reject(httpError({ status: 404 }))
+ : stubRegistry({ url }))
+
+ await runThroughRunner()
+
+ expect(await markedComplete()).toBe(true)
+ expect(await rowExists(removed[0].id)).toBe(true)
+ })
+
+ it('completes when cloud rejects many versions with a 4xx, since its validation will not change on retry', async () => {
+ const invalid = Array.from({ length: 6 }, (_, index) => officialPiece({ name: `@activepieces/piece-refetch-invalid-${index}`, version: '1.0.0-beta.1' }))
+ await seed({ pieces: invalid })
+ getMock.mockImplementation(async (url: string) => invalid.some((piece) => url.includes(piece.name))
+ ? Promise.reject(httpError({ status: 400 }))
+ : stubRegistry({ url }))
+
+ await runThroughRunner()
+
+ expect(await markedComplete()).toBe(true)
+ expect(await rowExists(invalid[0].id)).toBe(true)
+ })
+
+ it('treats rate limiting and timeouts as transient, so the run retries', async () => {
+ const throttled = Array.from({ length: 3 }, (_, index) => officialPiece({ name: `@activepieces/piece-refetch-throttled-${index}`, version: '1.0.0' }))
+ const timedOut = Array.from({ length: 3 }, (_, index) => officialPiece({ name: `@activepieces/piece-refetch-timedout-${index}`, version: '1.0.0' }))
+ await seed({ pieces: [...throttled, ...timedOut] })
+ getMock.mockImplementation(async (url: string) => {
+ if (throttled.some((piece) => url.includes(piece.name))) {
+ throw httpError({ status: 429 })
+ }
+ if (timedOut.some((piece) => url.includes(piece.name))) {
+ throw new AxiosError('timeout of 30000ms exceeded', 'ECONNABORTED')
+ }
+ return stubRegistry({ url })
+ })
+
+ await expect(runThroughRunner()).rejects.toThrow(/failed transiently/)
+ expect(await markedComplete()).toBe(false)
+ })
+
+ it('fails the run without restoring anything when the registry serves a translated version without translations', async () => {
+ getMock.mockImplementation(async (url: string) => url.includes(canary.name)
+ ? { data: { name: canary.name, version: canary.version } }
+ : stubRegistry({ url }))
+
+ await expect(runThroughRunner()).rejects.toThrow(/without translations/)
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual({})
+ expect(await markedComplete()).toBe(false)
+ })
+
+ it('fails the run without restoring anything when the canary request itself fails', async () => {
+ getMock.mockImplementation(async (url: string) => url.includes(canary.name)
+ ? Promise.reject(httpError({ status: 503 }))
+ : stubRegistry({ url }))
+
+ await expect(runThroughRunner()).rejects.toThrow(/canary/)
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual({})
+ expect(await markedComplete()).toBe(false)
+ })
+
+ it('tries the next translated version when cloud no longer serves the first canary', async () => {
+ getMock.mockImplementation(async (url: string) => url.includes(canary.name)
+ ? Promise.reject(httpError({ status: 404 }))
+ : stubRegistry({ url }))
+
+ await runThroughRunner()
+
+ expect(fetchedVersions()).toEqual(expect.arrayContaining([key(canary), key(lastGoodSync)]))
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ expect(await markedComplete()).toBe(true)
+ })
+
+ it('still fails the run when a later canary is served without translations', async () => {
+ getMock.mockImplementation(async (url: string) => {
+ if (url.includes(canary.name)) {
+ throw httpError({ status: 404 })
+ }
+ if (url.includes(lastGoodSync.name) && url.includes(`version=${lastGoodSync.version}`)) {
+ return { data: { name: lastGoodSync.name, version: lastGoodSync.version } }
+ }
+ return stubRegistry({ url })
+ })
+
+ await expect(runThroughRunner()).rejects.toThrow(/without translations/)
+ expect(await markedComplete()).toBe(false)
+ })
+
+ it('skips the canary check and still repairs when cloud serves none of the translated versions', async () => {
+ getMock.mockImplementation(async (url: string) => {
+ if (url.includes(canary.name) || (url.includes(lastGoodSync.name) && url.includes(`version=${lastGoodSync.version}`))) {
+ throw httpError({ status: 400 })
+ }
+ return stubRegistry({ url })
+ })
+
+ await runThroughRunner()
+
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ expect(await markedComplete()).toBe(true)
+ })
+
+ it('skips the canary check when no translated version is installed', async () => {
+ await databaseConnection().query('DELETE FROM "piece_metadata" WHERE "id" = ANY($1)', [[lastGoodSync.id, canary.id]])
+
+ await runThroughRunner()
+
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ expect(await translationsOf(untranslatedBeforeLastGoodSync.id)).toEqual(REGISTRY_I18N)
+ expect(await markedComplete()).toBe(true)
+ })
+
+ it('fetches untranslated versions that are not valid semver but never translated ones', async () => {
+ const translatedSemver = officialPiece({ name: '@activepieces/piece-refetch-nonsemver', version: '1.0.0', i18n: REGISTRY_I18N })
+ const translatedNonSemver = officialPiece({ name: '@activepieces/piece-refetch-nonsemver', version: 'nightly', i18n: REGISTRY_I18N })
+ const untranslatedNonSemver = officialPiece({ name: '@activepieces/piece-refetch-nonsemver', version: 'nightly-2' })
+ await seed({ pieces: [translatedSemver, translatedNonSemver, untranslatedNonSemver] })
+ getMock.mockImplementation(async (url: string) => {
+ const version = new URL(url).searchParams.get('version')
+ return url.includes('piece-refetch-nonsemver')
+ ? { data: { name: '@activepieces/piece-refetch-nonsemver', version, i18n: REGISTRY_I18N } }
+ : stubRegistry({ url })
+ })
+
+ await runThroughRunner()
+
+ expect(fetchedVersions()).toContain(key(untranslatedNonSemver))
+ expect(fetchedVersions()).not.toContain(key(translatedNonSemver))
+ expect(await translationsOf(untranslatedNonSemver.id)).toEqual(REGISTRY_I18N)
+ })
+
+ it('does not fetch custom pieces', async () => {
+ await runThroughRunner()
+
+ expect(fetchedVersions()).not.toContain(key(customPiece))
+ expect(await translationsOf(untranslatedUpstream.id)).toEqual({})
+ })
+
+ it('does nothing on cloud, which is the registry itself', async () => {
+ const edition = process.env.AP_EDITION
+ process.env.AP_EDITION = ApEdition.CLOUD
+ try {
+ await runThroughRunner()
+ }
+ finally {
+ process.env.AP_EDITION = edition
+ }
+
+ expect(getMock).not.toHaveBeenCalled()
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual({})
+ })
+
+ it('does nothing when piece sync is off', async () => {
+ process.env.AP_PIECES_SYNC_MODE = PieceSyncMode.NONE
+
+ await runThroughRunner()
+
+ expect(getMock).not.toHaveBeenCalled()
+ expect(await translationsOf(brokenAfterLastGoodSync.id)).toEqual({})
+ expect(await markedComplete()).toBe(true)
+ })
+})
+
+function officialPiece({ name, version, created, i18n }: { name: string, version: string, created?: string, i18n?: typeof REGISTRY_I18N }): ReturnType<typeof createMockPieceMetadata> {
+ return createMockPieceMetadata({ name, version, created, i18n, pieceType: PieceType.OFFICIAL, packageType: PackageType.REGISTRY, platformId: undefined })
+}
+
+async function seed({ pieces }: { pieces: ReturnType<typeof createMockPieceMetadata>[] }): Promise<void> {
+ for (const piece of pieces) {
+ await pieceMetadataService(app!.log).create({ pieceMetadata: piece, packageType: PackageType.REGISTRY, pieceType: piece.pieceType, publishCacheRefresh: false })
+ }
+}
+
+function stubRegistry({ url }: { url: string }): { data: unknown } {
+ const version = new URL(url).searchParams.get('version')
+ if (url.includes(brokenFetchFails.name)) {
+ throw new Error('connect ETIMEDOUT')
+ }
+ if (url.includes(brokenGoneFromCloud.name)) {
+ throw httpError({ status: 404 })
+ }
+ if (url.includes(brokenMismatched.name)) {
+ return { data: { name: brokenMismatched.name, version: '5.0.1', i18n: REGISTRY_I18N } }
+ }
+ if (url.includes(untranslatedUpstream.name)) {
+ return { data: { name: untranslatedUpstream.name, version: untranslatedUpstream.version } }
+ }
+ const piece = seeded.find((candidate) => url.includes(candidate.name) && candidate.version === version)
+ return { data: { name: piece?.name, version, i18n: REGISTRY_I18N } }
+}
+
+function httpError({ status }: { status: number }): AxiosError {
+ return Object.assign(new AxiosError(`Request failed with status code ${status}`), { response: { status } })
+}
+
+function key({ name, version }: { name: string, version: string }): string {
+ return `${name}@${version}`
+}
+
+function fetchedVersions(): string[] {
+ return getMock.mock.calls
+ .map(([url]) => new URL(String(url)))
+ .filter((url) => !url.pathname.endsWith('/registry'))
+ .map((url) => `${decodeURIComponent(url.pathname.replace('/api/v1/pieces/', ''))}@${url.searchParams.get('version')}`)
+}
+
+async function runThroughRunner(): Promise<void> {
+ await backgroundMigrationRunner.run({ log: app!.log, migrations: [RefetchMissingPieceTranslations1879000000000] })
+}
+
+async function translationsOf(pieceMetadataId: string): Promise<Record<string, unknown>> {
+ const rows: { locale: string, translations: unknown }[] = await databaseConnection().query(
+ 'SELECT "locale", "translations" FROM "piece_translation" WHERE "pieceMetadataId" = $1',
+ [pieceMetadataId],
+ )
+ return Object.fromEntries(rows.map((row) => [row.locale, row.translations]))
+}
+
+async function rowExists(id: string): Promise<boolean> {
+ const rows = await databaseConnection().query('SELECT 1 FROM "piece_metadata" WHERE "id" = $1', [id])
+ return rows.length > 0
+}
+
+async function markedComplete(): Promise<boolean> {
+ const rows = await databaseConnection().query(
+ 'SELECT 1 FROM "background_migrations" WHERE "name" = $1 AND "executed_at" IS NOT NULL',
+ [new RefetchMissingPieceTranslations1879000000000().name],
+ )
+ return rows.length > 0
+}
diff --git a/packages/server/api/test/integration/ce/pieces/piece-translation.test.ts b/packages/server/api/test/integration/ce/pieces/piece-translation.test.ts
index 9e4303f14fb..b0c6b62d0bb 100644
--- a/packages/server/api/test/integration/ce/pieces/piece-translation.test.ts
+++ b/packages/server/api/test/integration/ce/pieces/piece-translation.test.ts
@@ -57,6 +57,55 @@ describe('piece translations stored per locale', () => {
expect(germanSingle?.json().i18n).toBeUndefined()
})
+ it('are returned for every locale by the single route when no locale is requested, so self-hosted sync keeps them', async () => {
+ const saved = await pieceMetadataService(mockLog).create({
+ pieceMetadata: createMockPieceMetadata({
+ name: 'synced-with-all-locales',
+ pieceType: PieceType.OFFICIAL,
+ packageType: PackageType.REGISTRY,
+ description: 'Send a message',
+ i18n: {
+ [LocalesEnum.GERMAN]: { 'Send a message': 'Eine Nachricht senden' },
+ [LocalesEnum.JAPANESE]: { 'Send a message': 'メッセージを送信' },
+ },
+ }),
+ packageType: PackageType.REGISTRY,
+ pieceType: PieceType.OFFICIAL,
+ })
+ await databaseConnection().query('UPDATE "piece_metadata" SET "i18n" = NULL WHERE "id" = $1', [saved.id])
+ await pieceCache(mockLog).setup()
+ const headers = { authorization: `Bearer ${await generateMockToken({ type: PrincipalType.UNKNOWN, id: apId() })}` }
+
+ const [registryFetch, englishFetch] = await Promise.all([
+ app?.inject({ method: 'GET', url: `/api/v1/pieces/synced-with-all-locales?audience=all&version=${saved.version}`, headers }),
+ app?.inject({ method: 'GET', url: '/api/v1/pieces/synced-with-all-locales?locale=en', headers }),
+ ])
+
+ expect(registryFetch?.json().i18n).toEqual({
+ [LocalesEnum.GERMAN]: { 'Send a message': 'Eine Nachricht senden' },
+ [LocalesEnum.JAPANESE]: { 'Send a message': 'メッセージを送信' },
+ })
+ expect(registryFetch?.json().description).toBe('Send a message')
+ expect(englishFetch?.json().i18n).toBeUndefined()
+ })
+
+ it('fall back to the legacy i18n column for the single route when no piece_translation rows exist', async () => {
+ const saved = await pieceMetadataService(mockLog).create({
+ pieceMetadata: createMockPieceMetadata({ name: 'legacy-i18n-only', pieceType: PieceType.OFFICIAL, packageType: PackageType.REGISTRY }),
+ packageType: PackageType.REGISTRY,
+ pieceType: PieceType.OFFICIAL,
+ })
+ const legacyI18n = { [LocalesEnum.JAPANESE]: { 'Send a message': 'メッセージを送信' } }
+ await databaseConnection().query('UPDATE "piece_metadata" SET "i18n" = $1 WHERE "id" = $2', [legacyI18n, saved.id])
+ await databaseConnection().query('DELETE FROM "piece_translation" WHERE "pieceMetadataId" = $1', [saved.id])
+ await pieceCache(mockLog).setup()
+ const headers = { authorization: `Bearer ${await generateMockToken({ type: PrincipalType.UNKNOWN, id: apId() })}` }
+
+ const response = await app?.inject({ method: 'GET', url: '/api/v1/pieces/legacy-i18n-only', headers })
+
+ expect(response?.json().i18n).toEqual(legacyI18n)
+ })
+
it('are saved with characters Postgres jsonb rejects stripped from keys and values', async () => {
const saved = await pieceMetadataService(mockLog).create({
pieceMetadata: createMockPieceMetadata({
diff --git a/packages/web/src/features/pieces/api/pieces-api.ts b/packages/web/src/features/pieces/api/pieces-api.ts
index 6ed62fe2e5d..c6d0bd9d34e 100644
--- a/packages/web/src/features/pieces/api/pieces-api.ts
+++ b/packages/web/src/features/pieces/api/pieces-api.ts
@@ -1,3 +1,4 @@
+import { LocalesEnum } from '@activepieces/core-utils';
import {
PieceMetadataModel,
PieceMetadataModelSummary,
@@ -28,7 +29,7 @@ export const piecesApi = {
): Promise<PieceMetadataModel> {
return api.get<PieceMetadataModel>(`/v1/pieces/${request.name}`, {
version: request.version ?? undefined,
- locale: request.locale ?? undefined,
+ locale: request.locale ?? LocalesEnum.ENGLISH,
projectId: request.projectId ?? undefined,
audience: request.audience ?? undefined,
});