The open alternative to Salesforce, designed for AI.
TypeScript crmcrm-systemcustomergood-first-issuegraphqlhacktoberfestjavascriptmarketingmonoreponestjsopen-sourcepostgresqlreactreactjssalestypescriptweb
Add `yarn twenty pull` to write an installed application back to source (#25410)
Weiko92359d5a70de
diff --git a/packages/twenty-docs/developers/extend/apps/operations/cli.mdx b/packages/twenty-docs/developers/extend/apps/operations/cli.mdxindex ddfa00bc8e5..753204fdfcb 100644--- a/packages/twenty-docs/developers/extend/apps/operations/cli.mdx+++ b/packages/twenty-docs/developers/extend/apps/operations/cli.mdx@@ -11,6 +11,7 @@ The `yarn twenty` CLI is your interface to everything app-related. Full command| `dev` | Watch source files and live-sync changes | [Quick Start](/developers/extend/apps/getting-started/quick-start) || `plan` | Preview metadata changes without applying them | [Syncing & recovery](/developers/extend/apps/operations/sync-and-recovery#previewing-changes-plan) || `apply` | Apply metadata changes after showing the plan | [Syncing & recovery](/developers/extend/apps/operations/sync-and-recovery) |+| `pull` | **Experimental.** Write the installed application back to local source files | [Syncing & recovery](/developers/extend/apps/operations/sync-and-recovery#pulling-an-installed-app-back-to-source) || `dev:build` | Compile the app and generate the API client (`--tarball` to pack a `.tgz`) | [Publishing](/developers/extend/apps/operations/publishing) || `dev:typecheck` | Run TypeScript type checking | [Testing](/developers/extend/apps/operations/testing) || `dev:add` | Scaffold a new entity | [Scaffolding](/developers/extend/apps/getting-started/scaffolding) |diff --git a/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdx b/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdxindex 7e35bad1cfb..3706d534262 100644--- a/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdx+++ b/packages/twenty-docs/developers/extend/apps/operations/sync-and-recovery.mdx@@ -18,6 +18,7 @@ For day-to-day local iteration you almost always want `yarn twenty dev`. Deployi| Sync once and exit (CI, scripts, hooks) | `yarn twenty apply` | One build + sync, then exits. Add `--force` to skip the destructive-change confirmation. || Preview changes **without applying them** | `yarn twenty plan` | Computes and prints the diff; writes nothing. || Sync without deleting anything | `yarn twenty apply --no-delete` | Creates and updates only: entities that exist in the workspace but not in your source are left alone. Also accepted by `plan` and `dev`. |+| Regenerate local source from the workspace | `yarn twenty pull` | **Experimental.** Writes the installed app back to define files. See below. || Remove the app from the workspace | `yarn twenty app:uninstall` | Add `--yes` to skip the prompt. || Ship a tarball to a server | `yarn twenty app:publish --private` | Requires a **strictly higher** `package.json` version — see [Publishing](/developers/extend/apps/operations/publishing). || Publish to the marketplace (npm) | `yarn twenty app:publish` | — |@@ -94,6 +95,40 @@ A plan:A plan only previews **metadata** changes. It also works for an app that was never synced: the server evaluates the manifest against an empty application, so the plan lists everything your source would create.</Note>+## Pulling an installed app back to source++<Warning>+`yarn twenty pull` is **experimental**. It covers only part of an application today, its output shape may change between releases, and it overwrites the files that define the entities it pulls. Commit your work before running it, and review the diff.+</Warning>++`yarn twenty pull` is the inverse of `apply`: it reads an application out of the workspace and writes it back as define files.++```bash+yarn twenty pull -u <universalIdentifier>+```++Pull needs an existing project — scaffold one with `npx create-twenty-app@latest` first, then pull into it. Without `-u`, pull targets the application your local `defineApplication()` file declares. Add `-v` to expand the coverage sections of the report, which then name up to 20 identifiers per metadata type instead of only the totals.++**What it writes:**++- the application config;+- objects, with their fields inline;+- fields your app added to objects it does not own;+- authored indexes.++Each entity goes to the file that already defines it, so a second pull rewrites only what changed on the server. New entities are placed beside existing files of their kind, and a generated name that would land on an unrelated file is qualified rather than overwriting it.++**What it does not write:**++- **Not exported yet** — roles and permissions, views, page layouts, navigation, command menu items, logic functions and front components, and the stored source of your functions.+- **Derived by the engine** — system fields, default relations, default views and backing indexes. The server rebuilds them on apply. An object may still point its label identifier at one of them, as junction objects created in the UI do; the pointer is written and the build resolves it without adding a field.+- **Workspace runtime state** — member and API-key role assignments, webhooks and per-user navigation, all excluded by design.+- **Owned by another application** — reported, never pulled.++The report at the end of a pull lists all of it, grouped by reason. Everything it names as not written is still in your workspace, so push a pulled tree back with `yarn twenty apply --no-delete` until the missing kinds are covered. A pruning `yarn twenty apply` would delete them, and `yarn twenty plan` lists exactly which.++**The base file.** Pull records what it pulled in `.twenty/pull-base.json`, which is gitignored and per checkout. That file is what a later pull compares against to leave untouched the files whose entity did not change, and to delete the file of an entity that is gone from the workspace. A checkout without it simply writes everything and deletes nothing.+## Recovery ladderWhen local metadata looks wrong, escalate in this order and stop as soon as you're unblocked. Each step is more disruptive than the last.diff --git a/packages/twenty-sdk/src/cli/commands/dev/index.ts b/packages/twenty-sdk/src/cli/commands/dev/index.tsindex aac96ba4318..6cdc8ad6a24 100644--- a/packages/twenty-sdk/src/cli/commands/dev/index.ts+++ b/packages/twenty-sdk/src/cli/commands/dev/index.ts@@ -8,6 +8,7 @@ import { AppDevCommand } from './dev';import { AppDevOnceCommand } from './dev-once';import { registerDevFunctionCommands } from './function';import { AppGenerateClientCommand } from './generate-client';+import { AppPullCommand } from './pull';import { AppTranslationsExtractCommand } from './translations-extract';import { AppTypecheckCommand } from './typecheck';@@ -15,6 +16,7 @@ export const registerDevCommands = (program: Command): void => {const buildCommand = new AppBuildCommand();const devCommand = new AppDevCommand();const devOnceCommand = new AppDevOnceCommand();+ const pullCommand = new AppPullCommand();const typecheckCommand = new AppTypecheckCommand();const addCommand = new EntityAddCommand();const generateClientCommand = new AppGenerateClientCommand();@@ -146,6 +148,29 @@ export const registerDevCommands = (program: Command): void => {},);+ program+ .command('pull [appPath]')+ .description(+ 'Write the installed application back to local source files (experimental)',+ )+ .option(+ '-u, --universal-identifier <id>',+ 'Universal identifier of the application to pull',+ )+ .option('-v, --verbose', 'Show detailed logs')+ .action(+ async (+ appPath: string | undefined,+ options: { universalIdentifier?: string; verbose?: boolean },+ ) => {+ await pullCommand.execute({+ appPath: formatPath(appPath),+ universalIdentifier: options.universalIdentifier,+ verbose: options.verbose,+ });+ },+ );+program.command('dev:build [appPath]').description('Build and generate API client')diff --git a/packages/twenty-sdk/src/cli/commands/dev/pull.ts b/packages/twenty-sdk/src/cli/commands/dev/pull.tsnew file mode 100644index 00000000000..65a61769b1c--- /dev/null+++ b/packages/twenty-sdk/src/cli/commands/dev/pull.ts@@ -0,0 +1,65 @@+import { appPull } from '@/cli/operations/pull';+import { ConfigService } from '@/cli/utilities/config/config-service';+import { CURRENT_EXECUTION_DIRECTORY } from '@/cli/utilities/config/current-execution-directory';+import { formatPullReport } from '@/cli/utilities/pull/format-pull-report';+import { PULL_BASE_FILE_PATH } from '@/cli/utilities/pull/pull-base-file';+import { checkSdkVersionCompatibility } from '@/cli/utilities/version/check-sdk-version-compatibility';+import chalk from 'chalk';++export type AppPullCommandOptions = {+ appPath?: string;+ universalIdentifier?: string;+ verbose?: boolean;+};++export class AppPullCommand {+ async execute(options: AppPullCommandOptions): Promise<void> {+ const appPath = options.appPath ?? CURRENT_EXECUTION_DIRECTORY;++ await checkSdkVersionCompatibility(appPath);++ const remoteName = ConfigService.getActiveRemote();++ console.log(+ chalk.yellow(+ '⚠ pull is experimental\n' ++ ' It covers only part of an application, and it overwrites the files\n' ++ ' that define what it pulls. Commit your work before running it.\n',+ ),+ );+ console.log(chalk.blue(`Pulling application from ${remoteName}...`));+ console.log(chalk.gray(`App path: ${appPath}\n`));++ const result = await appPull({+ appPath,+ universalIdentifier: options.universalIdentifier,+ onProgress: (message) => console.log(chalk.gray(message)),+ });++ if (!result.success) {+ console.error(chalk.red(result.error.message));+ process.exit(1);+ }++ const report = formatPullReport({+ writes: result.data.writes,+ deletions: result.data.deletions,+ unchangedCount: result.data.unchangedCount,+ skipped: result.data.skipped,+ coverage: result.data.coverage,+ localOnlyRelativePaths: result.data.localOnlyRelativePaths,+ unreadableRelativePaths: result.data.unreadableRelativePaths,+ verbose: options.verbose,+ });++ console.log(`\n${report}\n`);++ console.log(+ chalk.green(+ `✓ Pulled ${result.data.applicationDisplayName} into ${appPath}`,+ ),+ );+ console.log(chalk.gray(`Base recorded in ${PULL_BASE_FILE_PATH}`));+ console.log(chalk.gray('Next: yarn twenty plan --no-delete'));+ }+}diff --git a/packages/twenty-sdk/src/cli/operations/index.ts b/packages/twenty-sdk/src/cli/operations/index.tsindex cae2abc386b..98f37b07787 100644--- a/packages/twenty-sdk/src/cli/operations/index.ts+++ b/packages/twenty-sdk/src/cli/operations/index.ts@@ -16,6 +16,8 @@ export type { AppDevOnceOptions, AppDevOnceResult } from './dev-once';export { appInstall } from './install';export type { AppInstallOptions } from './install';export { appPublish } from './publish';+export { appPull } from './pull';+export type { AppPullOptions, AppPullResult } from './pull';export type { AppPublishOptions, AppPublishResult } from './publish';export { appUninstall } from './uninstall';export type { AppUninstallOptions } from './uninstall';diff --git a/packages/twenty-sdk/src/cli/operations/pull.ts b/packages/twenty-sdk/src/cli/operations/pull.tsnew file mode 100644index 00000000000..1ae0ebfe477--- /dev/null+++ b/packages/twenty-sdk/src/cli/operations/pull.ts@@ -0,0 +1,197 @@+import { APP_ERROR_CODES, type CommandResult } from '@/cli/types';+import { ApiService } from '@/cli/utilities/api/api-service';+import { promptForReauthentication } from '@/cli/utilities/auth/reauth-helper';+import { ManifestEntityKey } from '@/cli/utilities/build/manifest/manifest-extract-config';+import { ConfigService } from '@/cli/utilities/config/config-service';+import {+ getGraphQLErrorMessage,+ hasGraphQLErrorSubCode,+} from '@/cli/utilities/error/parse-server-error';+import { pathExists } from '@/cli/utilities/file/fs-utils';+import { type ApplicationExportCoverageEntry } from '@/cli/utilities/pull/application-export-type';+import { applyPullWrites } from '@/cli/utilities/pull/apply-pull-writes';+import { type SkippedPullEntity } from '@/cli/utilities/pull/build-pull-entities';+import {+ planPullWrites,+ type PullDeletion,+ type PullWrite,+} from '@/cli/utilities/pull/plan-pull-writes';+import {+ readPullBaseManifest,+ writePullBaseManifest,+} from '@/cli/utilities/pull/pull-base-file';+import { scanProjectDefineFiles } from '@/cli/utilities/pull/scan-project-define-files';+import { runSafe } from '@/cli/utilities/run-safe';+import { join } from 'node:path';+import { isDefined } from 'twenty-shared/utils';++export type AppPullOptions = {+ appPath: string;+ universalIdentifier?: string;+ onProgress?: (message: string) => void;+};++export type AppPullResult = {+ applicationDisplayName: string;+ applicationUniversalIdentifier: string;+ writes: PullWrite[];+ deletions: PullDeletion[];+ unchangedCount: number;+ localOnlyRelativePaths: string[];+ skipped: SkippedPullEntity[];+ coverage: ApplicationExportCoverageEntry[];+ unreadableRelativePaths: string[];+ hadBase: boolean;+};++const EXPORT_REFUSAL_SUB_CODES = [+ 'APPLICATION_NOT_EXPORTABLE',+ 'STANDARD_APPLICATION_NOT_EXPORTABLE',+ 'APPLICATION_NOT_FOUND',+];++const innerAppPull = async (+ options: AppPullOptions,+): Promise<CommandResult<AppPullResult>> => {+ const { appPath, onProgress } = options;++ if (!(await pathExists(join(appPath, 'package.json')))) {+ return {+ success: false,+ error: {+ code: APP_ERROR_CODES.PULL_FAILED,+ message:+ `No package.json found in ${appPath}.\n\n` ++ ' Scaffold a project first, then pull into it:\n' ++ ' npx create-twenty-app@latest my-app',+ },+ };+ }++ onProgress?.('Checking server...');++ const apiService = new ApiService({ disableInterceptors: true });+ const validateAuth = await apiService.validateAuth();++ if (!validateAuth.serverUp) {+ return {+ success: false,+ error: {+ code: APP_ERROR_CODES.PULL_FAILED,+ message:+ 'Cannot reach Twenty server.\n\n' ++ ' Start a local server:\n' ++ ' yarn twenty docker:start\n\n' ++ ' Check server status:\n' ++ ' yarn twenty docker:status',+ },+ };+ }++ if (!validateAuth.authValid) {+ const outcome = await promptForReauthentication(+ ConfigService.getActiveRemote(),+ );++ if (outcome !== 'reauthenticated') {+ return {+ success: false,+ error: {+ code: APP_ERROR_CODES.PULL_FAILED,+ message:+ 'Authentication failed. Run `yarn twenty remote:add` to authenticate.',+ },+ };+ }+ }++ onProgress?.('Reading local source files...');++ const scannedFiles = await scanProjectDefineFiles(appPath);+ const localApplicationUniversalIdentifier = scannedFiles.find(+ (scannedFile) => scannedFile.entityKey === ManifestEntityKey.Application,+ )?.universalIdentifier;++ const universalIdentifier =+ options.universalIdentifier ?? localApplicationUniversalIdentifier;++ if (!isDefined(universalIdentifier)) {+ return {+ success: false,+ error: {+ code: APP_ERROR_CODES.PULL_FAILED,+ message:+ 'Could not tell which application to pull.\n\n' ++ ' Pass the identifier explicitly:\n' ++ ' yarn twenty pull -u <universalIdentifier>',+ },+ };+ }++ onProgress?.(`Exporting application ${universalIdentifier}...`);++ const exportResult = await apiService.exportApplication(universalIdentifier);++ if (!exportResult.success) {+ const isRefusal = EXPORT_REFUSAL_SUB_CODES.some((subCode) =>+ hasGraphQLErrorSubCode(exportResult.error, subCode),+ );++ return {+ success: false,+ error: {+ code: APP_ERROR_CODES.PULL_FAILED,+ message: isRefusal+ ? (getGraphQLErrorMessage(exportResult.error) ??+ 'The server refused to export this application')+ : `Export failed: ${getGraphQLErrorMessage(exportResult.error) ?? exportResult.message ?? 'Unknown error'}`,+ },+ };+ }++ const applicationExport = exportResult.data;+ const { manifest } = applicationExport;++ onProgress?.('Planning source writes...');++ const baseManifest = await readPullBaseManifest({+ appPath,+ applicationUniversalIdentifier: manifest.application.universalIdentifier,+ });++ const plan = planPullWrites({ manifest, baseManifest, scannedFiles });++ onProgress?.('Writing source files...');++ await applyPullWrites({+ appPath,+ writes: plan.writes,+ deletions: plan.deletions,+ });++ await writePullBaseManifest({ appPath, manifest });++ return {+ success: true,+ data: {+ applicationDisplayName: applicationExport.application.displayName,+ applicationUniversalIdentifier:+ applicationExport.application.universalIdentifier,+ writes: plan.writes,+ deletions: plan.deletions,+ unchangedCount: plan.unchanged.length,+ localOnlyRelativePaths: plan.localOnlyRelativePaths,+ skipped: plan.skipped,+ coverage: applicationExport.coverage,+ unreadableRelativePaths: scannedFiles+ .filter((scannedFile) => !scannedFile.isReadable)+ .map((scannedFile) => scannedFile.relativePath),+ hadBase: isDefined(baseManifest),+ },+ };+};++export const appPull = (+ options: AppPullOptions,+): Promise<CommandResult<AppPullResult>> =>+ runSafe(() => innerAppPull(options), APP_ERROR_CODES.PULL_FAILED);diff --git a/packages/twenty-sdk/src/cli/types.ts b/packages/twenty-sdk/src/cli/types.tsindex 47e8aeb33e3..1d7b0412f28 100644--- a/packages/twenty-sdk/src/cli/types.ts+++ b/packages/twenty-sdk/src/cli/types.ts@@ -26,6 +26,7 @@ export const APP_ERROR_CODES = {APPLY_ABORTED: 'APPLY_ABORTED',TYPECHECK_FAILED: 'TYPECHECK_FAILED',DEPLOY_FAILED: 'DEPLOY_FAILED',+ PULL_FAILED: 'PULL_FAILED',} as const;export const SERVER_ERROR_CODES = {diff --git a/packages/twenty-sdk/src/cli/utilities/api/api-service.ts b/packages/twenty-sdk/src/cli/utilities/api/api-service.tsindex a6159c9b8e5..2334dd5daa8 100644--- a/packages/twenty-sdk/src/cli/utilities/api/api-service.ts+++ b/packages/twenty-sdk/src/cli/utilities/api/api-service.ts@@ -4,6 +4,7 @@ import { ApplicationApi } from '@/cli/utilities/api/application-api';import { FileApi } from '@/cli/utilities/api/file-api';import { LogicFunctionApi } from '@/cli/utilities/api/logic-function-api';import { SchemaApi } from '@/cli/utilities/api/schema-api';+import { type ApplicationExport } from '@/cli/utilities/pull/application-export-type';import { type Manifest } from 'twenty-shared/application';import {type MetadataValidationErrorResponse,@@ -97,6 +98,12 @@ export class ApiService {return this.applicationApi.syncApplication(manifest, options);}+ exportApplication(+ universalIdentifier: string,+ ): Promise<ApiResponse<ApplicationExport>> {+ return this.applicationApi.exportApplication(universalIdentifier);+ }+uninstallApplication(universalIdentifier: string): Promise<ApiResponse> {return this.applicationApi.uninstallApplication(universalIdentifier);}diff --git a/packages/twenty-sdk/src/cli/utilities/api/application-api.ts b/packages/twenty-sdk/src/cli/utilities/api/application-api.tsindex 54c484f1678..b964cfc1edd 100644--- a/packages/twenty-sdk/src/cli/utilities/api/application-api.ts+++ b/packages/twenty-sdk/src/cli/utilities/api/application-api.ts@@ -1,4 +1,5 @@import { type ApiResponse } from '@/cli/utilities/api/api-response-type';+import { type ApplicationExport } from '@/cli/utilities/pull/application-export-type';import { serializeError } from '@/cli/utilities/error/serialize-error';import axios, { type AxiosInstance } from 'axios';import { type Manifest } from 'twenty-shared/application';@@ -105,6 +106,67 @@ export class ApplicationApi {}}+ async exportApplication(+ universalIdentifier: string,+ ): Promise<ApiResponse<ApplicationExport>> {+ try {+ const query = `+ query ExportApplication($universalIdentifier: UUID!) {+ exportApplication(universalIdentifier: $universalIdentifier) {+ application {+ universalIdentifier+ displayName+ sourceType+ }+ manifest+ coverage {+ metadataName+ universalIdentifier+ status+ reason+ }+ files {+ folder+ path+ content+ }+ }+ }+ `;++ const response = await this.client.post(+ '/metadata',+ {+ query,+ variables: { universalIdentifier },+ },+ {+ headers: {+ 'Content-Type': 'application/json',+ Accept: '*/*',+ },+ },+ );++ if (response.data.errors) {+ return {+ success: false,+ error: response.data.errors[0],+ };+ }++ return {+ success: true,+ data: response.data.data.exportApplication,+ };+ } catch (error) {+ return {+ success: false,+ error,+ };+ }+ }+async createApplicationRegistration(input: {name: string;universalIdentifier: string;diff --git a/packages/twenty-sdk/src/cli/utilities/build/manifest/utils/__tests__/get-default-fields-in-object-fields.spec.ts b/packages/twenty-sdk/src/cli/utilities/build/manifest/utils/__tests__/get-default-fields-in-object-fields.spec.tsnew file mode 100644index 00000000000..3b259f182ff--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/build/manifest/utils/__tests__/get-default-fields-in-object-fields.spec.ts@@ -0,0 +1,94 @@+import { getDefaultFieldsInObjectFields } from '@/cli/utilities/build/manifest/utils/get-default-fields-in-object-fields';+import { type ObjectConfig } from '@/sdk/define/objects/object-config';+import { getFieldUniversalIdentifier } from 'twenty-shared/application';+import { FieldMetadataType } from 'twenty-shared/types';+import { describe, expect, it } from 'vitest';++const APP_UID = '11111111-1111-4111-8111-111111111111';+const OBJECT_UID = '22222222-2222-4222-8222-222222222222';+const RELATION_FIELD_UID = '33333333-3333-4333-8333-333333333333';++const buildObjectConfig = (overrides: Partial<ObjectConfig>): ObjectConfig =>+ ({+ universalIdentifier: OBJECT_UID,+ nameSingular: 'petCareAgreement',+ namePlural: 'petCareAgreements',+ labelSingular: 'Pet care agreement',+ labelPlural: 'Pet care agreements',+ fields: [+ {+ universalIdentifier: RELATION_FIELD_UID,+ name: 'pet',+ label: 'Pet',+ type: FieldMetadataType.RELATION,+ },+ ],+ ...overrides,+ }) as unknown as ObjectConfig;++describe('getDefaultFieldsInObjectFields', () => {+ it('should append a name field when the object declares none', () => {+ const { objectFields } = getDefaultFieldsInObjectFields({+ objectConfig: buildObjectConfig({}),+ applicationUniversalIdentifier: APP_UID,+ });++ const nameField = objectFields.find((field) => field.name === 'name');++ expect(nameField?.universalIdentifier).toBe(+ getFieldUniversalIdentifier({+ applicationUniversalIdentifier: APP_UID,+ objectUniversalIdentifier: OBJECT_UID,+ name: 'name',+ }),+ );+ });++ it('should keep the name field the object declares itself', () => {+ const { objectFields } = getDefaultFieldsInObjectFields({+ objectConfig: buildObjectConfig({+ fields: [+ {+ universalIdentifier: 'a-hand-written-identifier',+ name: 'name',+ label: 'Name',+ type: FieldMetadataType.TEXT,+ },+ ],+ } as unknown as Partial<ObjectConfig>),+ applicationUniversalIdentifier: APP_UID,+ });++ expect(objectFields).toHaveLength(1);+ expect(objectFields[0].universalIdentifier).toBe(+ 'a-hand-written-identifier',+ );+ });++ it('should not append a name field when the label identifier names an engine-derived field', () => {+ const { objectFields } = getDefaultFieldsInObjectFields({+ objectConfig: buildObjectConfig({+ labelIdentifierFieldMetadataUniversalIdentifier:+ getFieldUniversalIdentifier({+ applicationUniversalIdentifier: APP_UID,+ objectUniversalIdentifier: OBJECT_UID,+ name: 'id',+ }),+ }),+ applicationUniversalIdentifier: APP_UID,+ });++ expect(objectFields.map((field) => field.name)).toEqual(['pet']);+ });++ it('should still append a name field when the label identifier names one of its own fields', () => {+ const { objectFields } = getDefaultFieldsInObjectFields({+ objectConfig: buildObjectConfig({+ labelIdentifierFieldMetadataUniversalIdentifier: RELATION_FIELD_UID,+ }),+ applicationUniversalIdentifier: APP_UID,+ });++ expect(objectFields.map((field) => field.name)).toEqual(['pet', 'name']);+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/build/manifest/utils/get-default-fields-in-object-fields.ts b/packages/twenty-sdk/src/cli/utilities/build/manifest/utils/get-default-fields-in-object-fields.tsindex 22136568f27..c26c5c915e2 100644--- a/packages/twenty-sdk/src/cli/utilities/build/manifest/utils/get-default-fields-in-object-fields.ts+++ b/packages/twenty-sdk/src/cli/utilities/build/manifest/utils/get-default-fields-in-object-fields.ts@@ -1,3 +1,4 @@+import { isEngineDerivedLabelIdentifier } from '@/sdk/define/objects/is-engine-derived-label-identifier';import type { ObjectConfig } from '@/sdk/define/objects/object-config';import {getFieldUniversalIdentifier,@@ -44,7 +45,16 @@ export const getDefaultFieldsInObjectFields = ({applicationUniversalIdentifier,});- if (!objectConfigFieldNames.includes(defaultNameObjectField.name)) {+ const labelIdentifiesAnEngineDerivedField = isEngineDerivedLabelIdentifier({+ fields: objectConfig.fields,+ labelIdentifierFieldMetadataUniversalIdentifier:+ objectConfig.labelIdentifierFieldMetadataUniversalIdentifier,+ });++ if (+ !objectConfigFieldNames.includes(defaultNameObjectField.name) &&+ !labelIdentifiesAnEngineDerivedField+ ) {objectFieldsWithDefaults.push(defaultNameObjectField);}diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__integration__/pull-round-trip.integration.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__integration__/pull-round-trip.integration.spec.tsnew file mode 100644index 00000000000..5d1d4c0a2a9--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/__integration__/pull-round-trip.integration.spec.ts@@ -0,0 +1,347 @@+import { buildManifest } from '@/cli/utilities/build/manifest/manifest-build';+import { buildPullEntities } from '@/cli/utilities/pull/build-pull-entities';+import { writeDefineFile } from '@/cli/utilities/pull/write-define-file';+import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises';+import { dirname, join, resolve } from 'node:path';+import {+ getFieldUniversalIdentifier,+ type Manifest,+} from 'twenty-shared/application';+import { STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS } from 'twenty-shared/metadata';+import { afterAll, beforeAll, describe, expect, it } from 'vitest';++const PACKAGE_ROOT = resolve(__dirname, '../../../../..');++const APP_UID = '11111111-1111-4111-8111-111111111111';+const PET_UID = '22222222-2222-4222-8222-222222222222';+const PET_NAME_FIELD_UID = '33333333-3333-4333-8333-333333333333';+const PET_AGE_FIELD_UID = '44444444-4444-4444-8444-444444444444';+const PET_STATUS_FIELD_UID = '55555555-5555-4555-8555-555555555555';+const PET_OWNER_FIELD_UID = '66666666-6666-4666-8666-666666666666';+const COMPANY_PETS_FIELD_UID = '77777777-7777-4777-8777-777777777777';+const COMPANY_TAGLINE_FIELD_UID = '88888888-8888-4888-8888-888888888888';+const INDEX_UID = '99999999-9999-4999-8999-999999999999';+const INDEX_FIELD_UID = 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa';+const JUNCTION_UID = 'bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb';+const JUNCTION_PET_FIELD_UID = 'cccccccc-cccc-4ccc-8ccc-cccccccccccc';+const PET_AGREEMENTS_FIELD_UID = 'dddddddd-dddd-4ddd-8ddd-dddddddddddd';++const EXPORTED_MANIFEST = {+ application: {+ universalIdentifier: APP_UID,+ displayName: 'Pet Care',+ description: 'Pets and the companies that care for them',+ defaultRoleUniversalIdentifier: '20202020-02c2-43f2-b94d-cab1f2b532eb',+ packageJsonChecksum: 'a-package-json-checksum',+ yarnLockChecksum: 'a-yarn-lock-checksum',+ },+ objects: [+ {+ universalIdentifier: PET_UID,+ nameSingular: 'pet',+ namePlural: 'pets',+ labelSingular: 'Pet',+ labelPlural: 'Pets',+ description: 'A pet',+ icon: 'IconPaw',+ color: null,+ isLabelSyncedWithName: false,+ isSearchable: true,+ isUICreatable: true,+ isUIEditable: true,+ writability: 'OPEN',+ openRecordIn: 'USER_CHOICE',+ imageIdentifierFieldMetadataUniversalIdentifier: null,+ labelIdentifierFieldMetadataUniversalIdentifier: PET_NAME_FIELD_UID,+ fields: [+ {+ universalIdentifier: PET_NAME_FIELD_UID,+ type: 'TEXT',+ name: 'name',+ label: 'Name',+ description: 'Name',+ icon: 'IconAbc',+ options: null,+ universalSettings: null,+ defaultValue: "''",+ isUIEditable: true,+ writability: 'OPEN',+ isNullable: false,+ isUnique: false,+ isLabelSyncedWithName: false,+ },+ {+ universalIdentifier: PET_AGE_FIELD_UID,+ type: 'NUMBER',+ name: 'age',+ label: 'Age',+ options: null,+ universalSettings: { dataType: 'bigint', type: 'number' },+ defaultValue: null,+ isUIEditable: true,+ writability: 'OPEN',+ isNullable: true,+ isUnique: false,+ isLabelSyncedWithName: false,+ },+ {+ universalIdentifier: PET_STATUS_FIELD_UID,+ type: 'SELECT',+ name: 'status',+ label: 'Status',+ options: [+ {+ id: 'eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee',+ value: 'HEALTHY',+ label: 'Healthy',+ color: 'green',+ position: 0,+ },+ ],+ universalSettings: null,+ defaultValue: "'HEALTHY'",+ isUIEditable: true,+ writability: 'OPEN',+ isNullable: true,+ isUnique: false,+ isLabelSyncedWithName: false,+ },+ {+ universalIdentifier: PET_OWNER_FIELD_UID,+ type: 'RELATION',+ name: 'owner',+ label: 'Owner',+ options: null,+ defaultValue: null,+ isUIEditable: true,+ writability: 'OPEN',+ isNullable: true,+ isUnique: false,+ isLabelSyncedWithName: false,+ relationTargetFieldMetadataUniversalIdentifier:+ COMPANY_PETS_FIELD_UID,+ relationTargetObjectMetadataUniversalIdentifier:+ STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS.company,+ universalSettings: {+ relationType: 'MANY_TO_ONE',+ onDelete: 'SET_NULL',+ joinColumnName: 'ownerId',+ },+ },+ ],+ },+ {+ universalIdentifier: JUNCTION_UID,+ nameSingular: 'petCareAgreement',+ namePlural: 'petCareAgreements',+ labelSingular: 'Pet care agreement',+ labelPlural: 'Pet care agreements',+ labelIdentifierFieldMetadataUniversalIdentifier:+ getFieldUniversalIdentifier({+ applicationUniversalIdentifier: APP_UID,+ objectUniversalIdentifier: JUNCTION_UID,+ name: 'id',+ }),+ fields: [+ {+ universalIdentifier: JUNCTION_PET_FIELD_UID,+ type: 'RELATION',+ name: 'pet',+ label: 'Pet',+ options: null,+ defaultValue: null,+ isUIEditable: true,+ writability: 'OPEN',+ isNullable: true,+ isUnique: false,+ isLabelSyncedWithName: false,+ relationTargetFieldMetadataUniversalIdentifier:+ PET_AGREEMENTS_FIELD_UID,+ relationTargetObjectMetadataUniversalIdentifier: PET_UID,+ universalSettings: {+ relationType: 'MANY_TO_ONE',+ onDelete: 'CASCADE',+ joinColumnName: 'petId',+ },+ },+ ],+ },+ ],+ fields: [+ {+ universalIdentifier: COMPANY_TAGLINE_FIELD_UID,+ type: 'TEXT',+ name: 'tagline',+ label: 'Tagline',+ options: null,+ universalSettings: null,+ defaultValue: null,+ isUIEditable: true,+ writability: 'OPEN',+ isNullable: true,+ isUnique: false,+ isLabelSyncedWithName: false,+ objectUniversalIdentifier: STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS.company,+ },+ ],+ indexes: [+ {+ universalIdentifier: INDEX_UID,+ objectUniversalIdentifier: PET_UID,+ indexType: 'BTREE',+ isUnique: false,+ fields: [+ {+ universalIdentifier: INDEX_FIELD_UID,+ fieldUniversalIdentifier: PET_NAME_FIELD_UID,+ },+ ],+ },+ ],+} as unknown as Manifest;++const canonicalize = (value: unknown): unknown =>+ Array.isArray(value)+ ? value.map(canonicalize)+ : value !== null && typeof value === 'object'+ ? Object.fromEntries(+ Object.keys(value as Record<string, unknown>)+ .sort()+ .map((key) => [+ key,+ canonicalize((value as Record<string, unknown>)[key]),+ ]),+ )+ : value;++const sortByUniversalIdentifier = <T extends { universalIdentifier: string }>(+ entries: T[],+): T[] =>+ [...entries].sort((left, right) =>+ left.universalIdentifier.localeCompare(right.universalIdentifier),+ );++const withSortedFields = (objectManifest: {+ fields: { universalIdentifier: string }[];+}) => ({+ ...objectManifest,+ fields: sortByUniversalIdentifier(objectManifest.fields),+});++describe('pull round trip', () => {+ let appPath: string;+ let builtManifest: Manifest | null;+ let buildErrors: string[];++ beforeAll(async () => {+ appPath = await mkdtemp(join(PACKAGE_ROOT, '.pull-round-trip-'));++ await writeFile(+ join(appPath, 'package.json'),+ `${JSON.stringify(+ { name: 'pull-round-trip-app', version: '1.0.0', private: true },+ null,+ 2,+ )}\n`,+ );++ const { entities, skipped } = buildPullEntities(EXPORTED_MANIFEST);++ expect(skipped).toEqual([]);++ for (const entity of entities) {+ const filePath = join(+ appPath,+ entity.defaultFolder,+ `${entity.fileBaseName}${entity.fileSuffix}`,+ );++ await mkdir(dirname(filePath), { recursive: true });+ await writeFile(+ filePath,+ writeDefineFile({+ definer: entity.definer,+ config: entity.config,+ enumBindings: entity.enumBindings,+ }),+ );+ }++ const buildResult = await buildManifest(appPath);++ builtManifest = buildResult.manifest;+ buildErrors = buildResult.errors;+ }, 60000);++ afterAll(async () => {+ await rm(appPath, { recursive: true, force: true });+ });++ it('should build the written source without errors', () => {+ expect(buildErrors).toEqual([]);+ expect(builtManifest).not.toBeNull();+ });++ it('should rebuild the exported objects, fields and indexes unchanged', () => {+ expect(+ canonicalize(+ sortByUniversalIdentifier(builtManifest?.objects ?? []).map(+ withSortedFields,+ ),+ ),+ ).toEqual(+ canonicalize(+ sortByUniversalIdentifier(EXPORTED_MANIFEST.objects).map(+ withSortedFields,+ ),+ ),+ );++ expect(+ canonicalize(sortByUniversalIdentifier(builtManifest?.fields ?? [])),+ ).toEqual(+ canonicalize(sortByUniversalIdentifier(EXPORTED_MANIFEST.fields)),+ );++ expect(+ canonicalize(sortByUniversalIdentifier(builtManifest?.indexes ?? [])),+ ).toEqual(+ canonicalize(sortByUniversalIdentifier(EXPORTED_MANIFEST.indexes ?? [])),+ );+ });++ it('should keep a junction object without adding a name field to it', () => {+ const junctionObject = (builtManifest?.objects ?? []).find(+ (objectManifest) => objectManifest.universalIdentifier === JUNCTION_UID,+ );++ expect(junctionObject?.fields.map((field) => field.name)).toEqual(['pet']);+ expect(+ junctionObject?.labelIdentifierFieldMetadataUniversalIdentifier,+ ).toBe(+ getFieldUniversalIdentifier({+ applicationUniversalIdentifier: APP_UID,+ objectUniversalIdentifier: JUNCTION_UID,+ name: 'id',+ }),+ );+ });++ it('should rebuild the application header, leaving the build to recompute its checksums', () => {+ const builtApplication = builtManifest?.application as unknown as Record<+ string,+ unknown+ >;++ expect(builtApplication.universalIdentifier).toBe(APP_UID);+ expect(builtApplication.displayName).toBe('Pet Care');+ expect(builtApplication.description).toBe(+ 'Pets and the companies that care for them',+ );+ expect(builtApplication.defaultRoleUniversalIdentifier).toBe(+ '20202020-02c2-43f2-b94d-cab1f2b532eb',+ );+ expect(builtApplication.packageJsonChecksum).toBeNull();+ expect(builtApplication.yarnLockChecksum).toBeNull();+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/apply-pull-writes.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/apply-pull-writes.spec.tsnew file mode 100644index 00000000000..eb1afbe1281--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/apply-pull-writes.spec.ts@@ -0,0 +1,159 @@+import { applyPullWrites } from '@/cli/utilities/pull/apply-pull-writes';+import { type PullWrite } from '@/cli/utilities/pull/plan-pull-writes';+import {+ mkdir,+ mkdtemp,+ readFile,+ readdir,+ rm,+ writeFile,+} from 'node:fs/promises';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { afterEach, beforeEach, describe, expect, it } from 'vitest';++const buildWrite = (relativePath: string, content: string): PullWrite => ({+ kind: 'object',+ universalIdentifier: relativePath,+ relativePath,+ content,+ isRegeneration: false,+});++describe('applyPullWrites', () => {+ let appPath: string;++ beforeEach(async () => {+ appPath = await mkdtemp(join(tmpdir(), 'apply-pull-writes-'));+ });++ afterEach(async () => {+ await rm(appPath, { recursive: true, force: true });+ });++ const readAppFile = (relativePath: string) =>+ readFile(join(appPath, relativePath), 'utf-8');++ it('should write the planned files and delete the planned ones', async () => {+ await mkdir(join(appPath, 'src'), { recursive: true });+ await writeFile(join(appPath, 'src/gone.ts'), 'previous');++ await applyPullWrites({+ appPath,+ writes: [buildWrite('src/pet.object.ts', 'written')],+ deletions: [{ universalIdentifier: 'gone', relativePath: 'src/gone.ts' }],+ });++ expect(await readAppFile('src/pet.object.ts')).toBe('written');+ await expect(readAppFile('src/gone.ts')).rejects.toThrow();+ });++ it('should leave no working directory behind', async () => {+ await applyPullWrites({+ appPath,+ writes: [buildWrite('src/pet.object.ts', 'written')],+ deletions: [],+ });++ const workDirectoryEntries = await readdir(join(appPath, '.twenty'));++ expect(+ workDirectoryEntries.filter((entry) =>+ /^pull-(staging|backup)-/.test(entry),+ ),+ ).toEqual([]);+ });++ it('should restore every file it had already replaced when a later write fails', async () => {+ await mkdir(join(appPath, 'src'), { recursive: true });+ await writeFile(join(appPath, 'src/first.ts'), 'original first');+ await writeFile(join(appPath, 'src/second.ts'), 'original second');+ await writeFile(+ join(appPath, 'src/nested'),+ 'a file where a folder is needed',+ );++ await expect(+ applyPullWrites({+ appPath,+ writes: [+ buildWrite('src/first.ts', 'new first'),+ buildWrite('src/second.ts', 'new second'),+ buildWrite('src/nested/deep.object.ts', 'never lands'),+ ],+ deletions: [],+ }),+ ).rejects.toThrow();++ expect(await readAppFile('src/first.ts')).toBe('original first');+ expect(await readAppFile('src/second.ts')).toBe('original second');+ });++ it('should put back a file it had already deleted when a later step fails', async () => {+ await mkdir(join(appPath, 'src'), { recursive: true });+ await writeFile(join(appPath, 'src/kept.ts'), 'original kept');+ await writeFile(+ join(appPath, 'src/nested'),+ 'a file where a folder is needed',+ );++ await expect(+ applyPullWrites({+ appPath,+ writes: [+ buildWrite('src/kept.ts', 'new kept'),+ buildWrite('src/nested/deep.object.ts', 'never lands'),+ ],+ deletions: [],+ }),+ ).rejects.toThrow();++ expect(await readAppFile('src/kept.ts')).toBe('original kept');+ });++ it('should remove a new file it created when a later write fails', async () => {+ await mkdir(join(appPath, 'src'), { recursive: true });+ await writeFile(+ join(appPath, 'src/nested'),+ 'a file where a folder is needed',+ );++ await expect(+ applyPullWrites({+ appPath,+ writes: [+ buildWrite('src/brand-new.object.ts', 'new content'),+ buildWrite('src/nested/deep.object.ts', 'never lands'),+ ],+ deletions: [],+ }),+ ).rejects.toThrow();++ await expect(readAppFile('src/brand-new.object.ts')).rejects.toThrow();+ });++ it('should refuse a plan whose two entities resolve to the same file', async () => {+ await expect(+ applyPullWrites({+ appPath,+ writes: [+ buildWrite('src/pet.object.ts', 'one'),+ buildWrite('src/pet.object.ts', 'two'),+ ],+ deletions: [],+ }),+ ).rejects.toThrow('same file path');+ });++ it('should refuse to delete a folder standing where a planned file goes', async () => {+ await mkdir(join(appPath, 'src/pet.object.ts'), { recursive: true });++ await expect(+ applyPullWrites({+ appPath,+ writes: [buildWrite('src/pet.object.ts', 'written')],+ deletions: [],+ }),+ ).rejects.toThrow('never deletes a folder');+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/build-pull-entities.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/build-pull-entities.spec.tsnew file mode 100644index 00000000000..ea824928a59--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/build-pull-entities.spec.ts@@ -0,0 +1,162 @@+import { buildPullEntities } from '@/cli/utilities/pull/build-pull-entities';+import { type Manifest } from 'twenty-shared/application';+import { STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS } from 'twenty-shared/metadata';+import { describe, expect, it } from 'vitest';++const APP_UID = '11111111-1111-4111-8111-111111111111';+const PET_UID = '22222222-2222-4222-8222-222222222222';+const PET_NAME_FIELD_UID = '33333333-3333-4333-8333-333333333333';+const COMPANY_FIELD_UID = '44444444-4444-4444-8444-444444444444';+const INDEX_UID = '55555555-5555-4555-8555-555555555555';+const JUNCTION_UID = '66666666-6666-4666-8666-666666666666';+const JUNCTION_ID_FIELD_UID = '77777777-7777-4777-8777-777777777777';++const buildManifest = (overrides: Partial<Manifest> = {}): Manifest =>+ ({+ application: {+ universalIdentifier: APP_UID,+ displayName: 'Pets',+ description: 'Pet tracking',+ defaultRoleUniversalIdentifier: 'role-uid',+ packageJsonChecksum: 'package-checksum',+ yarnLockChecksum: 'lock-checksum',+ },+ objects: [+ {+ universalIdentifier: PET_UID,+ nameSingular: 'pet',+ namePlural: 'pets',+ labelSingular: 'Pet',+ labelPlural: 'Pets',+ labelIdentifierFieldMetadataUniversalIdentifier: PET_NAME_FIELD_UID,+ fields: [+ {+ universalIdentifier: PET_NAME_FIELD_UID,+ name: 'name',+ label: 'Name',+ type: 'TEXT',+ writability: 'OPEN',+ },+ ],+ },+ ],+ fields: [+ {+ universalIdentifier: COMPANY_FIELD_UID,+ name: 'caredForPets',+ label: 'Cared for pets',+ type: 'TEXT',+ objectUniversalIdentifier:+ STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS.company,+ },+ ],+ indexes: [],+ ...overrides,+ }) as unknown as Manifest;++describe('buildPullEntities', () => {+ it('should strip the checksums the build recomputes from the application config', () => {+ const { entities } = buildPullEntities(buildManifest());+ const application = entities.find(+ (entity) => entity.kind === 'application',+ );++ expect(application?.config).toEqual({+ universalIdentifier: APP_UID,+ displayName: 'Pets',+ description: 'Pet tracking',+ defaultRoleUniversalIdentifier: 'role-uid',+ });+ expect(+ `${application?.defaultFolder}/${application?.fileBaseName}${application?.fileSuffix}`,+ ).toBe('src/application.config.ts');+ });++ it('should write an object verbatim, keeping its name field and label identifier', () => {+ const manifest = buildManifest();+ const { entities } = buildPullEntities(manifest);+ const object = entities.find((entity) => entity.kind === 'object');++ expect(object?.config).toEqual(manifest.objects[0]);+ expect(+ `${object?.defaultFolder}/${object?.fileBaseName}${object?.fileSuffix}`,+ ).toBe('src/objects/pet.object.ts');+ });++ it('should name a field on a standard object after that object', () => {+ const { entities } = buildPullEntities(buildManifest());+ const field = entities.find((entity) => entity.kind === 'field');++ expect(+ `${field?.defaultFolder}/${field?.fileBaseName}${field?.fileSuffix}`,+ ).toBe('src/fields/company-cared-for-pets.field.ts');+ });++ it('should name an index after its object and fields', () => {+ const { entities } = buildPullEntities(+ buildManifest({+ indexes: [+ {+ universalIdentifier: INDEX_UID,+ objectUniversalIdentifier: PET_UID,+ fields: [{ fieldUniversalIdentifier: PET_NAME_FIELD_UID }],+ },+ ],+ } as unknown as Partial<Manifest>),+ );+ const index = entities.find((entity) => entity.kind === 'index');++ expect(+ `${index?.defaultFolder}/${index?.fileBaseName}${index?.fileSuffix}`,+ ).toBe('src/indexes/pet-name.index.ts');+ });++ it('should write an object whose label identifier names an engine-derived field, keeping the pointer', () => {+ const junctionObject = {+ universalIdentifier: JUNCTION_UID,+ nameSingular: 'petCareAgreement',+ namePlural: 'petCareAgreements',+ labelSingular: 'Pet care agreement',+ labelPlural: 'Pet care agreements',+ labelIdentifierFieldMetadataUniversalIdentifier: JUNCTION_ID_FIELD_UID,+ fields: [+ {+ universalIdentifier: 'relation-field-uid',+ name: 'pet',+ label: 'Pet',+ type: 'RELATION',+ },+ ],+ };++ const { entities, skipped } = buildPullEntities(+ buildManifest({+ objects: [junctionObject],+ } as unknown as Partial<Manifest>),+ );+ const object = entities.find((entity) => entity.kind === 'object');++ expect(skipped).toEqual([]);+ expect(object?.config).toEqual(junctionObject);+ });++ it('should skip an index whose object is not part of the export', () => {+ const { entities, skipped } = buildPullEntities(+ buildManifest({+ objects: [],+ indexes: [+ {+ universalIdentifier: INDEX_UID,+ objectUniversalIdentifier: 'an-object-of-another-application',+ fields: [{ fieldUniversalIdentifier: 'unknown-field' }],+ },+ ],+ } as unknown as Partial<Manifest>),+ );++ expect(entities.some((entity) => entity.kind === 'index')).toBe(false);+ expect(+ skipped.find((entry) => entry.universalIdentifier === INDEX_UID)?.reason,+ ).toBe('its object is not part of the written source');+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/format-pull-report.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/format-pull-report.spec.tsnew file mode 100644index 00000000000..a2b162e9caf--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/format-pull-report.spec.ts@@ -0,0 +1,146 @@+import { type ApplicationExportCoverageEntry } from '@/cli/utilities/pull/application-export-type';+import { formatPullReport } from '@/cli/utilities/pull/format-pull-report';+import { describe, expect, it } from 'vitest';++const buildCoverage = (+ entries: Partial<ApplicationExportCoverageEntry>[],+): ApplicationExportCoverageEntry[] =>+ entries.map((entry, index) => ({+ metadataName: 'view',+ universalIdentifier: `identifier-${index}`,+ status: 'UNSUPPORTED',+ reason: null,+ ...entry,+ }));++const buildReport = (+ overrides: Partial<Parameters<typeof formatPullReport>[0]> = {},+) =>+ formatPullReport({+ writes: [],+ deletions: [],+ unchangedCount: 0,+ skipped: [],+ coverage: [],+ localOnlyRelativePaths: [],+ ...overrides,+ });++describe('formatPullReport', () => {+ it('should group the files it touched by what happened to them', () => {+ const report = buildReport({+ writes: [+ {+ kind: 'object',+ universalIdentifier: 'a',+ relativePath: 'src/objects/pet.object.ts',+ content: '',+ isRegeneration: false,+ },+ {+ kind: 'application',+ universalIdentifier: 'b',+ relativePath: 'src/application.config.ts',+ content: '',+ isRegeneration: true,+ },+ ],+ deletions: [+ {+ universalIdentifier: 'c',+ relativePath: 'src/objects/gone.object.ts',+ },+ ],+ unchangedCount: 3,+ });++ expect(report).toContain('written src/objects/pet.object.ts');+ expect(report).toContain('regenerated src/application.config.ts');+ expect(report).toContain('deleted src/objects/gone.object.ts');+ expect(report).toContain('unchanged 3 file(s)');+ });++ it('should count coverage rows per metadata name under a titled section', () => {+ const report = buildReport({+ coverage: buildCoverage([+ { metadataName: 'view' },+ { metadataName: 'view' },+ { metadataName: 'role' },+ { metadataName: 'roleTarget', status: 'EXCLUDED' },+ { metadataName: 'fieldMetadata', status: 'ENGINE_DERIVED' },+ { metadataName: 'fieldMetadata', status: 'FOREIGN_OWNED' },+ ]),+ });++ expect(report).toContain(+ 'Not written, unsupported by this version (3 row(s)):',+ );+ expect(report).toMatch(/role\s+1/);+ expect(report).toMatch(/view\s+2/);+ expect(report).toContain(+ 'Owned by another application, not written (1 row(s)):',+ );+ expect(report).toContain('Excluded as workspace runtime state (1 row(s)):');+ expect(report).toContain(+ 'Derived by the engine, rebuilt on apply (1 row(s)):',+ );+ });++ it('should omit a section with no rows', () => {+ const report = buildReport({+ coverage: buildCoverage([{ metadataName: 'view' }]),+ });++ expect(report).not.toContain('Excluded as workspace runtime state');+ expect(report).not.toContain('Derived by the engine');+ });++ it('should list identifiers only in verbose mode and cap the list', () => {+ const coverage = buildCoverage(+ Array.from({ length: 25 }, () => ({ metadataName: 'view' })),+ );++ expect(buildReport({ coverage })).not.toContain('identifier-0');++ const verboseReport = buildReport({ coverage, verbose: true });++ expect(verboseReport).toContain('identifier-0');+ expect(verboseReport).toContain('…and 5 more');+ });++ it('should name the entities the writer refused to write', () => {+ const report = buildReport({+ skipped: [+ {+ kind: 'object',+ universalIdentifier: 'junction-uid',+ reason:+ 'petCareAgreement: its label identifier field is engine-derived and not part of the export',+ },+ ],+ });++ expect(report).toContain('Not written by the writer:');+ expect(report).toContain('object petCareAgreement:');+ });++ it('should warn about a define file it could not read', () => {+ const report = buildReport({+ unreadableRelativePaths: ['src/objects/broken.object.ts'],+ });++ expect(report).toContain('Could not be read');+ expect(report).toContain('src/objects/broken.object.ts');+ });++ it('should list local entities the workspace does not have', () => {+ const report = buildReport({+ localOnlyRelativePaths: ['src/objects/unpushed.object.ts'],+ });++ expect(report).toContain(+ 'Local entities the workspace does not have (left untouched):',+ );+ expect(report).toContain('src/objects/unpushed.object.ts');+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/plan-pull-writes.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/plan-pull-writes.spec.tsnew file mode 100644index 00000000000..3aa155e6bd9--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/plan-pull-writes.spec.ts@@ -0,0 +1,351 @@+import { ManifestEntityKey } from '@/cli/utilities/build/manifest/manifest-extract-config';+import { planPullWrites } from '@/cli/utilities/pull/plan-pull-writes';+import { type ScannedDefineFile } from '@/cli/utilities/pull/scan-project-define-files';+import { type Manifest } from 'twenty-shared/application';+import { describe, expect, it } from 'vitest';++const APP_UID = '11111111-1111-4111-8111-111111111111';+const PET_UID = '22222222-2222-4222-8222-222222222222';+const NAME_FIELD_UID = '33333333-3333-4333-8333-333333333333';+const ROCKET_UID = '44444444-4444-4444-8444-444444444444';+const ROCKET_NAME_FIELD_UID = '55555555-5555-4555-8555-555555555555';++const buildObject = ({+ universalIdentifier,+ nameSingular,+ labelIdentifierFieldMetadataUniversalIdentifier,+ label = 'Name',+}: {+ universalIdentifier: string;+ nameSingular: string;+ labelIdentifierFieldMetadataUniversalIdentifier: string;+ label?: string;+}) => ({+ universalIdentifier,+ nameSingular,+ namePlural: `${nameSingular}s`,+ labelSingular: nameSingular,+ labelPlural: `${nameSingular}s`,+ labelIdentifierFieldMetadataUniversalIdentifier,+ fields: [+ {+ universalIdentifier: labelIdentifierFieldMetadataUniversalIdentifier,+ name: 'name',+ label,+ type: 'TEXT',+ },+ ],+});++const buildManifest = (objects: unknown[]): Manifest =>+ ({+ application: {+ universalIdentifier: APP_UID,+ displayName: 'Pets',+ description: '',+ defaultRoleUniversalIdentifier: 'role-uid',+ },+ objects,+ fields: [],+ indexes: [],+ }) as unknown as Manifest;++const MANIFEST = buildManifest([+ buildObject({+ universalIdentifier: PET_UID,+ nameSingular: 'pet',+ labelIdentifierFieldMetadataUniversalIdentifier: NAME_FIELD_UID,+ }),+]);++describe('planPullWrites', () => {+ it('should write every entity when the project has no source and no base', () => {+ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: null,+ scannedFiles: [],+ });++ expect(plan.writes.map((write) => write.relativePath)).toEqual([+ 'src/application.config.ts',+ 'src/objects/pet.object.ts',+ ]);+ expect(plan.writes.every((write) => !write.isRegeneration)).toBe(true);+ expect(plan.deletions).toEqual([]);+ });++ it('should regenerate the application config in the file that already declares one', () => {+ const scannedFiles: ScannedDefineFile[] = [+ {+ relativePath: 'src/application-config.ts',+ entityKey: ManifestEntityKey.Application,+ universalIdentifier: 'a-placeholder-identifier',+ isReadable: true,+ },+ ];++ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: null,+ scannedFiles,+ });+ const applicationWrite = plan.writes.find(+ (write) => write.kind === 'application',+ );++ expect(applicationWrite?.relativePath).toBe('src/application-config.ts');+ expect(applicationWrite?.isRegeneration).toBe(true);+ });++ it('should leave a file untouched when its entity has not changed since the base', () => {+ const scannedFiles: ScannedDefineFile[] = [+ {+ relativePath: 'src/application.config.ts',+ entityKey: ManifestEntityKey.Application,+ universalIdentifier: APP_UID,+ isReadable: true,+ },+ {+ relativePath: 'src/objects/pet.object.ts',+ entityKey: ManifestEntityKey.Objects,+ universalIdentifier: PET_UID,+ isReadable: true,+ },+ ];++ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: MANIFEST,+ scannedFiles,+ });++ expect(plan.writes).toEqual([]);+ expect(plan.unchanged).toHaveLength(2);+ });++ it('should rewrite only the entity that changed on the server', () => {+ const scannedFiles: ScannedDefineFile[] = [+ {+ relativePath: 'src/application.config.ts',+ entityKey: ManifestEntityKey.Application,+ universalIdentifier: APP_UID,+ isReadable: true,+ },+ {+ relativePath: 'src/objects/pet.object.ts',+ entityKey: ManifestEntityKey.Objects,+ universalIdentifier: PET_UID,+ isReadable: true,+ },+ ];++ const plan = planPullWrites({+ manifest: buildManifest([+ buildObject({+ universalIdentifier: PET_UID,+ nameSingular: 'pet',+ labelIdentifierFieldMetadataUniversalIdentifier: NAME_FIELD_UID,+ label: 'Renamed on the server',+ }),+ ]),+ baseManifest: MANIFEST,+ scannedFiles,+ });++ expect(plan.writes.map((write) => write.relativePath)).toEqual([+ 'src/objects/pet.object.ts',+ ]);+ expect(plan.writes[0].isRegeneration).toBe(true);+ expect(plan.writes[0].content).toContain('Renamed on the server');+ });++ it('should delete the file of an entity the base knew and the workspace no longer has', () => {+ const scannedFiles: ScannedDefineFile[] = [+ {+ relativePath: 'src/objects/rocket.object.ts',+ entityKey: ManifestEntityKey.Objects,+ universalIdentifier: ROCKET_UID,+ isReadable: true,+ },+ ];++ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: buildManifest([+ buildObject({+ universalIdentifier: PET_UID,+ nameSingular: 'pet',+ labelIdentifierFieldMetadataUniversalIdentifier: NAME_FIELD_UID,+ }),+ buildObject({+ universalIdentifier: ROCKET_UID,+ nameSingular: 'rocket',+ labelIdentifierFieldMetadataUniversalIdentifier:+ ROCKET_NAME_FIELD_UID,+ }),+ ]),+ scannedFiles,+ });++ expect(plan.deletions).toEqual([+ {+ universalIdentifier: ROCKET_UID,+ relativePath: 'src/objects/rocket.object.ts',+ },+ ]);+ });++ it('should report a local entity that neither the workspace nor the base knows', () => {+ const scannedFiles: ScannedDefineFile[] = [+ {+ relativePath: 'src/objects/unpushed.object.ts',+ entityKey: ManifestEntityKey.Objects,+ universalIdentifier: 'an-unpushed-identifier',+ isReadable: true,+ },+ ];++ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: null,+ scannedFiles,+ });++ expect(plan.localOnlyRelativePaths).toEqual([+ 'src/objects/unpushed.object.ts',+ ]);+ expect(plan.deletions).toEqual([]);+ });++ it('should place a new entity beside existing files of its kind', () => {+ const scannedFiles: ScannedDefineFile[] = [+ {+ relativePath: 'app/data-model/rocket.object.ts',+ entityKey: ManifestEntityKey.Objects,+ universalIdentifier: ROCKET_UID,+ isReadable: true,+ },+ ];++ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: null,+ scannedFiles,+ });++ expect(+ plan.writes.find((write) => write.kind === 'object')?.relativePath,+ ).toBe('app/data-model/pet.object.ts');+ });++ it('should qualify colliding file names with the name of each parent object', () => {+ const plan = planPullWrites({+ manifest: {+ ...MANIFEST,+ objects: [+ buildObject({+ universalIdentifier: PET_UID,+ nameSingular: 'pet',+ labelIdentifierFieldMetadataUniversalIdentifier: NAME_FIELD_UID,+ }),+ buildObject({+ universalIdentifier: ROCKET_UID,+ nameSingular: 'rocket',+ labelIdentifierFieldMetadataUniversalIdentifier:+ ROCKET_NAME_FIELD_UID,+ }),+ ],+ fields: [+ {+ universalIdentifier: 'field-one',+ name: 'notes',+ label: 'Notes',+ type: 'TEXT',+ objectUniversalIdentifier: PET_UID,+ },+ {+ universalIdentifier: 'field-two',+ name: 'notes',+ label: 'Notes',+ type: 'TEXT',+ objectUniversalIdentifier: ROCKET_UID,+ },+ ],+ } as unknown as Manifest,+ baseManifest: null,+ scannedFiles: [],+ });++ expect(+ plan.writes+ .filter((write) => write.kind === 'field')+ .map((write) => write.relativePath)+ .sort(),+ ).toEqual([+ 'src/fields/pet-notes.field.ts',+ 'src/fields/rocket-notes.field.ts',+ ]);+ });++ it('should never write over a file that belongs to another entity', () => {+ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: null,+ scannedFiles: [+ {+ relativePath: 'src/objects/pet.object.ts',+ entityKey: ManifestEntityKey.Objects,+ universalIdentifier: 'a-different-identifier',+ isReadable: true,+ },+ ],+ });++ const objectWrite = plan.writes.find((write) => write.kind === 'object');++ expect(objectWrite?.relativePath).not.toBe('src/objects/pet.object.ts');+ expect(objectWrite?.relativePath).toBe(+ `src/objects/${PET_UID.slice(0, 8)}-pet.object.ts`,+ );+ });++ it('should not claim a path that differs only by case', () => {+ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: null,+ scannedFiles: [+ {+ relativePath: 'src/objects/Pet.object.ts',+ entityKey: ManifestEntityKey.Objects,+ universalIdentifier: 'a-different-identifier',+ isReadable: true,+ },+ ],+ });++ expect(+ plan.writes+ .find((write) => write.kind === 'object')+ ?.relativePath.toLowerCase(),+ ).not.toBe('src/objects/pet.object.ts');+ });++ it('should keep a file whose define file could not be read', () => {+ const plan = planPullWrites({+ manifest: MANIFEST,+ baseManifest: null,+ scannedFiles: [+ {+ relativePath: 'src/objects/pet.object.ts',+ entityKey: null,+ universalIdentifier: null,+ isReadable: false,+ },+ ],+ });++ expect(plan.writes.map((write) => write.relativePath)).not.toContain(+ 'src/objects/pet.object.ts',+ );+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/print-typescript-value.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/print-typescript-value.spec.tsnew file mode 100644index 00000000000..6120842b9bb--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/print-typescript-value.spec.ts@@ -0,0 +1,49 @@+import { printTypescriptValue } from '@/cli/utilities/pull/print-typescript-value';+import { describe, expect, it } from 'vitest';++const evaluatePrinted = (printed: string): unknown =>+ new Function(`return (${printed});`)();++describe('printTypescriptValue', () => {+ it('should escape characters that would otherwise break the generated literal', () => {+ const value = {+ description: 'first\r\nsecond',+ separators: ' ',+ quoted: "Tim's",+ backslash: 'a\\b',+ };++ const printed = printTypescriptValue({ value });++ expect(printed).toContain("description: 'first\\r\\nsecond',");+ expect(printed).toContain("separators: '\\u2028\\u2029',");+ expect(evaluatePrinted(printed)).toEqual(value);+ });++ it('should keep a __proto__ key as an own property instead of setting the prototype', () => {+ const value = JSON.parse('{"__proto__": "not-a-prototype"}');++ const printed = printTypescriptValue({ value });++ expect(printed).toContain("['__proto__']: 'not-a-prototype',");++ const evaluated = evaluatePrinted(printed) as Record<string, unknown>;++ expect(Object.prototype.hasOwnProperty.call(evaluated, '__proto__')).toBe(+ true,+ );+ expect(Object.getPrototypeOf(evaluated)).toBe(Object.prototype);+ });++ it('should quote a key that is not a valid identifier', () => {+ expect(printTypescriptValue({ value: { 'fr-FR': 1 } })).toContain(+ "'fr-FR': 1,",+ );+ });++ it('should drop undefined properties and keep null ones', () => {+ expect(+ printTypescriptValue({ value: { kept: null, dropped: undefined } }),+ ).toBe('{\n kept: null,\n}');+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/pull/__tests__/write-define-file.spec.ts b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/write-define-file.spec.tsnew file mode 100644index 00000000000..aaa4ae15190--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/__tests__/write-define-file.spec.ts@@ -0,0 +1,125 @@+import {+ FIELD_ENUM_BINDINGS,+ INDEX_ENUM_BINDINGS,+ OBJECT_ENUM_BINDINGS,+ writeDefineFile,+} from '@/cli/utilities/pull/write-define-file';+import { describe, expect, it } from 'vitest';++describe('writeDefineFile', () => {+ it('should write an enum-valued property as a symbol and import it', () => {+ const file = writeDefineFile({+ definer: 'defineField',+ config: {+ universalIdentifier: 'field-uid',+ type: 'TEXT',+ name: 'title',+ writability: 'OPEN',+ },+ enumBindings: FIELD_ENUM_BINDINGS,+ });++ expect(file).toContain(+ "import { defineField, FieldType, MetadataWritability } from 'twenty-sdk/define';",+ );+ expect(file).toContain('type: FieldType.TEXT,');+ expect(file).toContain('writability: MetadataWritability.OPEN,');+ });++ it('should look up an enum member whose name differs from its value', () => {+ const file = writeDefineFile({+ definer: 'defineField',+ config: {+ universalIdentifier: 'field-uid',+ type: 'NUMBER',+ universalSettings: { dataType: 'bigint' },+ },+ enumBindings: FIELD_ENUM_BINDINGS,+ });++ expect(file).toContain('dataType: NumberDataType.BIGINT,');+ });++ it('should leave a string that is not a member of the bound enum as a literal', () => {+ const file = writeDefineFile({+ definer: 'defineField',+ config: { universalIdentifier: 'field-uid', type: 'NOT_A_FIELD_TYPE' },+ enumBindings: FIELD_ENUM_BINDINGS,+ });++ expect(file).toContain("type: 'NOT_A_FIELD_TYPE',");+ expect(file).toContain("import { defineField } from 'twenty-sdk/define';");+ });++ it('should only bind enum properties at the declared path', () => {+ const file = writeDefineFile({+ definer: 'defineIndex',+ config: {+ universalIdentifier: 'index-uid',+ indexType: 'BTREE',+ fields: [{ fieldUniversalIdentifier: 'field-uid', indexType: 'BTREE' }],+ },+ enumBindings: INDEX_ENUM_BINDINGS,+ });++ expect(file).toContain('indexType: IndexType.BTREE,');+ expect(file).toContain("indexType: 'BTREE',");+ });++ it('should wrap the import statement when it grows past one line', () => {+ const file = writeDefineFile({+ definer: 'defineObject',+ config: {+ writability: 'OPEN',+ openRecordIn: 'USER_CHOICE',+ fields: [+ {+ type: 'RELATION',+ universalSettings: {+ relationType: 'MANY_TO_ONE',+ onDelete: 'SET_NULL',+ },+ },+ ],+ },+ enumBindings: OBJECT_ENUM_BINDINGS,+ });++ expect(file.startsWith('import {\n defineObject,\n')).toBe(true);+ expect(file).toContain("} from 'twenty-sdk/define';");+ expect(file).toContain('relationType: RelationType.MANY_TO_ONE,');+ expect(file).toContain('onDelete: OnDeleteAction.SET_NULL,');+ });++ it('should quote a key that is not a valid identifier and escape strings', () => {+ const file = writeDefineFile({+ definer: 'defineApplication',+ config: {+ displayName: "Tim's app",+ translations: { 'fr-FR': { greeting: 'Bonjour' } },+ nothing: null,+ empty: [],+ },+ });++ expect(file).toContain("displayName: 'Tim\\'s app',");+ expect(file).toContain("'fr-FR': {");+ expect(file).toContain('nothing: null,');+ expect(file).toContain('empty: [],');+ });++ it('should end with a single default export of the definer call', () => {+ const file = writeDefineFile({+ definer: 'defineObject',+ config: { nameSingular: 'pet' },+ });++ expect(file).toBe(+ "import { defineObject } from 'twenty-sdk/define';\n" ++ '\n' ++ 'export default defineObject({\n' ++ " nameSingular: 'pet',\n" ++ '});\n',+ );+ });+});diff --git a/packages/twenty-sdk/src/cli/utilities/pull/application-export-type.ts b/packages/twenty-sdk/src/cli/utilities/pull/application-export-type.tsnew file mode 100644index 00000000000..cea26752192--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/application-export-type.ts@@ -0,0 +1,36 @@+import { type Manifest } from 'twenty-shared/application';++export const APPLICATION_EXPORT_COVERAGE_STATUSES = [+ 'EXPORTED',+ 'ENGINE_DERIVED',+ 'EXCLUDED',+ 'UNSUPPORTED',+ 'FOREIGN_OWNED',+] as const;++export type ApplicationExportCoverageStatus =+ (typeof APPLICATION_EXPORT_COVERAGE_STATUSES)[number];++export type ApplicationExportCoverageEntry = {+ metadataName: string;+ universalIdentifier: string;+ status: ApplicationExportCoverageStatus;+ reason: string | null;+};++export type ApplicationExportFile = {+ folder: string;+ path: string;+ content: string;+};++export type ApplicationExport = {+ application: {+ universalIdentifier: string;+ displayName: string;+ sourceType: string;+ };+ manifest: Manifest;+ coverage: ApplicationExportCoverageEntry[];+ files: ApplicationExportFile[];+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.ts b/packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.tsnew file mode 100644index 00000000000..fa5e580f3cd--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/apply-pull-writes.ts@@ -0,0 +1,180 @@+import {+ copy,+ ensureDir,+ pathExists,+ remove,+} from '@/cli/utilities/file/fs-utils';+import {+ type PullDeletion,+ type PullWrite,+} from '@/cli/utilities/pull/plan-pull-writes';+import { lstat, mkdtemp, rm, writeFile } from 'node:fs/promises';+import { dirname, join } from 'node:path';++const PULL_WORK_DIRECTORY = '.twenty';++const assertPlanIsApplicable = async ({+ appPath,+ writes,+ deletions,+}: {+ appPath: string;+ writes: PullWrite[];+ deletions: PullDeletion[];+}): Promise<void> => {+ const relativePaths = writes.map((write) => write.relativePath);++ if (new Set(relativePaths).size !== relativePaths.length) {+ throw new Error(+ 'Refusing to write: two entities resolved to the same file path',+ );+ }++ for (const relativePath of [+ ...relativePaths,+ ...deletions.map((deletion) => deletion.relativePath),+ ]) {+ const destinationPath = join(appPath, relativePath);++ if (!(await pathExists(destinationPath))) {+ continue;+ }++ const destinationStats = await lstat(destinationPath);++ if (destinationStats.isDirectory()) {+ throw new Error(+ `Refusing to write: ${relativePath} is a directory, and pull never deletes a folder`,+ );+ }+ }+};++const backUpExistingFile = async ({+ appPath,+ backupDirectory,+ relativePath,+}: {+ appPath: string;+ backupDirectory: string;+ relativePath: string;+}): Promise<boolean> => {+ const destinationPath = join(appPath, relativePath);++ if (!(await pathExists(destinationPath))) {+ return false;+ }++ const backupPath = join(backupDirectory, relativePath);++ await ensureDir(dirname(backupPath));+ await copy(destinationPath, backupPath);++ return true;+};++const restoreBackedUpFiles = async ({+ appPath,+ backupDirectory,+ backedUpRelativePaths,+ writtenRelativePaths,+}: {+ appPath: string;+ backupDirectory: string;+ backedUpRelativePaths: string[];+ writtenRelativePaths: string[];+}): Promise<void> => {+ for (const relativePath of writtenRelativePaths) {+ await rm(join(appPath, relativePath), { force: true });+ }++ for (const relativePath of backedUpRelativePaths) {+ const destinationPath = join(appPath, relativePath);++ await ensureDir(dirname(destinationPath));+ await rm(destinationPath, { force: true });+ await copy(join(backupDirectory, relativePath), destinationPath);+ }+};++export const applyPullWrites = async ({+ appPath,+ writes,+ deletions,+}: {+ appPath: string;+ writes: PullWrite[];+ deletions: PullDeletion[];+}): Promise<void> => {+ await assertPlanIsApplicable({ appPath, writes, deletions });++ const workDirectory = join(appPath, PULL_WORK_DIRECTORY);++ await ensureDir(workDirectory);++ const stagingDirectory = await mkdtemp(join(workDirectory, 'pull-staging-'));+ const backupDirectory = await mkdtemp(join(workDirectory, 'pull-backup-'));+ const backedUpRelativePaths: string[] = [];+ const writtenRelativePaths: string[] = [];++ try {+ for (const write of writes) {+ const stagedPath = join(stagingDirectory, write.relativePath);++ await ensureDir(dirname(stagedPath));+ await writeFile(stagedPath, write.content);+ }++ for (const relativePath of [+ ...writes.map((write) => write.relativePath),+ ...deletions.map((deletion) => deletion.relativePath),+ ]) {+ const wasBackedUp = await backUpExistingFile({+ appPath,+ backupDirectory,+ relativePath,+ });++ if (wasBackedUp) {+ backedUpRelativePaths.push(relativePath);+ }+ }++ for (const write of writes) {+ const destinationPath = join(appPath, write.relativePath);++ await ensureDir(dirname(destinationPath));+ writtenRelativePaths.push(write.relativePath);+ await copy(join(stagingDirectory, write.relativePath), destinationPath);+ }++ for (const deletion of deletions) {+ await rm(join(appPath, deletion.relativePath), { force: true });+ }+ } catch (error) {+ try {+ await restoreBackedUpFiles({+ appPath,+ backupDirectory,+ backedUpRelativePaths,+ writtenRelativePaths: writtenRelativePaths.filter(+ (relativePath) => !backedUpRelativePaths.includes(relativePath),+ ),+ });+ } catch (restoreError) {+ await remove(stagingDirectory);++ throw new Error(+ `Pull failed and could not restore every file it had replaced. The originals are kept in ${backupDirectory}.\nOriginal failure: ${error instanceof Error ? error.message : String(error)}\nRestore failure: ${restoreError instanceof Error ? restoreError.message : String(restoreError)}`,+ );+ }++ await remove(stagingDirectory);+ await remove(backupDirectory);++ throw error;+ }++ await remove(stagingDirectory);+ await remove(backupDirectory);+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/build-pull-entities.ts b/packages/twenty-sdk/src/cli/utilities/pull/build-pull-entities.tsnew file mode 100644index 00000000000..8a6af01f247--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/build-pull-entities.ts@@ -0,0 +1,237 @@+import {+ FIELD_ENUM_BINDINGS,+ INDEX_ENUM_BINDINGS,+ OBJECT_ENUM_BINDINGS,+ type EnumBinding,+} from '@/cli/utilities/pull/write-define-file';+import { kebabCase } from '@/cli/utilities/string/kebab-case';+import {+ type ApplicationManifest,+ type IndexManifest,+ type Manifest,+} from 'twenty-shared/application';+import { STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS } from 'twenty-shared/metadata';+import { isDefined } from 'twenty-shared/utils';++export const PULL_ENTITY_KINDS = [+ 'application',+ 'object',+ 'field',+ 'index',+] as const;++export type PullEntityKind = (typeof PULL_ENTITY_KINDS)[number];++export type PullEntity = {+ kind: PullEntityKind;+ universalIdentifier: string;+ definer: string;+ config: unknown;+ enumBindings: EnumBinding[];+ defaultFolder: string;+ fileSuffix: string;+ fileBaseName: string;+ parentName: string | null;+};++export type SkippedPullEntity = {+ kind: PullEntityKind;+ universalIdentifier: string;+ reason: string;+};++const APPLICATION_PROPERTIES_TO_STRIP = [+ 'packageJsonChecksum',+ 'yarnLockChecksum',+ 'requiredServerVersionRange',+ 'aboutDescription',+ 'postInstallLogicFunction',+ 'preInstallLogicFunction',+ 'uninstallLogicFunction',+ 'settingsFrontComponent',+ 'settingsCustomTabFrontComponentUniversalIdentifier',+ 'frontComponentSharedDependencies',+ 'logoUrl',+ 'screenshots',+] as const;++const APPLICATION_PROPERTY_NAMES_TO_STRIP = new Set<string>(+ APPLICATION_PROPERTIES_TO_STRIP,+);++const GENERATED_COVER_GALLERY_IMAGE = 'public/cover.generated.png';++const STANDARD_OBJECT_NAME_BY_UNIVERSAL_IDENTIFIER = new Map<string, string>(+ Object.entries(STANDARD_OBJECT_UNIVERSAL_IDENTIFIERS).map(+ ([name, universalIdentifier]) => [universalIdentifier, name] as const,+ ),+);++const buildApplicationConfig = (+ manifest: Manifest,+): Partial<ApplicationManifest> => {+ const applicationConfig = Object.fromEntries(+ Object.entries(manifest.application).filter(+ ([property]) => !APPLICATION_PROPERTY_NAMES_TO_STRIP.has(property),+ ),+ ) as Partial<ApplicationManifest>;++ const { galleryImages } = applicationConfig;++ if (+ !Array.isArray(galleryImages) ||+ galleryImages.length === 0 ||+ galleryImages.every((image) => image === GENERATED_COVER_GALLERY_IMAGE)+ ) {+ const { galleryImages: _galleryImages, ...withoutGalleryImages } =+ applicationConfig;++ return withoutGalleryImages;+ }++ return applicationConfig;+};++const getObjectName = ({+ objectUniversalIdentifier,+ manifest,+}: {+ objectUniversalIdentifier: string;+ manifest: Manifest;+}): string | null =>+ manifest.objects.find(+ (objectManifest) =>+ objectManifest.universalIdentifier === objectUniversalIdentifier,+ )?.nameSingular ??+ STANDARD_OBJECT_NAME_BY_UNIVERSAL_IDENTIFIER.get(objectUniversalIdentifier) ??+ null;++const buildIndexFileBaseName = ({+ indexManifest,+ objectName,+ fieldNameByUniversalIdentifier,+}: {+ indexManifest: IndexManifest;+ objectName: string | null;+ fieldNameByUniversalIdentifier: Map<string, string>;+}): string => {+ const fieldNames = indexManifest.fields+ .map(({ fieldUniversalIdentifier }) =>+ fieldNameByUniversalIdentifier.get(fieldUniversalIdentifier),+ )+ .filter(isDefined);++ const segments = [objectName, ...fieldNames].filter(isDefined);++ return segments.length > 0+ ? segments.map(kebabCase).join('-')+ : indexManifest.universalIdentifier.slice(0, 8);+};++export const buildPullEntities = (+ manifest: Manifest,+): { entities: PullEntity[]; skipped: SkippedPullEntity[] } => {+ const applicationUniversalIdentifier =+ manifest.application.universalIdentifier;+ const entities: PullEntity[] = [];+ const skipped: SkippedPullEntity[] = [];++ entities.push({+ kind: 'application',+ universalIdentifier: applicationUniversalIdentifier,+ definer: 'defineApplication',+ config: buildApplicationConfig(manifest),+ enumBindings: [],+ defaultFolder: 'src',+ fileSuffix: '.config.ts',+ fileBaseName: 'application',+ parentName: null,+ });++ const writtenObjectUniversalIdentifiers = new Set<string>();+ const fieldNameByUniversalIdentifier = new Map<string, string>();++ for (const objectManifest of manifest.objects) {+ for (const field of objectManifest.fields) {+ fieldNameByUniversalIdentifier.set(field.universalIdentifier, field.name);+ }++ writtenObjectUniversalIdentifiers.add(objectManifest.universalIdentifier);++ entities.push({+ kind: 'object',+ universalIdentifier: objectManifest.universalIdentifier,+ definer: 'defineObject',+ config: objectManifest,+ enumBindings: OBJECT_ENUM_BINDINGS,+ defaultFolder: 'src/objects',+ fileSuffix: '.object.ts',+ fileBaseName: kebabCase(objectManifest.nameSingular),+ parentName: null,+ });+ }++ for (const fieldManifest of manifest.fields) {+ fieldNameByUniversalIdentifier.set(+ fieldManifest.universalIdentifier,+ fieldManifest.name,+ );++ const objectName = getObjectName({+ objectUniversalIdentifier: fieldManifest.objectUniversalIdentifier,+ manifest,+ });++ entities.push({+ kind: 'field',+ universalIdentifier: fieldManifest.universalIdentifier,+ definer: 'defineField',+ config: fieldManifest,+ enumBindings: FIELD_ENUM_BINDINGS,+ defaultFolder: 'src/fields',+ fileSuffix: '.field.ts',+ fileBaseName: isDefined(objectName)+ ? `${kebabCase(objectName)}-${kebabCase(fieldManifest.name)}`+ : kebabCase(fieldManifest.name),+ parentName: objectName,+ });+ }++ for (const indexManifest of manifest.indexes ?? []) {+ if (+ !writtenObjectUniversalIdentifiers.has(+ indexManifest.objectUniversalIdentifier,+ )+ ) {+ skipped.push({+ kind: 'index',+ universalIdentifier: indexManifest.universalIdentifier,+ reason: 'its object is not part of the written source',+ });+ continue;+ }++ const objectName = getObjectName({+ objectUniversalIdentifier: indexManifest.objectUniversalIdentifier,+ manifest,+ });++ entities.push({+ kind: 'index',+ universalIdentifier: indexManifest.universalIdentifier,+ definer: 'defineIndex',+ config: indexManifest,+ enumBindings: INDEX_ENUM_BINDINGS,+ defaultFolder: 'src/indexes',+ fileSuffix: '.index.ts',+ fileBaseName: buildIndexFileBaseName({+ indexManifest,+ objectName,+ fieldNameByUniversalIdentifier,+ }),+ parentName: objectName,+ });+ }++ return { entities, skipped };+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/format-pull-report.ts b/packages/twenty-sdk/src/cli/utilities/pull/format-pull-report.tsnew file mode 100644index 00000000000..2ab16817fc7--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/format-pull-report.ts@@ -0,0 +1,166 @@+import { type ApplicationExportCoverageEntry } from '@/cli/utilities/pull/application-export-type';+import { type SkippedPullEntity } from '@/cli/utilities/pull/build-pull-entities';+import {+ type PullDeletion,+ type PullWrite,+} from '@/cli/utilities/pull/plan-pull-writes';++const MAX_LISTED_IDENTIFIERS = 20;++const COVERAGE_SECTIONS = [+ {+ status: 'UNSUPPORTED' as const,+ title: 'Not written, unsupported by this version',+ },+ {+ status: 'FOREIGN_OWNED' as const,+ title: 'Owned by another application, not written',+ },+ {+ status: 'EXCLUDED' as const,+ title: 'Excluded as workspace runtime state',+ },+ {+ status: 'ENGINE_DERIVED' as const,+ title: 'Derived by the engine, rebuilt on apply',+ },+] as const;++const formatFileGroup = ({+ title,+ relativePaths,+}: {+ title: string;+ relativePaths: string[];+}): string[] =>+ relativePaths.length === 0+ ? []+ : relativePaths+ .slice()+ .sort()+ .map((relativePath) => ` ${title.padEnd(13)}${relativePath}`);++const groupIdentifiersByMetadataName = (+ coverage: ApplicationExportCoverageEntry[],+): Map<string, string[]> => {+ const identifiersByMetadataName = new Map<string, string[]>();++ for (const entry of coverage) {+ const identifiers = identifiersByMetadataName.get(entry.metadataName) ?? [];++ identifiersByMetadataName.set(entry.metadataName, [+ ...identifiers,+ entry.universalIdentifier,+ ]);+ }++ return identifiersByMetadataName;+};++export const formatPullReport = ({+ writes,+ deletions,+ unchangedCount,+ skipped,+ coverage,+ localOnlyRelativePaths,+ unreadableRelativePaths = [],+ verbose = false,+}: {+ writes: PullWrite[];+ deletions: PullDeletion[];+ unchangedCount: number;+ skipped: SkippedPullEntity[];+ coverage: ApplicationExportCoverageEntry[];+ localOnlyRelativePaths: string[];+ unreadableRelativePaths?: string[];+ verbose?: boolean;+}): string => {+ const lines: string[] = [+ ...formatFileGroup({+ title: 'written',+ relativePaths: writes+ .filter((write) => !write.isRegeneration)+ .map((write) => write.relativePath),+ }),+ ...formatFileGroup({+ title: 'regenerated',+ relativePaths: writes+ .filter((write) => write.isRegeneration)+ .map((write) => write.relativePath),+ }),+ ...formatFileGroup({+ title: 'deleted',+ relativePaths: deletions.map((deletion) => deletion.relativePath),+ }),+ ];++ if (unchangedCount > 0) {+ lines.push(` unchanged ${unchangedCount} file(s)`);+ }++ if (skipped.length > 0) {+ lines.push('', 'Not written by the writer:');++ for (const skippedEntity of skipped) {+ lines.push(` ${skippedEntity.kind} ${skippedEntity.reason}`);+ }+ }++ if (unreadableRelativePaths.length > 0) {+ lines.push(+ '',+ 'Could not be read, so a second file may now define the same entity (run `yarn install` and pull again):',+ ...unreadableRelativePaths+ .slice()+ .sort()+ .map((relativePath) => ` ${relativePath}`),+ );+ }++ if (localOnlyRelativePaths.length > 0) {+ lines.push(+ '',+ 'Local entities the workspace does not have (left untouched):',+ ...localOnlyRelativePaths+ .slice()+ .sort()+ .map((relativePath) => ` ${relativePath}`),+ );+ }++ for (const section of COVERAGE_SECTIONS) {+ const sectionEntries = coverage.filter(+ (entry) => entry.status === section.status,+ );++ if (sectionEntries.length === 0) {+ continue;+ }++ lines.push('', `${section.title} (${sectionEntries.length} row(s)):`);++ const identifiersByMetadataName =+ groupIdentifiersByMetadataName(sectionEntries);++ for (const [metadataName, identifiers] of [+ ...identifiersByMetadataName.entries(),+ ].sort(([left], [right]) => left.localeCompare(right))) {+ lines.push(` ${metadataName.padEnd(34)}${identifiers.length}`);++ if (verbose) {+ for (const identifier of identifiers.slice(0, MAX_LISTED_IDENTIFIERS)) {+ lines.push(` ${identifier}`);+ }++ if (identifiers.length > MAX_LISTED_IDENTIFIERS) {+ lines.push(+ ` …and ${identifiers.length - MAX_LISTED_IDENTIFIERS} more`,+ );+ }+ }+ }+ }++ return lines.join('\n');+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.ts b/packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.tsnew file mode 100644index 00000000000..d9848a320eb--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/plan-pull-writes.ts@@ -0,0 +1,315 @@+import { ManifestEntityKey } from '@/cli/utilities/build/manifest/manifest-extract-config';+import {+ buildPullEntities,+ type PullEntity,+ type PullEntityKind,+} from '@/cli/utilities/pull/build-pull-entities';+import { type ScannedDefineFile } from '@/cli/utilities/pull/scan-project-define-files';+import { writeDefineFile } from '@/cli/utilities/pull/write-define-file';+import { dirname, posix } from 'node:path';+import { type Manifest } from 'twenty-shared/application';+import { isDefined } from 'twenty-shared/utils';++export type PullWrite = {+ kind: PullEntityKind;+ universalIdentifier: string;+ relativePath: string;+ content: string;+ isRegeneration: boolean;+};++export type PullDeletion = {+ universalIdentifier: string;+ relativePath: string;+};++export type PullWritePlan = {+ writes: PullWrite[];+ unchanged: PullEntity[];+ deletions: PullDeletion[];+ localOnlyRelativePaths: string[];+};++const toReservationKey = (relativePath: string): string =>+ relativePath.toLowerCase();++const ENTITY_KEY_BY_KIND: Record<PullEntityKind, ManifestEntityKey> = {+ application: ManifestEntityKey.Application,+ object: ManifestEntityKey.Objects,+ field: ManifestEntityKey.Fields,+ index: ManifestEntityKey.Indexes,+};++const toPosixPath = (value: string): string => value.split('\\').join('/');++const findExistingFolderForKind = ({+ scannedFiles,+ entityKey,+}: {+ scannedFiles: ScannedDefineFile[];+ entityKey: ManifestEntityKey;+}): string | null => {+ const folderCounts = new Map<string, number>();++ for (const scannedFile of scannedFiles) {+ if (scannedFile.entityKey !== entityKey) {+ continue;+ }++ const folder = toPosixPath(dirname(scannedFile.relativePath));++ folderCounts.set(folder, (folderCounts.get(folder) ?? 0) + 1);+ }++ const sortedFolders = [...folderCounts.entries()].sort(+ ([leftFolder, leftCount], [rightFolder, rightCount]) =>+ rightCount - leftCount || leftFolder.localeCompare(rightFolder),+ );++ return sortedFolders[0]?.[0] ?? null;+};++const resolveFileBaseNames = (entities: PullEntity[]): Map<string, string> => {+ const entitiesByCandidate = new Map<string, PullEntity[]>();++ for (const entity of entities) {+ const candidate = `${entity.defaultFolder}/${entity.fileBaseName}${entity.fileSuffix}`;+ const existing = entitiesByCandidate.get(candidate) ?? [];++ entitiesByCandidate.set(candidate, [...existing, entity]);+ }++ const fileBaseNameByUniversalIdentifier = new Map<string, string>();++ for (const collidingEntities of entitiesByCandidate.values()) {+ if (collidingEntities.length === 1) {+ fileBaseNameByUniversalIdentifier.set(+ collidingEntities[0].universalIdentifier,+ collidingEntities[0].fileBaseName,+ );+ continue;+ }++ const qualifiedNames = collidingEntities.map((entity) =>+ isDefined(entity.parentName)+ ? `${entity.parentName}-${entity.fileBaseName}`+ : entity.fileBaseName,+ );+ const hasUniqueQualifiedNames =+ new Set(qualifiedNames).size === qualifiedNames.length;++ collidingEntities.forEach((entity, index) => {+ fileBaseNameByUniversalIdentifier.set(+ entity.universalIdentifier,+ hasUniqueQualifiedNames+ ? qualifiedNames[index]+ : `${entity.universalIdentifier.slice(0, 8)}-${entity.fileBaseName}`,+ );+ });+ }++ return fileBaseNameByUniversalIdentifier;+};++const reserveRelativePath = ({+ folder,+ fileBaseName,+ fileSuffix,+ universalIdentifier,+ takenRelativePaths,+}: {+ folder: string;+ fileBaseName: string;+ fileSuffix: string;+ universalIdentifier: string;+ takenRelativePaths: Set<string>;+}): string => {+ const identifierPrefix = universalIdentifier.slice(0, 8);+ const candidates = [+ fileBaseName,+ `${identifierPrefix}-${fileBaseName}`,+ ...Array.from(+ { length: 100 },+ (_unused, index) => `${identifierPrefix}-${fileBaseName}-${index + 2}`,+ ),+ ];++ for (const candidate of candidates) {+ const candidatePath = posix.join(folder, `${candidate}${fileSuffix}`);++ if (!takenRelativePaths.has(toReservationKey(candidatePath))) {+ return candidatePath;+ }+ }++ throw new Error(+ `Could not find a free file name for ${universalIdentifier} in ${folder}`,+ );+};++const findExistingPath = ({+ entity,+ applicationFile,+ pathByUniversalIdentifier,+}: {+ entity: PullEntity;+ applicationFile: ScannedDefineFile | undefined;+ pathByUniversalIdentifier: Map<string, string>;+}): string | undefined => {+ if (entity.kind !== 'application') {+ return pathByUniversalIdentifier.get(entity.universalIdentifier);+ }++ return isDefined(applicationFile)+ ? toPosixPath(applicationFile.relativePath)+ : undefined;+};++const buildConfigByUniversalIdentifier = (+ manifest: Manifest | null,+): Map<string, string> => {+ if (!isDefined(manifest)) {+ return new Map();+ }++ return new Map(+ buildPullEntities(manifest).entities.map((entity) => [+ entity.universalIdentifier,+ JSON.stringify(entity.config),+ ]),+ );+};++export const planPullWrites = ({+ manifest,+ baseManifest,+ scannedFiles,+}: {+ manifest: Manifest;+ baseManifest: Manifest | null;+ scannedFiles: ScannedDefineFile[];+}): PullWritePlan & {+ skipped: ReturnType<typeof buildPullEntities>['skipped'];+} => {+ const { entities, skipped } = buildPullEntities(manifest);+ const baseConfigByUniversalIdentifier =+ buildConfigByUniversalIdentifier(baseManifest);+ const fileBaseNameByUniversalIdentifier = resolveFileBaseNames(entities);++ const pathByUniversalIdentifier = new Map<string, string>();+ const applicationFile = scannedFiles.find(+ (scannedFile) => scannedFile.entityKey === ManifestEntityKey.Application,+ );++ for (const scannedFile of scannedFiles) {+ if (isDefined(scannedFile.universalIdentifier)) {+ pathByUniversalIdentifier.set(+ scannedFile.universalIdentifier,+ toPosixPath(scannedFile.relativePath),+ );+ }+ }++ const writes: PullWrite[] = [];+ const unchanged: PullEntity[] = [];+ const usedRelativePaths = new Set<string>();+ const takenRelativePaths = new Set(+ scannedFiles.map((scannedFile) =>+ toReservationKey(toPosixPath(scannedFile.relativePath)),+ ),+ );++ for (const entity of entities) {+ const existingPath = findExistingPath({+ entity,+ applicationFile,+ pathByUniversalIdentifier,+ });++ const folder =+ findExistingFolderForKind({+ scannedFiles,+ entityKey: ENTITY_KEY_BY_KIND[entity.kind],+ }) ?? entity.defaultFolder;+ const fileBaseName =+ fileBaseNameByUniversalIdentifier.get(entity.universalIdentifier) ??+ entity.fileBaseName;+ const relativePath =+ existingPath ??+ reserveRelativePath({+ folder,+ fileBaseName,+ fileSuffix: entity.fileSuffix,+ universalIdentifier: entity.universalIdentifier,+ takenRelativePaths,+ });++ usedRelativePaths.add(relativePath);+ takenRelativePaths.add(toReservationKey(relativePath));++ const baseConfig = baseConfigByUniversalIdentifier.get(+ entity.universalIdentifier,+ );++ if (+ isDefined(existingPath) &&+ isDefined(baseConfig) &&+ baseConfig === JSON.stringify(entity.config)+ ) {+ unchanged.push(entity);+ continue;+ }++ writes.push({+ kind: entity.kind,+ universalIdentifier: entity.universalIdentifier,+ relativePath,+ content: writeDefineFile({+ definer: entity.definer,+ config: entity.config,+ enumBindings: entity.enumBindings,+ }),+ isRegeneration: isDefined(existingPath),+ });+ }++ const exportedUniversalIdentifiers = new Set(+ entities.map((entity) => entity.universalIdentifier),+ );+ const deletions: PullDeletion[] = [];++ for (const baseUniversalIdentifier of baseConfigByUniversalIdentifier.keys()) {+ if (exportedUniversalIdentifiers.has(baseUniversalIdentifier)) {+ continue;+ }++ const relativePath = pathByUniversalIdentifier.get(baseUniversalIdentifier);++ if (!isDefined(relativePath) || usedRelativePaths.has(relativePath)) {+ continue;+ }++ deletions.push({+ universalIdentifier: baseUniversalIdentifier,+ relativePath,+ });+ }++ const localOnlyRelativePaths = scannedFiles+ .filter(+ (scannedFile) =>+ isDefined(scannedFile.universalIdentifier) &&+ !exportedUniversalIdentifiers.has(scannedFile.universalIdentifier) &&+ !baseConfigByUniversalIdentifier.has(scannedFile.universalIdentifier) &&+ !usedRelativePaths.has(toPosixPath(scannedFile.relativePath)),+ )+ .map((scannedFile) => toPosixPath(scannedFile.relativePath));++ return {+ writes,+ unchanged,+ deletions,+ localOnlyRelativePaths,+ skipped,+ };+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/print-typescript-value.ts b/packages/twenty-sdk/src/cli/utilities/pull/print-typescript-value.tsnew file mode 100644index 00000000000..12aa138acbd--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/print-typescript-value.ts@@ -0,0 +1,97 @@+import { isDefined } from 'twenty-shared/utils';++const IDENTIFIER_PATTERN = /^[A-Za-z_$][A-Za-z0-9_$]*$/;++const PROTOTYPE_KEY = '__proto__';++export type EnumSymbolResolver = (params: {+ path: string[];+ value: string;+}) => string | undefined;++const printString = (value: string): string =>+ `'${value+ .replace(/\\/g, '\\\\')+ .replace(/'/g, "\\'")+ .replace(/\n/g, '\\n')+ .replace(/\r/g, '\\r')+ .replace(/\u2028/g, '\\u2028')+ .replace(/\u2029/g, '\\u2029')}'`;++const printKey = (key: string): string => {+ if (key === PROTOTYPE_KEY) {+ return `[${printString(key)}]`;+ }++ return IDENTIFIER_PATTERN.test(key) ? key : printString(key);+};++const indentOf = (depth: number): string => ' '.repeat(depth);++export const printTypescriptValue = ({+ value,+ depth = 0,+ path = [],+ resolveEnumSymbol,+}: {+ value: unknown;+ depth?: number;+ path?: string[];+ resolveEnumSymbol?: EnumSymbolResolver;+}): string => {+ if (value === null) {+ return 'null';+ }++ if (typeof value === 'string') {+ const enumSymbol = resolveEnumSymbol?.({ path, value });++ return isDefined(enumSymbol) ? enumSymbol : printString(value);+ }++ if (typeof value === 'number' || typeof value === 'boolean') {+ return String(value);+ }++ if (Array.isArray(value)) {+ if (value.length === 0) {+ return '[]';+ }++ const items = value.map(+ (item) =>+ `${indentOf(depth + 1)}${printTypescriptValue({+ value: item,+ depth: depth + 1,+ path: [...path, '[]'],+ resolveEnumSymbol,+ })},`,+ );++ return `[\n${items.join('\n')}\n${indentOf(depth)}]`;+ }++ if (typeof value === 'object') {+ const entries = Object.entries(value as Record<string, unknown>).filter(+ ([, entryValue]) => entryValue !== undefined,+ );++ if (entries.length === 0) {+ return '{}';+ }++ const properties = entries.map(+ ([key, entryValue]) =>+ `${indentOf(depth + 1)}${printKey(key)}: ${printTypescriptValue({+ value: entryValue,+ depth: depth + 1,+ path: [...path, key],+ resolveEnumSymbol,+ })},`,+ );++ return `{\n${properties.join('\n')}\n${indentOf(depth)}}`;+ }++ return 'null';+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/pull-base-file.ts b/packages/twenty-sdk/src/cli/utilities/pull/pull-base-file.tsnew file mode 100644index 00000000000..69f22132e78--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/pull-base-file.ts@@ -0,0 +1,98 @@+import {+ ensureDir,+ pathExists,+ readJson,+ writeJson,+} from '@/cli/utilities/file/fs-utils';+import { dirname, join } from 'node:path';+import { type Manifest } from 'twenty-shared/application';+import { isDefined } from 'twenty-shared/utils';++export const PULL_BASE_FILE_PATH = '.twenty/pull-base.json';++const PULL_BASE_FILE_VERSION = 1;++type PullBaseFile = {+ version: number;+ applicationUniversalIdentifier: string;+ manifest: Manifest;+};++const hasUniversalIdentifier = (value: unknown): boolean =>+ isDefined(value) &&+ typeof value === 'object' &&+ typeof (value as { universalIdentifier?: unknown }).universalIdentifier ===+ 'string';++const isEntityListWithFields = (value: unknown): boolean =>+ Array.isArray(value) &&+ value.every(+ (entry) =>+ hasUniversalIdentifier(entry) &&+ Array.isArray((entry as { fields?: unknown }).fields),+ );++const isUsableBaseManifest = (manifest: unknown): manifest is Manifest => {+ if (!isDefined(manifest) || typeof manifest !== 'object') {+ return false;+ }++ const { application, objects, fields, indexes } =+ manifest as Partial<Manifest>;++ return (+ hasUniversalIdentifier(application) &&+ isEntityListWithFields(objects) &&+ Array.isArray(fields) &&+ fields.every(hasUniversalIdentifier) &&+ (!isDefined(indexes) || isEntityListWithFields(indexes))+ );+};++export const readPullBaseManifest = async ({+ appPath,+ applicationUniversalIdentifier,+}: {+ appPath: string;+ applicationUniversalIdentifier: string;+}): Promise<Manifest | null> => {+ const baseFilePath = join(appPath, PULL_BASE_FILE_PATH);++ if (!(await pathExists(baseFilePath))) {+ return null;+ }++ try {+ const baseFile = await readJson<PullBaseFile>(baseFilePath);++ if (+ baseFile.version !== PULL_BASE_FILE_VERSION ||+ baseFile.applicationUniversalIdentifier !==+ applicationUniversalIdentifier ||+ !isUsableBaseManifest(baseFile.manifest)+ ) {+ return null;+ }++ return baseFile.manifest;+ } catch {+ return null;+ }+};++export const writePullBaseManifest = async ({+ appPath,+ manifest,+}: {+ appPath: string;+ manifest: Manifest;+}): Promise<void> => {+ const baseFilePath = join(appPath, PULL_BASE_FILE_PATH);++ await ensureDir(dirname(baseFilePath));+ await writeJson(baseFilePath, {+ version: PULL_BASE_FILE_VERSION,+ applicationUniversalIdentifier: manifest.application.universalIdentifier,+ manifest,+ } satisfies PullBaseFile);+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/scan-project-define-files.ts b/packages/twenty-sdk/src/cli/utilities/pull/scan-project-define-files.tsnew file mode 100644index 00000000000..de681947922--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/scan-project-define-files.ts@@ -0,0 +1,87 @@+import {+ TARGET_FUNCTION_TO_ENTITY_KEY_MAPPING,+ extractDefineEntity,+ type ManifestEntityKey,+} from '@/cli/utilities/build/manifest/manifest-extract-config';+import { extractManifestFromFile } from '@/cli/utilities/build/manifest/manifest-extract-config-from-file';+import { glob } from 'tinyglobby';+import { readFile } from 'node:fs/promises';+import { relative } from 'node:path';+import { isDefined } from 'twenty-shared/utils';++export type ScannedDefineFile = {+ relativePath: string;+ entityKey: ManifestEntityKey | null;+ universalIdentifier: string | null;+ isReadable: boolean;+};++type ExtractedConfig = {+ universalIdentifier?: unknown;+};++export const scanProjectDefineFiles = async (+ appPath: string,+): Promise<ScannedDefineFile[]> => {+ const filePaths = await glob(['**/*.ts', '**/*.tsx'], {+ cwd: appPath,+ absolute: true,+ ignore: ['**/node_modules/**', '**/*.d.ts', '**/dist/**', '**/.twenty/**'],+ onlyFiles: true,+ });++ const scannedFiles: ScannedDefineFile[] = [];++ for (const filePath of filePaths) {+ const relativePath = relative(appPath, filePath);++ let fileContent: string;++ try {+ fileContent = await readFile(filePath, 'utf-8');+ } catch {+ scannedFiles.push({+ relativePath,+ entityKey: null,+ universalIdentifier: null,+ isReadable: false,+ });+ continue;+ }++ const targetFunctionName = extractDefineEntity(fileContent);++ if (!isDefined(targetFunctionName)) {+ continue;+ }++ const entityKey = TARGET_FUNCTION_TO_ENTITY_KEY_MAPPING[targetFunctionName];++ let config: ExtractedConfig = {};+ let isReadable = true;++ try {+ const extract = await extractManifestFromFile<ExtractedConfig>({+ appPath,+ filePath,+ });++ config = extract.config ?? {};+ } catch {+ config = {};+ isReadable = false;+ }++ scannedFiles.push({+ relativePath,+ entityKey,+ universalIdentifier:+ typeof config.universalIdentifier === 'string'+ ? config.universalIdentifier+ : null,+ isReadable,+ });+ }++ return scannedFiles;+};diff --git a/packages/twenty-sdk/src/cli/utilities/pull/write-define-file.ts b/packages/twenty-sdk/src/cli/utilities/pull/write-define-file.tsnew file mode 100644index 00000000000..5f94c9e3c3e--- /dev/null+++ b/packages/twenty-sdk/src/cli/utilities/pull/write-define-file.ts@@ -0,0 +1,127 @@+import {+ type EnumSymbolResolver,+ printTypescriptValue,+} from '@/cli/utilities/pull/print-typescript-value';+import {+ DateDisplayFormat,+ FieldMetadataType,+ IndexType,+ MetadataWritability,+ NumberDataType,+ ObjectOpenRecordIn,+ RelationOnDeleteAction,+ RelationType,+} from 'twenty-shared/types';+import { isDefined } from 'twenty-shared/utils';++const MAX_IMPORT_LINE_LENGTH = 80;++export type EnumBinding = {+ path: string[];+ symbol: string;+ members: Record<string, string>;+};++const buildFieldEnumBindings = (prefix: string[]): EnumBinding[] => [+ {+ path: [...prefix, 'type'],+ symbol: 'FieldType',+ members: FieldMetadataType,+ },+ {+ path: [...prefix, 'writability'],+ symbol: 'MetadataWritability',+ members: MetadataWritability,+ },+ {+ path: [...prefix, 'universalSettings', 'relationType'],+ symbol: 'RelationType',+ members: RelationType,+ },+ {+ path: [...prefix, 'universalSettings', 'onDelete'],+ symbol: 'OnDeleteAction',+ members: RelationOnDeleteAction,+ },+ {+ path: [...prefix, 'universalSettings', 'dataType'],+ symbol: 'NumberDataType',+ members: NumberDataType,+ },+ {+ path: [...prefix, 'universalSettings', 'displayFormat'],+ symbol: 'DateDisplayFormat',+ members: DateDisplayFormat,+ },+];++export const FIELD_ENUM_BINDINGS: EnumBinding[] = buildFieldEnumBindings([]);++export const OBJECT_ENUM_BINDINGS: EnumBinding[] = [+ {+ path: ['writability'],+ symbol: 'MetadataWritability',+ members: MetadataWritability,+ },+ {+ path: ['openRecordIn'],+ symbol: 'ObjectOpenRecordIn',+ members: ObjectOpenRecordIn,+ },+ ...buildFieldEnumBindings(['fields', '[]']),+];++export const INDEX_ENUM_BINDINGS: EnumBinding[] = [+ { path: ['indexType'], symbol: 'IndexType', members: IndexType },+];++const isSamePath = (left: string[], right: string[]): boolean =>+ left.length === right.length &&+ left.every((segment, index) => segment === right[index]);++export const writeDefineFile = ({+ definer,+ config,+ enumBindings = [],+}: {+ definer: string;+ config: unknown;+ enumBindings?: EnumBinding[];+}): string => {+ const usedSymbols = new Set<string>();++ const resolveEnumSymbol: EnumSymbolResolver = ({ path, value }) => {+ const binding = enumBindings.find((candidate) =>+ isSamePath(candidate.path, path),+ );++ if (!isDefined(binding)) {+ return undefined;+ }++ const memberName = Object.keys(binding.members).find(+ (key) => binding.members[key] === value,+ );++ if (!isDefined(memberName)) {+ return undefined;+ }++ usedSymbols.add(binding.symbol);++ return `${binding.symbol}.${memberName}`;+ };++ const body = printTypescriptValue({ value: config, resolveEnumSymbol });+ const imported = [definer, ...[...usedSymbols].sort()];+ const singleLineImport = `import { ${imported.join(', ')} } from 'twenty-sdk/define';`;+ const importStatement =+ singleLineImport.length <= MAX_IMPORT_LINE_LENGTH+ ? singleLineImport+ : `import {\n${imported.map((name) => ` ${name},`).join('\n')}\n} from 'twenty-sdk/define';`;++ return `${importStatement}++export default ${definer}(${body});+`;+};diff --git a/packages/twenty-sdk/src/sdk/define/objects/__tests__/define-object.spec.ts b/packages/twenty-sdk/src/sdk/define/objects/__tests__/define-object.spec.tsindex 771a997ed08..89d04c80f34 100644--- a/packages/twenty-sdk/src/sdk/define/objects/__tests__/define-object.spec.ts+++ b/packages/twenty-sdk/src/sdk/define/objects/__tests__/define-object.spec.ts@@ -231,7 +231,7 @@ describe('defineObject', () => {expect(result.config.fields[0].options).toHaveLength(2);});- it('should return error when labelIdentifierFieldMetadataUniversalIdentifier references non-existent field', () => {+ it('should warn but accept a labelIdentifierFieldMetadataUniversalIdentifier that names an engine-derived field', () => {const config: ObjectManifest = {...validConfig,labelIdentifierFieldMetadataUniversalIdentifier:@@ -240,9 +240,10 @@ describe('defineObject', () => {const result = defineObject(config);- expect(result.success).toBe(false);- expect(result.errors).toContain(- 'labelIdentifierFieldMetadataUniversalIdentifier must reference a field defined in the fields array',+ expect(result.success).toBe(true);+ expect(result.errors).toEqual([]);+ expect(result.warnings).toContain(+ `labelIdentifierFieldMetadataUniversalIdentifier of "${validConfig.nameSingular}" names no field in its fields array; it must name a field the engine derives for this object, or the sync will fail to resolve it`,);});});diff --git a/packages/twenty-sdk/src/sdk/define/objects/define-object.ts b/packages/twenty-sdk/src/sdk/define/objects/define-object.tsindex c5f6d765c36..9ad34bad27a 100644--- a/packages/twenty-sdk/src/sdk/define/objects/define-object.ts+++ b/packages/twenty-sdk/src/sdk/define/objects/define-object.ts@@ -2,6 +2,7 @@ import { type DefineEntity } from '@/sdk/define/common/types/define-entity.type'import { createValidationResult } from '@/sdk/define/common/utils/create-validation-result';import { getFieldDefaultValueWarnings } from '@/sdk/define/fields/get-field-default-value-warnings';import { validateFields } from '@/sdk/define/fields/validate-fields';+import { isEngineDerivedLabelIdentifier } from '@/sdk/define/objects/is-engine-derived-label-identifier';import { type ObjectConfig } from '@/sdk/define/objects/object-config';import { isDefined } from 'twenty-shared/utils';@@ -32,20 +33,20 @@ export const defineObject: DefineEntity<ObjectConfig> = (config) => {errors.push(...fieldErrors);- if (- isDefined(config.labelIdentifierFieldMetadataUniversalIdentifier) &&- !config.fields.some(- (field) =>- field.universalIdentifier ===- config.labelIdentifierFieldMetadataUniversalIdentifier,- )- ) {- errors.push(- 'labelIdentifierFieldMetadataUniversalIdentifier must reference a field defined in the fields array',- );- }+ const labelIdentifiesAnEngineDerivedField = isEngineDerivedLabelIdentifier({+ fields: config.fields,+ labelIdentifierFieldMetadataUniversalIdentifier:+ config.labelIdentifierFieldMetadataUniversalIdentifier,+ });- const warnings = getFieldDefaultValueWarnings(config.fields);+ const warnings = [+ ...getFieldDefaultValueWarnings(config.fields),+ ...(labelIdentifiesAnEngineDerivedField+ ? [+ `labelIdentifierFieldMetadataUniversalIdentifier of "${config.nameSingular}" names no field in its fields array; it must name a field the engine derives for this object, or the sync will fail to resolve it`,+ ]+ : []),+ ];return createValidationResult({config,diff --git a/packages/twenty-sdk/src/sdk/define/objects/is-engine-derived-label-identifier.ts b/packages/twenty-sdk/src/sdk/define/objects/is-engine-derived-label-identifier.tsnew file mode 100644index 00000000000..85546001997--- /dev/null+++ b/packages/twenty-sdk/src/sdk/define/objects/is-engine-derived-label-identifier.ts@@ -0,0 +1,15 @@+import { isDefined } from 'twenty-shared/utils';++export const isEngineDerivedLabelIdentifier = ({+ fields,+ labelIdentifierFieldMetadataUniversalIdentifier,+}: {+ fields: { universalIdentifier?: string }[];+ labelIdentifierFieldMetadataUniversalIdentifier?: string;+}): boolean =>+ isDefined(labelIdentifierFieldMetadataUniversalIdentifier) &&+ !fields.some(+ (field) =>+ field.universalIdentifier ===+ labelIdentifierFieldMetadataUniversalIdentifier,+ );