Skip to content

Open-source manufacturing ERP, MES and QMS. Quoting, MRP, inventory, shop floor, quality and lot/serial traceability on one Postgres schema, with a REST API and MCP server. Self-host or use Carbon Cloud.

Ramp: card-liability-only requirement, health signal, and Sync Activity (#1707)

Brad Barbina82ca727f225
diff --git a/.claude/rules/accounting-sync-handlers.md b/.claude/rules/accounting-sync-handlers.md
index 1387bf2564..f1305b5e2f 100644
--- a/.claude/rules/accounting-sync-handlers.md
+++ b/.claude/rules/accounting-sync-handlers.md
@@ -51,7 +51,7 @@ Why tokens die: Xero and QBO rotate refresh tokens, and Carbon refreshes reactiv
### The operation ledger — enqueue, cooldown, truthful close
-Every entry point routes through the durable **`accountingSyncOperation`** ledger (`accounting-sync-operations.ts` in jobs + `core/operations.ts` in ee) between enqueue and `SyncFactory.getSyncer(...).pushBatch/pullBatch`:
+Every entry point routes through the durable **`accountingSyncOperation`** ledger (`accounting-sync-operations.ts` in jobs + `core/operations.ts` in ee) between enqueue and `SyncFactory.getSyncer(...).pushBatch/pullBatch`. The table is **no longer accounting-exclusive**: the Ramp (spend-management) sync writes terminal `Warning` rows here for its own failed/skipped items and deletes them on the next successful sync via `clearResolvedSyncOperations` (ee `core/operations.ts`) — see `ramp-integration.md` → "Sync Activity". Ramp does NOT use the enqueue/drain machinery below; it records dispositions directly.
- **Enqueue.** `enqueueSyncOperations` absorbs re-triggers into the live (Pending/In Flight) row. The v5 reconcile path enqueues with trigger `"reconcile"`, which is **never cooldown-gated** — a state-derived decision is idempotent, so an unchanged entity decides `nothing` instead of needing a window, and a changed one enqueues immediately. The 60s completed-row cooldown (`SYNC_OPERATION_COOLDOWN_MS`, `isCooldownTrigger`: `event`/`webhook`) survives ONLY on the inbound/webhook entry points (`sync-external-accounting`, the pull sweep), which don't route through the reconciler. `isStatusTransitionEvent` and `getPaymentPushDecision` were deleted with v5 (nothing to bypass; payment state lives in the decision core).
- **Drain.** `drainSyncOperations` claims Pending (+ stale In Flight) rows in groups of (entityType, direction); an op claimed more than `MAX_SYNC_OPERATION_ATTEMPTS` (10) times parks as `Failed` `ATTEMPTS_EXHAUSTED` instead of running again (a human Retry resets the loop deliberately).
diff --git a/.claude/rules/ramp-integration.md b/.claude/rules/ramp-integration.md
index a43f084f1f..b2764862a1 100644
--- a/.claude/rules/ramp-integration.md
+++ b/.claude/rules/ramp-integration.md
@@ -79,8 +79,11 @@ providers, which own the data and mirror it out.
category "Spend Management", active only when public `RAMP_CLIENT_ID` is configured).
The UI connection is the production OAuth `oauth` block; the form carries no customer
client credentials. `RampSettingsSchema` is flat: optional `entityId`, account mapping
- (`cardLiabilityAccountId` + `statementBankAccountId` **required**;
- `cashbackIncomeAccountId`, `reimbursementBankAccountId` optional), and five sync toggles
+ (`cardLiabilityAccountId` is the ONLY **required** account — every card journal credits
+ it, and it is the one side of the double entry Carbon cannot invent; `statementBankAccountId`,
+ `cashbackIncomeAccountId`, `reimbursementBankAccountId` optional — `statementBankAccountId`
+ is only the offset for statement payments/transfers, and those families self-gate on it),
+ and five sync toggles
(`pullTransactions`,
`pullBills`, `pullReimbursements`, `pushPurchaseOrders`, `pushInvoices`, all default
`"true"`). Renders `SetupInstructions` with the webhook URL
@@ -156,9 +159,11 @@ for stored-data compatibility, but the settings UI exposes only OAuth Connect.
`convergeRamp` runs on install and every settings save. It validates credentials with
`client.getBusiness()`, ensures the accounting connection, and best-effort registers the
-webhook first. A fresh OAuth callback has no required accounts, so it returns here: it does
-**not** push master data or start financial sync before both `cardLiabilityAccountId` and
-`statementBankAccountId` exist. Once configured, it validates an optional `entityId`, pushes
+webhook first. A fresh OAuth callback has no required account, so it returns here: it does
+**not** push master data or start financial sync before `cardLiabilityAccountId` exists
+(`statementBankAccountId` is NOT required here — it only offsets statement payments/transfers,
+and each of those families self-gates on it; coupling it here used to block card-charge sync
+on an account card charges never touch). Once configured, it validates an optional `entityId`, pushes
CoA/cost centers, and fires `trigger("ramp-sync", {companyId, reason})`. A reconnect whose
atomic OAuth patch preserved valid mappings may therefore converge immediately. The trigger
uses a lazy runtime `import("@carbon/jobs")` because `jobs → ee` is the dependency direction.
@@ -228,8 +233,11 @@ uses a lazy runtime `import("@carbon/jobs")` because `jobs → ee` is the depend
- `rampOnUninstall` best-effort deletes the webhook and the accounting connection
(tolerating already-gone), then `clearRampConnectionState` removes connection/webhook
ids and the webhook secret without disturbing OAuth credentials/settings/cursors.
- `rampHealthcheck` = `getBusiness()` succeeds AND at least one
- accounting connection is `linked`/`active`/`connected`.
+ `rampHealthcheck` = `cardLiabilityAccountId` is set AND `getBusiness()` succeeds AND at
+ least one accounting connection is `linked`/`active`/`connected`. A connected-but-unmapped
+ Ramp reads **unhealthy** rather than a green badge over a sync that silently does nothing;
+ `statementBankAccountId` is deliberately NOT checked (its absence is a healthy "that family
+ is off", not a broken connection).
## Metadata (`companyIntegration.metadata`, id `ramp`)
@@ -464,6 +472,38 @@ company with an ACTIVE `ramp` integration and fires one `carbon/ramp-sync`
disabled webhook delivery becomes ≤1h of staleness, never permanent loss. `ramp-sync` is
idempotent, so re-firing is safe.
+## Sync Activity (failure observability)
+
+Every family records a **`Warning`** row on the shared `accountingSyncOperation` ledger
+for each failed/skipped item, and clears it when that item later syncs — so a coded-but-
+unrecognized charge shows up in the integration's **Sync Activity** tab with its reason
+instead of vanishing into the Inngest logs. A successfully-synced item is NOT recorded (it
+already appears as a `cardTransaction`/`purchaseInvoice`/`payment` row); the tab is a
+"what didn't come through, and why" inbox, not a full audit log.
+
+- `recordRampSyncFailures(ctx, {entityType, direction, failures})` and
+ `resolveRampSyncOperations(ctx, {entityType, direction, entityIds})`
+ (`ramp-sync-shared.ts`) are the two write points, called once per family after its
+ drain. Both are **strictly best-effort** — every error (thrown OR returned) is swallowed
+ and logged; observability must never fail or pollute a family's sync result. `ctx` gained
+ `createdBy` (`integration.updatedBy ?? "system"`) and `trigger` (`webhook` vs `event`).
+- entityType/direction per family: card charges `cardTransaction`, transfers `transfer`,
+ cashbacks `cashback`, bills `bill`, bill payments `billPayment`, reimbursements
+ `reimbursement`, repayments `repayment` — all `pull-from-accounting`; PO push
+ `purchaseOrder` and draft-bill push `purchaseInvoice` — `push-to-accounting`. entityId is
+ the Ramp id inbound, the Carbon record id outbound (direction disambiguates the tuple).
+- Records are `Warning` because a Ramp failure is a config hole (`Warning` is what the tab's
+ `failingCount` badge counts) via the ee `insertTerminalSyncOperation`. Unlike accounting
+ journals — whose disposition is permanent — a Ramp inbound family RE-EVALUATES every run, so
+ the resolve-on-success delete (`clearResolvedSyncOperations`, ee `operations.ts`) is what
+ drops a recoded-and-posted charge out of the inbox. `mapped`/already-synced items also
+ resolve, so a fixed item that is now skipped-as-mapped still clears its old Warning.
+- The **Sync Activity tab renders for Ramp** (`producesSyncOperations` in the ERP
+ `integrations.$id.tsx` loader — accounting category OR Ramp), with the accounting-only
+ tie-out/reconciliation surfaces kept gated on `isAccountingInstalled`. Retention/compaction
+ needs nothing new — Ramp rows live in the same `accountingSyncOperation` table the nightly
+ passes already sweep.
+
## cardTransaction schema (migration `20260919152233_ramp-integration.sql`)
The forward, retry-safe reconciliation migration supersedes the three branch-only Ramp
diff --git a/apps/erp/app/routes/x+/settings+/integrations.$id.tsx b/apps/erp/app/routes/x+/settings+/integrations.$id.tsx
index 983796d448..f28399f14d 100644
--- a/apps/erp/app/routes/x+/settings+/integrations.$id.tsx
+++ b/apps/erp/app/routes/x+/settings+/integrations.$id.tsx
@@ -647,9 +647,17 @@ export async function loader({ request, params }: LoaderFunctionArgs) {
const isAccountingInstalled =
integration.category === "Accounting" && integrationData.data.active;
- // Sync-operation inbox for accounting integrations (RLS SELECT covers
- // employees, so the user-scoped client is enough). Params are prefixed
- // (syncStatus/syncPage) to avoid clashing with other search params.
+ // Ramp (Spend Management) also writes accountingSyncOperation rows for its
+ // inbound/outbound families, so it gets the same Sync Activity inbox — minus
+ // the accounting-only tie-out/reconciliation surfaces, which stay gated on
+ // isAccountingInstalled below.
+ const producesSyncOperations =
+ isAccountingInstalled ||
+ (integration.id === "ramp" && integrationData.data.active);
+
+ // Sync-operation inbox (RLS SELECT covers employees, so the user-scoped
+ // client is enough). Params are prefixed (syncStatus/syncPage) to avoid
+ // clashing with other search params.
let syncActivity: {
operations: SyncOperation[];
count: number;
@@ -672,7 +680,7 @@ export async function loader({ request, params }: LoaderFunctionArgs) {
} | null;
} | null = null;
- if (isAccountingInstalled) {
+ if (producesSyncOperations) {
const url = new URL(request.url);
const statusFilter = SyncOperationStatusSchema.safeParse(
url.searchParams.get("syncStatus")
@@ -699,13 +707,15 @@ export async function loader({ request, params }: LoaderFunctionArgs) {
status: ["Failed", "Warning"],
limit: 1
}),
- // Tie-out cells for this integration. The table is not in the
- // generated DB types yet — cast, same pattern as
- // @carbon/ee/accounting core/operations.ts.
- (client.from("accountingSyncTieOut" as any) as any)
- .select("internalDelta, externalDelta, computedAt")
- .eq("companyId", companyId)
- .eq("integration", integrationId)
+ // Tie-out cells for this integration — accounting-only (Ramp has no
+ // reconciliation tie-out). The table is not in the generated DB types
+ // yet — cast, same pattern as @carbon/ee/accounting core/operations.ts.
+ isAccountingInstalled
+ ? (client.from("accountingSyncTieOut" as any) as any)
+ .select("internalDelta, externalDelta, computedAt")
+ .eq("companyId", companyId)
+ .eq("integration", integrationId)
+ : Promise.resolve({ data: [], error: null })
]);
if (operations.error) {
diff --git a/packages/ee/src/accounting/core/operations.ts b/packages/ee/src/accounting/core/operations.ts
index 733a56e7b3..44a2c47903 100644
--- a/packages/ee/src/accounting/core/operations.ts
+++ b/packages/ee/src/accounting/core/operations.ts
@@ -348,6 +348,42 @@ export async function insertTerminalSyncOperation(
return { data: inserted.data as SyncOperation, error: null };
}
+/**
+ * Delete terminal disposition rows for entities that have since synced
+ * successfully. Accounting journals get a permanent disposition, but a
+ * re-evaluating INBOUND family (Ramp) can fail an entity one run — a charge
+ * coded to an unrecognized account records a Warning — and succeed the next,
+ * once it is recoded. Without this the resolved failure lingers in the Sync
+ * Activity inbox and keeps the tab badge lit, so on every successful sync the
+ * caller clears any prior disposition for those (entityType, entityId,
+ * direction) tuples. Defaults to `Warning` (the only status the Ramp recorder
+ * writes); a no-op when `entityIds` is empty. Service-role only (the table has
+ * no DELETE policy — jobs delete via service role).
+ */
+export async function clearResolvedSyncOperations(
+ client: SupabaseClient<Database>,
+ args: {
+ companyId: string;
+ integration: string;
+ entityType: string;
+ direction: SyncOperationDirection;
+ entityIds: string[];
+ statuses?: SyncOperationStatus[];
+ }
+): Promise<{ error: string | null }> {
+ if (args.entityIds.length === 0) return { error: null };
+ const statuses = args.statuses ?? ["Warning"];
+ const { error } = await syncOperationTable(client)
+ .delete()
+ .eq("companyId", args.companyId)
+ .eq("integration", args.integration)
+ .eq("entityType", args.entityType)
+ .eq("direction", args.direction)
+ .in("entityId", args.entityIds)
+ .in("status", statuses);
+ return { error: error ? (error as { message: string }).message : null };
+}
+
/**
* Claim up to `limit` operations for a drain: Pending rows plus "In Flight"
* rows whose lastAttemptAt is older than 10 minutes (abandoned by a crashed
diff --git a/packages/ee/src/ramp/config.tsx b/packages/ee/src/ramp/config.tsx
index f250547d23..9db3525eb8 100644
--- a/packages/ee/src/ramp/config.tsx
+++ b/packages/ee/src/ramp/config.tsx
@@ -4,21 +4,18 @@ import { isBrowser } from "@carbon/utils";
import type { ComponentProps } from "react";
import { z } from "zod";
import { defineIntegration } from "../fns";
+import { RAMP_AUTHORIZE_URL, RAMP_TOKEN_URL } from "./environment";
import { RAMP_OAUTH_SCOPES } from "./scopes";
/**
- * OAuth "Connect to Ramp" authorization-code flow (production) — the ONLY way to
- * connect Ramp. Declared inline (not imported from `./lib/client`, which pulls
- * `node:crypto` into the client bundle). `offline_access` requests a refresh
- * token; the app must have the Refresh Token grant enabled. IntegrationCard
- * builds the authorize redirect from this block.
+ * OAuth "Connect to Ramp" authorization-code flow — the ONLY way to connect
+ * Ramp. The authorize/token hosts come from `./environment` (a browser-safe
+ * module, not `./lib/client`, which pulls `node:crypto` into the client
+ * bundle), which carries the TEMPORARY sandbox/production switch used for
+ * customer testing. `offline_access` requests a refresh token; the app must
+ * have the Refresh Token grant enabled. IntegrationCard builds the authorize
+ * redirect from this block.
*/
-// The user-consent authorize endpoint lives on the APP host (app.ramp.com), NOT
-// the API host — hitting api.ramp.com/v1/authorize returns "Not Authorized" from
-// the API gateway before any consent screen. The token exchange DOES stay on the
-// API host (api.ramp.com/developer/v1/token).
-const RAMP_AUTHORIZE_URL = "https://app.ramp.com/v1/authorize";
-const RAMP_TOKEN_URL = "https://api.ramp.com/developer/v1/token";
/**
* Ramp settings form schema. Connection is exclusively via the "Connect to Ramp"
@@ -32,7 +29,7 @@ const RampSettingsSchema = z.object({
// Required non-empty (the card liability + statement bank accounts back every
// card journal); populated by the loader's dynamicOptions.
cardLiabilityAccountId: z.string().min(1),
- statementBankAccountId: z.string().min(1),
+ statementBankAccountId: z.string().optional(),
cashbackIncomeAccountId: z.string().optional(),
reimbursementBankAccountId: z.string().optional(),
// SwitchField posts a literal "true"/"false" string; stored flat as-is.
@@ -96,7 +93,7 @@ export const Ramp = defineIntegration({
name: "cardLiabilityAccountId",
label: "Card liability account",
description:
- "The liability account Ramp card charges credit (your Ramp card balance).",
+ "Pick the Liability account that tracks what you owe on your Ramp cards — your outstanding Ramp balance. Each card charge Carbon pulls in credits this account; paying a statement debits it back down. Required — no card transactions sync until this is set.",
group: "Accounts",
type: "options" as const,
listOptions: [],
@@ -107,11 +104,11 @@ export const Ramp = defineIntegration({
name: "statementBankAccountId",
label: "Statement bank account",
description:
- "The bank/asset account statement payments and transfers draw from.",
+ "Optional. Only needed if you want Carbon to book Ramp statement payments and transfers: Carbon credits this bank (Asset) account and debits the card liability above. Leave blank to skip statement-payment and transfer sync — card charges sync without it.",
group: "Accounts",
type: "options" as const,
listOptions: [],
- required: true,
+ required: false,
value: ""
},
{
diff --git a/packages/ee/src/ramp/environment.ts b/packages/ee/src/ramp/environment.ts
new file mode 100644
index 0000000000..a6ef57f336
--- /dev/null
+++ b/packages/ee/src/ramp/environment.ts
@@ -0,0 +1,36 @@
+/**
+ * TEMPORARY hard switch between the Ramp sandbox (demo) and production
+ * environments, for testing the integration with a customer. There is NO env
+ * var by design — flip this one constant and redeploy, then set it back to
+ * "production" when testing is done.
+ *
+ * While set to "sandbox", point the RAMP_CLIENT_ID / RAMP_CLIENT_SECRET env
+ * vars at your Ramp *demo* OAuth app's credentials — the sandbox is a separate
+ * Ramp OAuth app, so the production app's id/secret will NOT authorize against
+ * it. The stored `credentials.environment` (stamped by `exchangeRampOAuthCode`)
+ * carries this choice through to every subsequent Ramp API call and token
+ * refresh, so `RampClient` talks to demo-api.ramp.com automatically.
+ *
+ * This module is browser-safe (no node imports) so both the client-bundled
+ * `config.tsx` and the server `connection.ts` share the one switch.
+ */
+export type RampEnvironment = "production" | "sandbox";
+
+export const RAMP_ENVIRONMENT: RampEnvironment = "sandbox";
+
+const IS_SANDBOX = RAMP_ENVIRONMENT === "sandbox";
+
+/**
+ * User-consent (authorize) endpoint. Lives on the APP host, NOT the API host —
+ * hitting the api host's /v1/authorize returns "Not Authorized" before any
+ * consent screen. Production: app.ramp.com. Sandbox/demo: demo.ramp.com. (If
+ * the sandbox consent screen 404s, this demo host is the value to adjust.)
+ */
+export const RAMP_AUTHORIZE_URL = IS_SANDBOX
+ ? "https://demo.ramp.com/v1/authorize"
+ : "https://app.ramp.com/v1/authorize";
+
+/** Token-exchange endpoint — on the API host. */
+export const RAMP_TOKEN_URL = IS_SANDBOX
+ ? "https://demo-api.ramp.com/developer/v1/token"
+ : "https://api.ramp.com/developer/v1/token";
diff --git a/packages/ee/src/ramp/hooks.server.ts b/packages/ee/src/ramp/hooks.server.ts
index cf4bc8a424..26f0cf6d3e 100644
--- a/packages/ee/src/ramp/hooks.server.ts
+++ b/packages/ee/src/ramp/hooks.server.ts
@@ -58,9 +58,13 @@ async function convergeRamp(
}
// OAuth creates the credential-bearing integration row before the user maps
- // the two accounts required by the card families. Establish connectivity,
- // but do not push master data or launch finance until that setup is complete.
- if (!metadata.cardLiabilityAccountId || !metadata.statementBankAccountId) {
+ // the card liability account that every card journal credits. Establish
+ // connectivity, but do not push master data or launch finance until that one
+ // required account exists. statementBankAccountId is NOT required here — it is
+ // only the offset for statement payments and transfers, and each of those
+ // families self-gates on it (skipping when unset). Coupling it here blocked
+ // card-charge sync on an account card charges never touch.
+ if (!metadata.cardLiabilityAccountId) {
return;
}
@@ -253,6 +257,18 @@ export async function rampHealthcheck(
);
if (!integration) return false;
+ // A connected Ramp with no card liability account is not functional:
+ // convergeRamp returns early (no chart-of-accounts push, no sync) and the
+ // card-transaction sync gate skips every family without it. Report it as
+ // unhealthy rather than showing a green badge over a sync that silently does
+ // nothing — the required-field gap was invisible in the UI otherwise.
+ // statementBankAccountId is intentionally NOT checked: it is optional (only
+ // statement-payment/transfer sync needs it), so its absence is a healthy
+ // "that family is off", not a broken connection.
+ if (!integration.metadata.cardLiabilityAccountId) {
+ return false;
+ }
+
try {
await integration.client.getBusiness();
const connections = extractConnections(
diff --git a/packages/ee/src/ramp/lib/connection.ts b/packages/ee/src/ramp/lib/connection.ts
index 31b0322357..6c4f8c5c83 100644
--- a/packages/ee/src/ramp/lib/connection.ts
+++ b/packages/ee/src/ramp/lib/connection.ts
@@ -1,6 +1,7 @@
import type { Database } from "@carbon/database";
import type { SupabaseClient } from "@supabase/supabase-js";
import { resolveIntegrationSecrets } from "../../integrations/secrets";
+import { RAMP_ENVIRONMENT } from "../environment";
import { buildRampIdempotencyKey, RampClient } from "./client";
import {
RampAccountingConnectionSchema,
@@ -157,10 +158,12 @@ export async function getRampIntegration(
/**
* Exchange an OAuth authorization code (the Connect-flow callback) for oauth2
- * credentials, using Carbon's registered Ramp OAuth app. OAuth is the production
- * flow, so the returned credentials are pinned to `environment: "production"`.
- * The caller stores these via the atomic Ramp OAuth patch (which vaults the
- * access + refresh tokens) and then runs `rampOnInstall`.
+ * credentials, using Carbon's registered Ramp OAuth app. The returned
+ * credentials are stamped with `RAMP_ENVIRONMENT` (the TEMPORARY sandbox/
+ * production switch in `../environment`) so every subsequent Ramp API call and
+ * token refresh hits the matching host. The caller stores these via the atomic
+ * Ramp OAuth patch (which vaults the access + refresh tokens) and then runs
+ * `rampOnInstall`.
*/
export async function exchangeRampOAuthCode(
code: string,
@@ -173,10 +176,11 @@ export async function exchangeRampOAuthCode(
"Ramp OAuth is not configured (set RAMP_CLIENT_ID / RAMP_CLIENT_SECRET)"
);
}
- // A placeholder access token just to construct the client; the exchange only
- // uses the OAuth app credentials + host (production).
+ // A placeholder access token just to construct the client; the exchange uses
+ // the OAuth app credentials + the environment's host (the token POST goes to
+ // `${host}/developer/v1/token`, so the environment must be set here too).
const client = new RampClient(
- { type: "oauth2", accessToken: "", environment: "production" },
+ { type: "oauth2", accessToken: "", environment: RAMP_ENVIRONMENT },
{ oauthApp: { clientId, clientSecret } }
);
const tokens = await client.exchangeAuthorizationCode(code, redirectUri);
@@ -185,7 +189,7 @@ export async function exchangeRampOAuthCode(
accessToken: tokens.accessToken,
refreshToken: tokens.refreshToken,
expiresAt: tokens.expiresAt,
- environment: "production"
+ environment: RAMP_ENVIRONMENT
};
}
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.integration.test.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.integration.test.ts
index 9ea50cbf1c..d33baa4556 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.integration.test.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.integration.test.ts
@@ -167,6 +167,8 @@ describe.skipIf(process.env.RUN_RAMP_DB_TESTS !== "true")(
mapping: createMappingService(db, scope.companyId),
decimalsCache: new Map(),
exchangeRateCache: new Map(),
+ createdBy: "system",
+ trigger: "event",
metadata: { sync: { pullBills: true } } as RampSyncContext["metadata"]
};
const bill: RampBill = {
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.ts
index d5b30cce34..d1b13db157 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-bill.ts
@@ -27,6 +27,8 @@ import {
invoiceDeepLinkUrl,
normalizeVerifiedMinorAmount,
type RampSyncContext,
+ recordRampSyncFailures,
+ resolveRampSyncOperations,
type SyncItem,
stripSpecialCharacters,
verifyCostCenters,
@@ -486,6 +488,17 @@ export async function syncRampBills(
result.created = successful.length - reconfirmed;
result.reconfirmed = reconfirmed;
result.failed += failed.length;
+
+ await recordRampSyncFailures(ctx, {
+ entityType: "bill",
+ direction: "pull-from-accounting",
+ failures: failed
+ });
+ await resolveRampSyncOperations(ctx, {
+ entityType: "bill",
+ direction: "pull-from-accounting",
+ entityIds: successful.map((item) => item.id)
+ });
return result;
}
@@ -580,5 +593,16 @@ export async function syncRampBillPayments(
result.created = successful.length - reconfirmed;
result.reconfirmed = reconfirmed;
result.failed += failed.length;
+
+ await recordRampSyncFailures(ctx, {
+ entityType: "billPayment",
+ direction: "pull-from-accounting",
+ failures: failed
+ });
+ await resolveRampSyncOperations(ctx, {
+ entityType: "billPayment",
+ direction: "pull-from-accounting",
+ entityIds: successful.map((item) => item.id)
+ });
return result;
}
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.integration.test.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.integration.test.ts
index b7541016f1..7da6b92c8f 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.integration.test.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.integration.test.ts
@@ -324,7 +324,9 @@ describe.skipIf(!runDatabaseTests)("Ramp card staging (Postgres)", () => {
baseCurrency: fixture.baseCurrency,
companyGroupId: fixture.companyGroupId,
decimalsCache: new Map([[fixture.baseCurrency, 2]]),
- exchangeRateCache: new Map()
+ exchangeRateCache: new Map(),
+ createdBy: "system",
+ trigger: "event"
};
let transaction: RampTransaction = {
id: rampId,
@@ -449,7 +451,9 @@ describe.skipIf(!runDatabaseTests)("Ramp card staging (Postgres)", () => {
baseCurrency: fixture.baseCurrency,
companyGroupId: fixture.companyGroupId,
decimalsCache: new Map(),
- exchangeRateCache: new Map()
+ exchangeRateCache: new Map(),
+ createdBy: "system",
+ trigger: "event"
};
const args: Parameters<typeof createAndPostTransaction>[1] = {
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.ts
index 1d5d60432a..d5924e7217 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-card.ts
@@ -28,6 +28,8 @@ import {
getRampExchangeRate,
normalizeVerifiedMinorAmount,
type RampSyncContext,
+ recordRampSyncFailures,
+ resolveRampSyncOperations,
type SyncItem,
stripSpecialCharacters,
verifyCostCenters,
@@ -414,6 +416,8 @@ type CardFamilyGate =
type CardFamilyConfig<TItem extends RampCardListItem> = {
/** Sync toggle key (`metadata.sync.pull*`). */
family: RampInboundFamily;
+ /** `accountingSyncOperation.entityType` for this family's Sync Activity rows. */
+ entityType: string;
/** Human name used in the drain-failure log line. */
label: string;
/** Ramp confirm `sync_type` for this family. */
@@ -523,6 +527,22 @@ async function syncRampCardFamily<TItem extends RampCardListItem>(
result.reconfirmed += reconfirmed.successful.length;
result.created = successful.length - result.reconfirmed;
result.failed += failed.length;
+
+ // Sync Activity: record why each item failed, and clear a prior Warning for
+ // any item that synced (or is now already mapped) this run.
+ await recordRampSyncFailures(ctx, {
+ entityType: config.entityType,
+ direction: "pull-from-accounting",
+ failures: failed
+ });
+ await resolveRampSyncOperations(ctx, {
+ entityType: config.entityType,
+ direction: "pull-from-accounting",
+ entityIds: [
+ ...successful.map((item) => item.id),
+ ...mapped.map((item) => item.rampId)
+ ]
+ });
return result;
}
@@ -708,6 +728,7 @@ export async function syncRampCardTransactions(
cardLiabilityAccountId,
{
family: "transactions",
+ entityType: "cardTransaction",
label: "card transactions",
syncType: "TRANSACTION_SYNC",
gate: (ctx, cardLiabilityAccountId) => {
@@ -746,6 +767,7 @@ export async function syncRampTransfers(
cardLiabilityAccountId,
{
family: "transfers",
+ entityType: "transfer",
label: "transfers",
syncType: "TRANSFER_SYNC",
gate: (ctx, cardLiabilityAccountId) => {
@@ -780,6 +802,7 @@ export async function syncRampCashbacks(
cardLiabilityAccountId,
{
family: "cashbacks",
+ entityType: "cashback",
label: "cashbacks",
syncType: "STATEMENT_CREDIT_SYNC",
// Skip the family silently when no cashback income account is configured.
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-outbound.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-outbound.ts
index c24c5e89da..7f75cde432 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-outbound.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-outbound.ts
@@ -19,7 +19,12 @@ import {
recordRampFamilyError
} from "./ramp-sync-observability";
import { loadRampPurchaseOrderLines } from "./ramp-sync-outbound-lines";
-import type { RampSyncContext } from "./ramp-sync-shared";
+import {
+ type FailItem,
+ type RampSyncContext,
+ recordRampSyncFailures,
+ resolveRampSyncOperations
+} from "./ramp-sync-shared";
const OUTBOUND_PAGE_SIZE = 100;
const PO_PUSH_STATUSES: Database["public"]["Enums"]["purchaseOrderStatus"][] = [
@@ -262,6 +267,7 @@ export async function syncRampOutbound(
)
);
const failedIds = new Set<string>();
+ const poFailures: FailItem[] = [];
for (const row of poRows) {
try {
const action = await pushPurchaseOrder(
@@ -287,6 +293,11 @@ export async function syncRampOutbound(
} catch (poError) {
result.purchaseOrders.failed += 1;
failedIds.add(row.id);
+ poFailures.push({
+ id: row.id,
+ message:
+ poError instanceof Error ? poError.message : String(poError)
+ });
console.error(
`[RAMP SYNC] ${companyId}: purchase order ${row.purchaseOrderId} push failed`,
poError
@@ -294,6 +305,21 @@ export async function syncRampOutbound(
}
}
+ // Sync Activity: record why each PO push failed, and clear a prior
+ // Warning for every PO that pushed (or archived) cleanly this run.
+ await recordRampSyncFailures(ctx, {
+ entityType: "purchaseOrder",
+ direction: "push-to-accounting",
+ failures: poFailures
+ });
+ await resolveRampSyncOperations(ctx, {
+ entityType: "purchaseOrder",
+ direction: "push-to-accounting",
+ entityIds: poRows
+ .filter((row) => !failedIds.has(row.id))
+ .map((row) => row.id)
+ });
+
const cursorRows = poRows.filter(
(row): row is typeof row & { updatedAt: string } =>
Boolean(row.updatedAt)
@@ -369,6 +395,7 @@ export async function syncRampOutbound(
// Advance past EVERY fetched row (mapped / employee / pushed alike);
// only a throw holds the cursor back.
const failedIds = new Set<string>();
+ const invFailures: FailItem[] = [];
const candidates = invRows.filter(
(row) => row.id && !mappedInvoiceIds.has(row.id)
@@ -463,6 +490,13 @@ export async function syncRampOutbound(
} catch (invoiceError) {
result.invoices.failed += 1;
failedIds.add(invoiceRowId);
+ invFailures.push({
+ id: invoiceRowId,
+ message:
+ invoiceError instanceof Error
+ ? invoiceError.message
+ : String(invoiceError)
+ });
console.error(
`[RAMP SYNC] ${companyId}: invoice ${
row.invoiceId ?? invoiceRowId
@@ -473,6 +507,22 @@ export async function syncRampOutbound(
}
}
+ // Sync Activity: record why each draft-bill push failed, and clear a
+ // prior Warning for every fetched invoice that did not fail this run
+ // (pushed, already mapped, or employee-skipped).
+ await recordRampSyncFailures(ctx, {
+ entityType: "purchaseInvoice",
+ direction: "push-to-accounting",
+ failures: invFailures
+ });
+ await resolveRampSyncOperations(ctx, {
+ entityType: "purchaseInvoice",
+ direction: "push-to-accounting",
+ entityIds: invRows
+ .filter((row) => row.id && !failedIds.has(row.id))
+ .map((row) => row.id as string)
+ });
+
// The keyset advances on `createdAt` (mapped into the cursor's timestamp
// slot), since that is the column the invoice page is ordered by.
const cursorRows = invRows
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-reimbursement-family.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-reimbursement-family.ts
index aed44b0937..54ac79fb43 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-reimbursement-family.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-reimbursement-family.ts
@@ -18,6 +18,8 @@ import {
invoiceDeepLinkUrl,
normalizeVerifiedMinorAmount,
type RampSyncContext,
+ recordRampSyncFailures,
+ resolveRampSyncOperations,
type SyncItem
} from "./ramp-sync-shared";
@@ -96,5 +98,16 @@ export async function syncRampReimbursements(
result.created = successful.length;
result.failed += failed.length;
+
+ await recordRampSyncFailures(ctx, {
+ entityType: "reimbursement",
+ direction: "pull-from-accounting",
+ failures: failed
+ });
+ await resolveRampSyncOperations(ctx, {
+ entityType: "reimbursement",
+ direction: "pull-from-accounting",
+ entityIds: successful.map((item) => item.id)
+ });
return result;
}
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-repayment.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-repayment.ts
index 77d6db661d..f0667ba625 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-repayment.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-repayment.ts
@@ -11,10 +11,13 @@ import {
isRampInboundFamilyEnabled
} from "./ramp-sync-policy";
import {
+ type FailItem,
type FamilyResult,
getRampCurrencyDecimals,
normalizeVerifiedMinorAmount,
- type RampSyncContext
+ type RampSyncContext,
+ recordRampSyncFailures,
+ resolveRampSyncOperations
} from "./ramp-sync-shared";
const REPAYMENT_REPAID_STATUS = "REPAID";
@@ -86,6 +89,8 @@ export async function syncRampRepayments(
const processedRepaidAt: string[] = [];
const failedRepaidAt: string[] = [];
+ const repaymentFailures: FailItem[] = [];
+ const processedIds: string[] = [];
let created = 0;
let reconfirmed = 0;
let failed = 0;
@@ -98,10 +103,15 @@ export async function syncRampRepayments(
if (!isRampEntityInScope(entityId, repayment.entity_id)) continue;
if (repayment.status !== REPAYMENT_REPAID_STATUS) continue;
const repaidAt = repayment.repaid_at ?? null;
- if (repayment.funding_method !== REPAYMENT_BANK_FUNDING) {
+ const failRepayment = (message: string) => {
failed += 1;
if (repaidAt) failedRepaidAt.push(repaidAt);
- result.error ??= `Repayment ${repayment.id} has unsupported funding method: ${repayment.funding_method || "missing"}`;
+ repaymentFailures.push({ id: repayment.id, message });
+ };
+ if (repayment.funding_method !== REPAYMENT_BANK_FUNDING) {
+ const message = `Unsupported funding method: ${repayment.funding_method || "missing"}`;
+ failRepayment(message);
+ result.error ??= `Repayment ${repayment.id} has ${message}`;
continue;
}
@@ -113,6 +123,7 @@ export async function syncRampRepayments(
);
if (existing) {
reconfirmed += 1;
+ processedIds.push(repayment.id);
if (repaidAt) processedRepaidAt.push(repaidAt);
continue;
}
@@ -120,8 +131,7 @@ export async function syncRampRepayments(
// Resolve the ORIGINAL card transaction via its mapping.
const originalRampId = repayment.original_transaction_id;
if (!originalRampId) {
- failed += 1;
- if (repaidAt) failedRepaidAt.push(repaidAt);
+ failRepayment("No original_transaction_id on the repayment");
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} has no original_transaction_id — skipped`
);
@@ -133,8 +143,9 @@ export async function syncRampRepayments(
"cardTransaction"
);
if (!originalEntityId) {
- failed += 1;
- if (repaidAt) failedRepaidAt.push(repaidAt);
+ failRepayment(
+ `Original transaction ${originalRampId} is not synced yet`
+ );
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} original transaction ${originalRampId} is not synced yet — skipped`
);
@@ -148,8 +159,9 @@ export async function syncRampRepayments(
.eq("companyId", companyId)
.maybeSingle();
if (!original.data) {
- failed += 1;
- if (repaidAt) failedRepaidAt.push(repaidAt);
+ failRepayment(
+ `Original card transaction ${originalEntityId} no longer exists`
+ );
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} original card transaction ${originalEntityId} no longer exists — skipped`
);
@@ -162,8 +174,9 @@ export async function syncRampRepayments(
.eq("companyId", companyId)
.order("sequence", { ascending: true });
if (originalLines.error) {
- failed += 1;
- if (repaidAt) failedRepaidAt.push(repaidAt);
+ failRepayment(
+ `Failed to load original lines: ${originalLines.error.message}`
+ );
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} failed to load original lines`,
originalLines.error
@@ -179,8 +192,7 @@ export async function syncRampRepayments(
try {
decimals = await getRampCurrencyDecimals(ctx, currencyCode);
} catch (error) {
- failed += 1;
- if (repaidAt) failedRepaidAt.push(repaidAt);
+ failRepayment("Invalid currency precision");
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} has invalid currency precision`,
error
@@ -194,8 +206,7 @@ export async function syncRampRepayments(
"Repayment amount"
);
if (!normalizedAmount.ok) {
- failed += 1;
- if (repaidAt) failedRepaidAt.push(repaidAt);
+ failRepayment(`Amount is invalid: ${normalizedAmount.error}`);
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} amount is invalid: ${normalizedAmount.error}`
);
@@ -220,7 +231,7 @@ export async function syncRampRepayments(
const transactionDate = repaidAt?.slice(0, 10);
if (!transactionDate) {
- failed += 1;
+ failRepayment("No repaid_at date on the repayment");
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} has no repaid_at — skipped`
);
@@ -252,10 +263,10 @@ export async function syncRampRepayments(
});
if ("ok" in outcome) {
created += 1;
+ processedIds.push(repayment.id);
if (repaidAt) processedRepaidAt.push(repaidAt);
} else {
- failed += 1;
- if (repaidAt) failedRepaidAt.push(repaidAt);
+ failRepayment(outcome.fail.message);
console.error(
`[RAMP SYNC] ${companyId}: repayment ${repayment.id} failed — ${outcome.fail.message}`
);
@@ -280,5 +291,16 @@ export async function syncRampRepayments(
result.created = created;
result.reconfirmed = reconfirmed;
result.failed += failed;
+
+ await recordRampSyncFailures(ctx, {
+ entityType: "repayment",
+ direction: "pull-from-accounting",
+ failures: repaymentFailures
+ });
+ await resolveRampSyncOperations(ctx, {
+ entityType: "repayment",
+ direction: "pull-from-accounting",
+ entityIds: processedIds
+ });
return result;
}
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync-shared.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync-shared.ts
index f86bd2f0d0..461d8769fa 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync-shared.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync-shared.ts
@@ -1,6 +1,13 @@
import type { Database } from "@carbon/database";
-import type { createMappingService } from "@carbon/ee/accounting";
import {
+ clearResolvedSyncOperations,
+ type createMappingService,
+ insertTerminalSyncOperation,
+ type SyncOperationDirection,
+ type SyncOperationTrigger
+} from "@carbon/ee/accounting";
+import {
+ buildRampIdempotencyKey,
parseVerifiedRampMinorAmount,
type RampIntegrationMetadata,
rampMinorAmountToMajor,
@@ -40,8 +47,104 @@ export type RampSyncContext = {
companyGroupId: string | null;
decimalsCache: Map<string, number>;
exchangeRateCache: Map<string, number>;
+ /** User the recorded sync operations are attributed to (integration configurer, else "system"). */
+ createdBy: string;
+ /** How this sync run was triggered — stamped on recorded sync operations. */
+ trigger: SyncOperationTrigger;
};
+const RAMP_INTEGRATION_ID = "ramp";
+
+/**
+ * Record each failed/skipped sync item as a terminal `Warning` operation on the
+ * shared `accountingSyncOperation` ledger, so the integration's Sync Activity
+ * tab shows WHY a Ramp record did not come through instead of it vanishing into
+ * the Inngest logs. Idempotent per (entityType, entityId, direction): a
+ * persistently-failing record does not stack rows. Failures to record are
+ * logged, never thrown — observability must not fail the sync.
+ */
+export async function recordRampSyncFailures(
+ ctx: RampSyncContext,
+ args: {
+ entityType: string;
+ direction: SyncOperationDirection;
+ failures: FailItem[];
+ }
+): Promise<void> {
+ for (const failure of args.failures) {
+ // Observability is strictly best-effort: a record that throws (or returns
+ // an error) must never fail or pollute the family sync result.
+ try {
+ const { error } = await insertTerminalSyncOperation(ctx.client, {
+ companyId: ctx.companyId,
+ integration: RAMP_INTEGRATION_ID,
+ entityType: args.entityType,
+ entityId: failure.id,
+ direction: args.direction,
+ trigger: ctx.trigger,
+ status: "Warning",
+ errorCode: "RAMP_SYNC_FAILED",
+ errorMessage: failure.message,
+ idempotencyKey: buildRampIdempotencyKey({
+ companyId: ctx.companyId,
+ operation: `sync-fail:${args.entityType}:${args.direction}`,
+ scope: failure.id
+ }),
+ createdBy: ctx.createdBy
+ });
+ if (error) {
+ console.error(
+ `[RAMP SYNC] ${ctx.companyId}: failed to record ${args.entityType} sync failure for ${failure.id}`,
+ error
+ );
+ }
+ } catch (recordError) {
+ console.error(
+ `[RAMP SYNC] ${ctx.companyId}: recording ${args.entityType} sync failure for ${failure.id} threw`,
+ recordError
+ );
+ }
+ }
+}
+
+/**
+ * Clear any prior `Warning` operation for records that synced successfully this
+ * run — a Ramp charge recoded and posted after an earlier failure must drop out
+ * of the Sync Activity inbox (Ramp's inbound families re-evaluate every run,
+ * unlike accounting journals whose disposition is permanent). Logged, never
+ * thrown.
+ */
+export async function resolveRampSyncOperations(
+ ctx: RampSyncContext,
+ args: {
+ entityType: string;
+ direction: SyncOperationDirection;
+ entityIds: string[];
+ }
+): Promise<void> {
+ // Best-effort: clearing a resolved Warning must never fail the sync.
+ try {
+ const { error } = await clearResolvedSyncOperations(ctx.client, {
+ companyId: ctx.companyId,
+ integration: RAMP_INTEGRATION_ID,
+ entityType: args.entityType,
+ direction: args.direction,
+ entityIds: args.entityIds
+ });
+ if (error) {
+ console.error(
+ `[RAMP SYNC] ${ctx.companyId}: failed to clear resolved ${args.entityType} sync operations`,
+ error
+ );
+ }
+ } catch (resolveError) {
+ console.error(
+ `[RAMP SYNC] ${ctx.companyId}: clearing resolved ${args.entityType} sync operations threw`,
+ resolveError
+ );
+ }
+}
+
export async function verifyCostCenters(
ctx: RampSyncContext,
lines: ReadonlyArray<{ costCenterId: string | null }>
diff --git a/packages/jobs/src/inngest/functions/integrations/ramp-sync.ts b/packages/jobs/src/inngest/functions/integrations/ramp-sync.ts
index 991461f81b..a30422a154 100644
--- a/packages/jobs/src/inngest/functions/integrations/ramp-sync.ts
+++ b/packages/jobs/src/inngest/functions/integrations/ramp-sync.ts
@@ -79,7 +79,9 @@ export const rampSyncFunction = inngest.createFunction(
baseCurrency: company.data.baseCurrencyCode,
companyGroupId: company.data.companyGroupId,
decimalsCache: new Map(),
- exchangeRateCache: new Map()
+ exchangeRateCache: new Map(),
+ createdBy: integrationRow.data?.updatedBy ?? "system",
+ trigger: event.data.reason === "webhook" ? "webhook" : "event"
};
const cardLiabilityAccountId = metadata.cardLiabilityAccountId;
const entityId = metadata.entityId;