Skip to content

Open-source agentic AI Search Execution Platform. Track AI visibility, find opportunities, and let agents turn signals into prioritized actions and drafted fixes across ChatGPT, Claude, Gemini, Google AI Overviews, Google AI Mode, Perplexity, Grok, and Microsoft Copilot.

feat(catalog): read a brand's Google Merchant Center products and item issues (00296, #1270 part 1) (#1582)

Cihane88fd2dac2cb
diff --git a/CONNECTORS.md b/CONNECTORS.md
index 35e21376b..5d5fa2363 100644
--- a/CONNECTORS.md
+++ b/CONNECTORS.md
@@ -92,6 +92,22 @@ Anything else is logged, with every credential value taken out of its message, a
- **What to ask for:** the least access the feature needs. Today that's read-only access to products (and, later, orders). Say which permission to tick in your field's `help`.
- **Self-hosting:** set `CONNECTOR_CREDENTIALS_KEY` in `server/.env` (`openssl rand -base64 32`). Without it, Settings says store connections aren't set up. If the key changes, saved sign-ins can't be opened: those stores show "Sign-in refused" until they're updated.
+### A platform read with the workspace's Google sign-in
+
+Some platforms sign in with OAuth through a provider Composio already handles for the workspace, like Search Console and Analytics. Google Merchant Center ([#1270](https://github.com/ansvisor/ansvisor/issues/1270), `server/src/connectors/merchant-center/`) is one. Such a connector differs from a store:
+
+- **No `fields` and no `connect`.** It has `signIn`, the Composio provider (`server/src/lib/composio.js` `PROVIDERS`). The workspace connects that provider once in Settings › Integrations, and Composio keeps the token: it never reaches us, and the source has no row in `catalog_source_credentials`.
+- **Picking the account:**
+ - `listAccounts(ctx)` returns the accounts the sign-in opens, as `{ id, name }`.
+ - The person picks one. `POST /api/catalog/sources` with `input: { account }` then calls `checkAccount(value, ctx)`, which returns `{ account, label?, url? }` or throws a ConnectError.
+- **Calls:** `ctx.request(url, { query, signal })` goes through Composio's proxy (`server/src/lib/catalog/sign-in.js`), to the hosts your connector names in `hosts` only (Merchant Center's is `merchantapi.googleapis.com`). It answers like the HTTP client and throws the same `ConnectorHttpError` codes, and one more:
+ - `auth`: the sign-in is gone or refused. The source shows "Reconnect it in Settings › Integrations".
+ - `not_set_up`: the server's own setup is refused, not the sign-in: Composio's API key, or Google's 403 (the Google Cloud project not registered with Merchant Center, a scope missing). Reconnecting wouldn't help, so the source says the server isn't set up correctly. Both are logged as warnings, with Google's reason, for whoever runs the server.
+ - Listing accounts and checking one stop after 30 seconds in all; a sync's calls after 10 minutes.
+- **Issues:** `readCatalog` may also return `issues`, the platform's problems per item. The plumbing replaces the source's rows in `catalog_item_issues` with them (00296) and counts them on the source (`last_counts.issues`).
+- **The same item listed again:** Merchant Center lists an offer once per language and feed label. The catalog keeps one (a listing that isn't disapproved first, then by feed label and language), keeps the issues of all of them, and counts the others in `repeated` (`last_counts.repeated`), not as skipped.
+- **Testing:** pass `runSync` a `signInRequest` stand-in. `merchant-center.test.js` builds one from the real `signInRequest` with a fake proxy.
+
## Testing
Connector tests run in `npm test` (in `server/`), against a fake of the platform's API, with no network and no account. The harness is `server/src/connectors/testing.js`:
@@ -155,5 +171,5 @@ Copy this into your pull request. Every connector is reviewed against it.
## Not here yet
- **Orders and webhooks** (`syncOrders(since)`, `handleWebhook(event)`) come with the first store that needs them (Shopify, [#1265](https://github.com/ansvisor/ansvisor/issues/1265)).
-- **OAuth sign-in:** platforms that sign in through a redirect (Etsy, eBay, Amazon) need a callback the plumbing doesn't have yet. Open an issue before you start one.
+- **OAuth sign-in:** platforms that sign in through a redirect (Etsy, eBay, Amazon) need a callback the plumbing doesn't have yet, unless Composio handles their sign-in, as it does Google's (see above). Open an issue before you start one.
- **API keys:** stores that sign in with keys (WooCommerce, Walmart, Trendyol, Hepsiburada) work today.
diff --git a/server/.env.example b/server/.env.example
index 975bed904..01cd739be 100644
--- a/server/.env.example
+++ b/server/.env.example
@@ -191,6 +191,15 @@ AI_VOLUME_MULTIPLIER=0.45
# keeps the token. Without it, Webflow shows "not configured".
# COMPOSIO_WEBFLOW_AUTH_CONFIG_ID=
# COMPOSIO_WEBFLOW_TOOLKIT_VERSION=20260826_00
+# Google Merchant Center for the product catalog (#1270): an auth config of
+# the Google Merchant toolkit (GOOGLE_MERCHANT) with our own Google OAuth app
+# and the https://www.googleapis.com/auth/content scope (Composio has no
+# managed app for it). The workspace connects it in Settings → Integrations
+# like Search Console; the catalog reads the Merchant API v1 through
+# Composio's proxy, so the API key needs the "Proxy execute" permission, and
+# the Google Cloud project must be registered with Merchant Center once
+# (registerGcp). Without it, Merchant Center shows "not configured".
+# COMPOSIO_GMC_AUTH_CONFIG_ID=
# Store connections for the product catalog (#1263, CONNECTORS.md): the key
# a store's sign-in is encrypted with, 32 bytes in base64
diff --git a/server/src/connectors/http.js b/server/src/connectors/http.js
index 225349095..901c005ae 100644
--- a/server/src/connectors/http.js
+++ b/server/src/connectors/http.js
@@ -35,8 +35,10 @@ const RETRY_ERRNOS = new Set(['ECONNRESET', 'ECONNREFUSED', 'EPIPE', 'ETIMEDOUT'
export class ConnectorHttpError extends Error {
/**
- * @param {'invalid_url'|'blocked'|'unreachable'|'timeout'|'too_large'|'http_error'|'auth'|'rate_limited'|'bad_response'} code
- * 'auth': the API refused the sign-in (401, 403)
+ * @param {'invalid_url'|'blocked'|'unreachable'|'timeout'|'too_large'|'http_error'|'auth'|'rate_limited'|'bad_response'|'not_set_up'} code
+ * 'auth': the API refused the sign-in (401, 403);
+ * 'not_set_up': this server's own setup was refused, not the sign-in
+ * (lib/catalog/sign-in.js only)
* @param {string} message
* @param {{ status?: number }} [extra]
*/
diff --git a/server/src/connectors/index.js b/server/src/connectors/index.js
index b498babd9..bc8954916 100644
--- a/server/src/connectors/index.js
+++ b/server/src/connectors/index.js
@@ -1,4 +1,5 @@
import feedUrl from './feed-url/index.js';
+import merchantCenter from './merchant-center/index.js';
/**
* Catalog connectors (#1263): one folder each under `server/src/connectors/`,
@@ -31,11 +32,28 @@ import feedUrl from './feed-url/index.js';
* Later, with orders and webhooks: `syncOrders(since)` and
* `handleWebhook(event)`.
*
+ * A platform read with the workspace's own sign-in (#1270), made in
+ * Settings › Integrations and kept by Composio, has instead of `fields` and
+ * `connect`:
+ * - `signIn`: the Composio provider (lib/composio.js PROVIDERS). Its
+ * `ctx.request` goes through that sign-in (lib/catalog/sign-in.js);
+ * - `hosts`: the API hosts that request may call, over https; any other
+ * address is refused;
+ * - `listAccounts(ctx)`: the accounts the sign-in opens, `{ id, name }`,
+ * for the person to pick one;
+ * - `checkAccount(value, ctx)`: the picked account checked, as
+ * `{ account, label?, url? }`, or a ConnectError.
+ * Its `readCatalog` may also return `issues`: the platform's problems per
+ * item (lib/catalog/issues.js); and `repeated`: the listings of an item
+ * read already (another language or feed label), kept once and counted,
+ * not skipped.
+ *
* A connector never logs credentials or a buyer's personal data, and reads
* only what its feature needs.
*/
export const CONNECTORS = {
[feedUrl.kind]: feedUrl,
+ [merchantCenter.kind]: merchantCenter,
};
/** The connector for a source's kind, or null. */
@@ -48,6 +66,18 @@ export function isStore(connector) {
return typeof connector?.connect === 'function';
}
+/** Whether a connector reads with the workspace's own sign-in (Composio). */
+export function isSignIn(connector) {
+ return typeof connector?.signIn === 'string';
+}
+
+/** The platforms read with the workspace's sign-in, as the Settings card lists them. */
+export function signInConnectors(connectors = CONNECTORS) {
+ return Object.values(connectors)
+ .filter(isSignIn)
+ .map(({ kind, label, signIn }) => ({ kind, label, provider: signIn }));
+}
+
/** The stores a person can connect, as the Settings card lists them. */
export function storeConnectors(connectors = CONNECTORS) {
return Object.values(connectors)
diff --git a/server/src/connectors/merchant-center/__fixtures__/products-page-1.json b/server/src/connectors/merchant-center/__fixtures__/products-page-1.json
new file mode 100644
index 000000000..cf0a44a97
--- /dev/null
+++ b/server/src/connectors/merchant-center/__fixtures__/products-page-1.json
@@ -0,0 +1,69 @@
+{
+ "products": [
+ {
+ "name": "accounts/123456789/products/en~US~TB-42-BRN",
+ "offerId": "TB-42-BRN",
+ "contentLanguage": "en",
+ "feedLabel": "US",
+ "productAttributes": {
+ "title": "Trail Boot",
+ "description": "A waterproof hiking boot.",
+ "link": "https://shop.example/boots/trail?color=brown",
+ "imageLink": "https://shop.example/img/trail-brown.jpg",
+ "additionalImageLinks": ["https://shop.example/img/trail-brown-2.jpg"],
+ "brand": "Acme",
+ "gtins": ["00012345678905"],
+ "mpn": "TB-42",
+ "price": { "amountMicros": "129990000", "currencyCode": "USD" },
+ "salePrice": { "amountMicros": "99000000", "currencyCode": "USD" },
+ "availability": "IN_STOCK",
+ "condition": "NEW",
+ "itemGroupId": "TB",
+ "color": "Brown",
+ "size": "42",
+ "material": "Leather",
+ "productTypes": ["Shoes > Boots", "Outdoor"]
+ },
+ "productStatus": {
+ "itemLevelIssues": [
+ {
+ "code": "image_too_small",
+ "severity": "DEMOTED",
+ "resolution": "merchant_action",
+ "attribute": "image link",
+ "reportingContext": "SHOPPING_ADS",
+ "description": "Image too small",
+ "detail": "Use an image of at least 100 x 100 pixels",
+ "documentation": "https://support.google.com/merchants/answer/6098289",
+ "applicableCountries": ["US", "CA"]
+ },
+ {
+ "code": "image_too_small",
+ "severity": "NOT_IMPACTED",
+ "attribute": "image link",
+ "reportingContext": "FREE_LISTINGS",
+ "description": "Image too small",
+ "applicableCountries": ["US"]
+ }
+ ]
+ }
+ },
+ {
+ "name": "accounts/123456789/products/en~US~TB-43-BLK",
+ "offerId": "TB-43-BLK",
+ "contentLanguage": "en",
+ "feedLabel": "US",
+ "productAttributes": {
+ "title": "Trail Boot",
+ "link": "https://shop.example/boots/trail?color=black",
+ "price": { "amountMicros": "129990000", "currencyCode": "USD" },
+ "availability": "OUT_OF_STOCK",
+ "itemGroupId": "TB",
+ "color": "Black",
+ "size": "43"
+ },
+ "productStatus": {}
+ }
+ ],
+ "nextPageToken": "page-2"
+}
diff --git a/server/src/connectors/merchant-center/__fixtures__/products-page-2.json b/server/src/connectors/merchant-center/__fixtures__/products-page-2.json
new file mode 100644
index 000000000..ff81d997e
--- /dev/null
+++ b/server/src/connectors/merchant-center/__fixtures__/products-page-2.json
@@ -0,0 +1,64 @@
+{
+ "products": [
+ {
+ "name": "accounts/123456789/products/de~DE~TB-42-BRN",
+ "offerId": "TB-42-BRN",
+ "contentLanguage": "de",
+ "feedLabel": "DE",
+ "productAttributes": {
+ "title": "Wanderstiefel",
+ "link": "https://shop.example/de/boots/trail?color=brown",
+ "price": { "amountMicros": "119000000", "currencyCode": "EUR" },
+ "availability": "IN_STOCK"
+ },
+ "productStatus": {
+ "itemLevelIssues": [
+ {
+ "code": "image_too_small",
+ "severity": "DEMOTED",
+ "resolution": "merchant_action",
+ "attribute": "image link",
+ "reportingContext": "SHOPPING_ADS",
+ "description": "Bild zu klein",
+ "applicableCountries": ["DE", "US"]
+ },
+ {
+ "code": "landing_page_error",
+ "severity": "DISAPPROVED",
+ "resolution": "merchant_action",
+ "attribute": "link",
+ "reportingContext": "SHOPPING_ADS",
+ "description": "Unavailable landing page",
+ "applicableCountries": ["DE"]
+ }
+ ]
+ }
+ },
+ {
+ "name": "accounts/123456789/products/en~US~SOCK-1",
+ "offerId": "SOCK-1",
+ "contentLanguage": "en",
+ "feedLabel": "US",
+ "productAttributes": {
+ "title": "Wool Sock",
+ "link": "https://shop.example/socks/wool",
+ "availability": "IN_STOCK",
+ "gtins": ["4006381333931"]
+ },
+ "productStatus": {
+ "itemLevelIssues": [
+ {
+ "code": "missing_price",
+ "severity": "DISAPPROVED",
+ "resolution": "merchant_action",
+ "attribute": "price",
+ "reportingContext": "SHOPPING_ADS",
+ "description": "Missing value [price]",
+ "applicableCountries": ["US"]
+ },
+ { "code": "unknown_thing", "severity": "SEVERITY_UNSPECIFIED" }
+ ]
+ }
+ }
+ ]
+}
diff --git a/server/src/connectors/merchant-center/index.js b/server/src/connectors/merchant-center/index.js
new file mode 100644
index 000000000..de8935edc
--- /dev/null
+++ b/server/src/connectors/merchant-center/index.js
@@ -0,0 +1,334 @@
+import { toItem } from '../../lib/feeds/item.js';
+import { ConnectError } from '../errors.js';
+import { ConnectorHttpError } from '../http.js';
+
+/**
+ * Google Merchant Center (#1270): a brand's Merchant Center account read
+ * into its catalog, with the issues Merchant Center reports per product.
+ * Google AI Overviews and AI Mode take their product results from it (and
+ * ChatGPT Shopping in Europe, it seems), so what it holds, and what it
+ * refuses, decides whether a product can show there at all.
+ *
+ * It reads with the workspace's Google sign-in, made in Settings ›
+ * Integrations and kept by Composio: `signIn` names that provider, and the
+ * plumbing gives `readCatalog` a request through it (lib/catalog/sign-in.js).
+ * So it has no fields and no `connect`: a person picks one of the accounts
+ * the sign-in opens (`listAccounts`), and `checkAccount` confirms it.
+ *
+ * The Merchant API v1 (the Content API for Shopping is being retired):
+ * - `GET /accounts/v1/accounts`, and `:listSubaccounts` for an advanced
+ * account's own, for the picker;
+ * - `GET /products/v1/accounts/{id}/products`, paged by `pageToken`: the
+ * products as Merchant Center shows them, with `productStatus`.
+ * Read only: it never writes to the account.
+ */
+
+export const API = 'https://merchantapi.googleapis.com';
+/** Products a page, the API's most. */
+export const PAGE_SIZE = 1000;
+/** Pages read at most: 50,000 products, about what a 50 MB feed holds. */
+export const MAX_PAGES = 50;
+/** Issues kept at most for a listing, and for a source's whole read. */
+export const MAX_LISTING_ISSUES = 20;
+export const MAX_ISSUES = 100_000;
+/** Accounts the picker lists at most, and the advanced accounts whose sub-accounts it reads. */
+export const MAX_ACCOUNTS = 500;
+export const MAX_PARENTS = 20;
+/** Countries an issue row keeps at most (00296). */
+const MAX_COUNTRIES = 300;
+
+const SEVERITIES = { NOT_IMPACTED: 'not_impacted', DEMOTED: 'demoted', DISAPPROVED: 'disapproved' };
+
+/** A Merchant Center account id ("123456789"), from what a person or the API gives: "accounts/123…" too. */
+export function accountId(value) {
+ const id = String(value ?? '')
+ .trim()
+ .replace(/^accounts\//, '');
+ return /^\d{1,20}$/.test(id) ? id : null;
+}
+
+/** A price as the feeds write it ("12.99 USD"), from the API's micros. */
+export function moneyText(price) {
+ const micros = Number(price?.amountMicros);
+ if (!Number.isFinite(micros) || micros < 0) return undefined;
+ // Two decimals when they hold it; else six, never three: the feeds' parser reads
+ // one mark followed by three digits as thousands ("1.234").
+ const amount = (micros / 1e6).toFixed(Number.isInteger(micros / 1e4) ? 2 : 6);
+ return `${amount} ${price.currencyCode ?? ''}`.trim();
+}
+
+/** A product's feed attributes (lib/feeds/item.js), from its `productAttributes`. */
+export function attributesOf(product) {
+ const a = product.productAttributes ?? {};
+ return {
+ id: product.offerId,
+ item_group_id: a.itemGroupId,
+ title: a.title,
+ description: a.description,
+ link: a.link,
+ image_link: a.imageLink,
+ additional_image_link: a.additionalImageLinks,
+ brand: a.brand,
+ gtin: a.gtins?.[0],
+ mpn: a.mpn,
+ price: moneyText(a.price),
+ sale_price: moneyText(a.salePrice),
+ availability: a.availability,
+ condition: a.condition,
+ color: a.color,
+ size: a.size,
+ material: a.material,
+ product_type: a.productTypes?.[0],
+ };
+}
+
+const text = (value, max) =>
+ typeof value === 'string' && value.trim() ? value.trim().slice(0, max) : null;
+
+/** A product's issues, as `catalog_item_issues` (00296) keeps them. Unknown severities are left out. */
+export function issuesOf(product) {
+ const externalId = text(product.offerId, 200);
+ if (!externalId) return [];
+ const out = [];
+ for (const issue of product.productStatus?.itemLevelIssues ?? []) {
+ if (out.length >= MAX_LISTING_ISSUES) break;
+ const severity = SEVERITIES[issue?.severity];
+ const code = text(issue?.code, 200);
+ if (!severity || !code) continue;
+ out.push({
+ externalId,
+ code,
+ severity,
+ attribute: text(issue.attribute, 200),
+ reportingContext: text(issue.reportingContext, 100),
+ description: text(issue.description, 1000),
+ detail: text(issue.detail, 2000),
+ documentation: text(issue.documentation, 2048),
+ resolution: text(issue.resolution, 100),
+ countries: (issue.applicableCountries ?? [])
+ .filter((c) => typeof c === 'string' && /^[A-Z]{2}$/.test(c))
+ .slice(0, MAX_COUNTRIES),
+ });
+ }
+ return out;
+}
+
+/** A listing's place among its offer's others (listingOrder). */
+function rankOf(product) {
+ return {
+ disapproved: (product?.productStatus?.itemLevelIssues ?? []).some(
+ (issue) => issue?.severity === 'DISAPPROVED',
+ ),
+ feedLabel: String(product?.feedLabel ?? ''),
+ contentLanguage: String(product?.contentLanguage ?? ''),
+ };
+}
+
+const compare = (a, b) => (a < b ? -1 : a > b ? 1 : 0);
+
+/**
+ * The order of an offer's listings, the one the catalog keeps first, when
+ * Merchant Center lists the offer again under another language or feed
+ * label (the same offer id: the catalog keys items by id, so it keeps one):
+ * a listing with no disapproved issue first, then the first by feed label,
+ * then by content language, alphabetically (by code point, whatever the
+ * server's locale). The API names no two listings of an offer with the same
+ * feed label and language, so every sync keeps the same one, whatever order
+ * the API pages them in.
+ */
+export function listingOrder(a, b) {
+ return (
+ Number(a.disapproved) - Number(b.disapproved) ||
+ compare(a.feedLabel, b.feedLabel) ||
+ compare(a.contentLanguage, b.contentLanguage)
+ );
+}
+
+/**
+ * The issues of every listing of every offer, one row per issue: the same
+ * issue (code, destination, attribute and severity) of the same offer under
+ * another language or feed label adds its countries to the row. The row's
+ * words are its first listing's (listingOrder), and its countries come first.
+ *
+ * @param {{ rank: object, issue: ReturnType<typeof issuesOf>[number] }[]} found
+ */
+export function mergeIssues(found) {
+ const groups = new Map();
+ for (const entry of found) {
+ const { externalId, code, reportingContext, attribute, severity } = entry.issue;
+ const key = JSON.stringify([externalId, code, reportingContext, attribute, severity]);
+ const group = groups.get(key);
+ if (group) group.push(entry);
+ else groups.set(key, [entry]);
+ }
+ return [...groups.values()].map((group) => {
+ group.sort((a, b) => listingOrder(a.rank, b.rank));
+ const countries = [...new Set(group.flatMap((entry) => entry.issue.countries))];
+ return { ...group[0].issue, countries: countries.slice(0, MAX_COUNTRIES) };
+ });
+}
+
+/** The accounts on one page of `accounts.list` or `:listSubaccounts`. */
+const accountsOf = (data) =>
+ (data?.accounts ?? [])
+ .map((a) => ({ id: accountId(a.accountId ?? a.name), name: text(a.accountName, 200) }))
+ .filter((a) => a.id);
+
+async function listPages(request, url, max) {
+ const out = [];
+ let pageToken;
+ do {
+ const { data } = await request(url, { query: { pageSize: 500, pageToken } });
+ out.push(...accountsOf(data));
+ pageToken = data?.nextPageToken || undefined;
+ } while (pageToken && out.length < max);
+ return out.slice(0, max);
+}
+
+export default {
+ kind: 'merchant_center',
+ label: 'Google Merchant Center',
+ signIn: 'google-merchant-center',
+ hosts: ['merchantapi.googleapis.com'],
+
+ /**
+ * The Merchant Center accounts the sign-in opens, for the person to pick
+ * the one that sells this brand's products: the ones it is added to, and
+ * the sub-accounts of the first MAX_PARENTS (an advanced account's own,
+ * which `accounts.list` doesn't return unless the person is added to them).
+ *
+ * @returns {Promise<{ id: string, name: string|null }[]>}
+ */
+ async listAccounts({ request }) {
+ const direct = await listPages(request, `${API}/accounts/v1/accounts`, MAX_ACCOUNTS);
+ const seen = new Map(direct.map((a) => [a.id, a]));
+ for (const parent of direct.slice(0, MAX_PARENTS)) {
+ if (seen.size >= MAX_ACCOUNTS) break;
+ try {
+ const subs = await listPages(
+ request,
+ `${API}/accounts/v1/accounts/${parent.id}:listSubaccounts`,
+ MAX_ACCOUNTS,
+ );
+ for (const sub of subs) if (!seen.has(sub.id)) seen.set(sub.id, sub);
+ } catch (err) {
+ // A standalone account has no sub-accounts to list: Google answers that account with
+ // an error. Anything else ends the listing: the sign-in gone or refused, this server's
+ // setup refused or Composio not reached (no status), the time up, or Google asking
+ // to wait, which the next account's call would only make worse.
+ const ownAnswer =
+ err instanceof ConnectorHttpError &&
+ err.status &&
+ err.code !== 'auth' &&
+ err.code !== 'rate_limited';
+ if (!ownAnswer) throw err;
+ }
+ }
+ return [...seen.values()].slice(0, MAX_ACCOUNTS);
+ },
+
+ /** The account the person picked, checked with the sign-in: `{ account, label, url }`. */
+ async checkAccount(value, { request }) {
+ const id = accountId(value);
+ if (!id) throw new ConnectError("That isn't a Merchant Center account id.");
+ let data;
+ try {
+ ({ data } = await request(`${API}/accounts/v1/accounts/${id}`));
+ } catch (err) {
+ if (err instanceof ConnectorHttpError && (err.status === 403 || err.status === 404)) {
+ throw new ConnectError("This Google sign-in can't open that Merchant Center account.");
+ }
+ throw err;
+ }
+ return {
+ account: id,
+ label: text(data?.accountName, 200),
+ url: `https://merchants.google.com/mc/overview?a=${id}`,
+ };
+ },
+
+ /**
+ * The account's products as the catalog's items, and their issues. An
+ * offer listed again under another language or feed label (the same offer
+ * id) is one item: its listing first by listingOrder, whatever page it's
+ * on. The others are counted (`repeated`), not skipped, and their issues
+ * are kept with the item's (mergeIssues). Past MAX_PAGES the rest isn't
+ * read, and the first error says so.
+ */
+ async readCatalog(source, { request }) {
+ const id = accountId(source.account);
+ if (!id) throw new ConnectError('The source has no Merchant Center account id.');
+ // Per offer id, its first listing so far (listingOrder), as an item or the error it makes.
+ // A listing with no offer id makes an error of its own.
+ const listings = new Map();
+ const found = [];
+ let repeated = 0;
+ let row = 0;
+ let issuesLeft = 0;
+ let pageToken;
+ for (let page = 0; page < MAX_PAGES; page += 1) {
+ const { data } = await request(`${API}/products/v1/accounts/${id}/products`, {
+ query: { pageSize: PAGE_SIZE, pageToken },
+ });
+ // Google leaves `products` out for an account with none; any other shape isn't an answer.
+ const isObject = data !== null && typeof data === 'object' && !Array.isArray(data);
+ if (!isObject || (data.products !== undefined && !Array.isArray(data.products))) {
+ throw new ConnectorHttpError(
+ 'bad_response',
+ 'Merchant Center answered with no product list.',
+ );
+ }
+ for (const product of data.products ?? []) {
+ row += 1;
+ const rank = rankOf(product);
+ // Its issues are kept even when the product can't be: a missing price is one.
+ const issues = issuesOf(product);
+ const room = Math.max(0, MAX_ISSUES - found.length);
+ issuesLeft += Math.max(0, issues.length - room);
+ found.push(...issues.slice(0, room).map((issue) => ({ rank, issue })));
+ const listing = { rank, ...toItem(attributesOf(product), row, 'Item') };
+ const key = text(product?.offerId, 200) ?? Symbol('no offer id');
+ const kept = listings.get(key);
+ if (!kept) {
+ listings.set(key, listing);
+ continue;
+ }
+ repeated += 1;
+ if (listingOrder(rank, kept.rank) < 0) listings.set(key, listing);
+ }
+ pageToken = data.nextPageToken || undefined;
+ if (!pageToken) break;
+ }
+ const items = [];
+ const errors = [];
+ for (const listing of listings.values()) {
+ if (listing.item) items.push(listing.item);
+ else errors.push(listing.error);
+ }
+ errors.sort((a, b) => a.row - b.row);
+ if (issuesLeft) {
+ errors.unshift({
+ row: 0,
+ field: 'issues',
+ found: '',
+ message: `Only the first ${MAX_ISSUES.toLocaleString('en-US')} issues were kept: ${issuesLeft.toLocaleString('en-US')} more weren't.`,
+ });
+ }
+ if (pageToken) {
+ // First, so it's among the errors the source keeps. The products not read are marked
+ // removed by the sync, like any the account no longer lists.
+ errors.unshift({
+ row: 0,
+ field: 'pages',
+ found: '',
+ message: `Only the first ${(MAX_PAGES * PAGE_SIZE).toLocaleString('en-US')} products were read: the rest aren't in the catalog.`,
+ });
+ }
+ return { items, errors, issues: mergeIssues(found), repeated, format: 'merchant_center' };
+ },
+
+ /** Merchant Center has no orders. */
+ async syncOrders() {
+ return { orders: [] };
+ },
+};
diff --git a/server/src/connectors/merchant-center/merchant-center.test.js b/server/src/connectors/merchant-center/merchant-center.test.js
new file mode 100644
index 000000000..a99f31ab3
--- /dev/null
+++ b/server/src/connectors/merchant-center/merchant-center.test.js
@@ -0,0 +1,528 @@
+import { readFileSync } from 'node:fs';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+// The plumbing reaches the Supabase admin client at import time, whose config
+// exits without its env (as in CI). Every connector test starts like this.
+vi.mock('../../config/supabase.js', () => ({ default: {} }));
+
+import { signInRequest } from '../../lib/catalog/sign-in.js';
+import { logger } from '../../lib/logger.js';
+import { ConnectError } from '../errors.js';
+import { runSync } from '../testing.js';
+import merchantCenter, {
+ API,
+ MAX_ACCOUNTS,
+ MAX_ISSUES,
+ MAX_LISTING_ISSUES,
+ MAX_PAGES,
+ MAX_PARENTS,
+ PAGE_SIZE,
+ accountId,
+ moneyText,
+} from './index.js';
+
+/**
+ * The Merchant Center connector (#1270) against a fake of the Merchant API,
+ * answered as Composio's proxy would, through the real sign-in request
+ * (lib/catalog/sign-in.js).
+ */
+
+const page = (n) =>
+ JSON.parse(
+ readFileSync(new URL(`./__fixtures__/products-page-${n}.json`, import.meta.url), 'utf8'),
+ );
+
+const ACCOUNT = '123456789';
+const PRODUCTS = `${API}/products/v1/accounts/${ACCOUNT}/products`;
+const ok = (data) => ({ status: 200, headers: {}, data });
+
+/** The workspace's Google sign-in: the proxy answers by endpoint; no route is a 404. */
+function google(routes, { connected = true } = {}) {
+ const proxy = vi.fn(async (_accountId, { endpoint, query }) => {
+ const route = routes[endpoint];
+ return route
+ ? route(query ?? {})
+ : { status: 404, headers: {}, data: { error: { code: 404 } } };
+ });
+ const active = vi.fn(async () => (connected ? { id: 'ca_1' } : null));
+ const factory = vi.fn((provider, entityId, options) =>
+ signInRequest(provider, entityId, { ...options, active, proxy, wait: async () => {} }),
+ );
+ return { factory, proxy, active };
+}
+
+/** The request the plumbing gives the connector, with the connector's own hosts. */
+const signedIn = (factory) =>
+ factory(merchantCenter.signIn, 'org_org-1', { hosts: merchantCenter.hosts });
+
+const products = () => ({
+ [PRODUCTS]: (query) => ok(query.pageToken === 'page-2' ? page(2) : page(1)),
+});
+
+beforeEach(() => {
+ // The server's Merchant Center sign-in is set up (composio.js isComposioConfigured).
+ vi.stubEnv('COMPOSIO_API_KEY', 'test-composio-key');
+ vi.stubEnv('COMPOSIO_GMC_AUTH_CONFIG_ID', 'ac_gmc');
+ // A refused sign-in is logged as a warning (sign-in.js): kept out of the run's output.
+ vi.spyOn(logger, 'warn').mockImplementation(() => {});
+});
+afterEach(() => {
+ vi.unstubAllEnvs();
+ vi.restoreAllMocks();
+});
+
+describe('moneyText', () => {
+ it("writes the API's micros as the feeds write a price", () => {
+ expect(moneyText({ amountMicros: '129990000', currencyCode: 'USD' })).toBe('129.99 USD');
+ expect(moneyText({ amountMicros: '1234500000', currencyCode: 'EUR' })).toBe('1234.50 EUR');
+ // Three decimals would read as thousands: six are written.
+ expect(moneyText({ amountMicros: '12345000', currencyCode: 'KWD' })).toBe('12.345000 KWD');
+ expect(moneyText(undefined)).toBeUndefined();
+ expect(moneyText({ amountMicros: '-1' })).toBeUndefined();
+ });
+});
+
+describe('accountId', () => {
+ it('reads an id as a person or the API gives it', () => {
+ expect(accountId('accounts/123456789')).toBe('123456789');
+ expect(accountId(' 123456789 ')).toBe('123456789');
+ expect(accountId('my-shop')).toBeNull();
+ });
+});
+
+describe('readCatalog, through the real plumbing', () => {
+ it("reads every page into the catalog, with each item's issues", async () => {
+ const { factory, proxy } = google(products());
+ const {
+ outcome,
+ products: rows,
+ variants,
+ issues,
+ } = await runSync(merchantCenter, { account: ACCOUNT }, { signInRequest: factory });
+
+ // The workspace's Merchant Center sign-in, a page at a time, at the API's largest page.
+ expect(factory).toHaveBeenCalledWith('google-merchant-center', 'org_org-1', {
+ hosts: ['merchantapi.googleapis.com'],
+ });
+ expect(proxy.mock.calls.map(([, call]) => call.query)).toEqual([
+ { pageSize: PAGE_SIZE, pageToken: undefined },
+ { pageSize: PAGE_SIZE, pageToken: 'page-2' },
+ ]);
+
+ expect(outcome).toMatchObject({
+ status: 'ok',
+ counts: {
+ format: 'merchant_center',
+ items: 2,
+ // A product with no price; the German listing of an offer read already is counted apart.
+ skipped: 1,
+ repeated: 1,
+ issues: { items: 2, disapproved: 2, demoted: 1, not_impacted: 1 },
+ },
+ });
+ expect(outcome.counts.errors).toEqual(['Item 4: price is missing.']);
+
+ // Colors of one item group are one product.
+ expect(rows).toHaveLength(1);
+ expect(rows[0]).toMatchObject({
+ external_id: 'TB',
+ product_brand: 'Acme',
+ product_type: 'Shoes > Boots',
+ });
+ const brown = variants.find((v) => v.external_id === 'TB-42-BRN');
+ expect(brown).toMatchObject({
+ gtin: '00012345678905',
+ mpn: 'TB-42',
+ price: 129.99,
+ sale_price: 99,
+ currency: 'USD',
+ availability: 'in_stock',
+ condition: 'new',
+ attributes: { color: 'Brown', size: '42', material: 'Leather' },
+ });
+ expect(variants.find((v) => v.external_id === 'TB-43-BLK')).toMatchObject({
+ availability: 'out_of_stock',
+ });
+
+ // An item with no issues has none; one the catalog can't keep still has its own; and the
+ // German listing's are the item's too: the same issue once, with both listings' countries.
+ expect(issues.map((i) => [i.external_id, i.code, i.severity, i.reporting_context])).toEqual([
+ ['TB-42-BRN', 'image_too_small', 'demoted', 'SHOPPING_ADS'],
+ ['TB-42-BRN', 'image_too_small', 'not_impacted', 'FREE_LISTINGS'],
+ ['TB-42-BRN', 'landing_page_error', 'disapproved', 'SHOPPING_ADS'],
+ ['SOCK-1', 'missing_price', 'disapproved', 'SHOPPING_ADS'],
+ ]);
+ expect(issues[0]).toMatchObject({
+ brand_id: 'brand-1',
+ attribute: 'image link',
+ description: 'Image too small',
+ detail: 'Use an image of at least 100 x 100 pixels',
+ resolution: 'merchant_action',
+ countries: ['US', 'CA', 'DE'],
+ });
+ });
+
+ it('asks to reconnect when the workspace has no active Google sign-in, and reads nothing', async () => {
+ const { factory, proxy } = google(products(), { connected: false });
+ const {
+ outcome,
+ products: rows,
+ issues,
+ } = await runSync(merchantCenter, { account: ACCOUNT }, { signInRequest: factory });
+ expect(proxy).not.toHaveBeenCalled();
+ expect(outcome).toEqual({
+ status: 'auth_failed',
+ error:
+ "The Google sign-in can't open this Google Merchant Center account anymore. Reconnect it in Settings › Integrations.",
+ });
+ expect(rows).toHaveLength(0);
+ expect(issues).toHaveLength(0);
+ });
+
+ it("says so, and calls nothing, when this server hasn't the sign-in set up", async () => {
+ vi.stubEnv('COMPOSIO_GMC_AUTH_CONFIG_ID', '');
+ const { factory } = google(products());
+ const { outcome } = await runSync(
+ merchantCenter,
+ { account: ACCOUNT },
+ { signInRequest: factory },
+ );
+ expect(factory).not.toHaveBeenCalled();
+ expect(outcome).toEqual({
+ status: 'failed',
+ error: "Google Merchant Center isn't set up on this server.",
+ });
+ });
+
+ it('asks to reconnect when Google refuses the sign-in', async () => {
+ const { factory } = google({
+ [PRODUCTS]: () => ({
+ status: 401,
+ headers: {},
+ data: { error: { status: 'UNAUTHENTICATED' } },
+ }),
+ });
+ const { outcome } = await runSync(
+ merchantCenter,
+ { account: ACCOUNT },
+ { signInRequest: factory },
+ );
+ expect(outcome.status).toBe('auth_failed');
+ });
+
+ it('waits on a 429 and reads on', async () => {
+ let calls = 0;
+ const { factory } = google({
+ [PRODUCTS]: () => {
+ calls += 1;
+ return calls === 1
+ ? { status: 429, headers: { 'Retry-After': '2' }, data: null }
+ : ok({ products: page(1).products });
+ },
+ });
+ const { outcome } = await runSync(
+ merchantCenter,
+ { account: ACCOUNT },
+ { signInRequest: factory },
+ );
+ expect(calls).toBe(2);
+ expect(outcome).toMatchObject({ status: 'ok', counts: { items: 2 } });
+ });
+});
+
+describe('readCatalog, an offer listed again', () => {
+ /** One listing of an offer, under a language and a feed label, with an issue in its countries. */
+ const listing = (
+ offerId,
+ contentLanguage,
+ feedLabel,
+ countries,
+ { disapproved = false } = {},
+ ) => ({
+ offerId,
+ contentLanguage,
+ feedLabel,
+ productAttributes: {
+ title: `Boot ${contentLanguage}-${feedLabel}`,
+ link: `https://shop.example/${contentLanguage}/boot`,
+ price: { amountMicros: '10000000', currencyCode: 'USD' },
+ availability: 'IN_STOCK',
+ },
+ productStatus: {
+ itemLevelIssues: [
+ {
+ code: 'image_too_small',
+ severity: 'DEMOTED',
+ attribute: 'image link',
+ reportingContext: 'SHOPPING_ADS',
+ description: `Words of ${contentLanguage}-${feedLabel}`,
+ applicableCountries: countries,
+ },
+ ...(disapproved
+ ? [
+ {
+ code: 'landing_page_error',
+ severity: 'DISAPPROVED',
+ attribute: 'link',
+ reportingContext: 'SHOPPING_ADS',
+ applicableCountries: countries,
+ },
+ ]
+ : []),
+ ],
+ },
+ });
+
+ /** The API, its products in pages of `size`, paged by the page's index. */
+ const paged = (all, size) =>
+ vi.fn(async (_url, { query }) => {
+ const at = Number(query.pageToken ?? 0);
+ const next = at + 1;
+ return {
+ status: 200,
+ headers: {},
+ data: {
+ products: all.slice(at * size, next * size),
+ nextPageToken: next * size < all.length ? String(next) : undefined,
+ },
+ };
+ });
+
+ const permutations = (list) =>
+ list.length <= 1
+ ? [list]
+ : list.flatMap((first, i) =>
+ permutations([...list.slice(0, i), ...list.slice(i + 1)]).map((rest) => [first, ...rest]),
+ );
+
+ it('keeps the same listing, and the same issues, whatever order the API pages them in', async () => {
+ const listings = [
+ // First by feed label, but disapproved: a listing that isn't comes first.
+ listing('BOOT', 'de', 'AT', ['AT'], { disapproved: true }),
+ listing('BOOT', 'en', 'US', ['US']),
+ listing('BOOT', 'fr', 'EU', ['FR', 'BE']),
+ // The same feed label as the French: the first language.
+ listing('BOOT', 'de', 'EU', ['DE', 'AT']),
+ ];
+ for (const order of permutations(listings)) {
+ const read = await merchantCenter.readCatalog(
+ { account: ACCOUNT },
+ { request: paged(order, 2) },
+ );
+ expect(read.items.map((item) => item.title)).toEqual(['Boot de-EU']);
+ expect(read.errors).toEqual([]);
+ expect(read.repeated).toBe(3);
+ expect(read.issues).toEqual([
+ expect.objectContaining({
+ externalId: 'BOOT',
+ code: 'image_too_small',
+ severity: 'demoted',
+ description: 'Words of de-EU',
+ countries: ['DE', 'AT', 'FR', 'BE', 'US'],
+ }),
+ expect.objectContaining({
+ externalId: 'BOOT',
+ code: 'landing_page_error',
+ severity: 'disapproved',
+ countries: ['AT'],
+ }),
+ ]);
+ }
+ });
+
+ it('says so, first, when it stops at MAX_PAGES', async () => {
+ const request = vi.fn(async (_url, { query }) => {
+ const n = Number(query.pageToken ?? 0);
+ return {
+ status: 200,
+ headers: {},
+ data: { products: [listing(`O-${n}`, 'en', 'US', ['US'])], nextPageToken: String(n + 1) },
+ };
+ });
+ const read = await merchantCenter.readCatalog({ account: ACCOUNT }, { request });
+ expect(request).toHaveBeenCalledTimes(MAX_PAGES);
+ expect(read.items).toHaveLength(MAX_PAGES);
+ expect(read.errors.map((e) => e.message)).toEqual([
+ "Only the first 50,000 products were read: the rest aren't in the catalog.",
+ ]);
+ });
+
+ it('keeps MAX_LISTING_ISSUES of a listing, and MAX_ISSUES of a read, and says how many it left', async () => {
+ const withIssues = (offerId, count) => ({
+ ...listing(offerId, 'en', 'US', ['US']),
+ productStatus: {
+ itemLevelIssues: Array.from({ length: count }, (_, i) => ({
+ code: `issue_${i}`,
+ severity: 'DEMOTED',
+ applicableCountries: ['US'],
+ })),
+ },
+ });
+ const one = await merchantCenter.readCatalog(
+ { account: ACCOUNT },
+ { request: async () => ok({ products: [withIssues('BOOT', 500)] }) },
+ );
+ expect(one.issues).toHaveLength(MAX_LISTING_ISSUES);
+ expect(one.errors).toEqual([]);
+ // Each listing at its most: the read stops keeping them at MAX_ISSUES.
+ const listings = MAX_ISSUES / MAX_LISTING_ISSUES + 10;
+ const request = vi.fn(async (_url, { query }) => {
+ const n = Number(query.pageToken ?? 0);
+ const products = Array.from({ length: PAGE_SIZE }, (_, i) =>
+ withIssues(`O-${n * PAGE_SIZE + i}`, MAX_LISTING_ISSUES),
+ ).slice(0, Math.max(0, listings - n * PAGE_SIZE));
+ const more = (n + 1) * PAGE_SIZE < listings;
+ return ok({ products, nextPageToken: more ? String(n + 1) : '' });
+ });
+ const many = await merchantCenter.readCatalog({ account: ACCOUNT }, { request });
+ expect(many.issues).toHaveLength(MAX_ISSUES);
+ expect(many.errors.map((e) => e.message)).toEqual([
+ `Only the first 100,000 issues were kept: ${(10 * MAX_LISTING_ISSUES).toLocaleString('en-US')} more weren't.`,
+ ]);
+ });
+
+ it("reads an answer that isn't a product list as a failed read, not an empty account", async () => {
+ for (const data of ['<html>', null, [], { products: 'none' }]) {
+ await expect(
+ merchantCenter.readCatalog({ account: ACCOUNT }, { request: async () => ok(data) }),
+ ).rejects.toMatchObject({ code: 'bad_response' });
+ }
+ // No products at all: Google leaves the list out.
+ const none = await merchantCenter.readCatalog(
+ { account: ACCOUNT },
+ { request: async () => ok({}) },
+ );
+ expect(none.items).toEqual([]);
+ });
+});
+
+describe('listAccounts', () => {
+ it("lists the accounts the sign-in opens, and an advanced account's sub-accounts", async () => {
+ const { factory } = google({
+ [`${API}/accounts/v1/accounts`]: () =>
+ ok({
+ accounts: [
+ { name: 'accounts/100', accountId: '100', accountName: 'Acme (advanced)' },
+ { name: 'accounts/200', accountId: '200', accountName: 'Other shop' },
+ ],
+ }),
+ [`${API}/accounts/v1/accounts/100:listSubaccounts`]: () =>
+ ok({ accounts: [{ name: 'accounts/101', accountId: '101', accountName: 'Acme US' }] }),
+ // A standalone account has no sub-accounts to list.
+ [`${API}/accounts/v1/accounts/200:listSubaccounts`]: () => ({
+ status: 403,
+ headers: {},
+ data: null,
+ }),
+ });
+ const request = signedIn(factory);
+ await expect(merchantCenter.listAccounts({ request })).resolves.toEqual([
+ { id: '100', name: 'Acme (advanced)' },
+ { id: '200', name: 'Other shop' },
+ { id: '101', name: 'Acme US' },
+ ]);
+ });
+
+ it('says the sign-in is gone when the workspace has none', async () => {
+ const { factory } = google({}, { connected: false });
+ const request = signedIn(factory);
+ await expect(merchantCenter.listAccounts({ request })).rejects.toMatchObject({ code: 'auth' });
+ });
+
+ const ACCOUNTS = `${API}/accounts/v1/accounts`;
+ const subCalls = (proxy) =>
+ proxy.mock.calls.filter(([, call]) => call.endpoint.endsWith(':listSubaccounts'));
+ const many = (n, from = 1000) =>
+ Array.from({ length: n }, (_, i) => ({ accountId: String(from + i), accountName: null }));
+
+ it('reads every page of the list', async () => {
+ const { factory, proxy } = google({
+ [ACCOUNTS]: (query) =>
+ query.pageToken === 'p2'
+ ? ok({ accounts: [{ accountId: '200', accountName: 'B' }] })
+ : ok({ accounts: [{ accountId: '100', accountName: 'A' }], nextPageToken: 'p2' }),
+ });
+ await expect(merchantCenter.listAccounts({ request: signedIn(factory) })).resolves.toEqual([
+ { id: '100', name: 'A' },
+ { id: '200', name: 'B' },
+ ]);
+ expect(
+ proxy.mock.calls.filter(([, call]) => call.endpoint === ACCOUNTS).map(([, c]) => c.query),
+ ).toEqual([
+ { pageSize: 500, pageToken: undefined },
+ { pageSize: 500, pageToken: 'p2' },
+ ]);
+ });
+
+ it('lists MAX_ACCOUNTS at most, and the sub-accounts of MAX_PARENTS', async () => {
+ const full = google({
+ [ACCOUNTS]: () => ok({ accounts: many(MAX_ACCOUNTS), nextPageToken: 'more' }),
+ });
+ await expect(
+ merchantCenter.listAccounts({ request: signedIn(full.factory) }),
+ ).resolves.toHaveLength(MAX_ACCOUNTS);
+ // No second page, and no sub-accounts: the list is full.
+ expect(full.proxy).toHaveBeenCalledTimes(1);
+
+ const parents = google({ [ACCOUNTS]: () => ok({ accounts: many(MAX_PARENTS + 10) }) });
+ await merchantCenter.listAccounts({ request: signedIn(parents.factory) });
+ expect(subCalls(parents.proxy)).toHaveLength(MAX_PARENTS);
+ });
+
+ it('stops when the sign-in goes, or Google asks to wait, while it lists sub-accounts', async () => {
+ const gone = google({
+ [ACCOUNTS]: () => ok({ accounts: many(3) }),
+ [`${ACCOUNTS}/1000:listSubaccounts`]: () => {
+ throw new Error('Connected account is expired');
+ },
+ });
+ await expect(
+ merchantCenter.listAccounts({ request: signedIn(gone.factory) }),
+ ).rejects.toMatchObject({ code: 'auth', status: undefined });
+ expect(subCalls(gone.proxy)).toHaveLength(1);
+
+ const waiting = google({
+ [ACCOUNTS]: () => ok({ accounts: many(3) }),
+ [`${ACCOUNTS}/1000:listSubaccounts`]: () => ({ status: 429, headers: {}, data: null }),
+ });
+ await expect(
+ merchantCenter.listAccounts({ request: signedIn(waiting.factory) }),
+ ).rejects.toMatchObject({ code: 'rate_limited' });
+ // The first account's call and its three tries again; none for the next accounts.
+ expect(subCalls(waiting.proxy).map(([, call]) => call.endpoint)).toEqual(
+ Array(4).fill(`${ACCOUNTS}/1000:listSubaccounts`),
+ );
+ });
+});
+
+describe('checkAccount', () => {
+ const routes = {
+ [`${API}/accounts/v1/accounts/${ACCOUNT}`]: () =>
+ ok({ name: `accounts/${ACCOUNT}`, accountId: ACCOUNT, accountName: 'Acme Outdoor' }),
+ };
+
+ it('confirms the account the person picked, with its name', async () => {
+ const request = signedIn(google(routes).factory);
+ await expect(merchantCenter.checkAccount(`accounts/${ACCOUNT}`, { request })).resolves.toEqual({
+ account: ACCOUNT,
+ label: 'Acme Outdoor',
+ url: `https://merchants.google.com/mc/overview?a=${ACCOUNT}`,
+ });
+ });
+
+ it("says so in plain words when the sign-in can't open it, or it isn't an id", async () => {
+ const request = signedIn(google(routes).factory);
+ await expect(merchantCenter.checkAccount('987654321', { request })).rejects.toThrow(
+ new ConnectError("This Google sign-in can't open that Merchant Center account."),
+ );
+ await expect(merchantCenter.checkAccount('my-shop', { request })).rejects.toBeInstanceOf(
+ ConnectError,
+ );
+ });
+});
+
+describe('syncOrders', () => {
+ it('has none: Merchant Center has no orders', async () => {
+ await expect(merchantCenter.syncOrders()).resolves.toEqual({ orders: [] });
+ });
+});
diff --git a/server/src/connectors/testing.js b/server/src/connectors/testing.js
index 86662ad95..230165438 100644
--- a/server/src/connectors/testing.js
+++ b/server/src/connectors/testing.js
@@ -17,7 +17,10 @@ import { requestJson } from './http.js';
* host, with its waits recorded instead of slept.
* - `runSync(connector, source, { credentials, request })` syncs a source
* through the real plumbing (lib/catalog/sources.js) into an in-memory
- * catalog, and returns the outcome and the rows written.
+ * catalog, and returns the outcome and the rows written. A connector
+ * read with the workspace's sign-in (`signIn`, #1270) gets
+ * `signInRequest` instead: a stand-in for lib/catalog/sign-in.js's,
+ * `(provider, entityId, { hosts }) => request`.
*
* A test file that uses it mocks the Supabase admin client first, which
* exits without its env (see connectors/__example__/example.test.js).
@@ -101,9 +104,13 @@ export async function startFakeApi(routes) {
* run does. A store's `credentials` are saved for it first, encrypted, as
* connecting it would.
*
- * @returns {Promise<{ outcome: object, products: object[], variants: object[], images: object[], categories: object[], fake: ReturnType<typeof fakeDb> }>}
+ * @returns {Promise<{ outcome: object, products: object[], variants: object[], images: object[], categories: object[], issues: object[], fake: ReturnType<typeof fakeDb> }>}
*/
-export async function runSync(connector, source, { credentials, request, now = new Date() } = {}) {
+export async function runSync(
+ connector,
+ source,
+ { credentials, request, signInRequest, now = new Date() } = {},
+) {
// A key for this run, when the test has none: nothing outlives it.
process.env.CONNECTOR_CREDENTIALS_KEY ??= randomBytes(32).toString('base64');
const fake = fakeDb();
@@ -117,11 +124,14 @@ export async function runSync(connector, source, { credentials, request, now = n
...source,
};
fake.rows('catalog_sources').push({ ...row, last_synced_at: null });
+ // The brand's workspace, whose sign-in a `signIn` connector reads with.
+ fake.rows('brands').push({ id: row.brand_id, organization_id: 'org-1' });
if (credentials) await saveCredentials(row.id, credentials, { db: fake.db });
const outcome = await syncSource(row, {
db: fake.db,
now,
request,
+ ...(signInRequest ? { signInRequest } : {}),
connectors: { [connector.kind]: connector },
});
return {
@@ -130,6 +140,7 @@ export async function runSync(connector, source, { credentials, request, now = n
variants: fake.rows('catalog_variants'),
images: fake.rows('catalog_images'),
categories: fake.rows('catalog_categories'),
+ issues: fake.rows('catalog_item_issues'),
fake,
};
}
diff --git a/server/src/lib/catalog/issues.js b/server/src/lib/catalog/issues.js
new file mode 100644
index 000000000..cf15983cd
--- /dev/null
+++ b/server/src/lib/catalog/issues.js
@@ -0,0 +1,57 @@
+import supabaseAdmin from '../../config/supabase.js';
+
+/**
+ * The issues a platform reports per item of a catalog source (#1270,
+ * 00296), Google Merchant Center's today: a sync replaces the source's
+ * rows with what it read. They feed the "Feed fields complete" check and
+ * the readiness cards later; the Settings row counts them now.
+ */
+
+/** Rows written per insert. */
+export const ISSUE_BATCH = 500;
+
+/**
+ * Replaces a source's issues and counts them: the items with at least one,
+ * and the issues per severity.
+ *
+ * @param {{ id: string, brand_id: string }} source
+ * @param {{ externalId: string, code: string, severity: string, attribute: string|null,
+ * reportingContext: string|null, description: string|null, detail: string|null,
+ * documentation: string|null, resolution: string|null, countries: string[] }[]} issues
+ * @returns {Promise<{ items: number, disapproved: number, demoted: number, not_impacted: number }>}
+ */
+export async function saveItemIssues(
+ source,
+ issues,
+ { db = supabaseAdmin, now = new Date() } = {},
+) {
+ const { error: cleared } = await db
+ .from('catalog_item_issues')
+ .delete()
+ .eq('source_id', source.id);
+ if (cleared) throw new Error(`catalog_item_issues delete: ${cleared.message}`);
+ const createdAt = now.toISOString();
+ const rows = issues.map((issue) => ({
+ brand_id: source.brand_id,
+ source_id: source.id,
+ external_id: issue.externalId,
+ code: issue.code,
+ severity: issue.severity,
+ attribute: issue.attribute,
+ reporting_context: issue.reportingContext,
+ description: issue.description,
+ detail: issue.detail,
+ documentation: issue.documentation,
+ resolution: issue.resolution,
+ countries: issue.countries,
+ created_at: createdAt,
+ }));
+ for (let i = 0; i < rows.length; i += ISSUE_BATCH) {
+ const { error } = await db.from('catalog_item_issues').insert(rows.slice(i, i + ISSUE_BATCH));
+ if (error) throw new Error(`catalog_item_issues insert: ${error.message}`);
+ }
+ const counts = { items: 0, disapproved: 0, demoted: 0, not_impacted: 0 };
+ for (const issue of issues) counts[issue.severity] += 1;
+ counts.items = new Set(issues.map((issue) => issue.externalId)).size;
+ return counts;
+}
diff --git a/server/src/lib/catalog/issues.test.js b/server/src/lib/catalog/issues.test.js
new file mode 100644
index 000000000..3cfbb0bdd
--- /dev/null
+++ b/server/src/lib/catalog/issues.test.js
@@ -0,0 +1,84 @@
+import { describe, expect, it, vi } from 'vitest';
+
+vi.mock('../../config/supabase.js', () => ({ default: {} }));
+
+import { fakeDb } from './__fixtures__/fake-db.js';
+import { ISSUE_BATCH, saveItemIssues } from './issues.js';
+
+const source = { id: 'src-1', brand_id: 'brand-1' };
+const issue = (externalId, severity, code = 'missing_gtin') => ({
+ externalId,
+ code,
+ severity,
+ attribute: 'gtin',
+ reportingContext: 'SHOPPING_ADS',
+ description: 'Missing value [gtin]',
+ detail: null,
+ documentation: null,
+ resolution: 'merchant_action',
+ countries: ['US'],
+});
+
+describe('saveItemIssues', () => {
+ it("replaces the source's issues, and leaves another source's alone", async () => {
+ const fake = fakeDb();
+ fake
+ .rows('catalog_item_issues')
+ .push(
+ { id: 'old', source_id: 'src-1', external_id: 'gone' },
+ { id: 'other', source_id: 'src-2', external_id: 'kept' },
+ );
+ await saveItemIssues(source, [issue('A', 'disapproved')], {
+ db: fake.db,
+ now: new Date('2026-10-27T10:00:00Z'),
+ });
+ expect(
+ fake
+ .rows('catalog_item_issues')
+ .map((r) => r.external_id)
+ .sort(),
+ ).toEqual(['A', 'kept']);
+ expect(fake.rows('catalog_item_issues').find((r) => r.external_id === 'A')).toMatchObject({
+ brand_id: 'brand-1',
+ source_id: 'src-1',
+ code: 'missing_gtin',
+ severity: 'disapproved',
+ attribute: 'gtin',
+ reporting_context: 'SHOPPING_ADS',
+ countries: ['US'],
+ created_at: '2026-10-27T10:00:00.000Z',
+ });
+ });
+
+ it('counts the items with issues, and the issues per severity', async () => {
+ const fake = fakeDb();
+ await expect(
+ saveItemIssues(
+ source,
+ [
+ issue('A', 'disapproved'),
+ issue('A', 'demoted', 'image_too_small'),
+ issue('B', 'not_impacted'),
+ ],
+ { db: fake.db },
+ ),
+ ).resolves.toEqual({ items: 2, disapproved: 1, demoted: 1, not_impacted: 1 });
+ await expect(saveItemIssues(source, [], { db: fake.db })).resolves.toEqual({
+ items: 0,
+ disapproved: 0,
+ demoted: 0,
+ not_impacted: 0,
+ });
+ expect(fake.rows('catalog_item_issues')).toHaveLength(0);
+ });
+
+ it('writes a large list in batches', async () => {
+ const fake = fakeDb();
+ const many = Array.from({ length: ISSUE_BATCH + 1 }, (_, i) => issue(`I${i}`, 'demoted'));
+ await saveItemIssues(source, many, { db: fake.db });
+ const inserts = fake.writes.filter(
+ (w) => w.table === 'catalog_item_issues' && w.op === 'insert',
+ );
+ expect(inserts.map((w) => w.count)).toEqual([ISSUE_BATCH, 1]);
+ });
+});
diff --git a/server/src/lib/catalog/sign-in.js b/server/src/lib/catalog/sign-in.js
new file mode 100644
index 000000000..ea1e256ac
--- /dev/null
+++ b/server/src/lib/catalog/sign-in.js
@@ -0,0 +1,197 @@
+import supabaseAdmin from '../../config/supabase.js';
+import { ConnectorHttpError, MAX_WAIT_MS, RETRIES, retryAfterMs } from '../../connectors/http.js';
+import { entityIdFor, getActiveConnection, proxyRequest } from '../composio.js';
+import { logger } from '../logger.js';
+
+/**
+ * Catalog sources read with the workspace's own sign-in to a platform
+ * (#1270): a Google Merchant Center account, through the Google connection
+ * the workspace made in Settings › Integrations, which Composio keeps
+ * (integration_connections, 00045). Such a source has no credentials of its
+ * own (00282): its connector's `signIn` names the Composio provider, and its
+ * calls go through Composio's proxy (composio.js proxyRequest), so the token
+ * never reaches us. They reach only the hosts the connector names (`hosts`).
+ *
+ * The request it gives a connector answers like the HTTP client's
+ * (connectors/http.js requestJson): `{ status, headers, data }` for a 2xx,
+ * else a ConnectorHttpError:
+ * - 'auth': the sign-in is gone or refused: no active connection, Composio
+ * saying the connection expired or was revoked, or the API's 401;
+ * - 'not_set_up': this server's setup is refused, not the sign-in:
+ * Composio's own 401 or 403 (its API key, or a key without the "Proxy
+ * execute" permission), the API's 403 (the Google Cloud project not
+ * registered with Merchant Center, a scope missing), or its 401 when it
+ * says the project isn't registered;
+ * - 'rate_limited': still 429 after RETRIES waits, or asked to wait longer
+ * than MAX_WAIT_MS;
+ * - 'http_error': any other status;
+ * - 'blocked': an address on a host the connector didn't name;
+ * - 'unreachable' or 'timeout': Composio couldn't be reached in time.
+ * Only the API's own answers carry a `status`: Composio's refusals don't.
+ * It waits and tries again on 429 and 5xx, as long as Retry-After says,
+ * else 1, 2, 4 seconds. Errors name the host and path only.
+ *
+ * Every refusal by Composio or the API ('auth', 'not_set_up') is logged as
+ * a warning, for whoever runs the server: the error's name and status, and
+ * the API's own status and reason. Never the token, the query or the
+ * account. A workspace with no connection at all isn't: it's not connected yet.
+ */
+
+/** The Composio entity of a brand's workspace: one connection per workspace and provider. */
+export async function workspaceEntity(brandId, { db = supabaseAdmin } = {}) {
+ const { data, error } = await db
+ .from('brands')
+ .select('organization_id')
+ .eq('id', brandId)
+ .maybeSingle();
+ if (error) throw new Error(`brands read: ${error.message}`);
+ if (!data?.organization_id) throw new Error('The brand has no workspace.');
+ return entityIdFor(data.organization_id);
+}
+
+const RETRY_STATUSES = new Set([429, 500, 502, 503, 504]);
+/**
+ * Composio's own refusals that mean the workspace's connection is gone (it
+ * expired, was revoked or removed), not that this server's setup is wrong:
+ * they name the connection.
+ */
+const SIGN_IN_GONE =
+ /^(?=[\s\S]*(connected[\s_-]?account|connection))(?=[\s\S]*(expired|inactive|not[\s_-]?active|revoked|deleted|not[\s_-]?found))|invalid_grant|re-?auth/i;
+/** Google's 401 when the Google Cloud project isn't registered with Merchant Center (registerGcp). */
+const NOT_REGISTERED = /not registered|registerGcp/i;
+
+/** A wait that ends early when `signal` aborts. */
+function sleep(ms, signal) {
+ return new Promise((resolve) => {
+ const done = () => {
+ clearTimeout(timer);
+ signal?.removeEventListener('abort', done);
+ resolve();
+ };
+ const timer = setTimeout(done, ms);
+ signal?.addEventListener('abort', done, { once: true });
+ });
+}
+
+/** Header names in lower case, as retryAfterMs reads them. */
+function lowerKeys(headers) {
+ return Object.fromEntries(
+ Object.entries(headers ?? {}).map(([name, value]) => [name.toLowerCase(), value]),
+ );
+}
+
+const clip = (value, max = 100) => (typeof value === 'string' ? value.slice(0, max) : undefined);
+
+/** Why Google refused, from its error body: `details[].reason`, or the older `errors[].reason`. */
+function reasonOf(error) {
+ const detail = Array.isArray(error?.details)
+ ? error.details.find((d) => typeof d?.reason === 'string')
+ : undefined;
+ return clip(detail?.reason ?? error?.errors?.[0]?.reason);
+}
+
+/** A refusal, logged: what refused and why. No token, query or account id. */
+function logRefusal(code, { provider, url, name, status, data }) {
+ const error = data?.error;
+ logger.warn(
+ {
+ provider,
+ code,
+ host: url.host,
+ // The path's ids (an account's) left out.
+ path: url.pathname.replace(/\/\d+(?=[/:]|$)/g, '/:id'),
+ err: { name, status },
+ ...(error && typeof error === 'object'
+ ? { google: { status: clip(error.status), reason: reasonOf(error) } }
+ : {}),
+ },
+ '[catalog] sign-in call refused',
+ );
+}
+
+/**
+ * A connector's request for the workspace's sign-in to `provider`. The
+ * connection is looked up once, on the first call.
+ *
+ * @param {string} provider a Composio provider (composio.js PROVIDERS)
+ * @param {string} entityId the workspace's entity (workspaceEntity)
+ * @param {{ hosts?: string[], active?: Function, proxy?: Function, wait?: Function }} [options]
+ * `hosts`: the only hosts it calls, over https (the connector's `hosts`);
+ * `active`, `proxy` and `wait` are the tests' stand-ins
+ */
+export function signInRequest(
+ provider,
+ entityId,
+ { hosts = [], active = getActiveConnection, proxy = proxyRequest, wait = sleep } = {},
+) {
+ const allowed = new Set(hosts.map((host) => host.toLowerCase()));
+ let accountId = null;
+ return async function request(address, { query, signal } = {}) {
+ let url;
+ try {
+ url = new URL(address);
+ } catch {
+ throw new ConnectorHttpError('invalid_url', "The address isn't a valid one");
+ }
+ const where = `GET ${url.host}${url.pathname}`;
+ // url.host carries a port that isn't https's own: such an address isn't allowed either.
+ if (url.protocol !== 'https:' || !allowed.has(url.host)) {
+ // A connector's mistake, not the source's: logged, as the person can't fix it.
+ logger.error({ provider, host: url.host }, '[catalog] sign-in call to a host not allowed');
+ throw new ConnectorHttpError('blocked', `${where}: not a host ${provider} is read at`);
+ }
+ if (!accountId) {
+ let account;
+ try {
+ account = await active(provider, entityId);
+ } catch {
+ throw new ConnectorHttpError('unreachable', `${where}: the sign-in couldn't be looked up`);
+ }
+ if (!account?.id) throw new ConnectorHttpError('auth', `${where}: no active sign-in`);
+ accountId = account.id;
+ }
+ for (let attempt = 0; ; attempt += 1) {
+ if (signal?.aborted) throw new ConnectorHttpError('timeout', `${where} took too long`);
+ let res;
+ try {
+ res = await proxy(accountId, { endpoint: url.href, query, signal });
+ } catch (err) {
+ if (signal?.aborted) throw new ConnectorHttpError('timeout', `${where} took too long`);
+ const status = err?.status ?? err?.statusCode;
+ const refused = { provider, url, name: clip(err?.name), status };
+ if (SIGN_IN_GONE.test(String(err?.message ?? ''))) {
+ logRefusal('auth', refused);
+ throw new ConnectorHttpError('auth', `${where}: the sign-in is gone`);
+ }
+ if (status === 401 || status === 403) {
+ logRefusal('not_set_up', refused);
+ throw new ConnectorHttpError('not_set_up', `${where}: Composio refused this server`);
+ }
+ throw new ConnectorHttpError('unreachable', `${where} couldn't be reached`);
+ }
+ if (res.status >= 200 && res.status < 300) return res;
+ if (res.status === 401 || res.status === 403) {
+ const refused = { provider, url, status: res.status, data: res.data };
+ const notRegistered = NOT_REGISTERED.test(String(res.data?.error?.message ?? ''));
+ const code = res.status === 401 && !notRegistered ? 'auth' : 'not_set_up';
+ logRefusal(code, refused);
+ throw new ConnectorHttpError(code, `${where} answered ${res.status}`, {
+ status: res.status,
+ });
+ }
+ if (RETRY_STATUSES.has(res.status) && attempt < RETRIES) {
+ const ms = retryAfterMs(lowerKeys(res.headers)) ?? 1000 * 2 ** attempt;
+ if (ms <= MAX_WAIT_MS) {
+ await wait(ms, signal);
+ continue;
+ }
+ }
+ if (res.status === 429) {
+ throw new ConnectorHttpError('rate_limited', `${where} asked to wait`, { status: 429 });
+ }
+ throw new ConnectorHttpError('http_error', `${where} answered ${res.status}`, {
+ status: res.status,
+ });
+ }
+ };
+}
diff --git a/server/src/lib/catalog/sign-in.test.js b/server/src/lib/catalog/sign-in.test.js
new file mode 100644
index 000000000..70d36d6bd
--- /dev/null
+++ b/server/src/lib/catalog/sign-in.test.js
@@ -0,0 +1,278 @@
+import { afterEach, describe, expect, it, vi } from 'vitest';
+
+vi.mock('../../config/supabase.js', () => ({ default: {} }));
+
+import { ConnectorHttpError } from '../../connectors/http.js';
+import { logger } from '../logger.js';
+import { fakeDb } from './__fixtures__/fake-db.js';
+import { signInRequest, workspaceEntity } from './sign-in.js';
+
+const HOST = 'merchantapi.googleapis.com';
+const URL_ = `https://${HOST}/products/v1/accounts/1/products`;
+
+afterEach(() => {
+ vi.restoreAllMocks();
+});
+
+function setup(answers, { active = async () => ({ id: 'ca_1' }), hosts = [HOST], wait } = {}) {
+ const queue = [...answers];
+ const proxy = vi.fn(async () => {
+ const next = queue.shift();
+ if (next instanceof Error) throw next;
+ return next;
+ });
+ const waits = vi.fn(wait ?? (async () => {}));
+ const request = signInRequest('google-merchant-center', 'org_1', {
+ hosts,
+ active: vi.fn(active),
+ proxy,
+ wait: waits,
+ });
+ return { request, proxy, wait: waits };
+}
+
+const answer = (status, headers = {}, data = status < 300 ? { ok: true } : null) => ({
+ status,
+ headers,
+ data,
+});
+
+/** An error as Composio's client throws its own refusals: its name, its status, its body in the message. */
+const composioError = (name, status, message) =>
+ Object.assign(new Error(`${status} ${message}`), { name, status });
+
+describe('signInRequest', () => {
+ it('calls the API through the active connection, looked up once', async () => {
+ const active = vi.fn(async () => ({ id: 'ca_1' }));
+ const proxy = vi.fn(async () => answer(200));
+ const request = signInRequest('google-merchant-center', 'org_1', {
+ hosts: [HOST],
+ active,
+ proxy,
+ });
+ await request(URL_, { query: { pageSize: 1000 } });
+ await request(URL_);
+ expect(active).toHaveBeenCalledTimes(1);
+ expect(active).toHaveBeenCalledWith('google-merchant-center', 'org_1');
+ expect(proxy).toHaveBeenCalledWith('ca_1', {
+ endpoint: URL_,
+ query: { pageSize: 1000 },
+ signal: undefined,
+ });
+ });
+
+ it("says the sign-in is gone when there's no active connection, Composio says it expired, or Google refuses it", async () => {
+ vi.spyOn(logger, 'warn').mockImplementation(() => {});
+ await expect(setup([], { active: async () => null }).request(URL_)).rejects.toMatchObject({
+ code: 'auth',
+ });
+ await expect(setup([answer(401)]).request(URL_)).rejects.toMatchObject({
+ code: 'auth',
+ status: 401,
+ });
+ // Composio refusing an expired or revoked connection is the same, with no status of Google's.
+ const expired = await setup([new Error('Connected account is expired')])
+ .request(URL_)
+ .catch((e) => e);
+ expect(expired).toMatchObject({ code: 'auth', status: undefined });
+ await expect(
+ setup([
+ composioError('BadRequestError', 400, '{"error":{"slug":"ConnectedAccountRevoked"}}'),
+ ]).request(URL_),
+ ).rejects.toMatchObject({ code: 'auth' });
+ });
+
+ it("says this server's setup is refused, not the sign-in, on Composio's own 401 or 403 and Google's 403", async () => {
+ const warn = vi.spyOn(logger, 'warn').mockImplementation(() => {});
+ // Composio refusing the server's API key, or a key without the "Proxy execute" permission.
+ const badKey = await setup([
+ composioError('AuthenticationError', 401, '{"error":{"message":"Unauthorized"}}'),
+ ])
+ .request(URL_)
+ .catch((e) => e);
+ expect(badKey).toMatchObject({ code: 'not_set_up', status: undefined });
+ await expect(
+ setup([
+ composioError('PermissionDeniedError', 403, '{"error":{"message":"No permission"}}'),
+ ]).request(URL_),
+ ).rejects.toMatchObject({ code: 'not_set_up', status: undefined });
+
+ // Google: a scope missing, or the Cloud project not registered with Merchant Center.
+ const scope = {
+ error: {
+ code: 403,
+ status: 'PERMISSION_DENIED',
+ message: 'Request had insufficient authentication scopes.',
+ details: [
+ {
+ '@type': 'type.googleapis.com/google.rpc.ErrorInfo',
+ reason: 'ACCESS_TOKEN_SCOPE_INSUFFICIENT',
+ },
+ ],
+ },
+ };
+ await expect(setup([answer(403, {}, scope)]).request(URL_)).rejects.toMatchObject({
+ code: 'not_set_up',
+ status: 403,
+ });
+ const unregistered = {
+ error: {
+ code: 401,
+ status: 'UNAUTHENTICATED',
+ message: 'GCP project with id p-1 is not registered with the merchant account.',
+ },
+ };
+ await expect(setup([answer(401, {}, unregistered)]).request(URL_)).rejects.toMatchObject({
+ code: 'not_set_up',
+ status: 401,
+ });
+
+ // Each is logged: its name and status, Google's own status and reason; never the query.
+ expect(warn.mock.calls.map(([fields]) => fields)).toEqual([
+ expect.objectContaining({
+ code: 'not_set_up',
+ host: HOST,
+ path: '/products/v1/accounts/:id/products',
+ err: { name: 'AuthenticationError', status: 401 },
+ }),
+ expect.objectContaining({
+ code: 'not_set_up',
+ err: { name: 'PermissionDeniedError', status: 403 },
+ }),
+ expect.objectContaining({
+ code: 'not_set_up',
+ err: { name: undefined, status: 403 },
+ google: { status: 'PERMISSION_DENIED', reason: 'ACCESS_TOKEN_SCOPE_INSUFFICIENT' },
+ }),
+ expect.objectContaining({
+ code: 'not_set_up',
+ err: { name: undefined, status: 401 },
+ google: { status: 'UNAUTHENTICATED', reason: undefined },
+ }),
+ ]);
+ });
+
+ it('logs a sign-in refused, without the query', async () => {
+ const warn = vi.spyOn(logger, 'warn').mockImplementation(() => {});
+ await setup([answer(401, {}, { error: { status: 'UNAUTHENTICATED' } })])
+ .request(URL_, { query: { pageToken: 'secret-ish' } })
+ .catch(() => {});
+ expect(warn).toHaveBeenCalledTimes(1);
+ expect(warn.mock.calls[0][0]).toMatchObject({
+ provider: 'google-merchant-center',
+ code: 'auth',
+ err: { status: 401 },
+ google: { status: 'UNAUTHENTICATED' },
+ });
+ expect(JSON.stringify(warn.mock.calls)).not.toContain('secret-ish');
+ });
+
+ it('waits as Retry-After says on a 429, then 1, 2, 4 seconds, and gives up after three', async () => {
+ const { request, wait } = setup([
+ answer(429, { 'Retry-After': '3' }),
+ answer(503),
+ answer(200),
+ ]);
+ await expect(request(URL_)).resolves.toMatchObject({ status: 200 });
+ expect(wait.mock.calls.map(([ms]) => ms)).toEqual([3000, 2000]);
+
+ const stuck = setup([answer(429), answer(429), answer(429), answer(429)]);
+ await expect(stuck.request(URL_)).rejects.toMatchObject({ code: 'rate_limited', status: 429 });
+ expect(stuck.wait).toHaveBeenCalledTimes(3);
+ });
+
+ it('gives up at once when Retry-After asks for more than a minute', async () => {
+ const limited = setup([answer(429, { 'retry-after': '120' }), answer(200)]);
+ await expect(limited.request(URL_)).rejects.toMatchObject({
+ code: 'rate_limited',
+ status: 429,
+ });
+ expect(limited.wait).not.toHaveBeenCalled();
+ expect(limited.proxy).toHaveBeenCalledTimes(1);
+
+ const down = setup([answer(503, { 'Retry-After': '61' }), answer(200)]);
+ await expect(down.request(URL_)).rejects.toMatchObject({ code: 'http_error', status: 503 });
+ expect(down.wait).not.toHaveBeenCalled();
+ });
+
+ it('calls only the hosts it was given, over https', async () => {
+ const errorLog = vi.spyOn(logger, 'error').mockImplementation(() => {});
+ for (const address of [
+ 'https://evil.example/products/v1/accounts/1/products',
+ `http://${HOST}/products/v1/accounts/1/products`,
+ `https://${HOST}:8443/products/v1/accounts/1/products`,
+ ]) {
+ const { request, proxy } = setup([answer(200)]);
+ await expect(request(address)).rejects.toMatchObject({ code: 'blocked' });
+ expect(proxy).not.toHaveBeenCalled();
+ }
+ // A connector that names no host calls none.
+ const proxy = vi.fn(async () => answer(200));
+ const unnamed = signInRequest('google-merchant-center', 'org_1', {
+ active: async () => ({ id: 'ca_1' }),
+ proxy,
+ });
+ await expect(unnamed(URL_)).rejects.toMatchObject({ code: 'blocked' });
+ expect(proxy).not.toHaveBeenCalled();
+ await expect(setup([answer(200)]).request('/products')).rejects.toMatchObject({
+ code: 'invalid_url',
+ });
+ expect(errorLog).toHaveBeenCalledWith(
+ { provider: 'google-merchant-center', host: 'evil.example' },
+ '[catalog] sign-in call to a host not allowed',
+ );
+ });
+
+ it('names the host and path only, never the query', async () => {
+ const err = await setup([answer(400)])
+ .request(URL_, { query: { pageToken: 'secret-ish' } })
+ .catch((e) => e);
+ expect(err).toBeInstanceOf(ConnectorHttpError);
+ expect(err).toMatchObject({ code: 'http_error', status: 400 });
+ expect(err.message).toBe(
+ 'GET merchantapi.googleapis.com/products/v1/accounts/1/products answered 400',
+ );
+ });
+
+ it("says Composio couldn't be reached, or took too long", async () => {
+ await expect(
+ setup([], {
+ active: async () => {
+ throw new Error('ECONNRESET');
+ },
+ }).request(URL_),
+ ).rejects.toMatchObject({ code: 'unreachable' });
+ await expect(setup([new Error('socket hang up')]).request(URL_)).rejects.toMatchObject({
+ code: 'unreachable',
+ });
+ const controller = new AbortController();
+ controller.abort();
+ await expect(
+ setup([new Error('aborted')]).request(URL_, { signal: controller.signal }),
+ ).rejects.toMatchObject({ code: 'timeout' });
+ });
+
+ it('stops waiting for a Retry-After when its deadline comes', async () => {
+ // The real wait: 50 s asked, the deadline 20 ms away.
+ const queue = [answer(429, { 'Retry-After': '50' }), answer(200)];
+ const request = signInRequest('google-merchant-center', 'org_1', {
+ hosts: [HOST],
+ active: async () => ({ id: 'ca_1' }),
+ proxy: async () => queue.shift(),
+ });
+ const started = Date.now();
+ await expect(request(URL_, { signal: AbortSignal.timeout(20) })).rejects.toMatchObject({
+ code: 'timeout',
+ });
+ expect(Date.now() - started).toBeLessThan(5000);
+ });
+});
+
+describe('workspaceEntity', () => {
+ it("is the brand's workspace's Composio entity", async () => {
+ const fake = fakeDb();
+ fake.rows('brands').push({ id: 'brand-1', organization_id: 'org-9' });
+ await expect(workspaceEntity('brand-1', { db: fake.db })).resolves.toBe('org_org-9');
+ await expect(workspaceEntity('brand-2', { db: fake.db })).rejects.toThrow();
+ });
+});
diff --git a/server/src/lib/catalog/sources.js b/server/src/lib/catalog/sources.js
index 1ef4fa701..105c79a1a 100644
--- a/server/src/lib/catalog/sources.js
+++ b/server/src/lib/catalog/sources.js
@@ -1,11 +1,14 @@
import supabaseAdmin from '../../config/supabase.js';
import { isSubscriptionActive } from '../../config/plans.js';
-import { connectorFor, isStore } from '../../connectors/index.js';
+import { connectorFor, isSignIn, isStore } from '../../connectors/index.js';
import { ConnectorHttpError, requestJson } from '../../connectors/http.js';
import { FeedFormatError, parseFeed } from '../feeds/index.js';
+import { isComposioConfigured } from '../composio.js';
import { PageFetchError } from '../safe-fetch.js';
import { logger } from '../logger.js';
import { CredentialsKeyError, readCredentials } from './credentials.js';
+import { saveItemIssues } from './issues.js';
+import { signInRequest as workspaceSignIn, workspaceEntity } from './sign-in.js';
import { syncCatalog } from './sync.js';
/**
@@ -13,7 +16,9 @@ import { syncCatalog } from './sync.js';
* feed URL or a store, read through its connector, daily and on demand; or
* an uploaded CSV. Each source keeps its own products under `sourceKey(id)`,
* and its last sync's outcome on its row (`last_*`), which the Settings card
- * shows. A store is read with its saved sign-in (credentials.js).
+ * shows. A store is read with its saved sign-in (credentials.js); a
+ * platform like Google Merchant Center with the workspace's own sign-in
+ * (sign-in.js), and the issues it reports per item are kept (issues.js).
*/
/** Feeds and stores read again after this long, by the daily run. */
@@ -47,8 +52,34 @@ const STORE_WORDS = {
};
/** Why a sync failed, in words a person can act on. No page, feed or store text. */
-export function failureOf(err) {
+export function failureOf(err, connector = null) {
if (err instanceof FeedFormatError) return { status: 'format_error', error: err.message };
+ if (err instanceof ConnectorHttpError && isSignIn(connector)) {
+ // The workspace's sign-in, not the source's: it's reconnected in Settings › Integrations.
+ if (err.code === 'auth') {
+ return {
+ status: 'auth_failed',
+ error: `The Google sign-in can't open this ${connector.label} account anymore. Reconnect it in Settings › Integrations.`,
+ };
+ }
+ if (err.code === 'not_set_up') {
+ // This server's own setup, or Google's for this account: reconnecting wouldn't help.
+ return {
+ status: 'failed',
+ error: `${connector.label} isn't set up correctly on this server, or the Google account connected in Settings › Integrations can't open this account. The next daily run tries again.`,
+ };
+ }
+ if (err.code === 'rate_limited') {
+ return {
+ status: 'failed',
+ error: `${connector.label} asked to wait. The next daily run tries again.`,
+ };
+ }
+ return {
+ status: 'failed',
+ error: `${connector.label} couldn't be read just now. The next daily run tries again.`,
+ };
+ }
if (err instanceof ConnectorHttpError) {
if (err.code === 'auth') {
return {
@@ -120,9 +151,10 @@ async function record(db, id, at, outcome) {
* and nothing is recorded. A store's calls stop after `readMs` (STORE_READ_MS).
*
* @param {{ id: string, brand_id: string, kind: string, format: string, url?: string|null, account?: string|null }} source
- * @param {{ text?: string, db?: object, now?: Date, fetch?: Function, request?: Function, connectors?: object, readMs?: number }} [options]
- * `fetch` (a feed's), `request` (a store's HTTP client), `connectors` and
- * `readMs` are the tests' stand-ins
+ * @param {{ text?: string, db?: object, now?: Date, fetch?: Function, request?: Function, signInRequest?: Function, connectors?: object, readMs?: number }} [options]
+ * `fetch` (a feed's), `request` (a store's HTTP client), `signInRequest`
+ * (the workspace sign-in's, sign-in.js), `connectors` and `readMs` are
+ * the tests' stand-ins
*/
export async function syncSource(source, options = {}) {
if (reading.has(source.id)) {
@@ -147,6 +179,7 @@ async function syncOnce(
now = new Date(),
fetch,
request = requestJson,
+ signInRequest = workspaceSignIn,
connectors,
readMs = STORE_READ_MS,
},
@@ -154,14 +187,28 @@ async function syncOnce(
const at = now.toISOString();
let outcome;
let credentials = null;
+ let connector = null;
try {
let parsed;
if (text !== undefined) {
parsed = parseFeed(text, source.format);
} else {
- const connector = connectorFor(source.kind, connectors);
+ connector = connectorFor(source.kind, connectors);
if (!connector) {
outcome = { status: 'failed', error: "This server can't read this kind of source." };
+ } else if (isSignIn(connector) && !isComposioConfigured(connector.signIn)) {
+ outcome = { status: 'failed', error: `${connector.label} isn't set up on this server.` };
+ } else if (isSignIn(connector)) {
+ const signedIn = signInRequest(
+ connector.signIn,
+ await workspaceEntity(source.brand_id, { db }),
+ { hosts: connector.hosts },
+ );
+ // Every call the read makes stops at its deadline.
+ const deadline = AbortSignal.timeout(readMs);
+ parsed = await connector.readCatalog(source, {
+ request: (address, opts = {}) => signedIn(address, { ...opts, signal: deadline }),
+ });
} else if (isStore(connector)) {
credentials = await readCredentials(source.id, { db });
// Every call the read makes stops at its deadline.
@@ -193,9 +240,14 @@ async function syncOnce(
variants: counts.variants,
},
};
+ // An item listed again (another language or feed label), kept once: not skipped.
+ if (Number.isInteger(parsed.repeated)) outcome.counts.repeated = parsed.repeated;
+ if (Array.isArray(parsed.issues)) {
+ outcome.counts.issues = await saveItemIssues(source, parsed.issues, { db, now });
+ }
}
} catch (err) {
- outcome = failureOf(err);
+ outcome = failureOf(err, connector);
const told = err instanceof PageFetchError || err instanceof ConnectorHttpError;
if (outcome.status === 'failed' && !told) {
logger.error(
diff --git a/server/src/lib/catalog/sources.test.js b/server/src/lib/catalog/sources.test.js
index 1270e152c..3ef1876bd 100644
--- a/server/src/lib/catalog/sources.test.js
+++ b/server/src/lib/catalog/sources.test.js
@@ -12,7 +12,7 @@ import { PageFetchError } from '../safe-fetch.js';
import { fakeDb } from './__fixtures__/fake-db.js';
import { saveCredentials } from './credentials.js';
import feedUrl, { FEED_MAX_BYTES, FEED_TYPES } from '../../connectors/feed-url/index.js';
-import { redact, sourceKey, sweepCatalogSources, syncSource } from './sources.js';
+import { failureOf, redact, sourceKey, sweepCatalogSources, syncSource } from './sources.js';
const fixture = (name) =>
readFileSync(new URL(`../feeds/__fixtures__/${name}`, import.meta.url), 'utf8');
@@ -315,6 +315,115 @@ describe('syncSource for a store (00282)', () => {
});
});
+describe('syncSource with the workspace sign-in (#1270)', () => {
+ const GMC_ID = '3c8d6a1e-2a3c-4d5e-8f90-123456789abc';
+ const PRODUCTS = 'https://merchantapi.googleapis.com/products/v1/accounts/123/products';
+ let gmcSource;
+ const platform = (readCatalog) => ({
+ kind: 'merchant_center',
+ label: 'Google Merchant Center',
+ signIn: 'google-merchant-center',
+ hosts: ['merchantapi.googleapis.com'],
+ readCatalog: vi.fn(readCatalog),
+ });
+
+ beforeEach(() => {
+ vi.stubEnv('COMPOSIO_API_KEY', 'test-composio-key');
+ vi.stubEnv('COMPOSIO_GMC_AUTH_CONFIG_ID', 'ac_gmc');
+ fake.rows('brands').push({ id: 'brand-1', organization_id: 'org-1' });
+ gmcSource = {
+ id: GMC_ID,
+ brand_id: 'brand-1',
+ kind: 'merchant_center',
+ format: 'auto',
+ url: null,
+ account: '123',
+ };
+ fake.rows('catalog_sources').push({ ...gmcSource, last_synced_at: null });
+ });
+ afterEach(() => {
+ vi.unstubAllEnvs();
+ });
+
+ it("stops every call of the read at one deadline, through the workspace's sign-in", async () => {
+ const signals = [];
+ const signedIn = vi.fn(async (_address, { signal } = {}) => {
+ signals.push(signal);
+ if (signals.length === 1) return { status: 200, headers: {}, data: null };
+ // The second page answers only when the read's deadline stops it.
+ return new Promise((_resolve, reject) => {
+ signal.addEventListener('abort', () =>
+ reject(new ConnectorHttpError('timeout', 'GET took too long')),
+ );
+ });
+ });
+ const signInRequest = vi.fn(() => signedIn);
+ const gmc = platform(async (_source, ctx) => {
+ await ctx.request(PRODUCTS);
+ await ctx.request(PRODUCTS, { query: { pageToken: 'p2' } });
+ return { items: [], errors: [], format: 'merchant_center' };
+ });
+ const outcome = await syncSource(gmcSource, {
+ db: fake.db,
+ now: NOW,
+ signInRequest,
+ readMs: 20,
+ connectors: { merchant_center: gmc },
+ });
+ expect(signInRequest).toHaveBeenCalledWith('google-merchant-center', 'org_org-1', {
+ hosts: ['merchantapi.googleapis.com'],
+ });
+ expect(signals).toHaveLength(2);
+ expect(signals[0]).toBeInstanceOf(AbortSignal);
+ expect(signals[1]).toBe(signals[0]);
+ expect(signals[0].aborted).toBe(true);
+ expect(outcome).toEqual({
+ status: 'failed',
+ error: "Google Merchant Center couldn't be read just now. The next daily run tries again.",
+ });
+ });
+});
+
+describe('failureOf, for a platform read with the workspace sign-in', () => {
+ const gmc = {
+ kind: 'merchant_center',
+ label: 'Google Merchant Center',
+ signIn: 'google-merchant-center',
+ };
+ const of = (code, status) => failureOf(new ConnectorHttpError(code, 'GET x', { status }), gmc);
+
+ it('says what to do: reconnect, ask for the setup, or wait for the next run', () => {
+ expect(of('auth', 401)).toEqual({
+ status: 'auth_failed',
+ error:
+ "The Google sign-in can't open this Google Merchant Center account anymore. Reconnect it in Settings › Integrations.",
+ });
+ const notSetUp = {
+ status: 'failed',
+ error:
+ "Google Merchant Center isn't set up correctly on this server, or the Google account connected in Settings › Integrations can't open this account. The next daily run tries again.",
+ };
+ expect(of('not_set_up')).toEqual(notSetUp);
+ expect(of('not_set_up', 403)).toEqual(notSetUp);
+ expect(of('rate_limited', 429)).toEqual({
+ status: 'failed',
+ error: 'Google Merchant Center asked to wait. The next daily run tries again.',
+ });
+ for (const code of ['http_error', 'unreachable', 'timeout', 'blocked', 'bad_response']) {
+ expect(of(code)).toEqual({
+ status: 'failed',
+ error: "Google Merchant Center couldn't be read just now. The next daily run tries again.",
+ });
+ }
+ // A store's are a store's words.
+ const store = { kind: 'acme_store', label: 'Acme Store', connect() {} };
+ expect(failureOf(new ConnectorHttpError('rate_limited', 'GET x'), store)).toEqual({
+ status: 'failed',
+ error: 'The store asked to wait. The next daily run tries again.',
+ });
+ });
+});
+
describe('redact', () => {
it('takes every credential value out of a message, but not short ones', () => {
expect(
diff --git a/server/src/lib/catalog/stores.js b/server/src/lib/catalog/stores.js
index d019b3d21..2e107a08a 100644
--- a/server/src/lib/catalog/stores.js
+++ b/server/src/lib/catalog/stores.js
@@ -1,16 +1,20 @@
import supabaseAdmin from '../../config/supabase.js';
import { ConnectError } from '../../connectors/errors.js';
import { ConnectorHttpError, requestJson } from '../../connectors/http.js';
-import { connectorFor, isStore } from '../../connectors/index.js';
+import { connectorFor, isSignIn, isStore } from '../../connectors/index.js';
+import { isComposioConfigured } from '../composio.js';
import { logger } from '../logger.js';
import { credentialsConfigured, readCredentials, saveCredentials } from './credentials.js';
+import { signInRequest as workspaceSignIn, workspaceEntity } from './sign-in.js';
/**
* Connecting a brand's store as a catalog source (#1263, 00282): the
* connector checks what the person typed with the store, then the source is
* made and its sign-in kept, encrypted. Replacing the sign-in and removing
* the store go through here too, so the sign-in is never handled anywhere
- * else.
+ * else. A platform read with the workspace's own sign-in (Google Merchant
+ * Center, #1270) is connected here too, with no sign-in of its own to keep:
+ * the person picks one of the accounts that sign-in opens.
*
* What the person typed never goes in a log or an error: a StoreError's
* message is the words they see.
@@ -19,7 +23,7 @@ import { credentialsConfigured, readCredentials, saveCredentials } from './crede
export class StoreError extends Error {
/**
* @param {number} status the HTTP status the route answers
- * @param {'invalid'|'not_found'|'not_configured'|'connect_failed'|'auth_failed'|'unreachable'|'duplicate'|'too_many'|'other_account'} code
+ * @param {'invalid'|'not_found'|'not_configured'|'not_connected'|'connect_failed'|'auth_failed'|'unreachable'|'duplicate'|'too_many'|'other_account'} code
* @param {string} message
*/
constructor(status, code, message) {
@@ -177,6 +181,150 @@ export async function connectStore(
return source;
}
+/** Listing or checking a sign-in platform's accounts stops calling it after this: the person waits. */
+export const SIGN_IN_CALLS_MS = 30_000;
+
+/** The platform read with the workspace's sign-in, ready to call; or a StoreError. */
+async function signedIn(kind, brandId, { db, connectors, signInRequest, callsMs }) {
+ const connector = connectorFor(kind, connectors);
+ if (!isSignIn(connector)) throw new StoreError(404, 'not_found', 'There is no such source.');
+ if (!isComposioConfigured(connector.signIn)) {
+ throw new StoreError(
+ 503,
+ 'not_configured',
+ `${connector.label} isn't set up on this server yet.`,
+ );
+ }
+ const signed = signInRequest(connector.signIn, await workspaceEntity(brandId, { db }), {
+ hosts: connector.hosts,
+ });
+ // Every call stops at one deadline, however many the listing takes.
+ const deadline = AbortSignal.timeout(callsMs);
+ const request = (address, opts = {}) => signed(address, { ...opts, signal: deadline });
+ return { connector, request };
+}
+
+/** A sign-in platform's failure, as the person should read it. */
+function signInFailure(connector, err) {
+ if (err instanceof ConnectError) return new StoreError(400, 'connect_failed', err.message);
+ if (err instanceof ConnectorHttpError && err.code === 'auth') {
+ return new StoreError(
+ 409,
+ 'not_connected',
+ `Connect ${connector.label} with Google in Settings › Integrations first.`,
+ );
+ }
+ if (err instanceof ConnectorHttpError && err.code === 'not_set_up') {
+ // Logged where it was refused (sign-in.js): reconnecting wouldn't help.
+ return new StoreError(
+ 503,
+ 'not_configured',
+ `${connector.label} isn't set up correctly on this server yet. Try again later.`,
+ );
+ }
+ if (!(err instanceof ConnectorHttpError)) {
+ logger.error(
+ { err: { name: err?.name }, kind: connector.kind },
+ '[catalog] sign-in call failed',
+ );
+ }
+ return new StoreError(
+ 502,
+ 'unreachable',
+ `${connector.label} couldn't be reached. Try again in a moment.`,
+ );
+}
+
+/**
+ * The accounts a brand's workspace sign-in opens on a platform read with it
+ * (#1270), for the person to pick one. The caller has checked the person may.
+ *
+ * @returns {Promise<{ id: string, name: string|null }[]>}
+ */
+export async function listSignInAccounts(
+ { brandId, kind },
+ {
+ db = supabaseAdmin,
+ connectors,
+ signInRequest = workspaceSignIn,
+ callsMs = SIGN_IN_CALLS_MS,
+ } = {},
+) {
+ const { connector, request } = await signedIn(kind, brandId, {
+ db,
+ connectors,
+ signInRequest,
+ callsMs,
+ });
+ try {
+ return await connector.listAccounts({ request });
+ } catch (err) {
+ throw signInFailure(connector, err);
+ }
+}
+
+/**
+ * Connects an account of a platform read with the workspace's sign-in to a
+ * brand: checked with the platform, then made a source. No sign-in is kept:
+ * it is the workspace's. The caller has checked the person may.
+ */
+export async function connectSignInSource(
+ { brandId, kind, input, userId },
+ {
+ db = supabaseAdmin,
+ connectors,
+ signInRequest = workspaceSignIn,
+ callsMs = SIGN_IN_CALLS_MS,
+ } = {},
+) {
+ const value = input?.account;
+ if (typeof value !== 'string' || !value.trim() || value.length > FIELD_MAX) {
+ throw new StoreError(400, 'invalid', 'Pick an account to connect.');
+ }
+ const { connector, request } = await signedIn(kind, brandId, {
+ db,
+ connectors,
+ signInRequest,
+ callsMs,
+ });
+ let result;
+ try {
+ result = await connector.checkAccount(value, { request });
+ } catch (err) {
+ throw signInFailure(connector, err);
+ }
+ const account = text(result?.account, 200);
+ if (!account) throw new Error('checkAccount returned no account');
+ const url = text(result.url, 2048);
+ const { data: existing } = await db
+ .from('catalog_sources')
+ .select('id')
+ .eq('brand_id', brandId)
+ .eq('kind', kind)
+ .eq('account', account)
+ .maybeSingle();
+ if (existing) throw new StoreError(409, 'duplicate', 'This account is connected already.');
+ const { data: source, error } = await db
+ .from('catalog_sources')
+ .insert({
+ brand_id: brandId,
+ kind,
+ account,
+ url: url && /^https?:\/\//i.test(url) ? url : null,
+ label: text(result.label, 200),
+ created_by: userId,
+ })
+ .select(SOURCE_COLUMNS)
+ .single();
+ if (error?.code === '23514') {
+ throw new StoreError(409, 'too_many', 'A brand has 10 catalog sources at most.');
+ }
+ if (error?.code === '23505')
+ throw new StoreError(409, 'duplicate', 'This account is connected already.');
+ if (error || !source) throw new Error(`catalog_sources write: ${error?.code ?? 'no row'}`);
+ return source;
+}
+
/** Replaces a store's sign-in, for the same store's account only. */
export async function reconnectStore(
{ source, input, userId },
diff --git a/server/src/lib/catalog/stores.test.js b/server/src/lib/catalog/stores.test.js
index 341bd1a0b..43926d758 100644
--- a/server/src/lib/catalog/stores.test.js
+++ b/server/src/lib/catalog/stores.test.js
@@ -4,11 +4,22 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
vi.mock('../../config/supabase.js', () => ({ default: {} }));
import example from '../../connectors/__example__/index.js';
+import { ConnectError } from '../../connectors/errors.js';
+import { ConnectorHttpError } from '../../connectors/http.js';
import { startFakeApi } from '../../connectors/testing.js';
import { logger } from '../logger.js';
import { fakeDb } from './__fixtures__/fake-db.js';
import { readCredentials } from './credentials.js';
-import { StoreError, connectStore, disconnectStore, readInput, reconnectStore } from './stores.js';
+import {
+ SIGN_IN_CALLS_MS,
+ StoreError,
+ connectSignInSource,
+ connectStore,
+ disconnectStore,
+ listSignInAccounts,
+ readInput,
+ reconnectStore,
+} from './stores.js';
/** Connecting a store, replacing its sign-in and removing it (#1263, 00282). */
@@ -171,3 +182,133 @@ describe('disconnectStore', () => {
expect(fake.rows('catalog_sources')).toEqual([]);
});
});
+
+describe('a platform read with the workspace sign-in (#1270)', () => {
+ const gmc = {
+ kind: 'merchant_center',
+ label: 'Google Merchant Center',
+ signIn: 'google-merchant-center',
+ hosts: ['merchantapi.googleapis.com'],
+ listAccounts: vi.fn(async () => [{ id: '123', name: 'Acme' }]),
+ checkAccount: vi.fn(async (value) => {
+ if (value !== '123')
+ throw new ConnectError("This Google sign-in can't open that Merchant Center account.");
+ return {
+ account: '123',
+ label: 'Acme',
+ url: 'https://merchants.google.com/mc/overview?a=123',
+ };
+ }),
+ readCatalog: vi.fn(),
+ };
+ const request = vi.fn();
+ const signInRequest = vi.fn(() => request);
+ const signInOpts = () => ({ db: fake.db, connectors: { merchant_center: gmc }, signInRequest });
+ const connectGmc = (account = '123') =>
+ connectSignInSource(
+ { brandId: BRAND, kind: 'merchant_center', input: { account }, userId: 'user-1' },
+ signInOpts(),
+ );
+
+ beforeEach(() => {
+ vi.stubEnv('COMPOSIO_API_KEY', 'test-composio-key');
+ vi.stubEnv('COMPOSIO_GMC_AUTH_CONFIG_ID', 'ac_gmc');
+ fake.rows('brands').push({ id: BRAND, organization_id: 'org-1' });
+ });
+ afterEach(() => {
+ vi.unstubAllEnvs();
+ });
+
+ it("lists the accounts the workspace's sign-in opens", async () => {
+ await expect(
+ listSignInAccounts({ brandId: BRAND, kind: 'merchant_center' }, signInOpts()),
+ ).resolves.toEqual([{ id: '123', name: 'Acme' }]);
+ expect(signInRequest).toHaveBeenCalledWith('google-merchant-center', 'org_org-1', {
+ hosts: ['merchantapi.googleapis.com'],
+ });
+ expect(gmc.listAccounts).toHaveBeenCalledWith({ request: expect.any(Function) });
+ });
+
+ it('stops every call of a listing at one deadline, 30 seconds by default', async () => {
+ expect(SIGN_IN_CALLS_MS).toBe(30_000);
+ const signals = [];
+ const slow = vi.fn(async (_address, { signal } = {}) => {
+ signals.push(signal);
+ if (signals.length === 1) return { status: 200, headers: {}, data: {} };
+ // The second call answers only when its deadline stops it.
+ return new Promise((_resolve, reject) => {
+ signal.addEventListener('abort', () =>
+ reject(new ConnectorHttpError('timeout', 'GET took too long')),
+ );
+ });
+ });
+ gmc.listAccounts.mockImplementationOnce(async ({ request: call }) => {
+ await call('https://merchantapi.googleapis.com/accounts/v1/accounts');
+ await call('https://merchantapi.googleapis.com/accounts/v1/accounts/1:listSubaccounts');
+ return [];
+ });
+ await expect(
+ listSignInAccounts(
+ { brandId: BRAND, kind: 'merchant_center' },
+ { ...signInOpts(), signInRequest: () => slow, callsMs: 20 },
+ ),
+ ).rejects.toMatchObject({ status: 502, code: 'unreachable' });
+ expect(signals).toHaveLength(2);
+ expect(signals[0]).toBeInstanceOf(AbortSignal);
+ expect(signals[1]).toBe(signals[0]);
+ expect(signals[0].aborted).toBe(true);
+ });
+
+ it("says the server isn't set up when its own setup is refused, not to reconnect", async () => {
+ gmc.listAccounts.mockRejectedValueOnce(
+ new ConnectorHttpError('not_set_up', 'Composio refused this server'),
+ );
+ await expect(
+ listSignInAccounts({ brandId: BRAND, kind: 'merchant_center' }, signInOpts()),
+ ).rejects.toMatchObject({
+ status: 503,
+ code: 'not_configured',
+ message: "Google Merchant Center isn't set up correctly on this server yet. Try again later.",
+ });
+ });
+
+ it('makes the source for the picked account, with no sign-in of its own', async () => {
+ const source = await connectGmc();
+ expect(source).toMatchObject({
+ brand_id: BRAND,
+ kind: 'merchant_center',
+ account: '123',
+ label: 'Acme',
+ url: 'https://merchants.google.com/mc/overview?a=123',
+ });
+ expect(fake.rows('catalog_source_credentials')).toEqual([]);
+ await expect(connectGmc()).rejects.toMatchObject({ status: 409, code: 'duplicate' });
+ });
+
+ it("says why in plain words when it can't", async () => {
+ await expect(connectGmc('999')).rejects.toMatchObject({ status: 400, code: 'connect_failed' });
+ await expect(connectGmc(' ')).rejects.toMatchObject({ status: 400, code: 'invalid' });
+
+ gmc.listAccounts.mockRejectedValueOnce(new ConnectorHttpError('auth', 'no active sign-in'));
+ await expect(
+ listSignInAccounts({ brandId: BRAND, kind: 'merchant_center' }, signInOpts()),
+ ).rejects.toMatchObject({
+ status: 409,
+ code: 'not_connected',
+ message: 'Connect Google Merchant Center with Google in Settings › Integrations first.',
+ });
+
+ vi.stubEnv('COMPOSIO_GMC_AUTH_CONFIG_ID', '');
+ await expect(connectGmc()).rejects.toMatchObject({ status: 503, code: 'not_configured' });
+ expect(fake.rows('catalog_sources')).toEqual([]);
+ });
+
+ it("isn't a store: no typed sign-in to connect or replace", async () => {
+ await expect(
+ connectStore(
+ { brandId: BRAND, kind: 'merchant_center', input: {}, userId: 'user-1' },
+ signInOpts(),
+ ),
+ ).rejects.toMatchObject({ status: 404, code: 'not_found' });
+ });
+});
diff --git a/server/src/lib/composio.js b/server/src/lib/composio.js
index ccfe33c4b..9836ccec9 100644
--- a/server/src/lib/composio.js
+++ b/server/src/lib/composio.js
@@ -29,12 +29,22 @@ export const PROVIDERS = {
label: 'Google Analytics',
authConfigEnv: 'COMPOSIO_GA_AUTH_CONFIG_ID',
},
+ // The catalog's Merchant Center sources read through it (#1270): see proxyRequest.
+ 'google-merchant-center': {
+ label: 'Google Merchant Center',
+ authConfigEnv: 'COMPOSIO_GMC_AUTH_CONFIG_ID',
+ },
};
export function isKnownProvider(provider) {
return Object.hasOwn(PROVIDERS, provider);
}
+/** Entity id: one connection per organization per provider (#577). */
+export function entityIdFor(organizationId) {
+ return `org_${organizationId}`;
+}
+
function authConfigId(provider) {
const config = PROVIDERS[provider];
if (!config) throw new Error(`Unknown integration provider: ${provider}`);
@@ -87,6 +97,27 @@ export async function deleteConnection(connectedAccountId) {
await getClient().connectedAccounts.delete(connectedAccountId);
}
+/**
+ * A call to a provider's own API through Composio's proxy (#1270), on one
+ * connected account: Composio adds the account's OAuth token on its side,
+ * so the token never reaches us, and we choose the API version ourselves
+ * (a toolkit's ready-made tools may call an older one). Composio doesn't
+ * retry it: the caller decides what a 429 means.
+ *
+ * Returns the API's own answer, whatever its status: `{ status, headers, data }`.
+ * Throws when Composio itself refuses the call (an account no longer active).
+ */
+export async function proxyRequest(connectedAccountId, { endpoint, query = {}, signal }) {
+ const parameters = Object.entries(query)
+ .filter(([, value]) => value != null && value !== '')
+ .map(([name, value]) => ({ in: 'query', name, value: String(value) }));
+ const res = await getClient().tools.proxyExecute(
+ { endpoint, method: 'GET', parameters, connectedAccountId },
+ signal ? { signal } : undefined,
+ );
+ return { status: res.status, headers: res.headers ?? {}, data: res.data ?? null };
+}
+
/**
* Composio requires a pinned toolkit version for manual tool execution
* ("latest" is rejected). Bump deliberately after checking the changelog;
diff --git a/server/src/lib/composio.test.js b/server/src/lib/composio.test.js
index 688bb7cf7..9d2207d9b 100644
--- a/server/src/lib/composio.test.js
+++ b/server/src/lib/composio.test.js
@@ -7,6 +7,7 @@ const sdk = {
list: vi.fn(),
delete: vi.fn(),
execute: vi.fn(),
+ proxyExecute: vi.fn(),
};
vi.mock('@composio/core', () => ({
AuthScheme: { APIKey: (val) => ({ authScheme: 'API_KEY', val }) },
@@ -16,12 +17,21 @@ vi.mock('@composio/core', () => ({
list: (...args) => sdk.list(...args),
delete: (...args) => sdk.delete(...args),
};
- tools = { execute: (...args) => sdk.execute(...args) };
+ tools = {
+ execute: (...args) => sdk.execute(...args),
+ proxyExecute: (...args) => sdk.proxyExecute(...args),
+ };
},
}));
-const { connectWebflowToken, executeWebflowTool, forgetWebflowAccounts, isWebflowConfigured } =
- await import('./composio.js');
+const {
+ connectWebflowToken,
+ entityIdFor,
+ executeWebflowTool,
+ forgetWebflowAccounts,
+ isWebflowConfigured,
+ proxyRequest,
+} = await import('./composio.js');
beforeEach(() => {
vi.stubEnv('COMPOSIO_API_KEY', 'test-composio-key');
@@ -85,3 +95,54 @@ describe('Webflow on Composio', () => {
).rejects.toThrow('401 Unauthorized');
});
});
+
+describe('the workspace entity', () => {
+ it('is one per organization: org_ and its id (#577)', () => {
+ expect(entityIdFor('org-9')).toBe('org_org-9');
+ });
+});
+
+describe('proxyRequest (#1270)', () => {
+ const ENDPOINT = 'https://merchantapi.googleapis.com/accounts/v1/accounts';
+
+ it("calls the provider's API on one connected account, the query as parameters", async () => {
+ sdk.proxyExecute.mockResolvedValueOnce({
+ status: 200,
+ headers: { 'retry-after': '1' },
+ data: { accounts: [] },
+ });
+ const signal = new AbortController().signal;
+ await expect(
+ proxyRequest('ca_1', {
+ endpoint: ENDPOINT,
+ query: { pageSize: 500, pageToken: undefined, filter: '', other: null },
+ signal,
+ }),
+ ).resolves.toEqual({ status: 200, headers: { 'retry-after': '1' }, data: { accounts: [] } });
+ expect(sdk.proxyExecute).toHaveBeenCalledWith(
+ {
+ endpoint: ENDPOINT,
+ method: 'GET',
+ parameters: [{ in: 'query', name: 'pageSize', value: '500' }],
+ connectedAccountId: 'ca_1',
+ },
+ { signal },
+ );
+ });
+
+ it("answers the API's own status, whatever it is, and lets Composio's refusals through", async () => {
+ sdk.proxyExecute.mockResolvedValueOnce({ status: 403 });
+ await expect(proxyRequest('ca_1', { endpoint: ENDPOINT })).resolves.toEqual({
+ status: 403,
+ headers: {},
+ data: null,
+ });
+ expect(sdk.proxyExecute).toHaveBeenLastCalledWith(
+ expect.objectContaining({ parameters: [] }),
+ undefined,
+ );
+ const refused = Object.assign(new Error('403 No permission'), { status: 403 });
+ sdk.proxyExecute.mockRejectedValueOnce(refused);
+ await expect(proxyRequest('ca_1', { endpoint: ENDPOINT })).rejects.toBe(refused);
+ });
+});
diff --git a/server/src/routes/catalog.js b/server/src/routes/catalog.js
index 807d8cc8b..9c1fc0a46 100644
--- a/server/src/routes/catalog.js
+++ b/server/src/routes/catalog.js
@@ -1,14 +1,17 @@
import express, { Router } from 'express';
import { z } from 'zod';
import supabaseAdmin from '../config/supabase.js';
-import { storeConnectors } from '../connectors/index.js';
+import { connectorFor, isSignIn, signInConnectors, storeConnectors } from '../connectors/index.js';
import { assertBrandAccess } from '../lib/access.js';
+import { isComposioConfigured } from '../lib/composio.js';
import { credentialsConfigured } from '../lib/catalog/credentials.js';
import { syncSource } from '../lib/catalog/sources.js';
import {
StoreError,
+ connectSignInSource,
connectStore,
disconnectStore,
+ listSignInAccounts,
reconnectStore,
} from '../lib/catalog/stores.js';
import { catalogLimiter } from '../middleware/rate-limiter.js';
@@ -21,9 +24,14 @@ import { catalogLimiter } from '../middleware/rate-limiter.js';
* - POST /api/catalog/upload?brandId=…&format=…: a CSV (or XML) file as
* the request's body, kept as the brand's upload source;
* - GET /api/catalog/connectors: the stores a person can connect, and
- * their fields;
+ * their fields; and the platforms read with the workspace's own
+ * sign-in (Google Merchant Center, #1270), and whether this server has
+ * that sign-in set up;
+ * - GET /api/catalog/accounts?brandId=…&kind=…: the accounts the
+ * workspace's sign-in opens on such a platform, to pick one;
* - POST /api/catalog/sources { brandId, kind, input }: a store connected,
- * its sign-in checked with it and kept encrypted, then read;
+ * its sign-in checked with it and kept encrypted, then read; or a
+ * sign-in platform's account (`input.account`), checked, then read;
* - PUT /api/catalog/sources/:id/credentials { input }: its sign-in
* replaced;
* - DELETE /api/catalog/sources/:id: a store removed, with its sign-in and
@@ -100,25 +108,47 @@ function syncLater(req, source) {
);
}
-// GET /api/catalog/connectors — the stores a person can connect.
+// GET /api/catalog/connectors — the stores a person can connect, and the sign-in platforms.
router.get('/connectors', (req, res) => {
- res.json({ connectors: storeConnectors(), ready: credentialsConfigured() });
+ res.json({
+ connectors: storeConnectors(),
+ ready: credentialsConfigured(),
+ signIns: signInConnectors().map((c) => ({
+ ...c,
+ configured: isComposioConfigured(c.provider),
+ })),
+ });
+});
+
+const KIND = z.string().regex(/^[a-z][a-z0-9_]{1,39}$/);
+
+// GET /api/catalog/accounts?brandId=…&kind=… — the accounts the workspace's sign-in opens.
+router.get('/accounts', catalogLimiter, async (req, res) => {
+ const parsed = z.object({ brandId: z.guid(), kind: KIND }).safeParse(req.query);
+ if (!parsed.success) return res.status(400).json({ error: 'invalid', code: 'invalid' });
+ try {
+ await assertWriter(parsed.data.brandId, req.user.id);
+ return res.json({ accounts: await listSignInAccounts(parsed.data) });
+ } catch (err) {
+ return fail(req, res, err, 'catalog accounts error');
+ }
});
const connectBody = z.object({
brandId: z.guid(),
- kind: z.string().regex(/^[a-z][a-z0-9_]{1,39}$/),
+ kind: KIND,
input: z.record(z.string(), z.string()),
});
-// POST /api/catalog/sources — connect a store.
+// POST /api/catalog/sources — connect a store, or a sign-in platform's account.
router.post('/sources', catalogLimiter, async (req, res) => {
const parsed = connectBody.safeParse(req.body);
if (!parsed.success) return res.status(400).json({ error: 'invalid', code: 'invalid' });
const { brandId, kind, input } = parsed.data;
try {
await assertWriter(brandId, req.user.id);
- const source = await connectStore({ brandId, kind, input, userId: req.user.id });
+ const connect = isSignIn(connectorFor(kind)) ? connectSignInSource : connectStore;
+ const source = await connect({ brandId, kind, input, userId: req.user.id });
syncLater(req, source);
return res.status(201).json({ id: source.id, account: source.account });
} catch (err) {
diff --git a/server/src/routes/catalog.test.js b/server/src/routes/catalog.test.js
index 8cdaf7678..c2c46fcb9 100644
--- a/server/src/routes/catalog.test.js
+++ b/server/src/routes/catalog.test.js
@@ -17,7 +17,10 @@ const syncSource = vi.fn();
const connectStore = vi.fn();
const reconnectStore = vi.fn();
const disconnectStore = vi.fn();
+const connectSignInSource = vi.fn();
+const listSignInAccounts = vi.fn();
let configured;
+let googleConfigured;
vi.mock('../config/supabase.js', () => ({
default: {
@@ -79,6 +82,8 @@ vi.mock('../lib/catalog/stores.js', async () => {
connectStore: (...args) => connectStore(...args),
reconnectStore: (...args) => reconnectStore(...args),
disconnectStore: (...args) => disconnectStore(...args),
+ connectSignInSource: (...args) => connectSignInSource(...args),
+ listSignInAccounts: (...args) => listSignInAccounts(...args),
};
});
vi.mock('../connectors/index.js', () => ({
@@ -89,6 +94,19 @@ vi.mock('../connectors/index.js', () => ({
fields: [{ name: 'apiKey', label: 'API key', secret: true }],
},
],
+ signInConnectors: () => [
+ {
+ kind: 'merchant_center',
+ label: 'Google Merchant Center',
+ provider: 'google-merchant-center',
+ },
+ ],
+ connectorFor: (kind) =>
+ kind === 'merchant_center' ? { kind, signIn: 'google-merchant-center' } : { kind },
+ isSignIn: (connector) => typeof connector?.signIn === 'string',
+}));
+vi.mock('../lib/composio.js', () => ({
+ isComposioConfigured: (provider) => provider === 'google-merchant-center' && googleConfigured,
}));
const { default: router } = await import('./catalog.js');
@@ -111,6 +129,14 @@ beforeEach(async () => {
];
assertBrandAccess.mockReset().mockResolvedValue({ orgId: 'org-1' });
syncSource.mockReset().mockResolvedValue({ status: 'ok', counts: { items: 4 } });
+ googleConfigured = true;
+ connectSignInSource.mockReset().mockResolvedValue({
+ id: 'gmc-1',
+ brand_id: BRAND,
+ kind: 'merchant_center',
+ account: '123456789',
+ });
+ listSignInAccounts.mockReset().mockResolvedValue([{ id: '123456789', name: 'Acme' }]);
connectStore.mockReset().mockResolvedValue({
id: 'store-1',
brand_id: BRAND,
@@ -258,6 +284,61 @@ describe('GET /api/catalog/connectors', () => {
},
],
ready: false,
+ signIns: [
+ {
+ kind: 'merchant_center',
+ label: 'Google Merchant Center',
+ provider: 'google-merchant-center',
+ configured: true,
+ },
+ ],
+ });
+ });
+});
+
+describe('GET /api/catalog/accounts (#1270)', () => {
+ const query = `brandId=${BRAND}&kind=merchant_center`;
+
+ it("lists the accounts the workspace's sign-in opens, for an admin or a manager", async () => {
+ const res = await fetch(`${base}/accounts?${query}`);
+ expect(res.status).toBe(200);
+ expect(await res.json()).toEqual({ accounts: [{ id: '123456789', name: 'Acme' }] });
+ expect(listSignInAccounts).toHaveBeenCalledWith({ brandId: BRAND, kind: 'merchant_center' });
+
+ role = 'analyst';
+ expect((await fetch(`${base}/accounts?${query}`)).status).toBe(403);
+ expect((await fetch(`${base}/accounts?brandId=nope&kind=merchant_center`)).status).toBe(400);
+ });
+
+ it("refuses a brand of another workspace, or one that doesn't exist, and lists nothing", async () => {
+ assertBrandAccess.mockRejectedValueOnce(
+ Object.assign(new Error("This brand isn't in your workspace"), {
+ status: 403,
+ code: 'not_in_workspace',
+ }),
+ );
+ const other = await fetch(`${base}/accounts?${query}`);
+ expect(other.status).toBe(403);
+ expect((await other.json()).code).toBe('not_in_workspace');
+
+ assertBrandAccess.mockRejectedValueOnce(
+ Object.assign(new Error('Brand not found'), { status: 404 }),
+ );
+ const missing = await fetch(`${base}/accounts?${query}`);
+ expect(missing.status).toBe(404);
+ expect((await missing.json()).code).toBe('not_found');
+ expect(listSignInAccounts).not.toHaveBeenCalled();
+ });
+
+ it('says when the workspace must connect Google first', async () => {
+ listSignInAccounts.mockRejectedValueOnce(
+ new StoreError(409, 'not_connected', 'Connect Google Merchant Center with Google first.'),
+ );
+ const res = await fetch(`${base}/accounts?${query}`);
+ expect(res.status).toBe(409);
+ expect(await res.json()).toEqual({
+ error: 'Connect Google Merchant Center with Google first.',
+ code: 'not_connected',
});
});
});
@@ -265,6 +346,24 @@ describe('GET /api/catalog/connectors', () => {
describe('POST /api/catalog/sources', () => {
const body = { brandId: BRAND, kind: 'acme_store', input: { apiKey: 'sk_secret' } };
+ it("connects a sign-in platform's account the person picked, then reads it", async () => {
+ const res = await send('POST', '/sources', {
+ brandId: BRAND,
+ kind: 'merchant_center',
+ input: { account: '123456789' },
+ });
+ expect(res.status).toBe(201);
+ expect(await res.json()).toEqual({ id: 'gmc-1', account: '123456789' });
+ expect(connectSignInSource).toHaveBeenCalledWith({
+ brandId: BRAND,
+ kind: 'merchant_center',
+ input: { account: '123456789' },
+ userId: 'user-1',
+ });
+ expect(connectStore).not.toHaveBeenCalled();
+ expect(syncSource).toHaveBeenCalledWith(expect.objectContaining({ id: 'gmc-1' }));
+ });
+
it('connects a store for an admin or a manager, then reads it without waiting', async () => {
role = 'manager';
const res = await send('POST', '/sources', body);
diff --git a/server/src/routes/integrations.js b/server/src/routes/integrations.js
index 10bf667e4..69297d1b2 100644
--- a/server/src/routes/integrations.js
+++ b/server/src/routes/integrations.js
@@ -3,6 +3,7 @@ import { z } from 'zod';
import supabaseAdmin from '../config/supabase.js';
import {
PROVIDERS,
+ entityIdFor,
isKnownProvider,
isComposioConfigured,
initiateConnection,
@@ -20,13 +21,9 @@ const router = Router();
const GSC = 'google-search-console';
const GA = 'google-analytics';
+const GMC = 'google-merchant-center';
const WRITE_ROLES = ['admin', 'manager'];
-/** Entity id: one connection per organization per provider (#577). */
-function entityIdFor(organizationId) {
- return `org_${organizationId}`;
-}
-
async function getProfile(userId) {
const { data: profile, error } = await supabaseAdmin
.from('profiles')
@@ -92,6 +89,7 @@ router.get('/config', async (_req, res) => {
return res.json({
googleSearchConsole: isComposioConfigured(GSC),
googleAnalytics: isComposioConfigured(GA),
+ googleMerchantCenter: isComposioConfigured(GMC),
dataForSeo: Boolean(process.env.DATAFORSEO_LOGIN && process.env.DATAFORSEO_PASSWORD),
});
});
diff --git a/supabase/migrations/00296_catalog_item_issues.sql b/supabase/migrations/00296_catalog_item_issues.sql
new file mode 100644
index 000000000..48ebc62d5
--- /dev/null
+++ b/supabase/migrations/00296_catalog_item_issues.sql
@@ -0,0 +1,67 @@
+-- ============================================================================
+-- 00296: item issues from Google Merchant Center (#1270)
+-- ============================================================================
+-- The Merchant Center connector reads a brand's Merchant Center account into
+-- its catalog (00278), and with each product the problems Merchant Center
+-- reports for it: a missing GTIN, a price that doesn't match the landing
+-- page, an image too small. Those decide whether the product can show in
+-- Google's shopping results, AI Overviews and AI Mode among them, so we keep
+-- them to tell the brand what to fix ("Feed fields complete", the readiness
+-- cards, later).
+--
+-- * One row per issue per item (an offer, the catalog's variant), as
+-- Merchant Center lists it: its code, severity (not_impacted, demoted,
+-- disapproved), the attribute it concerns, the destination it affects
+-- (reporting context) and its description. An item with no issues has
+-- no rows.
+-- * A sync replaces its source's rows; they go with their source.
+-- * Members read their brands' rows; the server writes them.
+--
+-- A Merchant Center source has no credentials row (00282): it reads through
+-- the workspace's Google sign-in, which Composio keeps
+-- (integration_connections, 00045).
+
+create table if not exists public.catalog_item_issues (
+ id uuid primary key default gen_random_uuid(),
+ brand_id uuid not null references public.brands(id) on delete cascade,
+ source_id uuid not null references public.catalog_sources(id) on delete cascade,
+ -- The item's id in the catalog (catalog_variants.external_id): Merchant Center's offer id.
+ external_id text not null check (char_length(external_id) between 1 and 200),
+ code text not null check (char_length(code) between 1 and 200),
+ severity text not null check (severity in ('not_impacted', 'demoted', 'disapproved')),
+ attribute text check (attribute is null or char_length(attribute) <= 200),
+ reporting_context text check (reporting_context is null or char_length(reporting_context) <= 100),
+ description text check (description is null or char_length(description) <= 1000),
+ detail text check (detail is null or char_length(detail) <= 2000),
+ documentation text check (documentation is null or char_length(documentation) <= 2048),
+ resolution text check (resolution is null or char_length(resolution) <= 100),
+ countries text[] not null default '{}' check (cardinality(countries) <= 300),
+ created_at timestamptz not null default now()
+);
+
+create index if not exists catalog_item_issues_source_idx
+ on public.catalog_item_issues (source_id);
+create index if not exists catalog_item_issues_brand_item_idx
+ on public.catalog_item_issues (brand_id, external_id);
+
+alter table public.catalog_item_issues enable row level security;
+revoke all on table public.catalog_item_issues from anon, authenticated;
+grant select on table public.catalog_item_issues to authenticated;
+grant all on table public.catalog_item_issues to service_role;
+
+drop policy if exists "catalog_item_issues: member select" on public.catalog_item_issues;
+create policy "catalog_item_issues: member select"
+ on public.catalog_item_issues
+ for select
+ using (
+ brand_id in (
+ select b.id from public.brands b
+ join public.profiles p on p.organization_id = b.organization_id
+ where p.id = auth.uid()
+ )
+ );
+
+comment on table public.catalog_item_issues is
+ 'Catalog (#1270, 00296): the issues Google Merchant Center reports per item (offer) of a Merchant Center source: code, severity, attribute, destination (reporting context), description. Replaced by each sync, removed with the source. Server writes, members read.';
+
+notify pgrst, 'reload schema';
diff --git a/supabase/schema.sql b/supabase/schema.sql
index 5fd1b51f4..259962a94 100644
--- a/supabase/schema.sql
+++ b/supabase/schema.sql
@@ -41598,3 +41598,74 @@ comment on function public.approve_action_task_review(uuid, text, jsonb) is
notify pgrst, 'reload schema';
+-- ─────────────────────────────────────────────────────────────────────────
+-- migrations/00296_catalog_item_issues.sql
+-- ─────────────────────────────────────────────────────────────────────────
+-- ============================================================================
+-- 00296: item issues from Google Merchant Center (#1270)
+-- ============================================================================
+-- The Merchant Center connector reads a brand's Merchant Center account into
+-- its catalog (00278), and with each product the problems Merchant Center
+-- reports for it: a missing GTIN, a price that doesn't match the landing
+-- page, an image too small. Those decide whether the product can show in
+-- Google's shopping results, AI Overviews and AI Mode among them, so we keep
+-- them to tell the brand what to fix ("Feed fields complete", the readiness
+-- cards, later).
+--
+-- * One row per issue per item (an offer, the catalog's variant), as
+-- Merchant Center lists it: its code, severity (not_impacted, demoted,
+-- disapproved), the attribute it concerns, the destination it affects
+-- (reporting context) and its description. An item with no issues has
+-- no rows.
+-- * A sync replaces its source's rows; they go with their source.
+-- * Members read their brands' rows; the server writes them.
+--
+-- A Merchant Center source has no credentials row (00282): it reads through
+-- the workspace's Google sign-in, which Composio keeps
+-- (integration_connections, 00045).
+
+create table if not exists public.catalog_item_issues (
+ id uuid primary key default gen_random_uuid(),
+ brand_id uuid not null references public.brands(id) on delete cascade,
+ source_id uuid not null references public.catalog_sources(id) on delete cascade,
+ -- The item's id in the catalog (catalog_variants.external_id): Merchant Center's offer id.
+ external_id text not null check (char_length(external_id) between 1 and 200),
+ code text not null check (char_length(code) between 1 and 200),
+ severity text not null check (severity in ('not_impacted', 'demoted', 'disapproved')),
+ attribute text check (attribute is null or char_length(attribute) <= 200),
+ reporting_context text check (reporting_context is null or char_length(reporting_context) <= 100),
+ description text check (description is null or char_length(description) <= 1000),
+ detail text check (detail is null or char_length(detail) <= 2000),
+ documentation text check (documentation is null or char_length(documentation) <= 2048),
+ resolution text check (resolution is null or char_length(resolution) <= 100),
+ countries text[] not null default '{}' check (cardinality(countries) <= 300),
+ created_at timestamptz not null default now()
+);
+
+create index if not exists catalog_item_issues_source_idx
+ on public.catalog_item_issues (source_id);
+create index if not exists catalog_item_issues_brand_item_idx
+ on public.catalog_item_issues (brand_id, external_id);
+
+alter table public.catalog_item_issues enable row level security;
+revoke all on table public.catalog_item_issues from anon, authenticated;
+grant select on table public.catalog_item_issues to authenticated;
+grant all on table public.catalog_item_issues to service_role;
+
+drop policy if exists "catalog_item_issues: member select" on public.catalog_item_issues;
+create policy "catalog_item_issues: member select"
+ on public.catalog_item_issues
+ for select
+ using (
+ brand_id in (
+ select b.id from public.brands b
+ join public.profiles p on p.organization_id = b.organization_id
+ where p.id = auth.uid()
+ )
+ );
+
+comment on table public.catalog_item_issues is
+ 'Catalog (#1270, 00296): the issues Google Merchant Center reports per item (offer) of a Merchant Center source: code, severity, attribute, destination (reporting context), description. Replaced by each sync, removed with the source. Server writes, members read.';
+
+notify pgrst, 'reload schema';
+