Open-source manufacturing ERP, MES and QMS. Quoting, MRP, inventory, shop floor, quality and lot/serial traceability on one Postgres schema, with a REST API and MCP server. Self-host or use Carbon Cloud.
TypeScript ai-agentsbill-of-materialserpinventory-managementleanmanufacturingmanufacturing-execution-systemmcp-servermesmrpopen-source-erppostgresqlproduction-planningqmsquality-management-systemreact-routerself-hostedsupabasetraceabilitytypescript
fix(tables): a failed later page shows Retry instead of the error screen, export compose survives a retry, unordered RPC lists keep their order
aashuf2856f16cbf2
diff --git a/.claude/rules/table-csv-export.md b/.claude/rules/table-csv-export.mdindex 88baa535ad..4a4e2af560 100644--- a/.claude/rules/table-csv-export.md+++ b/.claude/rules/table-csv-export.md@@ -158,7 +158,8 @@ list and re-read every loaded page. The steps:If `trigger` fails, the route deletes the lock.5. Step `check-permission`: the job reads the user's claims again(`canReadList`, `utils/table-jobs.server.ts`). If the user lost the list's- permission, it notifies "Your export did not run" in-app and stops.+ permission, it notifies "Your export did not run" in-app and stops. A+ failed claims read throws, so the step retries instead.6. Steps `write-part-0`, `write-part-1`, …: each step extends the lock, thenreads at most 20 pages of 500 rows (`TABLE_JOB_ROWS_PER_STEP`, 10,000,`table-jobs/paging.ts`) as the user (`getUserScopedClient`, `fetchListPage`).@@ -168,11 +169,12 @@ list and re-read every loaded page. The steps:7. The parts stop at the end of the list, at 100,000 rows (`MAX_EXPORT_ROWS`),or after the page that passes 200 MB (`MAX_EXPORT_BYTES`, `table-jobs/csv.ts`).8. Step `compose` uploads one `ReadableStream` that downloads the parts one at- a time (`composeExport`, `concatParts`). Only one part is in memory. Then- it deletes the parts.-9. Step `notify` sends `NotificationEvent.TableExport`. The title says "first- N rows" when the export stopped at a limit.-10. Step `release-lock` deletes the lock.+ a time (`composeExport`, `concatParts`). Only one part is in memory.+9. Step `remove-parts` deletes the parts (`removeExportParts`). It is its own+ step so a retried `compose` still finds them.+10. Step `notify` sends `NotificationEvent.TableExport`. The title says "first+ N rows" when the export stopped at a limit.+11. Step `release-lock` deletes the lock.| File | Key (`table-jobs/storage.ts`, bucket `temp-staging`) ||---|---|diff --git a/.claude/rules/table-infinite-scroll.md b/.claude/rules/table-infinite-scroll.mdindex 481fcb5fde..9e87149398 100644--- a/.claude/rules/table-infinite-scroll.md+++ b/.claude/rules/table-infinite-scroll.md@@ -72,6 +72,9 @@ search the query reads. On an error:- A later page logs it and returns `{ [dataKey]: [], nextCursor: null, listError }`(`InfiniteListError`). The Table shows "Couldn't load more rows" with Retry.+ A later page that throws (not a `Response`) is caught and answered the same+ way: a fetcher whose loader throws replaces the list with the route's error+ screen.- Page 1 throws `redirect(onError.redirectTo)` with an error flash, or shows norows without `onError`.@@ -108,9 +111,10 @@ infinite={{ dataKey: "customers", nextCursor, listKey: "customers" }}- 🛑 Pass the loader's `data` array to `<Table infinite>`, or a memoized one.`useInfiniteRows` reads a new array as a revalidation and re-reads everyloaded page. A copy built in the render body (`data.map(…)`) re-reads them- on every render. `test/infinite-table-data-identity.test.ts` checks this, but- its regex reads only the `<Table …>` elements with no `>` inside: most- tables slip past it (open, see the 2026-10-14 review).+ on every render. `test/infinite-table-data-identity.test.ts` checks this: it+ parses every `.tsx` with the TypeScript compiler and fails on an infinite+ `<Table>` whose `data` is a `.map`/`.filter`/`.slice`/`.concat`/`.toSorted`+ call or a `[...x]` spread, inline or in a variable.- Column widths: only the rows in view are rendered (`useVirtualRows`, 10rows of overscan), so table layout alone would narrow and widen a column asrows scroll by. `useGrowOnlyColumnWidths` holds each column at its widest sodiff --git a/apps/erp/app/components/Table/infiniteRowsStore.ts b/apps/erp/app/components/Table/infiniteRowsStore.tsindex ac75479ae4..60a32ce313 100644--- a/apps/erp/app/components/Table/infiniteRowsStore.ts+++ b/apps/erp/app/components/Table/infiniteRowsStore.ts@@ -204,7 +204,9 @@ export type PageResult<T> =/*** Reads one fetched page from the fetcher's data. It is an error when:* - the data is still the object seen before the request (a loader that- * threw leaves no new data: the router drops the fetcher),+ * threw leaves no new data: the router drops the fetcher and shows the+ * route's error screen, so `loadListPage` turns a later page's throw into+ * `listError`),* - the loader returned `listError` (`InfiniteListError`),* - the data has no rows array.*diff --git a/apps/erp/app/utils/list-registry/load.server.ts b/apps/erp/app/utils/list-registry/load.server.tsindex 84415fe440..caf2b18d76 100644--- a/apps/erp/app/utils/list-registry/load.server.ts+++ b/apps/erp/app/utils/list-registry/load.server.ts@@ -132,43 +132,63 @@ export async function loadListPage<await options.guard?.(context);if (options.params) context.params = options.params(context);- const [page, extra] = await Promise.all([- fetchListPage(- list,- auth.client,- auth.companyId,- context.params,- undefined,- { companyGroupId: auth.companyGroupId }- ),- after ? undefined : options.firstPage?.(context)- ]);+ const loadPage = async () => {+ const [page, extra] = await Promise.all([+ fetchListPage(+ list,+ auth.client,+ auth.companyId,+ context.params,+ undefined,+ { companyGroupId: auth.companyGroupId }+ ),+ after ? undefined : options.firstPage?.(context)+ ]);- if (page.error) {+ if (page.error) {+ logger.error("Failed to load a page of the list {listKey}", {+ listKey: list.key,+ companyId: auth.companyId,+ laterPage: Boolean(after),+ error: page.error+ });+ if (after) return listErrorPage(options.dataKey);+ if (options.onError) {+ throw redirect(+ options.onError.redirectTo,+ await flash(request, error(page.error, options.onError.message))+ );+ }+ }++ const rows =+ options.mapRows && !page.error+ ? await options.mapRows(page.rows, context)+ : page.rows;++ return {+ ...extra,+ [options.dataKey]: rows,+ count: page.count,+ nextCursor: page.nextCursor+ } as ListLoaderData<TKey, TOut> & TExtra;+ };++ if (!after) return loadPage();+ // A later page is read by the Table's fetcher, and a fetcher whose loader+ // throws replaces the whole list with the route's error screen. So any+ // error becomes `listError` ("Couldn't load more rows" with Retry). A thrown+ // Response (a redirect, a refusal) is the router's to handle.+ try {+ return await loadPage();+ } catch (thrown) {+ if (thrown instanceof Response) throw thrown;logger.error("Failed to load a page of the list {listKey}", {listKey: list.key,companyId: auth.companyId,- laterPage: Boolean(after),- error: page.error+ laterPage: true,+ error: thrown});- if (after) return listErrorPage(options.dataKey);- if (options.onError) {- throw redirect(- options.onError.redirectTo,- await flash(request, error(page.error, options.onError.message))- );- }+ return listErrorPage(options.dataKey);}-- const rows =- options.mapRows && !page.error- ? await options.mapRows(page.rows, context)- : page.rows;-- return {- ...extra,- [options.dataKey]: rows,- count: page.count,- nextCursor: page.nextCursor- } as ListLoaderData<TKey, TOut> & TExtra;}diff --git a/apps/erp/app/utils/query.test.ts b/apps/erp/app/utils/query.test.tsindex fcfc072730..d4a51b5b5d 100644--- a/apps/erp/app/utils/query.test.ts+++ b/apps/erp/app/utils/query.test.ts@@ -258,6 +258,51 @@ describe("setGenericQueryFilters paging", () => {expect(query.limit).toHaveBeenCalledWith(101);});+ it("breaks ties by the id after the list's order", () => {+ const query = createQueryStub();+ setGenericQueryFilters(query as never, { limit: 100, cursor: null }, sorts);+ expect(query.order.mock.calls).toEqual([+ ["createdAt", { ascending: false, foreignTable: undefined }],+ ["id", { ascending: true }]+ ]);+ });++ it("breaks ties by the list's own id column", () => {+ const query = createQueryStub();+ setGenericQueryFilters(+ query as never,+ { sorts: [{ sortBy: "name", sortAsc: true }], limit: 50, offset: 0 },+ { defaultSorts: sorts, idColumn: "itemId" }+ );+ expect(query.order.mock.calls).toEqual([+ ["name", { ascending: true, foreignTable: undefined }],+ ["itemId", { ascending: true }]+ ]);+ });++ it("adds no second id order when the order already ends with the id", () => {+ const query = createQueryStub();+ setGenericQueryFilters(query as never, { limit: 100, cursor: null }, [+ { column: "id", ascending: false }+ ]);+ expect(query.order.mock.calls).toEqual([+ ["id", { ascending: false, foreignTable: undefined }]+ ]);+ });++ it("orders by the id alone when a cursor list has no sorts", () => {+ const query = createQueryStub();+ setGenericQueryFilters(query as never, { limit: 100, cursor: null });+ expect(query.order.mock.calls).toEqual([["id", { ascending: true }]]);+ });++ it("leaves a query nothing orders in its own order (an RPC's ORDER BY)", () => {+ const query = createQueryStub();+ setGenericQueryFilters(query as never, { limit: 1000, offset: 0 });+ expect(query.order).not.toHaveBeenCalled();+ expect(query.range).toHaveBeenCalledWith(0, 999);+ });+it("does not page at all without a limit", () => {const query = createQueryStub();setGenericQueryFilters(query as never, {}, sorts);diff --git a/apps/erp/app/utils/query.ts b/apps/erp/app/utils/query.tsindex 284cde9d10..e7f5419e3a 100644--- a/apps/erp/app/utils/query.ts+++ b/apps/erp/app/utils/query.ts@@ -237,7 +237,10 @@ export function setGenericQueryFilters<});// The id breaks ties, so the order is total and every row has one place.- if (keyset[keyset.length - 1]?.column !== idColumn) {+ // A query nothing orders keeps its own order (an RPC's ORDER BY), unless+ // it pages by cursor, which needs the id order.+ const ordered = orders.length > 0 || args.cursor !== undefined;+ if (ordered && keyset[keyset.length - 1]?.column !== idColumn) {query = query.order(idColumn, { ascending: true });}diff --git a/apps/erp/test/infinite-table-data-identity.test.ts b/apps/erp/test/infinite-table-data-identity.test.tsindex 68ec4a1dc9..36176a6e87 100644--- a/apps/erp/test/infinite-table-data-identity.test.ts+++ b/apps/erp/test/infinite-table-data-identity.test.ts@@ -4,6 +4,7 @@import { readdirSync, readFileSync, statSync } from "node:fs";import { join, relative } from "node:path";+import ts from "typescript";import { describe, expect, it } from "vitest";/**@@ -16,6 +17,8 @@ import { describe, expect, it } from "vitest";const APP = join(__dirname, "..", "app");+const COPY_METHODS = new Set(["map", "filter", "slice", "concat", "toSorted"]);+function tsxFiles(dir: string): string[] {return readdirSync(dir).flatMap((name) => {const path = join(dir, name);@@ -24,24 +27,93 @@ function tsxFiles(dir: string): string[] {});}-/** The identifier passed as `data={…}` on a JSX element that has `infinite=`. */-function infiniteDataNames(source: string): string[] {- const names: string[] = [];- for (const match of source.matchAll(/<Table\b[^>]*?>/gs)) {- const element = match[0];- if (!/\binfinite=\{/.test(element)) continue;- const data = /\bdata=\{\s*([A-Za-z_$][\w$]*)\s*\}/.exec(element);- if (data) names.push(data[1]);+function parse(source: string) {+ return ts.createSourceFile(+ "file.tsx",+ source,+ ts.ScriptTarget.Latest,+ true,+ ts.ScriptKind.TSX+ );+}++function unwrap(node: ts.Expression): ts.Expression {+ let current = node;+ while (+ ts.isParenthesizedExpression(current) ||+ ts.isAsExpression(current) ||+ ts.isNonNullExpression(current) ||+ ts.isSatisfiesExpression(current)+ ) {+ current = current.expression;}- return names;+ return current;}-/** True when `name` is built in the render body by copying an array. */-function isRenderCopy(source: string, name: string): boolean {- const copy = new RegExp(- String.raw`const\s+${name}\s*=\s*(?:[\w$.?]+\s*\.\s*(?:map|filter|slice|concat|toSorted)\s*\(|\[\s*\.\.\.)`+/** A new array on every evaluation: `x.map(…)` and the like, or `[...x]`. */+function isArrayCopy(node: ts.Expression): boolean {+ const expression = unwrap(node);+ if (ts.isArrayLiteralExpression(expression)) {+ return expression.elements.some(ts.isSpreadElement);+ }+ return (+ ts.isCallExpression(expression) &&+ ts.isPropertyAccessExpression(expression.expression) &&+ COPY_METHODS.has(expression.expression.name.text));- return copy.test(source);+}++function visit(node: ts.Node, onNode: (node: ts.Node) => void) {+ onNode(node);+ ts.forEachChild(node, (child) => visit(child, onNode));+}++/** The `data={…}` expression of every `<Table>` element that has `infinite`. */+function infiniteDataExpressions(file: ts.SourceFile): ts.Expression[] {+ const expressions: ts.Expression[] = [];+ visit(file, (node) => {+ if (!ts.isJsxOpeningElement(node) && !ts.isJsxSelfClosingElement(node)) {+ return;+ }+ if (node.tagName.getText(file) !== "Table") return;+ const attribute = (name: string) =>+ node.attributes.properties.find(+ (property): property is ts.JsxAttribute =>+ ts.isJsxAttribute(property) && property.name.getText(file) === name+ );+ const data = attribute("data")?.initializer;+ if (!attribute("infinite") || !data || !ts.isJsxExpression(data)) return;+ if (data.expression) expressions.push(data.expression);+ });+ return expressions;+}++/** The names of the variables in the file that hold a render-body copy. */+function renderCopyNames(file: ts.SourceFile): Set<string> {+ const names = new Set<string>();+ visit(file, (node) => {+ if (+ ts.isVariableDeclaration(node) &&+ ts.isIdentifier(node.name) &&+ node.initializer &&+ isArrayCopy(node.initializer)+ ) {+ names.add(node.name.text);+ }+ });+ return names;+}++/** How each infinite table's `data` is written that copies during render. */+function renderCopies(source: string): string[] {+ const file = parse(source);+ const copies = renderCopyNames(file);+ return infiniteDataExpressions(file).flatMap((expression) => {+ const data = unwrap(expression);+ if (isArrayCopy(data)) return [data.getText(file)];+ if (ts.isIdentifier(data) && copies.has(data.text)) return [data.text];+ return [];+ });}describe("infinite tables pass a stable data array", () => {@@ -49,34 +121,63 @@ describe("infinite tables pass a stable data array", () => {readFileSync(file, "utf8").includes("infinite={"));- it("finds the infinite tables", () => {- expect(files.length).toBeGreaterThan(50);+ it("reads the data of every infinite table", () => {+ const tables = files.reduce(+ (total, file) =>+ total + infiniteDataExpressions(parse(readFileSync(file, "utf8"))).length,+ 0+ );+ // Most files with `infinite={` render one infinite `<Table>`.+ expect(tables).toBeGreaterThan(100);});it("never passes an array copied during render", () => {- const offenders = files.flatMap((file) => {- const source = readFileSync(file, "utf8");- return infiniteDataNames(source)- .filter((name) => isRenderCopy(source, name))- .map((name) => `${relative(APP, file)}: data={${name}}`);- });+ const offenders = files.flatMap((file) =>+ renderCopies(readFileSync(file, "utf8")).map(+ (data) => `${relative(APP, file)}: data={${data}}`+ )+ );expect(offenders).toEqual([]);});- it("detects a copy built with .map", () => {+ it("detects a copy built with .map, behind props that contain =>", () => {const source = `const rows = data.map((row) => ({ ...row }));- return <Table data={rows} infinite={{ dataKey: "x", nextCursor }} />;+ return (+ <Table<Row>+ onRowClick={(row) => navigate(row.id)}+ data={rows}+ infinite={{ dataKey: "x", nextCursor }}+ />+ );`;- expect(infiniteDataNames(source)).toEqual(["rows"]);- expect(isRenderCopy(source, "rows")).toBe(true);+ expect(renderCopies(source)).toEqual(["rows"]);+ });++ it("detects a copy written inline and a spread copy", () => {+ expect(+ renderCopies(+ `<Table data={data.filter((r) => r.ok)} infinite={{ dataKey: "x" }} />`+ )+ ).toEqual(["data.filter((r) => r.ok)"]);+ expect(+ renderCopies(`+ const rows = [...data];+ <Table data={rows} infinite={{ dataKey: "x" }}></Table>+ `)+ ).toEqual(["rows"]);});- it("accepts a memoized array", () => {+ it("accepts a memoized array and the loader's own array", () => {const source = `const rows = useMemo(() => data.map((row) => ({ ...row })), [data]);- return <Table data={rows} infinite={{ dataKey: "x", nextCursor }} />;+ <Table data={rows} infinite={{ dataKey: "x", nextCursor }} />;+ <Table data={data} infinite={{ dataKey: "y", nextCursor }} />;`;- expect(isRenderCopy(source, "rows")).toBe(false);+ expect(renderCopies(source)).toEqual([]);+ });++ it("ignores a table without infinite", () => {+ expect(renderCopies(`<Table data={data.map((r) => r)} />`)).toEqual([]);});});diff --git a/packages/jobs/src/inngest/functions/tasks/table-export.ts b/packages/jobs/src/inngest/functions/tasks/table-export.tsindex 6e7c3e1937..954cf6a58d 100644--- a/packages/jobs/src/inngest/functions/tasks/table-export.ts+++ b/packages/jobs/src/inngest/functions/tasks/table-export.ts@@ -8,6 +8,7 @@ import { MAX_EXPORT_ROWS } from "../../../table-jobs/csv";import {composeExport,type ExportDeps,+ removeExportParts,writeExportPart} from "../../../table-jobs/export-run";import {@@ -162,11 +163,13 @@ export const tableExportFunction = inngest.createFunction(if (!cursor) break;}+ const exportKey = tableExportStorageKey(companyId, userId, exportId);await step.run("compose", () =>- composeExport(deps(), {- partKeys,- key: tableExportStorageKey(companyId, userId, exportId)- })+ composeExport(deps(), { partKeys, key: exportKey })+ );+ // Its own step: a compose retried after the parts were gone would fail.+ await step.run("remove-parts", () =>+ removeExportParts(deps(), { partKeys, key: exportKey }));await step.run("notify", () =>diff --git a/packages/jobs/src/table-jobs/export-run.ts b/packages/jobs/src/table-jobs/export-run.tsindex 350de9ec04..9e398775ea 100644--- a/packages/jobs/src/table-jobs/export-run.ts+++ b/packages/jobs/src/table-jobs/export-run.ts@@ -116,17 +116,26 @@ export function concatParts(});}-/** Streams the parts into the export file, then deletes the parts. A part- * that cannot be deleted is logged; the nightly cleanup removes it. */+/** Streams the parts into the export file. The parts stay, so a retried+ * compose writes the same file again; `removeExportParts` deletes them in+ * the step after it. */export async function composeExport(deps: Pick<ExportDeps, "storage">,args: { partKeys: string[]; key: string }-): Promise<{ partsRemoved: boolean }> {+): Promise<void> {await deps.storage.upload(args.key,concatParts(deps.storage, args.partKeys),CSV_CONTENT_TYPE);+}++/** Deletes the parts of a composed export. A part that cannot be deleted is+ * logged; the nightly cleanup removes it. */+export async function removeExportParts(+ deps: Pick<ExportDeps, "storage">,+ args: { partKeys: string[]; key: string }+): Promise<{ partsRemoved: boolean }> {try {await deps.storage.remove(args.partKeys);return { partsRemoved: true };diff --git a/packages/jobs/src/table-jobs/run.test.ts b/packages/jobs/src/table-jobs/run.test.tsindex eb50e16b09..6b1b843b64 100644--- a/packages/jobs/src/table-jobs/run.test.ts+++ b/packages/jobs/src/table-jobs/run.test.ts@@ -6,7 +6,11 @@ import { describe, expect, it } from "vitest";import { planIdSlices } from "./bulk";import { applyIdSlice, collectIds, storeSelectedIds } from "./bulk-run";import { MAX_EXPORT_ROWS } from "./csv";-import { composeExport, writeExportPart } from "./export-run";+import {+ composeExport,+ removeExportParts,+ writeExportPart+} from "./export-run";import type {TableBulkActionCounts,TableJobHandlers,@@ -177,6 +181,7 @@ describe("the export steps", () => {if (!cursor) break;}await composeExport({ storage }, { partKeys, key: "export.csv" });+ await removeExportParts({ storage }, { partKeys, key: "export.csv" });return { files, text, partKeys, rows };}@@ -195,6 +200,18 @@ describe("the export steps", () => {expect([...files.keys()]).toEqual(["export.csv"]);});+ it("writes the same file when compose runs again before the parts are removed", async () => {+ const { storage, text } = memoryStorage();+ const partKeys = ["parts/0.csv", "parts/1.csv"];+ await storage.upload(partKeys[0]!, "Part,Name\r\nrow_0,Row 0", "text/csv");+ await storage.upload(partKeys[1]!, "\r\nrow_1,Row 1", "text/csv");++ await composeExport({ storage }, { partKeys, key: "export.csv" });+ await composeExport({ storage }, { partKeys, key: "export.csv" });++ expect(text("export.csv")).toBe("Part,Name\r\nrow_0,Row 0\r\nrow_1,Row 1");+ });+it("writes the header alone for an empty list", async () => {const { text } = await runExport(0);expect(text("export.csv")).toBe("Part,Name");