AI-native design editor. Open-source Figma alternative.
perf(fig): save edited .fig files by rewriting only their changed records (#993)
Danila Poyarkov78b1eada5049
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 68880778c..c312e8d7f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -138,10 +138,10 @@- Keep a layer's `.fig` ID the same across saves (#890). Layers created in OpenPencil were numbered in tree order on each save until the file was reopened, so inserting a layer shifted the IDs of the layers after it; variable collections and modes could also be renumbered.- Store crash recovery snapshots of any size. Snapshots of documents over 127 MiB failed to save to IndexedDB and stayed in memory for the rest of the session.- Keep an opened `.fig` file saved until it is edited. Laying out its first page, which recomputes auto-layout sizes and positions, and showing another page for the first time, which loads its layers from the file, marked it unsaved, so closing it asked to save changes nobody made.+- Save variable edits to an opened `.fig` file. After changing only variables, variable collections, or the document's color profile or libraries, saving wrote the original file unchanged.- Run crash recovery and autosave after edits, not whenever the canvas redraws. Opening a document, laying out a page, or loading a font started a recovery snapshot or an autosave, which encoded the whole document again once another page had loaded. In Safari, where every opened file gets recovery snapshots, a large page froze the browser for minutes after it first appeared.- Open image-heavy `.fig` files without the canvas running out of memory (#924). Decoded images stay within a fixed budget, and documents with many large images draw previews sized to the view, decoded in the background a few at a time, while exports keep the full images.- Show tables in AI chat replies at the chat's text size and weight, with a light header and copy as their only action, and task lists with a checkbox in place of the bullet. Tooltips, the copy menu, and the confirmation before opening a link follow the app's style, and links no longer load each site's favicon.-- Keep an opened `.fig` file saved until it is edited. Laying out its first page, which recomputes auto-layout sizes and positions, marked it unsaved, so closing it asked to save changes nobody made.- Fill one axis of a grid cell: a grid child set to fill its width or its height no longer fills both, and an auto-layout child of a grid keeps its own size unless it fills, as in Figma. Design JSX renders and exports `w="fill"` and `h="fill"` in grids, and HTML and Tailwind export leave out the size a child fills, so it stretches in the browser too.- Resize auto-layout frames that fill across their parent with it: one with a fixed size on that axis kept its old size. A filling child still counts toward a hugging parent's size, as in Figma.- Set `layoutSizingHorizontal` and `layoutSizingVertical` in scripts as Figma does: Fill is recorded on the child for that axis instead of on the frame's own sizing, which `.fig` export dropped, text switches its `textAutoResize`, and sizing Figma refuses, such as Hug on a frame without auto layout, throws Figma's error.@@ -237,6 +237,7 @@### Performance+- Save an edited `.fig` file by rewriting only the layers that changed and copying the rest from the file it was opened from, as it is stored, without decoding it; in the browser that work runs off the main thread. Saving the 109 MB Preline UI kit after an edit takes about 3 seconds instead of over two minutes, and crash recovery no longer freezes Safari after the first edit to a large file. Editing a component also rewrites its instances on pages that were not opened yet.- Open `.fig` design kits with less memory: layers inside instances hold their component layers' paints, glyphs, and other unchanged values instead of a copy each, about a third fewer objects and an eighth less memory per loaded page.- Save large `.fig` documents with less memory: once every page is loaded, the first save releases the decoded original file, and saves write from the document instead of a copy of it, about 1.5 GB less on a 74 MB design kit.- Save edited `.fig` documents with many component instances faster: each overridden layer is addressed and its variables resolved once, overrides merge through an index instead of a scan per override, and the components Figma keeps off-canvas are read once instead of on every save. Saving a 74 MB design kit after an edit takes about 8 seconds instead of 13, and later saves about 7.diff --git a/packages/core/src/canvas/renderer/fonts.ts b/packages/core/src/canvas/renderer/fonts.tsindex 6a0d821c1..55e86e5cb 100644--- a/packages/core/src/canvas/renderer/fonts.ts+++ b/packages/core/src/canvas/renderer/fonts.ts@@ -141,6 +141,6 @@ export async function prepareForExport(syncFontGeneration(r)withTextMeasurer((node, maxWidth) => r.measureTextNode(node, maxWidth),- () => computeAllLayouts(graph, pageId)+ () => graph.applyDerivedLayoutDuring(() => computeAllLayouts(graph, pageId)))}diff --git a/packages/core/src/editor/pages.ts b/packages/core/src/editor/pages.tsindex 98d5f74e6..21605fc3a 100644--- a/packages/core/src/editor/pages.ts+++ b/packages/core/src/editor/pages.ts@@ -176,7 +176,7 @@ export function createPageActions(ctx: EditorContext) {if (generation !== pageSwitchGeneration) return nullif (ctx.getRenderer() || populated) {options.onProgress?.({ phase: 'layout', detail: page.name })- computeAllLayouts(ctx.graph, pageId)+ ctx.graph.applyDerivedLayoutDuring(() => computeAllLayouts(ctx.graph, pageId))}throwIfAborted(options.signal)return generation === pageSwitchGeneration ? { pageId, generation } : null@@ -213,7 +213,7 @@ export function createPageActions(ctx: EditorContext) {if (populated === null || graph !== ctx.graph) return falseawait resolvePageFonts(page.id, page.name, {})if (graph !== ctx.graph) return false- computeAllLayouts(graph, page.id)+ graph.applyDerivedLayoutDuring(() => computeAllLayouts(graph, page.id))return true}diff --git a/packages/core/src/io/formats/fig/export-setup.ts b/packages/core/src/io/formats/fig/export-setup.tsnew file mode 100644index 000000000..791b8bb28--- /dev/null+++ b/packages/core/src/io/formats/fig/export-setup.ts@@ -0,0 +1,505 @@+import type { CanvasKit } from 'canvaskit-wasm'+import { deflateSync, inflateSync } from 'fflate'+import { toUint8Array } from 'js-base64'++import {+ buildComponentPropIndex,+ exportCanvasGuides,+ importCanvasGuides,+ stringToGuid,+ type FigNodeChangeExportRuntime+} from '@open-pencil/fig/node-change'+import { getCompiledSchema, getSchemaBytes } from '@open-pencil/kiwi/fig/codec'+import type { NodeChange } from '@open-pencil/kiwi/fig/codec'+import { decodeBinarySchema, compileSchema, ByteBuffer } from '@open-pencil/kiwi/schema-runtime'+import {+ ownsSlotContent,+ readBehaviour,+ renameBehaviourProperties,+ withBehaviour,+ type PluginDataEntry,+ type SceneGraph+} from '@open-pencil/scene-graph'+import { fractionalPosition } from '@open-pencil/scene-graph/order-keys'+import type { GUID } from '@open-pencil/scene-graph/primitives'++import type { SkiaRenderer } from '#core/canvas'+import { CANVAS_BG_COLOR, IS_BROWSER, IS_TAURI } from '#core/constants'+import { applyEnabledLibrariesPluginData } from '#core/io/formats/fig/library-metadata'+import { findFigThumbnailPageId } from '#core/io/formats/fig/thumbnail-page'+import { renderThumbnail } from '#core/io/formats/raster'+import {+ makeCanvasNodeChange,+ makeDocumentNodeChange,+ settleFontDigestMap+} from '#core/kiwi/fig/node-change/serialize'++import { assignVariableGuid, assignVariableGuids } from './variable-export'++/**+ * What a `.fig` write shares however many records it writes: the GUID maps, caches and+ * canvases, the schema, and the thumbnail and images beside the records.+ */+const THUMBNAIL_1X1 = toUint8Array(+ 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=='+)++export type KiwiNodeChange = NodeChange & Record<string, unknown>+export type FigExportPage = ReturnType<SceneGraph['getPages']>[number]++export interface CanvasExportEntry {+ page: FigExportPage+ canvasGuid: GUID+ canvasNc: KiwiNodeChange+}++function collectImageEntries(graph: SceneGraph): Array<{ name: string; data: Uint8Array }> {+ const entries: Array<{ name: string; data: Uint8Array }> = []+ for (const [hash, data] of graph.images) {+ entries.push({ name: `images/${hash}`, data })+ }+ return entries+}++const THUMBNAIL_WIDTH = 512+const THUMBNAIL_HEIGHT = 512++async function renderFigThumbnail(+ graph: SceneGraph,+ pageId: string | undefined,+ ck?: CanvasKit,+ renderer?: SkiaRenderer,+ renderHeadless = false+): Promise<Uint8Array> {+ if (!pageId) return THUMBNAIL_1X1+ if (ck && renderer) {+ return (+ renderThumbnail(ck, renderer, graph, pageId, THUMBNAIL_WIDTH, THUMBNAIL_HEIGHT) ??+ THUMBNAIL_1X1+ )+ }+ if (!renderHeadless || IS_BROWSER || IS_TAURI) return THUMBNAIL_1X1+ const { headlessRenderThumbnail } = await import('#core/io/formats/raster')+ return (+ (await headlessRenderThumbnail(graph, pageId, THUMBNAIL_WIDTH, THUMBNAIL_HEIGHT)) ??+ THUMBNAIL_1X1+ )+}++/**+ * The GUID a layer that was never opened from a file is saved under: its graph ID, which has+ * Figma's `sessionID:localID` shape and never changes, so later saves keep it. Saved GUIDs of+ * opened layers and GUIDs already written win over it.+ */+function ownGuid(+ id: string,+ sourceGuidValues: ReadonlySet<string>,+ assignedGuidValues: ReadonlySet<string>+): GUID | null {+ if (!/^\d+:\d+$/.test(id)) return null+ const guid = stringToGuid(id)+ const key = `${guid.sessionID}:${guid.localID}`+ return sourceGuidValues.has(key) || assignedGuidValues.has(key) ? null : guid+}++function assignOwnLayerGuids(+ graph: SceneGraph,+ sourceGuidValues: ReadonlySet<string>,+ nodeIdToGuid: Map<string, GUID>,+ assignedGuidValues: Set<string>+): void {+ for (const node of graph.nodes.values()) {+ if (node.source.id || node.id === graph.rootId || nodeIdToGuid.has(node.id)) continue+ const guid = ownGuid(node.id, sourceGuidValues, assignedGuidValues)+ if (!guid) continue+ nodeIdToGuid.set(node.id, guid)+ assignedGuidValues.add(`${guid.sessionID}:${guid.localID}`)+ }+}++/**+ * The first local ID the counter can mint in sessions 0 and 1: past every saved GUID and every+ * ID that layers, collections, modes, variables and component properties are saved under.+ */+function firstUnclaimedLocalId(graph: SceneGraph, propertyIds: readonly string[]): number {+ const ids = [...propertyIds]+ for (const node of graph.nodes.values()) ids.push(node.source.id || node.id)+ for (const [collectionId, collection] of graph.variableCollections) {+ ids.push(collectionId, ...collection.modes.map((mode) => mode.modeId))+ ids.push(...collection.variableIds)+ }+ let last = 0+ for (const id of ids) {+ if (!/^\d+:\d+$/.test(id)) continue+ const guid = stringToGuid(id)+ if (guid.sessionID <= 1) last = Math.max(last, guid.localID)+ }+ return last + 1+}++function collectComponentPropertyIds(graph: SceneGraph): string[] {+ const ids = new Set<string>()+ for (const node of graph.getAllNodes()) {+ for (const definition of node.componentPropertyDefinitions) ids.add(definition.id)+ for (const reference of node.componentPropertyReferences) ids.add(reference.propertyId)+ for (const propertyId of Object.keys(node.componentPropertyAssignments)) ids.add(propertyId)+ for (const spec of node.variantPropSpecs) ids.add(spec.propDefId)+ }+ return [...ids]+}++function assignComponentPropertyGuids(+ propertyIds: readonly string[],+ localIdCounter: { value: number },+ propertyIdToGuid: Map<string, GUID>,+ assignedGuidValues: Set<string>,+ nodeSourceGuidValues: Set<string>+): void {+ for (const propertyId of propertyIds) {+ const guid = assignVariableGuid(+ propertyId,+ localIdCounter,+ assignedGuidValues,+ nodeSourceGuidValues+ )+ propertyIdToGuid.set(propertyId, guid)+ }+}++/**+ * Behaviours bind component properties by id, so they follow the ids' new GUIDs. The renamed+ * plugin data is written in place of the node's own; the document keeps its ids.+ */+function renamedBehaviourPluginData(+ graph: SceneGraph,+ propertyIdToGuid: Map<string, GUID>+): Map<string, PluginDataEntry[]> {+ const rename = (propertyId: string) => {+ const guid = propertyIdToGuid.get(propertyId)+ return guid ? `${guid.sessionID}:${guid.localID}` : propertyId+ }+ const renamed = new Map<string, PluginDataEntry[]>()+ for (const node of graph.getAllNodes()) {+ const behaviour = readBehaviour(node)+ if (behaviour)+ renamed.set(node.id, withBehaviour(node, renameBehaviourProperties(behaviour, rename)))+ }+ return renamed+}++function applyImportedCanvasFields(page: FigExportPage, canvasNc: KiwiNodeChange): void {+ if ('backgroundColor' in page.source.fig.rawNodeFields) {+ canvasNc.backgroundColor = structuredClone(page.source.fig.rawNodeFields.backgroundColor)+ }+ if ('backgroundPaints' in page.source.fig.rawNodeFields) {+ canvasNc.backgroundPaints = structuredClone(+ page.source.fig.rawNodeFields.backgroundPaints+ ) as NodeChange['backgroundPaints']+ }+ if (!page.source.id) return+ if (!('pageType' in page.source.fig.rawNodeFields)) delete canvasNc.pageType+ if (page.guides.length > 0) {+ const normalized = exportCanvasGuides(page.guides)+ const raw = page.source.fig.rawNodeFields.guides+ canvasNc.guides =+ Array.isArray(raw) && JSON.stringify(importCanvasGuides(raw)) === JSON.stringify(page.guides)+ ? structuredClone(raw)+ : normalized+ }+ const strokeJoin = page.source.fig.rawNodeFields.strokeJoin+ if (typeof strokeJoin === 'string') canvasNc.strokeJoin = strokeJoin+ const strokeWeight = page.source.fig.rawNodeFields.strokeWeight+ if (typeof strokeWeight === 'number') canvasNc.strokeWeight = strokeWeight+}++function buildCanvasEntries(+ graph: SceneGraph,+ pages: FigExportPage[],+ docGuid: GUID,+ localIdCounter: { value: number },+ nodeIdToGuid: Map<string, GUID>,+ assignedGuidValues: Set<string>,+ sourceGuidValues: ReadonlySet<string>,+ ownGuids = true+): { canvasEntries: CanvasExportEntry[]; internalCanvasGuid: GUID | null } {+ const canvasEntries: CanvasExportEntry[] = []+ let internalCanvasGuid: GUID | null = null+ for (let p = 0; p < pages.length; p++) {+ const page = pages[p]+ const canvasGuid = (() => {+ if (!page.source.id) {+ return (+ (ownGuids ? ownGuid(page.id, sourceGuidValues, assignedGuidValues) : null) ?? {+ sessionID: 0,+ localID: localIdCounter.value+++ }+ )+ }++ const importedGuid = stringToGuid(page.source.id)+ const key = `${importedGuid.sessionID}:${importedGuid.localID}`++ if (!assignedGuidValues.has(key)) return importedGuid++ return { sessionID: 0, localID: localIdCounter.value++ }+ })()+ // Advance counter past any source.id-derived GUID to prevent collisions+ // with subsequently generated variable/collection GUIDs.+ if (page.source.id && canvasGuid.sessionID === 0) {+ localIdCounter.value = Math.max(localIdCounter.value, canvasGuid.localID + 1)+ }+ nodeIdToGuid.set(page.id, canvasGuid)+ assignedGuidValues.add(`${canvasGuid.sessionID}:${canvasGuid.localID}`)+ if (page.internalOnly) internalCanvasGuid = canvasGuid++ const canvasNc = makeCanvasNodeChange(+ canvasGuid,+ docGuid,+ page.source.orderKey ?? fractionalPosition(p),+ page.name,+ {+ backgroundOpacity: 1,+ backgroundColor: { ...CANVAS_BG_COLOR },+ backgroundEnabled: true+ }+ )+ applyImportedCanvasFields(page, canvasNc)+ if (page.internalOnly) canvasNc.internalOnly = true+ canvasEntries.push({ page, canvasGuid, canvasNc })+ }++ const hasSharedStyles = [...graph.nodes.values()].some((node) => node.sharedStyleType !== null)+ const hasSlotContent = [...graph.nodes.values()].some((node) => ownsSlotContent(graph, node))+ if (+ (graph.variableCollections.size > 0 || hasSharedStyles || hasSlotContent) &&+ internalCanvasGuid === null+ ) {+ internalCanvasGuid = { sessionID: 0, localID: localIdCounter.value++ }+ assignedGuidValues.add(`${internalCanvasGuid.sessionID}:${internalCanvasGuid.localID}`)+ canvasEntries.push({+ page: { id: '', name: 'Internal Only Canvas', internalOnly: true } as FigExportPage,+ canvasGuid: internalCanvasGuid,+ canvasNc: makeCanvasNodeChange(+ internalCanvasGuid,+ docGuid,+ fractionalPosition(canvasEntries.length),+ 'Internal Only Canvas',+ { internalOnly: true }+ )+ })+ }++ return { canvasEntries, internalCanvasGuid }+}++/**+ * When the document was imported from a .fig file, preserve the original kiwi schema for both+ * encoding and embedding. Figma's schema has more types and fields than ours, and encoding with+ * ours would produce field IDs that don't align with the embedded schema.+ */+export function exportSchema(graph: SceneGraph): {+ compiled: ReturnType<typeof getCompiledSchema>+ schemaDeflated: Uint8Array+} {+ if (graph.figSchemaDeflated) {+ const schemaBytes = inflateSync(graph.figSchemaDeflated)+ const figSchema = decodeBinarySchema(new ByteBuffer(schemaBytes))+ return {+ compiled: compileSchema(figSchema) as ReturnType<typeof getCompiledSchema>,+ schemaDeflated: graph.figSchemaDeflated+ }+ }+ return { compiled: getCompiledSchema(), schemaDeflated: deflateSync(getSchemaBytes()) }+}++/** GUID maps, caches and canvases one write shares across every record it serializes. */+export interface FigExportSetup {+ graph: SceneGraph+ runtime: FigNodeChangeExportRuntime+ docGuid: GUID+ documentNc: KiwiNodeChange+ localIdCounter: { value: number }+ blobs: Uint8Array[]+ pages: FigExportPage[]+ canvasEntries: CanvasExportEntry[]+ internalCanvasGuid: GUID | null+ nodeIdToGuid: Map<string, GUID>+ assignedGuidValues: Set<string>+ varIdToGuid: Map<string, GUID>+ modeIdToGuid: Map<string, GUID>+ propertyIdToGuid: Map<string, GUID>+ fontDigestMap: Map<string, Uint8Array>+ glyphBlobMap: Map<string, number>+ blobIndexByHex: Map<string, number>+ componentPropertyDefinitionsById: ReturnType<typeof buildComponentPropIndex>+ pluginDataOverrides: ReadonlyMap<string, PluginDataEntry[]>+ recordOwners?: ReadonlyMap<string, string>+}++export interface FigExportSetupOptions {+ /** The first local ID a new GUID may take, above every GUID the target archive holds. */+ nextLocalId?: number+ /** The target archive's blobs, so the indices of new blobs follow theirs. */+ blobs?: Uint8Array[]+ /**+ * The layer each archive GUID belongs to; only that layer writes the archive's record. Writing+ * into an archive also gives new layers and pages counter GUIDs, past every GUID it holds,+ * rather than their graph IDs, which a record on a page not loaded may already use.+ */+ recordOwners?: ReadonlyMap<string, string>+ /** Writing into an archive: the variables, collections and modes it holds keep their GUIDs. */+ archiveResourceIds?: ReadonlySet<string>+}++/**+ * The shared state of one write. The last await before the records: after it every record comes+ * from one synchronous pass over the document, and the font digests cover what that pass reads.+ */+export async function prepareFigExport(+ graph: SceneGraph,+ runtime: FigNodeChangeExportRuntime,+ options: FigExportSetupOptions = {}+): Promise<FigExportSetup> {+ const fontDigestMap = await settleFontDigestMap(graph)+ const docGuid = { sessionID: 0, localID: 0 }+ const localIdCounter = { value: 2 }++ const documentNc = makeDocumentNodeChange(docGuid, graph.documentColorSpace)+ const rootNode = graph.getNode(graph.rootId)+ if (rootNode) Object.assign(documentNc, rootNode.source.fig.rawNodeFields)+ applyEnabledLibrariesPluginData(documentNc, graph)++ const pages = graph.getPages(true)+ const nodeIdToGuid = new Map<string, GUID>()+ const assignedGuidValues = new Set<string>()+ // Reserve the document GUID to prevent imported nodes with source.id "0:0"+ // from reusing the document's own GUID slot.+ assignedGuidValues.add(`${docGuid.sessionID}:${docGuid.localID}`)+ const varIdToGuid = new Map<string, GUID>()+ const modeIdToGuid = new Map<string, GUID>()+ const propertyIdToGuid = new Map<string, GUID>()++ const nodeSourceGuidValues = new Set<string>()+ for (const node of graph.nodes.values()) {+ if (node.source.id) nodeSourceGuidValues.add(node.source.id)+ }+ const propertyIds = collectComponentPropertyIds(graph)+ // Before any canvas, variable or layer takes a counter GUID, so none collides.+ localIdCounter.value = Math.max(+ localIdCounter.value,+ firstUnclaimedLocalId(graph, propertyIds),+ options.nextLocalId ?? 0+ )++ const { canvasEntries, internalCanvasGuid } = buildCanvasEntries(+ graph,+ pages,+ docGuid,+ localIdCounter,+ nodeIdToGuid,+ assignedGuidValues,+ nodeSourceGuidValues,+ !options.recordOwners+ )+ // Archive records keep their GUIDs: claim them before any copy that kept one can.+ for (const [guid, nodeId] of options.recordOwners ?? []) {+ if (nodeIdToGuid.has(nodeId) || !graph.nodes.has(nodeId)) continue+ nodeIdToGuid.set(nodeId, stringToGuid(guid))+ assignedGuidValues.add(guid)+ }++ // Assign variable GUIDs AFTER canvas entries so that source.id-derived+ // canvas GUIDs don't collide with generated variable GUIDs.+ const archived = options.archiveResourceIds+ assignVariableGuids(+ graph,+ localIdCounter,+ varIdToGuid,+ modeIdToGuid,+ assignedGuidValues,+ nodeSourceGuidValues,+ archived ? (id) => archived.has(id) : undefined+ )++ assignComponentPropertyGuids(+ propertyIds,+ localIdCounter,+ propertyIdToGuid,+ assignedGuidValues,+ nodeSourceGuidValues+ )+ const pluginDataOverrides = renamedBehaviourPluginData(graph, propertyIdToGuid)+ if (!options.recordOwners)+ assignOwnLayerGuids(graph, nodeSourceGuidValues, nodeIdToGuid, assignedGuidValues)++ return {+ graph,+ runtime,+ docGuid,+ documentNc,+ localIdCounter,+ blobs: options.blobs ?? [],+ pages,+ canvasEntries,+ internalCanvasGuid,+ nodeIdToGuid,+ assignedGuidValues,+ varIdToGuid,+ modeIdToGuid,+ propertyIdToGuid,+ fontDigestMap,+ glyphBlobMap: new Map(),+ blobIndexByHex: new Map(),+ componentPropertyDefinitionsById: buildComponentPropIndex(graph),+ pluginDataOverrides,+ recordOwners: options.recordOwners+ }+}++/** The serializer options a setup's records share. */+export function figExportRecordOptions(setup: FigExportSetup) {+ return {+ nodeIdToGuid: setup.nodeIdToGuid,+ fontDigestMap: setup.fontDigestMap,+ varIdToGuid: setup.varIdToGuid,+ glyphBlobMap: setup.glyphBlobMap,+ blobIndexByHex: setup.blobIndexByHex,+ assignedGuidValues: setup.assignedGuidValues,+ componentPropertyDefinitionsById: setup.componentPropertyDefinitionsById,+ modeIdToGuid: setup.modeIdToGuid,+ propertyIdToGuid: setup.propertyIdToGuid,+ pluginDataOverrides: setup.pluginDataOverrides,+ recordOwners: setup.recordOwners,+ runtime: setup.runtime+ }+}++export interface FigFileExtras {+ thumbnailPNG: Uint8Array+ metaJSON: string+ images: Array<{ name: string; data: Uint8Array }>+}++/** The thumbnail, metadata and images a written `.fig` carries besides its records. */+export async function figFileExtras(+ graph: SceneGraph,+ pageId: string | undefined,+ ck: CanvasKit | undefined,+ renderer: SkiaRenderer | undefined,+ renderHeadlessThumbnail: boolean+): Promise<FigFileExtras> {+ const currentPageId = pageId ?? findFigThumbnailPageId(graph.getPages(true))+ const thumbnailPNG = await renderFigThumbnail(+ graph,+ currentPageId,+ ck,+ renderer,+ renderHeadlessThumbnail+ )+ const metaJSON = JSON.stringify({+ version: 1,+ app: 'OpenPencil',+ createdAt: new Date().toISOString()+ })+ return { thumbnailPNG, metaJSON, images: collectImageEntries(graph) }+}diff --git a/packages/core/src/io/formats/fig/export.ts b/packages/core/src/io/formats/fig/export.tsindex 1d092ce96..d06ad3211 100644--- a/packages/core/src/io/formats/fig/export.ts+++ b/packages/core/src/io/formats/fig/export.ts@@ -1,43 +1,26 @@-/* eslint-disable max-lines -- FIG export orchestration keeps shared GUID state in one pipeline */import type { CanvasKit } from 'canvaskit-wasm'-import { deflateSync, inflateSync } from 'fflate'-import { toUint8Array } from 'js-base64'import { compressFigDataSync } from '@open-pencil/fig'-import {- buildComponentPropIndex,- placeSlotContent,- exportCanvasGuides,- importCanvasGuides,- stringToGuid,- type FigNodeChangeExportRuntime-} from '@open-pencil/fig/node-change'-import { initCodec, getCompiledSchema, getSchemaBytes } from '@open-pencil/kiwi/fig/codec'+import { placeSlotContent, type FigNodeChangeExportRuntime } from '@open-pencil/fig/node-change'+import { initCodec } from '@open-pencil/kiwi/fig/codec'import type { NodeChange } from '@open-pencil/kiwi/fig/codec'-import { decodeBinarySchema, compileSchema, ByteBuffer } from '@open-pencil/kiwi/schema-runtime'-import {- ownsSlotContent,- readBehaviour,- renameBehaviourProperties,- withBehaviour,- type PluginDataEntry,- type SceneGraph-} from '@open-pencil/scene-graph'+import type { SceneGraph } from '@open-pencil/scene-graph'import { fractionalPosition } from '@open-pencil/scene-graph/order-keys'import type { GUID } from '@open-pencil/scene-graph/primitives'import type { SkiaRenderer } from '#core/canvas'import { withFigExportRuntime } from '#core/canvas/text/shape'-import { CANVAS_BG_COLOR, IS_BROWSER, IS_TAURI } from '#core/constants'-import { applyEnabledLibrariesPluginData } from '#core/io/formats/fig/library-metadata'-import { findFigThumbnailPageId } from '#core/io/formats/fig/thumbnail-page'-import { renderThumbnail } from '#core/io/formats/raster'+import { IS_BROWSER, IS_TAURI } from '#core/constants'import {- sceneNodeToKiwi,- settleFontDigestMap,- makeDocumentNodeChange,- makeCanvasNodeChange-} from '#core/kiwi/fig/node-change/serialize'+ exportSchema,+ figExportRecordOptions,+ figFileExtras,+ prepareFigExport,+ type FigExportSetup,+ type KiwiNodeChange+} from '#core/io/formats/fig/export-setup'+import { writePatchedFigFile } from '#core/io/formats/fig/patch-export'+import { sceneNodeToKiwi } from '#core/kiwi/fig/node-change/serialize'import { cloneSceneGraphForFigExport } from '#core/kiwi/fig/parse/transfer'import {hasPendingReaderPages,@@ -49,280 +32,10 @@ import { originalFigArchive } from '#core/kiwi/fig/session/original-archive'import {appendVariableNodeChanges,sequentialPositions,- assignSharedStyleGuids,- assignVariableGuid,- assignVariableGuids+ assignSharedStyleGuids} from './variable-export'-const THUMBNAIL_1X1 = toUint8Array(- 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg=='-)--type KiwiNodeChange = NodeChange & Record<string, unknown>-type FigExportPage = ReturnType<SceneGraph['getPages']>[number]--interface CanvasExportEntry {- page: FigExportPage- canvasGuid: GUID- canvasNc: KiwiNodeChange-}--function collectImageEntries(graph: SceneGraph): Array<{ name: string; data: Uint8Array }> {- const entries: Array<{ name: string; data: Uint8Array }> = []- for (const [hash, data] of graph.images) {- entries.push({ name: `images/${hash}`, data })- }- return entries-}--const THUMBNAIL_WIDTH = 512-const THUMBNAIL_HEIGHT = 512--async function renderFigThumbnail(- graph: SceneGraph,- pageId: string | undefined,- ck?: CanvasKit,- renderer?: SkiaRenderer,- renderHeadless = false-): Promise<Uint8Array> {- if (!pageId) return THUMBNAIL_1X1- if (ck && renderer) {- return (- renderThumbnail(ck, renderer, graph, pageId, THUMBNAIL_WIDTH, THUMBNAIL_HEIGHT) ??- THUMBNAIL_1X1- )- }- if (!renderHeadless || IS_BROWSER || IS_TAURI) return THUMBNAIL_1X1- const { headlessRenderThumbnail } = await import('#core/io/formats/raster')- return (- (await headlessRenderThumbnail(graph, pageId, THUMBNAIL_WIDTH, THUMBNAIL_HEIGHT)) ??- THUMBNAIL_1X1- )-}--/**- * The GUID a layer that was never opened from a file is saved under: its graph ID, which has- * Figma's `sessionID:localID` shape and never changes, so later saves keep it. Saved GUIDs of- * opened layers and GUIDs already written win over it.- */-function ownGuid(- id: string,- sourceGuidValues: ReadonlySet<string>,- assignedGuidValues: ReadonlySet<string>-): GUID | null {- if (!/^\d+:\d+$/.test(id)) return null- const guid = stringToGuid(id)- const key = `${guid.sessionID}:${guid.localID}`- return sourceGuidValues.has(key) || assignedGuidValues.has(key) ? null : guid-}--function assignOwnLayerGuids(- graph: SceneGraph,- sourceGuidValues: ReadonlySet<string>,- nodeIdToGuid: Map<string, GUID>,- assignedGuidValues: Set<string>-): void {- for (const node of graph.nodes.values()) {- if (node.source.id || node.id === graph.rootId || nodeIdToGuid.has(node.id)) continue- const guid = ownGuid(node.id, sourceGuidValues, assignedGuidValues)- if (!guid) continue- nodeIdToGuid.set(node.id, guid)- assignedGuidValues.add(`${guid.sessionID}:${guid.localID}`)- }-}--/**- * The first local ID the counter can mint in sessions 0 and 1: past every saved GUID and every- * ID that layers, collections, modes, variables and component properties are saved under.- */-function firstUnclaimedLocalId(graph: SceneGraph, propertyIds: readonly string[]): number {- const ids = [...propertyIds]- for (const node of graph.nodes.values()) ids.push(node.source.id || node.id)- for (const [collectionId, collection] of graph.variableCollections) {- ids.push(collectionId, ...collection.modes.map((mode) => mode.modeId))- ids.push(...collection.variableIds)- }- let last = 0- for (const id of ids) {- if (!/^\d+:\d+$/.test(id)) continue- const guid = stringToGuid(id)- if (guid.sessionID <= 1) last = Math.max(last, guid.localID)- }- return last + 1-}--function collectComponentPropertyIds(graph: SceneGraph): string[] {- const ids = new Set<string>()- for (const node of graph.getAllNodes()) {- for (const definition of node.componentPropertyDefinitions) ids.add(definition.id)- for (const reference of node.componentPropertyReferences) ids.add(reference.propertyId)- for (const propertyId of Object.keys(node.componentPropertyAssignments)) ids.add(propertyId)- for (const spec of node.variantPropSpecs) ids.add(spec.propDefId)- }- return [...ids]-}--function assignComponentPropertyGuids(- propertyIds: readonly string[],- localIdCounter: { value: number },- propertyIdToGuid: Map<string, GUID>,- assignedGuidValues: Set<string>,- nodeSourceGuidValues: Set<string>-): void {- for (const propertyId of propertyIds) {- const guid = assignVariableGuid(- propertyId,- localIdCounter,- assignedGuidValues,- nodeSourceGuidValues- )- propertyIdToGuid.set(propertyId, guid)- }-}--/**- * Behaviours bind component properties by id, so they follow the ids' new GUIDs. The renamed- * plugin data is written in place of the node's own; the document keeps its ids.- */-function renamedBehaviourPluginData(- graph: SceneGraph,- propertyIdToGuid: Map<string, GUID>-): Map<string, PluginDataEntry[]> {- const rename = (propertyId: string) => {- const guid = propertyIdToGuid.get(propertyId)- return guid ? `${guid.sessionID}:${guid.localID}` : propertyId- }- const renamed = new Map<string, PluginDataEntry[]>()- for (const node of graph.getAllNodes()) {- const behaviour = readBehaviour(node)- if (behaviour)- renamed.set(node.id, withBehaviour(node, renameBehaviourProperties(behaviour, rename)))- }- return renamed-}--function applyImportedCanvasFields(page: FigExportPage, canvasNc: KiwiNodeChange): void {- if ('backgroundColor' in page.source.fig.rawNodeFields) {- canvasNc.backgroundColor = structuredClone(page.source.fig.rawNodeFields.backgroundColor)- }- if ('backgroundPaints' in page.source.fig.rawNodeFields) {- canvasNc.backgroundPaints = structuredClone(- page.source.fig.rawNodeFields.backgroundPaints- ) as NodeChange['backgroundPaints']- }- if (!page.source.id) return- if (!('pageType' in page.source.fig.rawNodeFields)) delete canvasNc.pageType- if (page.guides.length > 0) {- const normalized = exportCanvasGuides(page.guides)- const raw = page.source.fig.rawNodeFields.guides- canvasNc.guides =- Array.isArray(raw) && JSON.stringify(importCanvasGuides(raw)) === JSON.stringify(page.guides)- ? structuredClone(raw)- : normalized- }- const strokeJoin = page.source.fig.rawNodeFields.strokeJoin- if (typeof strokeJoin === 'string') canvasNc.strokeJoin = strokeJoin- const strokeWeight = page.source.fig.rawNodeFields.strokeWeight- if (typeof strokeWeight === 'number') canvasNc.strokeWeight = strokeWeight-}--function buildCanvasEntries(- graph: SceneGraph,- pages: FigExportPage[],- docGuid: GUID,- localIdCounter: { value: number },- nodeIdToGuid: Map<string, GUID>,- assignedGuidValues: Set<string>,- sourceGuidValues: ReadonlySet<string>-): { canvasEntries: CanvasExportEntry[]; internalCanvasGuid: GUID | null } {- const canvasEntries: CanvasExportEntry[] = []- let internalCanvasGuid: GUID | null = null- for (let p = 0; p < pages.length; p++) {- const page = pages[p]- const canvasGuid = (() => {- if (!page.source.id) {- return (- ownGuid(page.id, sourceGuidValues, assignedGuidValues) ?? {- sessionID: 0,- localID: localIdCounter.value++- }- )- }-- const importedGuid = stringToGuid(page.source.id)- const key = `${importedGuid.sessionID}:${importedGuid.localID}`-- if (!assignedGuidValues.has(key)) return importedGuid-- return { sessionID: 0, localID: localIdCounter.value++ }- })()- // Advance counter past any source.id-derived GUID to prevent collisions- // with subsequently generated variable/collection GUIDs.- if (page.source.id && canvasGuid.sessionID === 0) {- localIdCounter.value = Math.max(localIdCounter.value, canvasGuid.localID + 1)- }- nodeIdToGuid.set(page.id, canvasGuid)- assignedGuidValues.add(`${canvasGuid.sessionID}:${canvasGuid.localID}`)- if (page.internalOnly) internalCanvasGuid = canvasGuid-- const canvasNc = makeCanvasNodeChange(- canvasGuid,- docGuid,- page.source.orderKey ?? fractionalPosition(p),- page.name,- {- backgroundOpacity: 1,- backgroundColor: { ...CANVAS_BG_COLOR },- backgroundEnabled: true- }- )- applyImportedCanvasFields(page, canvasNc)- if (page.internalOnly) canvasNc.internalOnly = true- canvasEntries.push({ page, canvasGuid, canvasNc })- }-- const hasSharedStyles = [...graph.nodes.values()].some((node) => node.sharedStyleType !== null)- const hasSlotContent = [...graph.nodes.values()].some((node) => ownsSlotContent(graph, node))- if (- (graph.variableCollections.size > 0 || hasSharedStyles || hasSlotContent) &&- internalCanvasGuid === null- ) {- internalCanvasGuid = { sessionID: 0, localID: localIdCounter.value++ }- assignedGuidValues.add(`${internalCanvasGuid.sessionID}:${internalCanvasGuid.localID}`)- canvasEntries.push({- page: { id: '', name: 'Internal Only Canvas', internalOnly: true } as FigExportPage,- canvasGuid: internalCanvasGuid,- canvasNc: makeCanvasNodeChange(- internalCanvasGuid,- docGuid,- fractionalPosition(canvasEntries.length),- 'Internal Only Canvas',- { internalOnly: true }- )- })- }-- return { canvasEntries, internalCanvasGuid }-}--interface InternalResourceContext {- graph: SceneGraph- nodeChanges: KiwiNodeChange[]- internalCanvasGuid: GUID | null- localIdCounter: { value: number }- blobs: Uint8Array[]- nodeIdToGuid: Map<string, GUID>- fontDigestMap: Map<string, Uint8Array>- varIdToGuid: Map<string, GUID>- modeIdToGuid: Map<string, GUID>- glyphBlobMap: Map<string, number>- blobIndexByHex: Map<string, number>- assignedGuidValues: Set<string>- componentPropertyDefinitionsById: ReturnType<typeof buildComponentPropIndex>- propertyIdToGuid: Map<string, GUID>- pluginDataOverrides: ReadonlyMap<string, PluginDataEntry[]>- runtime: FigNodeChangeExportRuntime-}+type InternalResourceContext = FigExportSetup & { nodeChanges: KiwiNodeChange[] }/*** Children already written under a canvas. Shared styles, variables and the canvas's own@@ -395,8 +108,16 @@ export async function exportFigFile(): Promise<Uint8Array> {const originalArchive = await originalFigArchive(sourceGraph)if (originalArchive) return originalArchive.slice()- return withFigExportRuntime(sourceGraph, ck, (runtime) =>- writeFigFile(sourceGraph, runtime, ck, renderer, pageId, renderHeadlessThumbnail)+ return withFigExportRuntime(+ sourceGraph,+ ck,+ async (runtime) =>+ (await writePatchedFigFile(sourceGraph, runtime, {+ ck,+ renderer,+ pageId,+ renderHeadlessThumbnail+ })) ?? writeFigFile(sourceGraph, runtime, ck, renderer, pageId, renderHeadlessThumbnail))}@@ -417,116 +138,20 @@ async function writeFigFile(graph = cloneSceneGraphForFigExport(sourceGraph)populateReaderExport(sourceGraph, graph)}- // The last await before the records: from here to the encoded message every record comes from- // one synchronous pass over the document, and the digests cover every font that pass reads.- const fontDigestMap = await settleFontDigestMap(graph)-- // When the document was imported from a .fig file, preserve the original- // kiwi schema for both encoding and embedding. For the current version of- // Figma, likely for quite some time, schema has more types/fields than our- // subset, and using our schema to encode would produce field IDs that don't- // align with the embedded schema. By compiling and using the original- // schema, we improve the roundtrip-ability... This requires further work.- let compiled: ReturnType<typeof getCompiledSchema>- let schemaDeflated: Uint8Array- if (graph.figSchemaDeflated) {- const schemaBytes = inflateSync(graph.figSchemaDeflated)- const figSchema = decodeBinarySchema(new ByteBuffer(schemaBytes))- compiled = compileSchema(figSchema) as ReturnType<typeof getCompiledSchema>- schemaDeflated = graph.figSchemaDeflated- } else {- compiled = getCompiledSchema()- schemaDeflated = deflateSync(getSchemaBytes())- }-- const docGuid = { sessionID: 0, localID: 0 }- const localIdCounter = { value: 2 }-- const documentNc = makeDocumentNodeChange(docGuid, graph.documentColorSpace)- const rootNode = graph.getNode(graph.rootId)- if (rootNode) Object.assign(documentNc, rootNode.source.fig.rawNodeFields)- applyEnabledLibrariesPluginData(documentNc, graph)- const nodeChanges: KiwiNodeChange[] = [documentNc]-- const blobs: Uint8Array[] = []- const pages = graph.getPages(true)- const nodeIdToGuid = new Map<string, GUID>()- const assignedGuidValues = new Set<string>()- // Reserve the document GUID to prevent imported nodes with source.id "0:0"- // from reusing the document's own GUID slot.- assignedGuidValues.add(`${docGuid.sessionID}:${docGuid.localID}`)- const varIdToGuid = new Map<string, GUID>()- const modeIdToGuid = new Map<string, GUID>()- const propertyIdToGuid = new Map<string, GUID>()- const glyphBlobMap = new Map<string, number>()- const blobIndexByHex = new Map<string, number>()- const componentPropertyDefinitionsById = buildComponentPropIndex(graph)-- const nodeSourceGuidValues = new Set<string>()- for (const node of graph.nodes.values()) {- if (node.source.id) nodeSourceGuidValues.add(node.source.id)- }- const propertyIds = collectComponentPropertyIds(graph)- // Before any canvas, variable or layer takes a counter GUID, so none collides.- localIdCounter.value = Math.max(localIdCounter.value, firstUnclaimedLocalId(graph, propertyIds))-- const { canvasEntries, internalCanvasGuid } = buildCanvasEntries(- graph,- pages,- docGuid,- localIdCounter,- nodeIdToGuid,- assignedGuidValues,- nodeSourceGuidValues- )-- // Assign variable GUIDs AFTER canvas entries so that source.id-derived- // canvas GUIDs don't collide with generated variable GUIDs.- assignVariableGuids(- graph,- localIdCounter,- varIdToGuid,- modeIdToGuid,- assignedGuidValues,- nodeSourceGuidValues- )-- assignComponentPropertyGuids(- propertyIds,- localIdCounter,- propertyIdToGuid,- assignedGuidValues,- nodeSourceGuidValues- )- const pluginDataOverrides = renamedBehaviourPluginData(graph, propertyIdToGuid)- assignOwnLayerGuids(graph, nodeSourceGuidValues, nodeIdToGuid, assignedGuidValues)-+ const setup = await prepareFigExport(graph, runtime)+ const { compiled, schemaDeflated } = exportSchema(graph)+ const { canvasEntries, internalCanvasGuid, localIdCounter, blobs } = setup+ const nodeChanges: KiwiNodeChange[] = [setup.documentNc]for (const entry of canvasEntries) nodeChanges.push(entry.canvasNc)- appendInternalResources({- graph,- nodeChanges,- internalCanvasGuid,- localIdCounter,- blobs,- nodeIdToGuid,- fontDigestMap,- varIdToGuid,- modeIdToGuid,- glyphBlobMap,- blobIndexByHex,- assignedGuidValues,- componentPropertyDefinitionsById,- propertyIdToGuid,- pluginDataOverrides,- runtime- })+ appendInternalResources({ ...setup, nodeChanges })const orderedCanvasEntries = [...canvasEntries.filter((entry) => entry.page.internalOnly),...canvasEntries.filter((entry) => !entry.page.internalOnly)]const slotContentRecords: KiwiNodeChange[] = []+ const recordOptions = { ...figExportRecordOptions(setup), slotContentRecords }for (const { page, canvasGuid } of orderedCanvasEntries) {const children = graph.getChildren(page.id)@@ -534,20 +159,15 @@ async function writeFigFile(const base = countCanvasChildren(nodeChanges, canvasGuid)for (let i = 0; i < children.length; i++) {nodeChanges.push(- ...sceneNodeToKiwi(children[i], canvasGuid, base + i, localIdCounter, graph, blobs, {- nodeIdToGuid,- fontDigestMap,- varIdToGuid,- glyphBlobMap,- blobIndexByHex,- assignedGuidValues,- componentPropertyDefinitionsById,- modeIdToGuid,- propertyIdToGuid,- slotContentRecords,- pluginDataOverrides,- runtime- })+ ...sceneNodeToKiwi(+ children[i],+ canvasGuid,+ base + i,+ localIdCounter,+ graph,+ blobs,+ recordOptions+ ))}}@@ -569,23 +189,7 @@ async function writeFigFile(}const kiwiData = compiled.encodeMessage(msg)-- const currentPageId = pageId ?? findFigThumbnailPageId(pages)- const thumbnailPNG = await renderFigThumbnail(- graph,- currentPageId,- ck,- renderer,- renderHeadlessThumbnail- )-- const metaJSON = JSON.stringify({- version: 1,- app: 'OpenPencil',- createdAt: new Date().toISOString()- })-- const imageEntries = collectImageEntries(graph)+ const extras = await figFileExtras(graph, pageId, ck, renderer, renderHeadlessThumbnail)const version = graph.figKiwiVersion ?? undefined@@ -595,15 +199,22 @@ async function writeFigFile(await invoke<ArrayBuffer>('build_fig_file', {schemaDeflated: Array.from(schemaDeflated),kiwiData: Array.from(kiwiData),- thumbnailPng: Array.from(thumbnailPNG),- metaJson: metaJSON,- images: imageEntries.map((e) => ({ name: e.name, data: Array.from(e.data) })),+ thumbnailPng: Array.from(extras.thumbnailPNG),+ metaJson: extras.metaJSON,+ images: extras.images.map((e) => ({ name: e.name, data: Array.from(e.data) })),figKiwiVersion: version}))}- return compressFigData(schemaDeflated, kiwiData, thumbnailPNG, metaJSON, imageEntries, version)+ return compressFigData(+ schemaDeflated,+ kiwiData,+ extras.thumbnailPNG,+ extras.metaJSON,+ extras.images,+ version+ )}export { compressFigDataSync } from '@open-pencil/fig'diff --git a/packages/core/src/io/formats/fig/patch-export.ts b/packages/core/src/io/formats/fig/patch-export.tsnew file mode 100644index 000000000..d41a819d6--- /dev/null+++ b/packages/core/src/io/formats/fig/patch-export.ts@@ -0,0 +1,340 @@+import type { CanvasKit } from 'canvaskit-wasm'++import {+ figCheckpointRecords,+ type FigArchiveRecordInfo,+ type FigRecordPatch+} from '@open-pencil/fig'+import type { FigNodeChangeExportRuntime } from '@open-pencil/fig/node-change'+import { stringToGuid } from '@open-pencil/kiwi/fig/guid'+import { ownsSlotContent, type SceneGraph, type SceneNode } from '@open-pencil/scene-graph'+import { siblingOrderKeys } from '@open-pencil/scene-graph/order-keys'+import type { GUID } from '@open-pencil/scene-graph/primitives'++import type { SkiaRenderer } from '#core/canvas'+import {+ figExportRecordOptions,+ figFileExtras,+ prepareFigExport,+ type FigExportSetup,+ type KiwiNodeChange+} from '#core/io/formats/fig/export-setup'+import { sceneNodeToKiwi } from '#core/kiwi/fig/node-change/serialize'+import { figArchive, type FigArchiveAccess } from '#core/kiwi/fig/session/archive'+import { archiveChanges, type ArchiveChangeSummary } from '#core/kiwi/fig/session/archive-changes'+import {+ isReaderPagePending,+ populateFigPage,+ readerCheckpoint+} from '#core/kiwi/fig/session/document-state'++import { appendVariableNodeChanges } from './variable-export'++export interface PatchedFigFileOptions {+ ck?: CanvasKit+ renderer?: SkiaRenderer+ pageId?: string+ renderHeadlessThumbnail?: boolean+}++/** Slot content an instance writes as records of its own, which this path leaves alone. */+class UnpatchableChangeError extends Error {+ override name = 'UnpatchableChangeError'+}++/**+ * The layer whose record holds a layer: itself, or the outermost instance it sits in, since an+ * instance writes its contents as overrides and derived data on its own record.+ */+function recordOwner(graph: SceneGraph, id: string): string | null {+ const start = graph.getNode(id)+ if (!start) return null+ let owner: SceneNode = start+ for (let node: SceneNode | undefined = start; node;) {+ if (ownsSlotContent(graph, node)) throw new UnpatchableChangeError()+ if (node.type === 'INSTANCE') owner = node+ if (node.type === 'CANVAS') return owner.id+ node = node.parentId ? graph.getNode(node.parentId) : undefined+ }+ return null+}++/** The component definition a record sits in, if any. */+function enclosingComponent(graph: SceneGraph, id: string): SceneNode | undefined {+ for (let node = graph.getNode(id); node;) {+ if (node.type === 'COMPONENT') return node+ if (node.type === 'CANVAS') return undefined+ node = node.parentId ? graph.getNode(node.parentId) : undefined+ }+ return undefined+}++interface RecordPlan {+ /** Layers whose records are written: pages, and layers outside instances. */+ records: Set<string>+ orderKeys: Map<string, string>+ removed: GUID[]+ replaceVariables: boolean+}++/** The layer each archive record became, by record GUID, and the record each such layer is. */+interface ArchiveRecords {+ byGuid: ReadonlyMap<string, string>+ byNode: ReadonlyMap<string, string>+}++function archiveRecords(graph: SceneGraph): ArchiveRecords | null {+ const checkpoint = readerCheckpoint(graph)+ if (!checkpoint) return null+ const byGuid = figCheckpointRecords(checkpoint)+ return { byGuid, byNode: new Map([...byGuid].map(([guid, nodeId]) => [nodeId, guid])) }+}++/** An edited component changes every instance of it, and so every component nesting one. */+function addComponentInstances(+ graph: SceneGraph,+ records: Set<string>,+ add: (id: string) => string | null+): void {+ const visited = new Set<string>()+ const pending = Array.from(records)+ for (let id = pending.pop(); id !== undefined; id = pending.pop()) {+ const component = enclosingComponent(graph, id)+ if (!component || visited.has(component.id)) continue+ visited.add(component.id)+ for (const instanceId of graph.instanceIndex.get(component.id) ?? []) {+ const owner = add(instanceId)+ if (owner) pending.push(owner)+ }+ }+}++/** Records carry the values of the variables they use, and the modes they set. */+function addVariableUsers(+ graph: SceneGraph,+ changes: ArchiveChangeSummary,+ add: (id: string) => unknown+): void {+ for (const node of graph.nodes.values()) {+ const usesChanged = Object.values(node.boundVariables).some((id) =>+ changes.changedVariables.has(id)+ )+ const setsModes = changes.collectionsChanged && Object.keys(node.variableModes).length > 0+ if (usesChanged || setsModes) add(node.id)+ }+}++/** Siblings keep the positions the archive saved; one that cannot moves, and so is written. */+function siblingPositions(graph: SceneGraph, records: Set<string>): Map<string, string> {+ const orderKeys = new Map<string, string>()+ const parents = new Set<string>()+ for (const id of records) {+ const node = graph.getNode(id)+ if (node && node.type !== 'CANVAS' && node.parentId) parents.add(node.parentId)+ }+ for (const parentId of parents) {+ const siblings = graph.getChildren(parentId)+ const keys = siblingOrderKeys(siblings.map((sibling) => sibling.source.orderKey))+ siblings.forEach((sibling, index) => {+ orderKeys.set(sibling.id, keys[index])+ if (keys[index] !== sibling.source.orderKey) records.add(sibling.id)+ })+ }+ return orderKeys+}++function planRecords(+ graph: SceneGraph,+ changes: ArchiveChangeSummary,+ archived: ArchiveRecords+): RecordPlan {+ const records = new Set<string>()+ const add = (id: string) => {+ const owner = recordOwner(graph, id)+ if (owner) records.add(owner)+ return owner+ }+ for (const id of changes.touched) add(id)+ addComponentInstances(graph, records, add)+ const replaceVariables = changes.changedVariables.size > 0 || changes.collectionsChanged+ if (replaceVariables) addVariableUsers(graph, changes, add)+ const orderKeys = siblingPositions(graph, records)+ const removed: GUID[] = []+ for (const [guid, nodeId] of archived.byGuid)+ if (!graph.nodes.has(nodeId)) removed.push(stringToGuid(guid))+ return { records, orderKeys, removed, replaceVariables }+}++/** Components holding edited layers, by archive GUID, for finding the pages that use them. */+function editedComponents(+ graph: SceneGraph,+ changes: ArchiveChangeSummary,+ archived: ArchiveRecords+): string[] {+ const components = new Set<string>()+ for (const id of changes.touched) {+ const component = enclosingComponent(graph, id)+ const guid = component ? archived.byNode.get(component.id) : undefined+ if (guid) components.add(guid)+ }+ return [...components]+}++/**+ * Instances on pages not loaded yet still carry the overrides and sizes derived from their+ * components as saved. Loading those pages lets the edits reach them, and their records follow.+ */+async function loadComponentUsePages(+ graph: SceneGraph,+ archive: FigArchiveAccess,+ changes: ArchiveChangeSummary,+ archived: ArchiveRecords+): Promise<boolean> {+ const components = editedComponents(graph, changes, archived)+ if (components.length === 0) return false+ let loaded = false+ for (const pageGuid of await archive.componentPages(components)) {+ const pageId = archived.byGuid.get(pageGuid)+ if (pageId && isReaderPagePending(graph, pageId))+ loaded = populateFigPage(graph, pageId) || loaded+ }+ return loaded+}++function writeRecords(setup: FigExportSetup, plan: RecordPlan): KiwiNodeChange[] | null {+ const { graph } = setup+ const canvasGuids = new Map(setup.canvasEntries.map((entry) => [entry.page.id, entry]))+ const slotContentRecords: KiwiNodeChange[] = []+ const options = {+ ...figExportRecordOptions(setup),+ slotContentRecords,+ orderKeys: plan.orderKeys,+ writeChildren: false+ }+ const depth = (id: string) => {+ let count = 0+ for (let node = graph.getNode(id); node?.parentId; node = graph.getNode(node.parentId)) count+++ return count+ }+ // Parents first, so a new parent has its GUID before its children name it.+ const ordered = [...plan.records].toSorted((left, right) => depth(left) - depth(right))+ const written: KiwiNodeChange[] = []+ for (const id of ordered) {+ const node = graph.getNode(id)+ if (!node) continue+ if (node.type === 'CANVAS') {+ const entry = canvasGuids.get(node.id)+ if (entry) written.push(entry.canvasNc)+ continue+ }+ const parent = node.parentId ? graph.getNode(node.parentId) : undefined+ if (!parent) continue+ const parentGuid =+ parent.type === 'CANVAS'+ ? canvasGuids.get(parent.id)?.canvasGuid+ : setup.nodeIdToGuid.get(parent.id)+ if (!parentGuid) return null+ const index = graph.getChildren(parent.id).findIndex((child) => child.id === node.id)+ written.push(+ ...sceneNodeToKiwi(node, parentGuid, index, setup.localIdCounter, graph, setup.blobs, options)+ )+ }+ // Instance slot content is written as records of its own on the internal canvas.+ return slotContentRecords.length > 0 ? null : written+}++function variableRecords(setup: FigExportSetup, archiveInternalCanvas: boolean): KiwiNodeChange[] {+ const internal = setup.canvasEntries.find((entry) => entry.page.internalOnly)+ if (!internal) return []+ const records: KiwiNodeChange[] = archiveInternalCanvas ? [] : [internal.canvasNc]+ appendVariableNodeChanges(+ setup.graph,+ records,+ internal.canvasGuid,+ setup.varIdToGuid,+ setup.modeIdToGuid+ )+ return records+}++async function buildPatch(+ source: SceneGraph,+ runtime: FigNodeChangeExportRuntime,+ info: FigArchiveRecordInfo,+ changes: ArchiveChangeSummary,+ archived: ArchiveRecords+): Promise<{ patch: FigRecordPatch; graph: SceneGraph } | null> {+ // Serializing reads the document and writes nothing to it, so the records come from it directly.+ const graph = source+ const plan = planRecords(graph, changes, archived)+ const setup = await prepareFigExport(graph, runtime, {+ nextLocalId: info.nextLocalId,+ blobs: Array.from({ length: info.blobCount }, () => new Uint8Array(0)),+ recordOwners: archived.byGuid,+ archiveResourceIds: changes.resourceIds+ })+ const records = writeRecords(setup, plan)+ if (!records) return null+ // The document's own fields, its plugin data among them, are written on the document record.+ if (changes.documentChanged || changes.touched.has(graph.rootId)) {+ const guid = info.documentId ? stringToGuid(info.documentId) : setup.docGuid+ records.unshift({ ...setup.documentNc, guid })+ }+ if (plan.replaceVariables) {+ const internalPage = graph.getPages(true).find((page) => page.internalOnly)+ for (const record of variableRecords(setup, !!internalPage)) records.push(record)+ }+ return {+ graph,+ patch: {+ records,+ blobs: setup.blobs.slice(info.blobCount),+ removed: plan.removed,+ replaceVariables: plan.replaceVariables+ }+ }+}++/**+ * Write a document opened from a `.fig` as that archive with only its changed records+ * rewritten. The main thread serializes what changed; merging and compressing the rest happen+ * where the archive is held, the reader's worker in the browser. Null when the document did+ * not come from an archive, or changed in a way this path does not write.+ */+export async function writePatchedFigFile(+ source: SceneGraph,+ runtime: FigNodeChangeExportRuntime,+ options: PatchedFigFileOptions = {}+): Promise<Uint8Array | null> {+ const archive = figArchive(source)+ const changes = archiveChanges(source)+ const loaded = archiveRecords(source)+ if (!archive || !changes || !loaded) return null+ // Pages loaded for the components add their records, so the map is read again after them.+ const archived = (await loadComponentUsePages(source, archive, changes, loaded))+ ? archiveRecords(source)+ : loaded+ if (!archived) return null+ const info = await archive.info()+ let built: Awaited<ReturnType<typeof buildPatch>>+ try {+ built = await buildPatch(source, runtime, info, changes, archived)+ } catch (error) {+ if (error instanceof UnpatchableChangeError) return null+ throw error+ }+ if (!built) return null+ const extras = await figFileExtras(+ built.graph,+ options.pageId,+ options.ck,+ options.renderer,+ options.renderHeadlessThumbnail ?? false+ )+ const archivedImages = new Set(info.imageHashes.map((hash) => `images/${hash}`))+ return archive.patch(built.patch, {+ thumbnailPNG: extras.thumbnailPNG,+ metaJSON: extras.metaJSON,+ images: extras.images.filter((image) => !archivedImages.has(image.name))+ })+}diff --git a/packages/core/src/io/formats/fig/read.ts b/packages/core/src/io/formats/fig/read.tsindex ec1680f0d..e81097ce5 100644--- a/packages/core/src/io/formats/fig/read.ts+++ b/packages/core/src/io/formats/fig/read.ts@@ -1,6 +1,5 @@import type { FigPageManifestEntry } from '@open-pencil/kiwi/fig'import type { SceneGraph } from '@open-pencil/scene-graph'-import { randomHex } from '@open-pencil/scene-graph/random'import { IS_BROWSER } from '#core/constants'import { deserializeSceneGraph } from '#core/kiwi/fig/parse/transfer'@@ -8,6 +7,11 @@ import {registerFigPopulationWorker,registerOriginalArchiveRequest} from '#core/kiwi/fig/population/client'+import {+ localFigArchive,+ registerFigArchive,+ workerFigArchive+} from '#core/kiwi/fig/session/archive'import { createFigSessionWorker } from '#core/kiwi/fig/session/client'import {registerReaderRecovery,@@ -33,7 +37,11 @@ function parseFigFileSync(buffer: ArrayBuffer, options: ParseFigFileOptions = {}options.signal?.throwIfAborted()const bytes = buffer.slice(0)registerReaderSession(bytes, reader.session, reader.diagnostics)- registerOriginalArchiveRequest(reader.graph, async () => new Uint8Array(bytes.slice(0)))+ // Taken now, while the records are decoded: the archive outlives the reader's session.+ const archiveInfo = reader.session.archiveRecordInfo()+ const archive = localFigArchive(bytes, () => archiveInfo)+ registerFigArchive(reader.graph, archive)+ registerOriginalArchiveRequest(reader.graph, () => archive.original())return reader.graph}@@ -45,7 +53,6 @@ export function parseFigFileViaWorker(options.signal?.throwIfAborted()const worker = createFigSessionWorker()const channel = new MessageChannel()- const pendingArchives = new Map<string, (bytes: Uint8Array) => void>()const abort = () => {channel.port1.postMessage({ type: 'dispose' })channel.port1.close()@@ -55,15 +62,6 @@ export function parseFigFileViaWorker(options.signal?.addEventListener('abort', abort, { once: true })const cleanupAbort = () => options.signal?.removeEventListener('abort', abort)- // A listener of its own: registerFigPopulationWorker takes over port1.onmessage- // once the graph arrives, and archive requests are made after that.- channel.port1.addEventListener('message', (e: MessageEvent<FigSessionResponse>) => {- if (e.data.type !== 'original-archive-result') return- const resolveArchive = pendingArchives.get(e.data.requestId)- if (!resolveArchive) return- pendingArchives.delete(e.data.requestId)- resolveArchive(e.data.bytes)- })channel.port1.onmessage = (e: MessageEvent<FigSessionResponse>) => {if (e.data.type === 'page-manifest') {options.onPages?.(e.data.pages)@@ -81,18 +79,17 @@ export function parseFigFileViaWorker(const graph = deserializeSceneGraph(e.data.graph)if (options.populate === 'first-page' || options.populate === 'none') {cleanupAbort()- if (!e.data.checkpoint) throw new Error('Missing reader checkpoint')- registerReaderRecovery(graph, buffer.slice(0), e.data.checkpoint)- registerFigPopulationWorker(graph, worker, channel.port1)- registerOriginalArchiveRequest(- graph,- () =>- new Promise<Uint8Array>((resolveArchive) => {- const requestId = randomHex()- pendingArchives.set(requestId, resolveArchive)- channel.port1.postMessage({ type: 'original-archive', requestId })- })+ const { checkpoint, archiveInfo } = e.data+ if (!checkpoint || !archiveInfo) throw new Error('Missing reader checkpoint')+ const bytes = buffer.slice(0)+ registerReaderRecovery(graph, bytes, checkpoint)+ // The worker writes the archive back; without it the main thread does, from its copy.+ const archive = workerFigArchive(channel.port1, worker, () =>+ localFigArchive(bytes, () => archiveInfo))+ registerFigPopulationWorker(graph, worker, channel.port1, () => archive.dropWorker())+ registerFigArchive(graph, archive)+ registerOriginalArchiveRequest(graph, () => archive.original())} else {cleanupAbort()channel.port1.close()diff --git a/packages/core/src/io/formats/fig/variable-export.ts b/packages/core/src/io/formats/fig/variable-export.tsindex c93254308..3ed17bf4b 100644--- a/packages/core/src/io/formats/fig/variable-export.ts+++ b/packages/core/src/io/formats/fig/variable-export.ts@@ -23,9 +23,15 @@ export function assignVariableGuid(id: string,localIdCounter: { value: number },assignedGuidValues: Set<string>,- nodeSourceGuidValues: Set<string>+ nodeSourceGuidValues: Set<string>,+ keepsGuid: (id: string) => boolean = () => true): GUID {- if (/^\d+:\d+$/.test(id) && !assignedGuidValues.has(id) && !nodeSourceGuidValues.has(id)) {+ if (+ keepsGuid(id) &&+ /^\d+:\d+$/.test(id) &&+ !assignedGuidValues.has(id) &&+ !nodeSourceGuidValues.has(id)+ ) {const guid = stringToGuid(id)assignedGuidValues.add(id)return guid@@ -41,14 +47,16 @@ export function assignVariableGuids(varIdToGuid: Map<string, GUID>,modeIdToGuid: Map<string, GUID>,assignedGuidValues: Set<string>,- nodeSourceGuidValues: Set<string>+ nodeSourceGuidValues: Set<string>,+ keepsGuid?: (id: string) => boolean): void {for (const [colId, col] of graph.variableCollections) {const colGuid = assignVariableGuid(colId,localIdCounter,assignedGuidValues,- nodeSourceGuidValues+ nodeSourceGuidValues,+ keepsGuid)varIdToGuid.set(colId, colGuid)for (const mode of col.modes) {@@ -56,7 +64,8 @@ export function assignVariableGuids(mode.modeId,localIdCounter,assignedGuidValues,- nodeSourceGuidValues+ nodeSourceGuidValues,+ keepsGuid)modeIdToGuid.set(mode.modeId, modeGuid)}@@ -65,7 +74,8 @@ export function assignVariableGuids(varId,localIdCounter,assignedGuidValues,- nodeSourceGuidValues+ nodeSourceGuidValues,+ keepsGuid)varIdToGuid.set(varId, varGuid)}diff --git a/packages/core/src/kiwi/fig/population/client.ts b/packages/core/src/kiwi/fig/population/client.tsindex 508fc55c0..25b4a54d1 100644--- a/packages/core/src/kiwi/fig/population/client.ts+++ b/packages/core/src/kiwi/fig/population/client.ts@@ -1,6 +1,8 @@import type { SceneGraph } from '@open-pencil/scene-graph'import { randomHex } from '@open-pencil/scene-graph/random'+import { releaseFigArchive } from '#core/kiwi/fig/session/archive'+import { archiveChanges, trackArchiveChanges } from '#core/kiwi/fig/session/archive-changes'import { updateReaderRecovery, releaseReaderRecovery } from '#core/kiwi/fig/session/document-state'import type { FigSessionResponse } from '#core/kiwi/fig/session/protocol'@@ -37,7 +39,8 @@ function emitTelemetry(detail: FigPopulationWorkerTelemetry): void {export function registerFigPopulationWorker(graph: SceneGraph,worker: Worker,- port?: MessagePort+ port?: MessagePort,+ stopWorker?: () => void): void {if (graph.nodes.size > MAX_FIG_POPULATION_WORKER_NODES) {emitTelemetry({ event: 'fallback', reason: 'oversized' })@@ -45,10 +48,10 @@ export function registerFigPopulationWorker(worker.terminate()return}- populationWorkers.set(graph, createDisposalOnlyWorker(worker, port))+ populationWorkers.set(graph, createRetiredWorker(worker, port, !!stopWorker))return}- const client = createPopulationWorkerClient(graph, worker, port)+ const client = createPopulationWorkerClient(graph, worker, port, stopWorker)populationWorkers.set(graph, client)emitTelemetry({ event: 'registered' })}@@ -74,11 +77,21 @@ export function registerOriginalArchiveRequest(reordered: invalidate})originalArchiveRequests.set(graph, entry)+ trackArchiveChanges(graph)+}++/** Variables and document settings change without layer events. */+function divergedOutsideLayers(graph: SceneGraph): boolean {+ const changes = archiveChanges(graph)+ return (+ !!changes &&+ (changes.changedVariables.size > 0 || changes.collectionsChanged || changes.documentChanged)+ )}export async function requestOriginalArchive(graph: SceneGraph): Promise<Uint8Array | null> {const entry = originalArchiveRequests.get(graph)- if (!entry?.valid) return null+ if (!entry?.valid || divergedOutsideLayers(graph)) return nullconst archive = await entry.request()return originalArchiveRequests.get(graph)?.valid === true &&originalArchiveRequests.get(graph) === entry@@ -92,19 +105,30 @@ export function releaseFigPopulationWorker(graph: SceneGraph): void {populationWorkers.delete(graph)originalArchiveRequests.get(graph)?.unbind()originalArchiveRequests.delete(graph)+ releaseFigArchive(graph)}export interface FigPopulationWorker {populate: (pageId: string, signal?: AbortSignal) => Promise<boolean | null>+ /**+ * Stop loading pages through the worker. The worker itself keeps serving the document's+ * archive until the document closes, which releases it with the archive.+ */terminate: () => void}-function createDisposalOnlyWorker(worker: Worker, port: MessagePort): FigPopulationWorker {+/** A worker too large to load pages through; one nothing else holds is disposed with it. */+function createRetiredWorker(+ worker: Worker,+ port: MessagePort,+ sharedWorker: boolean+): FigPopulationWorker {+ if (sharedWorker) port.postMessage({ type: 'retire' })let disposed = falsereturn {populate: () => Promise.resolve(null),terminate() {- if (disposed) return+ if (disposed || sharedWorker) returndisposed = trueemitTelemetry({ event: 'terminated' })port.postMessage({ type: 'dispose' })@@ -119,11 +143,19 @@ export function createFigPopulationWorker(graph: SceneGraph): FigPopulationWorkereturn populationWorkers.get(graph) ?? null}+/**+ * The client owns the worker unless `stopWorker` is given: a document whose archive the worker+ * also holds passes the archive's, so that retiring page loading leaves the worker serving it+ * and stopping the worker lets the archive carry on without it.+ */export function createPopulationWorkerClient(graph: SceneGraph,worker: Pick<Worker, 'postMessage' | 'terminate' | 'onerror' | 'onmessage'>,- port?: Pick<MessagePort, 'postMessage' | 'start' | 'close' | 'onmessage'>+ port?: Pick<MessagePort, 'postMessage' | 'start' | 'close' | 'onmessage'>,+ stopWorker?: () => void): FigPopulationWorker {+ const sharedWorker = !!stopWorker+ const stopWorkerNow = stopWorker ?? (() => worker.terminate())const pending = new Map<string,{@@ -152,7 +184,11 @@ export function createPopulationWorkerClient(unbind?.()unbind = undefined}- const fail = (emit = true) => {+ /**+ * Stop loading pages here. A graph that diverged only retires the worker's session, so the+ * worker keeps serving the archive; an abandoned or failed load stops the worker itself.+ */+ const fail = (emit = true, stop = true) => {stale = trueif (emit) emitTelemetry({ event: 'fallback', reason: 'worker-error' })for (const request of pending.values()) {@@ -162,7 +198,8 @@ export function createPopulationWorkerClient(}pending.clear()releaseSubscription()- worker.terminate()+ if (stop || !sharedWorker) stopWorkerNow()+ else port?.postMessage({ type: 'retire' })populationWorkers.delete(graph)}unbind = graph.onNodeEvents({@@ -173,7 +210,7 @@ export function createPopulationWorkerClient(reordered: invalidate})const receive = (result: WorkerResult) => {- if (result.type === 'population-error') return fail()+ if (result.type === 'population-error') return fail(true, false)const request = pending.get(result.requestId)if (!request) returnclearTimeout(request.timeout)@@ -244,9 +281,11 @@ export function createPopulationWorkerClient(if (disposed) returndisposed = trueemitTelemetry({ event: 'terminated' })- port?.postMessage({ type: 'dispose' })- port?.close()- fail(false)+ if (!sharedWorker) {+ port?.postMessage({ type: 'dispose' })+ port?.close()+ }+ fail(false, false)}}}diff --git a/packages/core/src/kiwi/fig/session/archive-changes.ts b/packages/core/src/kiwi/fig/session/archive-changes.tsnew file mode 100644index 000000000..ae1dc7a8b--- /dev/null+++ b/packages/core/src/kiwi/fig/session/archive-changes.ts@@ -0,0 +1,99 @@+import { isEqual } from 'es-toolkit/predicate'++import type { SceneGraph } from '@open-pencil/scene-graph'++/**+ * How a document opened from a `.fig` archive has diverged from it, for writing only what+ * changed. Layers loading from the archive are not changes, nor is the layout a page gets as+ * it loads or as its fonts arrive; the layout an edit causes is, since it moves the layers+ * around the edit.+ */+interface ArchiveChanges {+ touched: Set<string>+ /** The variables, collections and modes the archive holds. */+ resourceIds: Set<string>+ variables: Map<string, unknown>+ collections: unknown+ settings: unknown+}++const changes = new WeakMap<SceneGraph, ArchiveChanges>()++function settingsSnapshot(graph: SceneGraph): unknown {+ return structuredClone({+ colorSpace: graph.documentColorSpace,+ libraries: [...graph.enabledLibraries]+ })+}++function collectionsSnapshot(graph: SceneGraph): unknown {+ return structuredClone({+ collections: [...graph.variableCollections],+ activeMode: [...graph.activeMode]+ })+}++function resourceIdsOf(graph: SceneGraph): Set<string> {+ const ids = new Set(graph.variables.keys())+ for (const [id, collection] of graph.variableCollections) {+ ids.add(id)+ for (const mode of collection.modes) ids.add(mode.modeId)+ }+ return ids+}++export function trackArchiveChanges(graph: SceneGraph): void {+ const entry: ArchiveChanges = {+ touched: new Set(),+ resourceIds: resourceIdsOf(graph),+ variables: new Map(+ [...graph.variables].map(([id, variable]) => [id, structuredClone(variable)])+ ),+ collections: collectionsSnapshot(graph),+ settings: settingsSnapshot(graph)+ }+ const touch = (id: string) => {+ if (!graph.isApplyingImportedState && !graph.isApplyingDerivedLayout) entry.touched.add(id)+ }+ graph.onNodeEvents({+ created: (node) => touch(node.id),+ updated: touch,+ reparented: touch,+ reordered: touch,+ // A removed layer has no record of its own to write: the instance or component holding it+ // changes, and the parent leads there.+ deleted: (_id, parentId) => {+ if (parentId) touch(parentId)+ }+ })+ changes.set(graph, entry)+}++export interface ArchiveChangeSummary {+ /**+ * Layers created, updated or moved since the archive opened, and the parents of layers+ * removed; some may be gone since.+ */+ touched: ReadonlySet<string>+ resourceIds: ReadonlySet<string>+ /** Variables whose value, binding or presence changed. */+ changedVariables: ReadonlySet<string>+ collectionsChanged: boolean+ documentChanged: boolean+}++export function archiveChanges(graph: SceneGraph): ArchiveChangeSummary | undefined {+ const entry = changes.get(graph)+ if (!entry) return undefined+ const changedVariables = new Set<string>()+ for (const [id, variable] of graph.variables)+ if (!isEqual(entry.variables.get(id), variable)) changedVariables.add(id)+ for (const id of entry.variables.keys()) if (!graph.variables.has(id)) changedVariables.add(id)+ return {+ touched: entry.touched,+ resourceIds: entry.resourceIds,+ changedVariables,+ collectionsChanged: !isEqual(entry.collections, collectionsSnapshot(graph)),+ documentChanged: !isEqual(entry.settings, settingsSnapshot(graph))+ }+}diff --git a/packages/core/src/kiwi/fig/session/archive.ts b/packages/core/src/kiwi/fig/session/archive.tsnew file mode 100644index 000000000..50497a408--- /dev/null+++ b/packages/core/src/kiwi/fig/session/archive.ts@@ -0,0 +1,202 @@+import {+ figArchiveComponentUsePages,+ patchFigArchive,+ type FigArchiveRecordInfo,+ type FigPatchedArchiveInput,+ type FigRecordPatch+} from '@open-pencil/fig'+import type { SceneGraph } from '@open-pencil/scene-graph'+import { randomHex } from '@open-pencil/scene-graph/random'++import type { FigSessionRequest, FigSessionResponse } from '#core/kiwi/fig/session/protocol'++/**+ * The `.fig` archive a document was opened from, for writing the document back as that archive+ * with only its changed records replaced. In the browser the reader's worker holds the archive+ * and does this work; elsewhere, or once that worker is gone, it runs where it is asked.+ */+export interface FigArchiveAccess {+ original(): Promise<Uint8Array>+ info(): Promise<FigArchiveRecordInfo>+ /** Archive GUIDs of the pages whose instances use these components. */+ componentPages(componentIds: string[]): Promise<string[]>+ patch(patch: FigRecordPatch, input: FigPatchedArchiveInput): Promise<Uint8Array>+ /** The document closed: release whatever holds the archive. */+ close(): void+}++const archives = new WeakMap<SceneGraph, FigArchiveAccess>()++export function registerFigArchive(graph: SceneGraph, archive: FigArchiveAccess): void {+ archives.set(graph, archive)+}++export function figArchive(graph: SceneGraph): FigArchiveAccess | undefined {+ return archives.get(graph)+}++export function releaseFigArchive(graph: SceneGraph): void {+ archives.get(graph)?.close()+ archives.delete(graph)+}++/**+ * The archive operations themselves, over bytes nothing else writes to. They read the archive+ * as bytes, so they outlive the reader session that opened it; only the record info comes from+ * that session, taken while its records are decoded.+ */+export function createFigArchiveOperations(bytes: ArrayBuffer, info: () => FigArchiveRecordInfo) {+ return {+ original: () => new Uint8Array(bytes.slice(0)),+ info,+ componentPages: (componentIds: string[]) => figArchiveComponentUsePages(bytes, componentIds),+ patch: (patch: FigRecordPatch, input: FigPatchedArchiveInput) =>+ patchFigArchive(bytes, patch, input)+ }+}++export function localFigArchive(+ bytes: ArrayBuffer,+ info: () => FigArchiveRecordInfo+): FigArchiveAccess {+ const operations = createFigArchiveOperations(bytes, info)+ return {+ original: async () => operations.original(),+ info: async () => operations.info(),+ componentPages: async (componentIds) => operations.componentPages(componentIds),+ patch: async (patch, input) => operations.patch(patch, input),+ close: () => undefined+ }+}++type ArchiveResponse = Extract<+ FigSessionResponse,+ {+ type:+ | 'original-archive-result'+ | 'archive-info-result'+ | 'component-pages-result'+ | 'patch-archive-result'+ | 'archive-error'+ }+>++const ARCHIVE_RESPONSES = new Set<FigSessionResponse['type']>([+ 'original-archive-result',+ 'archive-info-result',+ 'component-pages-result',+ 'patch-archive-result',+ 'archive-error'+])++type ArchivePort = Pick<MessagePort, 'postMessage' | 'addEventListener' | 'close'>+type ArchiveWorker = Pick<Worker, 'terminate' | 'addEventListener'>+type DistributiveOmit<T, K extends PropertyKey> = T extends unknown ? Omit<T, K> : never++export interface WorkerFigArchiveAccess extends FigArchiveAccess {+ /** Stop the worker now, as an abandoned page load does; requests continue on the fallback. */+ dropWorker(): void+}++interface WaitingRequest {+ retry: (access: FigArchiveAccess) => void+ resolve: (response: ArchiveResponse) => void+ reject: (error: Error) => void+}++/**+ * Archive operations answered by the reader's worker over its session port. The worker serves+ * the archive until the document closes, even after it stops loading pages. If it fails or is+ * stopped, requests, those waiting included, run on the fallback instead.+ */+export function workerFigArchive(+ port: ArchivePort,+ worker: ArchiveWorker,+ fallback: () => FigArchiveAccess+): WorkerFigArchiveAccess {+ const pending = new Map<string, WaitingRequest>()+ let local: FigArchiveAccess | undefined+ let closed = false+ const toFallback = (): FigArchiveAccess => {+ if (local) return local+ const access = fallback()+ local = access+ worker.terminate()+ for (const waiting of pending.values()) waiting.retry(access)+ pending.clear()+ return access+ }+ worker.addEventListener('error', () => toFallback())+ port.addEventListener('message', (event: MessageEvent<FigSessionResponse>) => {+ if (!ARCHIVE_RESPONSES.has(event.data.type)) return+ const response = event.data as ArchiveResponse+ const waiting = pending.get(response.requestId)+ if (!waiting) return+ pending.delete(response.requestId)+ if (response.type === 'archive-error') waiting.reject(new Error(response.error))+ else waiting.resolve(response)+ })+ const ask = <T extends ArchiveResponse['type'], Result>(+ message: DistributiveOmit<FigSessionRequest, 'requestId'>,+ type: T,+ read: (response: Extract<ArchiveResponse, { type: T }>) => Result,+ onFallback: (access: FigArchiveAccess) => Promise<Result>+ ): Promise<Result> => {+ if (closed) return Promise.reject(new Error('The document closed'))+ if (local) return onFallback(local)+ return new Promise<Result>((resolve, reject) => {+ const requestId = randomHex()+ pending.set(requestId, {+ retry: (access) => void onFallback(access).then(resolve, reject),+ resolve: (response) => {+ if (response.type === type)+ resolve(read(response as Extract<ArchiveResponse, { type: T }>))+ else reject(new Error(`Unexpected ${response.type} for ${message.type}`))+ },+ reject+ })+ port.postMessage({ ...message, requestId })+ })+ }+ return {+ original: () =>+ ask(+ { type: 'original-archive' },+ 'original-archive-result',+ (response) => response.bytes,+ (access) => access.original()+ ),+ info: () =>+ ask(+ { type: 'archive-info' },+ 'archive-info-result',+ (response) => response.info,+ (access) => access.info()+ ),+ componentPages: (componentIds) =>+ ask(+ { type: 'component-pages', componentIds },+ 'component-pages-result',+ (response) => response.pageIds,+ (access) => access.componentPages(componentIds)+ ),+ patch: (patch, input) =>+ ask(+ { type: 'patch-archive', patch, input },+ 'patch-archive-result',+ (response) => response.bytes,+ (access) => access.patch(patch, input)+ ),+ dropWorker: () => void toFallback(),+ close() {+ if (closed) return+ closed = true+ for (const waiting of pending.values()) waiting.reject(new Error('The document closed'))+ pending.clear()+ if (local) return+ port.postMessage({ type: 'dispose' })+ port.close()+ worker.terminate()+ }+ }+}diff --git a/packages/core/src/kiwi/fig/session/document-state.ts b/packages/core/src/kiwi/fig/session/document-state.tsindex 3060db059..1000b426e 100644--- a/packages/core/src/kiwi/fig/session/document-state.ts+++ b/packages/core/src/kiwi/fig/session/document-state.ts@@ -52,6 +52,12 @@ export function isReaderPagePending(graph: SceneGraph, pageId: string): boolean: !checkpoint.loadedPageIds.includes(sourceId)}+/** Where the reader stands: which records became which layers, and which pages it loaded. */+export function readerCheckpoint(graph: SceneGraph): FigSessionCheckpoint | undefined {+ const state = states.get(graph)+ return state?.session?.checkpoint() ?? state?.checkpoint+}+export function hasReaderSession(graph: SceneGraph): boolean {return states.has(graph)}@@ -69,16 +75,16 @@ export function populateAllFigPages(graph: SceneGraph): boolean {}/**- * Drop the session, its checkpoint, and the archive once every page is in the graph, internal- * ones included, which the first save loads. The session keeps every record of the file decoded,- * which for a large design kit is gigabytes, and nothing reads it again: no page is pending, and- * later saves write from the document.+ * Drop the session and the archive once every page is in the graph, internal ones included, which+ * the first full save loads. The session keeps every record of the file decoded, which for a+ * large design kit is gigabytes, and no page is pending. The checkpoint stays: it says which+ * record each layer came from, which saving only the changed records still reads.*/function releaseLoadedReader(state: ReaderState): void {const session = state.sessionif (!session?.pages.every((page) => session.loadedPageIds.has(page.id))) return+ state.checkpoint = session.checkpoint()state.session = undefined- state.checkpoint = undefinedstate.bytes = undefinedstate.complete = true}diff --git a/packages/core/src/kiwi/fig/session/protocol.ts b/packages/core/src/kiwi/fig/session/protocol.tsindex caca6e9d4..03a791970 100644--- a/packages/core/src/kiwi/fig/session/protocol.ts+++ b/packages/core/src/kiwi/fig/session/protocol.ts@@ -1,4 +1,9 @@-import type { FigSessionCheckpoint } from '@open-pencil/fig'+import type {+ FigArchiveRecordInfo,+ FigPatchedArchiveInput,+ FigRecordPatch,+ FigSessionCheckpoint+} from '@open-pencil/fig'import type { FigPageManifestEntry } from '@open-pencil/kiwi/fig'import type { ParseFigFileOptions } from '#core/io/formats/fig/read'@@ -25,6 +30,30 @@ export interface FigSessionOriginalArchiveRequest {requestId: string}+export interface FigSessionArchiveInfoRequest {+ type: 'archive-info'+ requestId: string+}++export interface FigSessionComponentPagesRequest {+ type: 'component-pages'+ requestId: string+ /** Archive GUIDs of the components, as `session:local`. */+ componentIds: string[]+}++export interface FigSessionPatchArchiveRequest {+ type: 'patch-archive'+ requestId: string+ patch: FigRecordPatch+ input: FigPatchedArchiveInput+}++/** The graph diverged from the worker's: stop loading pages there, keep the archive. */+export interface FigSessionRetireRequest {+ type: 'retire'+}+export interface FigSessionCancelRequest {type: 'cancel'requestId?: string@@ -37,6 +66,10 @@ export interface FigSessionDisposeRequest {export type FigSessionRequest =| FigSessionPopulateRequest| FigSessionOriginalArchiveRequest+ | FigSessionArchiveInfoRequest+ | FigSessionComponentPagesRequest+ | FigSessionPatchArchiveRequest+ | FigSessionRetireRequest| FigSessionCancelRequest| FigSessionDisposeRequest@@ -46,6 +79,8 @@ export type FigSessionResponse =type: 'graph'graph?: SerializedSceneGraphcheckpoint?: FigSessionCheckpoint+ /** Taken while the records are decoded, for writing the archive back later. */+ archiveInfo?: FigArchiveRecordInfoerror?: string}| {@@ -58,4 +93,8 @@ export type FigSessionResponse =}| { type: 'population-error'; requestId?: string; error: string }| { type: 'original-archive-result'; requestId: string; bytes: Uint8Array }+ | { type: 'archive-info-result'; requestId: string; info: FigArchiveRecordInfo }+ | { type: 'component-pages-result'; requestId: string; pageIds: string[] }+ | { type: 'patch-archive-result'; requestId: string; bytes: Uint8Array }+ | { type: 'archive-error'; requestId: string; error: string }| { type: 'disposed' }diff --git a/packages/core/src/kiwi/fig/session/worker.ts b/packages/core/src/kiwi/fig/session/worker.tsindex a2238e5ec..abb284294 100644--- a/packages/core/src/kiwi/fig/session/worker.ts+++ b/packages/core/src/kiwi/fig/session/worker.ts@@ -1,4 +1,5 @@import { serializeSceneGraph } from '#core/kiwi/fig/parse/transfer'+import { createFigArchiveOperations } from '#core/kiwi/fig/session/archive'import type {FigSessionOpenRequest,FigSessionRequest,@@ -7,7 +8,7 @@ import type {import { openReaderSession } from '#core/kiwi/fig/session/reader'let session: ReturnType<typeof openReaderSession> | undefined-let originalArchive: Uint8Array | undefined+let archive: ReturnType<typeof createFigArchiveOperations> | undefinedlet port: MessagePort | undefinedfunction respond(message: FigSessionResponse): void {@@ -25,19 +26,50 @@ function populate(request: Extract<FigSessionRequest, { type: 'populate' }>): vo})}-function handleRequest(request: FigSessionRequest): void {+type ArchiveRequest = Extract<+ FigSessionRequest,+ { type: 'original-archive' | 'archive-info' | 'component-pages' | 'patch-archive' }+>++function answerArchiveRequest(request: ArchiveRequest): void {+ const { requestId } = requesttry {+ if (!archive) throw new Error('FIG session has no original archive')if (request.type === 'original-archive') {- if (!originalArchive) throw new Error('FIG session has no original archive')- const bytes = originalArchive.slice()- port?.postMessage({ type: 'original-archive-result', requestId: request.requestId, bytes }, [- bytes.buffer- ])+ const bytes = archive.original()+ port?.postMessage({ type: 'original-archive-result', requestId, bytes }, [bytes.buffer])+ } else if (request.type === 'archive-info') {+ respond({ type: 'archive-info-result', requestId, info: archive.info() })+ } else if (request.type === 'component-pages') {+ const pageIds = archive.componentPages(request.componentIds)+ respond({ type: 'component-pages-result', requestId, pageIds })+ } else {+ const bytes = archive.patch(request.patch, request.input)+ port?.postMessage({ type: 'patch-archive-result', requestId, bytes }, [bytes.buffer])+ }+ } catch (error) {+ respond({+ type: 'archive-error',+ requestId,+ error: error instanceof Error ? error.message : String(error)+ })+ }+}++function handleRequest(request: FigSessionRequest): void {+ try {+ if (+ request.type === 'original-archive' ||+ request.type === 'archive-info' ||+ request.type === 'component-pages' ||+ request.type === 'patch-archive'+ ) {+ answerArchiveRequest(request)return}if (request.type === 'dispose') {session = undefined- originalArchive = undefined+ archive = undefinedrespond({ type: 'disposed' })port?.close()port = undefined@@ -45,6 +77,10 @@ function handleRequest(request: FigSessionRequest): void {return}if (request.type === 'cancel') return+ if (request.type === 'retire') {+ session = undefined+ return+ }populate(request)} catch (error) {respond({@@ -61,9 +97,11 @@ self.onmessage = (event: MessageEvent<FigSessionOpenRequest>) => {port.onmessage = (message: MessageEvent<FigSessionRequest>) => handleRequest(message.data)port.start()// The worker copies the archive itself, so the main thread sends the file once.- originalArchive = new Uint8Array(request.buffer.slice(0))+ const archiveBytes = request.buffer.slice(0)try {const opened = openReaderSession(request.buffer, request.options?.populate)+ const archiveInfo = opened.session.archiveRecordInfo()+ archive = createFigArchiveOperations(archiveBytes, () => archiveInfo)respond({ type: 'page-manifest', pages: opened.pages })session =request.options?.populate === 'first-page' || request.options?.populate === 'none'@@ -72,7 +110,8 @@ self.onmessage = (event: MessageEvent<FigSessionOpenRequest>) => {respond({type: 'graph',graph: serializeSceneGraph(opened.graph),- checkpoint: opened.checkpoint()+ checkpoint: opened.checkpoint(),+ archiveInfo})} catch (error) {respond({ type: 'graph', error: error instanceof Error ? error.message : String(error) })diff --git a/packages/core/tests/io/formats/fig/export/archive-patch.test.ts b/packages/core/tests/io/formats/fig/export/archive-patch.test.tsnew file mode 100644index 000000000..e57633bbe--- /dev/null+++ b/packages/core/tests/io/formats/fig/export/archive-patch.test.ts@@ -0,0 +1,313 @@+import { describe, expect, setDefaultTimeout, test } from 'bun:test'++import { isEqual } from 'es-toolkit/predicate'++import { exportFigFile, initCodec, parseFigFile, SceneGraph } from '@open-pencil/core'+import { parseFigBuffer } from '@open-pencil/fig'+import { populateAllFigPages, populateFigPage } from '@open-pencil/core/io/formats/fig'+import type { NodeChange } from '@open-pencil/kiwi/fig/codec'+import { guidToString } from '@open-pencil/kiwi/fig/guid'+import type { SceneNode } from '@open-pencil/scene-graph'+import { cloneNodeProps } from '@open-pencil/scene-graph/copy'++import { releaseFigArchive } from '#core/kiwi/fig/session/archive'+import { expectDefined } from '#core-tests/helpers/assert'+import { hasPendingReaderPages, populateFigInternalPages } from '#core/kiwi/fig/session/document-state'++setDefaultTimeout(60_000)++/** A document over three pages: a frame of layers, a component, and a page of its instances. */+async function fixtureBytes(): Promise<Uint8Array> {+ await initCodec()+ const source = new SceneGraph()+ const [first] = source.getPages()+ const frame = source.createNode('FRAME', first.id, { name: 'Frame', width: 200, height: 200 })+ for (const [index, name] of ['A', 'B', 'C'].entries())+ source.createNode('RECTANGLE', frame.id, { name, x: index * 30, width: 20, height: 20 })+ source.createNode('ELLIPSE', first.id, { name: 'Dot', x: 300, width: 10, height: 10 })+ const library = source.addPage('Library')+ const component = source.createNode('COMPONENT', library.id, {+ name: 'Card',+ width: 100,+ height: 40+ })+ source.createNode('RECTANGLE', component.id, { name: 'Body', width: 100, height: 40 })+ const used = source.addPage('Used')+ const overridden = source.createInstance(component.id, used.id)+ source.createInstance(component.id, used.id, { x: 200 })+ // An override on the component's layer, which the instance's record addresses by its GUID.+ const [overriddenBody] = source.getChildren(overridden?.id ?? '')+ source.updateNode(overriddenBody.id, { opacity: 0.5 })+ const collection = source.createCollection('Tokens')+ const brand = source.createVariable('brand', 'COLOR', collection.id, { r: 1, g: 0, b: 0, a: 1 })+ source.createNode('RECTANGLE', source.addPage('Bound').id, {+ name: 'Branded',+ width: 10,+ height: 10,+ fills: [+ {+ type: 'SOLID',+ color: { r: 1, g: 0, b: 0, a: 1 },+ opacity: 1,+ visible: true,+ blendMode: 'NORMAL'+ }+ ],+ boundVariables: { 'fills/0/color': brand.id }+ })+ return exportFigFile(source)+}++const open = (bytes: Uint8Array) =>+ parseFigFile(bytes.slice().buffer as ArrayBuffer, { populate: 'first-page' })++function pageNamed(graph: SceneGraph, name: string): SceneNode {+ const page = graph.getPages().find((candidate) => candidate.name === name)+ if (!page) throw new Error(`Missing page ${name}`)+ return page+}++function layerNamed(graph: SceneGraph, name: string): SceneNode {+ const layer = [...graph.nodes.values()].find((node) => node.name === name)+ if (!layer) throw new Error(`Missing layer ${name}`)+ return layer+}++interface LayerShape {+ name: string+ type: string+ bounds: number[]+ children: LayerShape[]+}++/** What a reopened document shows, independent of the IDs one session gave its layers. */+async function shapeOf(bytes: Uint8Array): Promise<Record<string, LayerShape[]>> {+ const graph = await open(bytes)+ populateAllFigPages(graph)+ const shape = (node: SceneNode): LayerShape => ({+ name: node.name,+ type: node.type,+ bounds: [node.x, node.y, node.width, node.height].map((value) => Math.round(value * 100)),+ children: graph.getChildren(node.id).map(shape)+ })+ return Object.fromEntries(+ graph.getPages().map((page) => [page.name, graph.getChildren(page.id).map(shape)])+ )+}++const guidKey = (record: NodeChange) =>+ record.guid ? guidToString(record.guid) : ''++/** The records of an archive by GUID. */+function recordsOf(bytes: Uint8Array) {+ const { nodeChanges } = parseFigBuffer(bytes.slice().buffer as ArrayBuffer)+ return new Map(+ nodeChanges.flatMap((record) => (record.guid ? [[guidToString(record.guid), record]] : []))+ )+}++/**+ * Export one edit both ways: writing the changed records into the archive, and the whole+ * document as before. Both must reopen the same.+ */+async function exportBothWays(edit: (graph: SceneGraph) => void) {+ const bytes = await fixtureBytes()+ const patchedGraph = await open(bytes)+ edit(patchedGraph)+ const patched = await exportFigFile(patchedGraph)+ const fullGraph = await open(bytes)+ edit(fullGraph)+ releaseFigArchive(fullGraph)+ const full = await exportFigFile(fullGraph)+ return { bytes, patched, full }+}++describe('writing an edited document into its archive', () => {+ test('rewrites only the record an edit changed', async () => {+ const { bytes, patched } = await exportBothWays((graph) =>+ graph.updateNode(layerNamed(graph, 'B').id, { name: 'B renamed' })+ )+ const before = recordsOf(bytes)+ const after = recordsOf(patched)+ expect([...after.keys()].toSorted()).toEqual([...before.keys()].toSorted())+ const changed = [...after].filter(([guid, record]) => !isEqual(before.get(guid), record))+ expect(changed.map(([, record]) => record.name)).toEqual(['B renamed'])+ })++ test('still rewrites only the edited record once every page is loaded', async () => {+ const bytes = await fixtureBytes()+ const graph = await open(bytes)+ // Loading the last page releases the decoded reader; the archive keeps serving saves.+ populateAllFigPages(graph)+ populateFigInternalPages(graph)+ expect(hasPendingReaderPages(graph)).toBe(false)+ graph.updateNode(layerNamed(graph, 'B').id, { name: 'B renamed' })+ const before = recordsOf(bytes)+ const after = recordsOf(await exportFigFile(graph))+ const changed = [...after].filter(([guid, record]) => !isEqual(before.get(guid), record))+ expect(changed.map(([, record]) => record.name)).toEqual(['B renamed'])+ })++ const edits: Array<[string, (graph: SceneGraph) => void]> = [+ ['moving a layer', (graph) => graph.updateNode(layerNamed(graph, 'A').id, { x: 120, y: 90 })],+ [+ 'adding a layer between siblings',+ (graph) => {+ const frame = layerNamed(graph, 'Frame')+ const added = graph.createNode('RECTANGLE', frame.id, { name: 'New', width: 5, height: 5 })+ graph.insertChildAt(added.id, frame.id, 1)+ }+ ],+ ['deleting a frame and its layers', (graph) => graph.deleteNode(layerNamed(graph, 'Frame').id)],+ [+ 'moving a layer out of a frame it then deletes',+ (graph) => {+ const page = pageNamed(graph, graph.getPages()[0].name)+ graph.reparentNode(layerNamed(graph, 'C').id, page.id)+ graph.deleteNode(layerNamed(graph, 'Frame').id)+ }+ ],+ [+ 'reordering siblings',+ (graph) => graph.insertChildAt(layerNamed(graph, 'C').id, layerNamed(graph, 'Frame').id, 0)+ ],+ [+ 'copying a layer with its source GUID, as a paste does',+ (graph) => {+ const dot = layerNamed(graph, 'Dot')+ graph.createNode(dot.type, graph.getPages()[0].id, {+ ...cloneNodeProps(dot, null),+ name: 'Dot copy',+ x: 400+ })+ }+ ],+ [+ 'editing a component whose instances are on a page not loaded yet',+ (graph) => {+ populateFigPage(graph, pageNamed(graph, 'Library').id)+ graph.updateNode(layerNamed(graph, 'Body').id, { height: 60 })+ graph.updateNode(layerNamed(graph, 'Card').id, { height: 60 })+ }+ ],+ [+ 'overriding a layer inside an instance',+ (graph) => {+ const used = pageNamed(graph, 'Used')+ populateFigPage(graph, used.id)+ const [instance] = graph.getChildren(used.id)+ const [body] = graph.getChildren(instance.id)+ graph.updateNode(body.id, { opacity: 0.5, name: 'Body override' })+ }+ ],+ [+ 'deleting a layer of a component whose instances are on a page not loaded yet',+ (graph) => {+ const library = pageNamed(graph, 'Library')+ populateFigPage(graph, library.id)+ const [body] = graph.getChildren(layerNamed(graph, 'Card').id)+ graph.deleteNode(body.id)+ }+ ],+ ['adding a page', (graph) => graph.addPage('Added')]+ ]+ for (const [name, edit] of edits) {+ test(`reopens as the full export does after ${name}`, async () => {+ const { patched, full } = await exportBothWays(edit)+ expect(await shapeOf(patched)).toEqual(await shapeOf(full))+ })+ }++ test("keeps the document's plugin data an edit sets on it", async () => {+ const { patched } = await exportBothWays((graph) => {+ const root = expectDefined(graph.getNode(graph.rootId))+ graph.updateNode(root.id, {+ pluginData: [...root.pluginData, { pluginId: 'probe', key: 'note', value: 'kept' }]+ })+ })+ const reopened = await open(patched)+ expect(reopened.getNode(reopened.rootId)?.pluginData).toContainEqual({+ pluginId: 'probe',+ key: 'note',+ value: 'kept'+ })+ })++ test('keeps every archive record its own when new pages and variables take IDs it uses', async () => {+ const bytes = await fixtureBytes()+ const graph = await open(bytes)+ // New IDs that happen to equal records on a page not loaded yet, as a file saved before can+ // hold: the records must stay theirs.+ const unloaded = [...recordsOf(bytes)].filter(([, record]) =>+ ['Bound', 'Branded'].includes(record.name ?? '')+ )+ const [pageGuid, variableGuid] = unloaded.map(([guid]) => guid)+ graph.createNodeWithId(pageGuid, 'CANVAS', graph.rootId, { name: 'Added page' })+ const [collection] = graph.variableCollections.values()+ const [existing] = graph.variables.values()+ graph.addVariable({ ...structuredClone(existing), id: variableGuid, name: 'added' })+ collection.variableIds.push(variableGuid)+ const patched = await exportFigFile(graph)+ const before = recordsOf(bytes)+ const after = recordsOf(patched)+ for (const [guid, record] of before) {+ expect(after.get(guid)?.type).toBe(record.type)+ expect(after.get(guid)?.name).toBe(record.name)+ }+ const added = [...after.values()].filter((record) => !before.has(guidKey(record)))+ expect(added.map((record) => record.name).toSorted()).toEqual(['Added page', 'added'])+ })++ test('keeps layers on unopened pages bound to a variable whose value changes', async () => {+ const { patched } = await exportBothWays((graph) => {+ const [variable] = graph.variables.values()+ const [modeId] = Object.keys(variable.valuesByMode)+ variable.valuesByMode[modeId] = { r: 0, g: 0, b: 1, a: 1 }+ })+ const reopened = await open(patched)+ populateFigPage(reopened, pageNamed(reopened, 'Bound').id)+ const branded = layerNamed(reopened, 'Branded')+ const variableId = branded.boundVariables['fills/0/color']+ expect(reopened.variables.get(variableId)?.name).toBe('brand')+ expect(Object.values(reopened.variables.get(variableId)?.valuesByMode ?? {})[0]).toEqual({+ r: 0,+ g: 0,+ b: 1,+ a: 1+ })+ })++ test('rewrites instances on unopened pages when their component loses a layer', async () => {+ const bytes = await fixtureBytes()+ const graph = await open(bytes)+ populateFigPage(graph, pageNamed(graph, 'Library').id)+ const [body] = graph.getChildren(layerNamed(graph, 'Card').id)+ graph.deleteNode(body.id)+ const before = recordsOf(bytes)+ const after = recordsOf(await exportFigFile(graph))+ const [bodyGuid, bodyRecord] = expectDefined(+ [...before].find(([, record]) => record.name === 'Body')+ )+ const instances = (records: Map<string, NodeChange>) =>+ [...records.values()].filter((record) => record.type === 'INSTANCE')+ // Instances address the component's layer by its GUID in their derived geometry.+ const addressesBody = (record: NodeChange) =>+ JSON.stringify(record).includes(JSON.stringify(bodyRecord.guid))+ expect(instances(before).every(addressesBody)).toBe(true)+ expect(after.has(bodyGuid)).toBe(false)+ expect(instances(after)).toHaveLength(2)+ expect(instances(after).some(addressesBody)).toBe(false)+ })++ test('rewrites the variables when one changes', async () => {+ const { patched } = await exportBothWays((graph) => {+ const [variable] = graph.variables.values()+ const [modeId] = Object.keys(variable.valuesByMode)+ variable.valuesByMode[modeId] = { r: 0, g: 0, b: 1, a: 1 }+ })+ const reopened = await open(patched)+ const [variable] = reopened.variables.values()+ expect(Object.values(variable.valuesByMode)[0]).toEqual({ r: 0, g: 0, b: 1, a: 1 })+ expect(reopened.variables.size).toBe(1)+ })+})diff --git a/packages/core/tests/io/formats/fig/roundtrip/slots.test.ts b/packages/core/tests/io/formats/fig/roundtrip/slots.test.tsindex b9deabe43..2292a87e2 100644--- a/packages/core/tests/io/formats/fig/roundtrip/slots.test.ts+++ b/packages/core/tests/io/formats/fig/roundtrip/slots.test.ts@@ -1,5 +1,7 @@import { beforeAll, describe, expect, test } from 'bun:test'+import { releaseFigPopulationWorker } from '#core/kiwi/fig/population/client'+import { FIXTURES } from '#core-tests/helpers/fig/fixtures'import {@@ -43,10 +45,8 @@ function slotContents(graph: SceneGraph): Record<string, string[]> {beforeAll(async () => {await initCodec()original = await parseFigFile(await Bun.file(`${FIXTURES}/slots.fig`).arrayBuffer())- // An unedited document exports its original archive; an edit makes the export re-encode.- const untouched = original.getPages().find((page) => page.name === 'Page 1')- if (!untouched) throw new Error('Missing Page 1')- original.updateNode(untouched.id, { name: 'Page 1 (edited)' })+ // Without its archive the export encodes every record, which is what this checks.+ releaseFigPopulationWorker(original)const exported = await exportFigFile(original)expect(exported).not.toEqual(new Uint8Array(await Bun.file(`${FIXTURES}/slots.fig`).arrayBuffer())diff --git a/packages/core/tests/io/formats/fig/session/archive-changes.test.ts b/packages/core/tests/io/formats/fig/session/archive-changes.test.tsnew file mode 100644index 000000000..a5c96e015--- /dev/null+++ b/packages/core/tests/io/formats/fig/session/archive-changes.test.ts@@ -0,0 +1,48 @@+import { expect, test } from 'bun:test'++import { computeAllLayouts, exportFigFile, initCodec, parseFigFile, SceneGraph } from '@open-pencil/core'++import { archiveChanges } from '#core/kiwi/fig/session/archive-changes'++/** A row whose saved child positions are not where auto layout puts them. */+async function openRow() {+ await initCodec()+ const source = new SceneGraph()+ const row = source.createNode('FRAME', source.getPages()[0].id, {+ name: 'Row',+ layoutMode: 'HORIZONTAL',+ itemSpacing: 10,+ width: 300,+ height: 40+ })+ for (const name of ['First', 'Second', 'Third'])+ source.createNode('RECTANGLE', row.id, { name, x: 0, width: 50, height: 20 })+ const graph = await parseFigFile((await exportFigFile(source)).slice().buffer as ArrayBuffer)+ const page = graph.getPages()[0]+ const layer = (name: string) => {+ const found = [...graph.nodes.values()].find((node) => node.name === name)+ if (!found) throw new Error(`Missing ${name}`)+ return found+ }+ return { graph, page, layer }+}++test('the layout a page gets as it loads is no change to its archive', async () => {+ const { graph, page, layer } = await openRow()+ graph.applyDerivedLayoutDuring(() => computeAllLayouts(graph, page.id))+ expect(layer('Second').x).toBe(60)+ expect([...(archiveChanges(graph)?.touched ?? [])]).toEqual([])+})++test('the layout an edit causes changes the layers it moves', async () => {+ const { graph, page, layer } = await openRow()+ graph.applyDerivedLayoutDuring(() => computeAllLayouts(graph, page.id))+ graph.updateNode(layer('First').id, { width: 80 })+ computeAllLayouts(graph, page.id)+ const touched = new Set(archiveChanges(graph)?.touched)+ expect(['First', 'Second', 'Third'].filter((name) => touched.has(layer(name).id))).toEqual([+ 'First',+ 'Second',+ 'Third'+ ])+})diff --git a/packages/core/tests/io/formats/fig/session/main-thread.test.ts b/packages/core/tests/io/formats/fig/session/main-thread.test.tsindex 2abe81f49..e36fa6ab8 100644--- a/packages/core/tests/io/formats/fig/session/main-thread.test.ts+++ b/packages/core/tests/io/formats/fig/session/main-thread.test.ts@@ -84,11 +84,10 @@ test('edited export loads missing visible pages on an isolated graph', async ()const bytes = await exportFigFile(graph)expectSaveLoadedOnlyInternalPages(graph, before)expect(graph.getChildren(pages[1].id)).toHaveLength(0)+ // The edit is written into the archive, whose internal content is copied as it is saved.const internalPage = graph.getPages(true).find((page) => page.internalOnly)- expect(graph.getChildren(internalPage?.id ?? '').map((node) => node.name)).toEqual([- 'Retained internal content'- ])- // A second save finds the internal page already loaded and leaves the document as it was.+ expect(graph.getChildren(internalPage?.id ?? '')).toEqual([])+ // A second save leaves the document as it was.const loaded = snapshotNodes(graph)await exportFigFile(graph)expect(new Map(graph.nodes)).toEqual(loaded)@@ -107,17 +106,12 @@ test('edited export loads missing visible pages on an isolated graph', async ()releaseFigPopulationWorker(reopened)})-test('after the first save, later saves keep internal content without the reader', async () => {+test('later saves keep internal content they never load', async () => {const graph = await parseFigFile(await fixture(), { populate: 'first-page' })populateAllFigPages(graph)- // The first save of an edit reads the internal page into the document; every page is loaded- // after it. An unedited document is saved as the original file and reads nothing.+ // An edited save writes the edit into the archive and copies the internal content from it.graph.updateNode(graph.getPages()[0].id, { name: 'Edited page' })await exportFigFile(graph)- const internal = graph.getPages(true).find((page) => page.internalOnly)- expect(graph.getChildren(internal?.id ?? '').map((node) => node.name)).toEqual([- 'Retained internal content'- ])const second = graph.getPages()[1]graph.updateNode(graph.getChildren(second.id)[0].id, { text: 'Edited after loading' })diff --git a/packages/core/tests/io/formats/fig/session/worker-original-archive.test.ts b/packages/core/tests/io/formats/fig/session/worker-original-archive.test.tsindex 5cd14c3ab..9aadb9d93 100644--- a/packages/core/tests/io/formats/fig/session/worker-original-archive.test.ts+++ b/packages/core/tests/io/formats/fig/session/worker-original-archive.test.ts@@ -1,6 +1,9 @@import { expect, test } from 'bun:test'+import { isEqual } from 'es-toolkit/predicate'+import { exportFigFile } from '@open-pencil/core/io'+import { parseFigBuffer } from '@open-pencil/fig'import { initCodec } from '@open-pencil/core/kiwi'import { SceneGraph } from '@open-pencil/scene-graph'@@ -42,3 +45,74 @@ test('loading a page through the worker keeps saving the original bytes', asyncreleaseFigPopulationWorker(opened)}}, 20000)++test('an edited worker-opened .fig is written by the worker with only its edit re-encoded', async () => {+ await initCodec()+ const source = new SceneGraph()+ source.createNode('RECTANGLE', source.getPages()[0].id, { name: 'Kept' })+ source.createNode('RECTANGLE', source.getPages()[0].id, { name: 'Renamed' })+ source.createNode('TEXT', source.addPage('Second').id, { text: 'Second' })+ const bytes = await exportFigFile(source)++ const opened = await parseFigFileViaWorker(bytes.slice().buffer, { populate: 'first-page' })+ try {+ const renamed = opened.getChildren(opened.getPages()[0].id)[1]+ opened.updateNode(renamed.id, { name: 'Renamed again' })+ const written = parseFigBuffer((await exportFigFile(opened)).slice().buffer).nodeChanges+ const original = parseFigBuffer(bytes.slice().buffer).nodeChanges+ expect(written.map((record) => record.name)).toEqual(+ original.map((record) => (record.name === 'Renamed' ? 'Renamed again' : record.name))+ )+ expect(written.filter((record, index) => !isEqual(record, original[index]))).toHaveLength(1)+ } finally {+ releaseFigPopulationWorker(opened)+ }+}, 20000)++async function openViaWorker() {+ await initCodec()+ const source = new SceneGraph()+ source.createNode('RECTANGLE', source.getPages()[0].id, { name: 'Edited' })+ source.createNode('TEXT', source.addPage('Second').id, { text: 'Second' })+ source.createNode('TEXT', source.addPage('Third').id, { text: 'Third' })+ const bytes = await exportFigFile(source)+ return parseFigFileViaWorker(bytes.slice().buffer, { populate: 'first-page' })+}++function edit(graph: SceneGraph) {+ const [edited] = graph.getChildren(graph.getPages()[0].id)+ graph.updateNode(edited.id, { name: 'Edited again' })+}++const names = (bytes: Uint8Array) =>+ parseFigBuffer(bytes.slice().buffer).nodeChanges.flatMap((record) => record.name ?? [])++test('a page load that finds the graph diverged leaves the worker writing the archive', async () => {+ const opened = await openViaWorker()+ edit(opened)+ try {+ const worker = createFigPopulationWorker(opened)+ expect(await worker?.populate(opened.getPages()[1].id)).toBeNull()+ worker?.terminate()+ expect(names(await exportFigFile(opened))).toContain('Edited again')+ } finally {+ releaseFigPopulationWorker(opened)+ }+}, 20000)++test('an abandoned page load stops the worker, and the archive is written without it', async () => {+ const opened = await openViaWorker()+ try {+ const controller = new AbortController()+ const loading = createFigPopulationWorker(opened)?.populate(+ opened.getPages()[2].id,+ controller.signal+ )+ controller.abort()+ await expect(loading).rejects.toThrow('Aborted')+ edit(opened)+ expect(names(await exportFigFile(opened))).toContain('Edited again')+ } finally {+ releaseFigPopulationWorker(opened)+ }+}, 20000)diff --git a/packages/fig/AGENTS.md b/packages/fig/AGENTS.mdindex 019363a0f..e484b0b33 100644--- a/packages/fig/AGENTS.md+++ b/packages/fig/AGENTS.md@@ -6,6 +6,7 @@- Figma clipboard envelope encoding, decoding, and bounds belong to `@open-pencil/fig/clipboard`; converting those records to a SceneGraph is `materializeFigFragment` in `packages/fig/src/document/fragment.ts`, which Core drives from `packages/core/src/clipboard/fig-import.ts`. Core prepares runtime fonts and text and owns editor placement and history; browser and Tauri adapters own system clipboard I/O. Do not add platform clipboard APIs here.- One reader serves every `.fig` path — parse, worker, page population, session recovery, export, and paste — through the document sessions in `packages/fig/src/document/`. Read what it could not resolve with `readerDiagnostics(graph)` from `packages/core/src/kiwi/fig/session/document-state.ts`; do not add a second import path.- Browser `.fig` export uses fflate and `@open-pencil/fig`; Tauri uses `build_fig_file` in `desktop/`.+- Saving a document opened from a `.fig` rewrites only the records its edits changed and copies the rest from the archive as bytes (`packages/core/src/io/formats/fig/patch-export.ts`, `patchFigMessage` in `packages/fig/src/record-patch.ts`); the full export remains for documents with no archive. A change is what the archive tracker sees outside page loads and outside `applyDerivedLayoutDuring`, which wraps layout derived from the file or its fonts; layout an edit causes counts (`packages/core/tests/io/formats/fig/export/archive-patch.test.ts`).- Vector networks use the reverse-engineered `vectorNetworkBlob`; codecs live under `packages/core/src/vector/` and types in Scene Graph.- Changes to `.fig` behavior require round-trip validation in Figma. `packages/docs/development/roadmap.md` tracks raw metadata coverage and the code map for import/export mapping and schema files.- Measure a reader change over a whole archive, not one frame: capture before and after with `visual-oracles compare digest --file F --out baseline.json` then `--baseline baseline.json`, and account for every node it reports. `compare interpreted-document` checks one frame against live Figma and will report no change while a rule breaks pages it does not cover.diff --git a/packages/fig/src/archive.ts b/packages/fig/src/archive.tsindex 6d5b228b1..e00210b5f 100644--- a/packages/fig/src/archive.ts+++ b/packages/fig/src/archive.ts@@ -3,7 +3,7 @@ import { unzipSync, zipSync, type Unzipped, type Zippable } from 'fflate'import type { FigPageManifestEntry } from '@open-pencil/kiwi/fig'import type { NodeChange } from '@open-pencil/kiwi/fig/codec'import { buildFigKiwi, parseFigKiwiChunks } from '@open-pencil/kiwi/fig/container'-import { decodeFigKiwiCanvas } from '@open-pencil/kiwi/fig/parse'+import { decodeFigKiwiCanvas, parseFigKiwiContainer } from '@open-pencil/kiwi/fig/parse'import { hasPNGSignature } from './thumbnail'@@ -105,6 +105,58 @@ export function parseFigBuffer(}}+/** A `.fig` file's parts with its Kiwi message still encoded, for copying records as bytes. */+export interface FigArchiveParts {+ schemaDeflated: Uint8Array+ /** The inflated Kiwi message. */+ dataRaw: Uint8Array+ figKiwiVersion: number+ images: Array<[string, Uint8Array]>+ thumbnailPNG: Uint8Array | null+ metaJSON: string | null+}++function canvasParts(canvasData: Uint8Array) {+ const payload = parseFigKiwiContainer(canvasData)+ if (!payload) throw new Error('Invalid fig-kiwi container')+ return {+ schemaDeflated: payload.schemaDeflated,+ dataRaw: payload.dataRaw,+ figKiwiVersion: payload.version+ }+}++export function readFigArchiveParts(buffer: ArrayBuffer): FigArchiveParts {+ const bytes = new Uint8Array(buffer)+ const chunks = parseFigKiwiChunks(bytes)+ if (chunks) {+ const thumbnailPNG = chunks.slice(2).find(hasPNGSignature) ?? null+ return { ...canvasParts(bytes), images: [], thumbnailPNG, metaJSON: null }+ }+ const canvasArchive = unzipSync(bytes, { filter: ({ name }) => isCanonicalCanvasEntry(name) })+ const canonical = findCanvasData(canvasArchive)+ const archive = unzipSync(+ bytes,+ canonical ? { filter: ({ name }) => !isCanonicalCanvasEntry(name) } : undefined+ )+ const canvasData = canonical ?? findCanvasData(archive)+ if (!canvasData)+ throw new Error(+ `No canvas data found in .fig file. Entries: ${Object.keys(archive).join(', ')}`+ )+ const images = Object.entries(archive)+ .filter(([name]) => name.startsWith('images/') && name !== 'images/')+ .map(([name, data]) => [name.slice('images/'.length), data] as [string, Uint8Array])+ return {+ ...canvasParts(canvasData),+ images,+ thumbnailPNG: archive['thumbnail.png'] ?? null,+ metaJSON: Object.hasOwn(archive, 'meta.json')+ ? new TextDecoder().decode(archive['meta.json'])+ : null+ }+}+/** Assemble a complete zipped `.fig` archive from an encoded Kiwi message and resources. */export function writeFigArchive(input: WriteFigArchiveInput): Uint8Array {const canvasData = buildFigKiwi(input.schemaDeflated, input.kiwiData, input.figKiwiVersion)diff --git a/packages/fig/src/document/materialize.ts b/packages/fig/src/document/materialize.tsindex fea158bcb..049bdcee1 100644--- a/packages/fig/src/document/materialize.ts+++ b/packages/fig/src/document/materialize.ts@@ -17,6 +17,7 @@ import {type MaterializedComponentOccurrence} from '../instance-overrides/source-children'import { nodeChangeToProps } from '../node-change'+import { figArchiveRecordInfo, type FigArchiveRecordInfo } from '../record-patch'import { applyDocumentLayoutBindings } from './bindings/layout'import { applyDocumentPaintBindings } from './bindings/paint'import type { BindingReferenceDiagnostic } from './bindings/references'@@ -74,6 +75,30 @@ export interface FigSessionCheckpoint {components: Array<[string, ComponentCheckpoint]>}+/**+ * The layer each archive record became, by record GUID: layers, pages, instances, and the+ * layers inside components down to the instances they nest, whose contents are no records.+ */+export function figCheckpointRecords(checkpoint: FigSessionCheckpoint): Map<string, string> {+ const records = new Map(checkpoint.sources)+ for (const [componentId, component] of checkpoint.components) {+ records.set(componentId, component.rootId)+ const instances: string[][] = []+ for (const entry of component.nodes) {+ const sourceId = entry.path.at(-1)+ if (!sourceId) continue+ const inInstance = instances.some(+ (prefix) =>+ prefix.length < entry.path.length && prefix.every((id, index) => entry.path[index] === id)+ )+ if (inInstance) continue+ records.set(sourceId, entry.nodeId)+ if (entry.mainComponentId !== null) instances.push(entry.path)+ }+ }+ return records+}+export interface FigSessionResume {graph: SceneGraphcheckpoint: FigSessionCheckpoint@@ -111,6 +136,7 @@ export function createFigDocumentSession(state.graph.figKiwiVersion = archive.figKiwiVersionstate.graph.figSchemaDeflated = archive.figSchemaDeflatedconst loaded = new Set<string>(resume?.checkpoint.loadedPageIds)+ let recordInfo: FigArchiveRecordInfo | undefinedreturn {checkpoint(): FigSessionCheckpoint {return structuredClone({@@ -131,6 +157,15 @@ export function createFigDocumentSession(: undefined},pages: archive.reader.pages,+ /** Where a patch of this archive starts its new GUIDs and blob indices. */+ archiveRecordInfo(): FigArchiveRecordInfo {+ recordInfo ??= figArchiveRecordInfo({+ records: archive.records,+ blobCount: archive.blobs.length,+ imageHashes: archive.images.map(([hash]) => hash)+ })+ return recordInfo+ },loadPage(id: string): void {if (loaded.has(id)) returnconst reader = archive.reader.selectPages(new Set([id]))diff --git a/packages/fig/src/document/read.ts b/packages/fig/src/document/read.tsindex a77ff3ef3..1c3c0e0e0 100644--- a/packages/fig/src/document/read.ts+++ b/packages/fig/src/document/read.ts@@ -32,7 +32,9 @@ export function createArchiveDocumentReader(bytes: ArrayBuffer, pageIds?: ReadonfigSchemaDeflated: parsed.figSchemaDeflated,reader: createReader(parsed.nodeChanges, 'transfer', pageIds),blobs: parsed.blobs,- images: parsed.images+ images: parsed.images,+ /** The archive's records; the reader shares them, so keeping them costs nothing more. */+ records: parsed.nodeChanges}}diff --git a/packages/fig/src/index.ts b/packages/fig/src/index.tsindex fc85f5c95..c29d5dd30 100644--- a/packages/fig/src/index.ts+++ b/packages/fig/src/index.ts@@ -2,6 +2,7 @@ export { materializeFigFragment, type FragmentMaterializationOptions } from './dexport {materializeDocument,materializeFigArchive,+ figCheckpointRecords,type FigSessionCheckpoint,type FigSessionResume,createFigDocumentSession,@@ -15,6 +16,16 @@ export {type FigParseResult,type WriteFigArchiveInput} from './archive'+export {+ figArchiveComponentUsePages,+ figArchiveRecordInfo,+ figComponentUsePages,+ patchFigArchive,+ patchFigRecords,+ type FigArchiveRecordInfo,+ type FigPatchedArchiveInput,+ type FigRecordPatch+} from './record-patch'export {extractFigThumbnailFromReader,type FigRangeReader,diff --git a/packages/fig/src/node-change/export/context.ts b/packages/fig/src/node-change/export/context.tsindex b7c06924f..0f3e10346 100644--- a/packages/fig/src/node-change/export/context.ts+++ b/packages/fig/src/node-change/export/context.ts@@ -62,6 +62,15 @@ export interface SceneNodeToKiwiContext {* variantPropSpecs pointing at the same property reuse the same GUID. */propertyIdToGuid: Map<string, GUID>componentPropertyDefinitionsById: ReadonlyMap<string, ComponentPropertyDefinition>+ /**+ * Writing records into the archive a document came from: the layer each archive GUID+ * belongs to, so a copy that kept its original's `source.id` gets a GUID of its own.+ */+ recordOwners?: ReadonlyMap<string, string>+ /** Positions the caller decided, kept so untouched siblings keep theirs. */+ orderKeys?: ReadonlyMap<string, string>+ /** False to write a layer's record alone, as when only some records change. */+ writeChildren?: boolean/*** Receives the content frames of instance slots. Figma stores them on the internal canvas,* so the caller re-parents the `isSlotContent` roots there after serializing its nodes.@@ -130,7 +139,9 @@ export function getOrCreateNodeGuid(if (!node) return undefinedconst existing = context.nodeIdToGuid?.get(nodeId)if (existing) return existing- const importedGuid = node.source.id ? parseGuidOrNull(node.source.id) : null+ const ownsSource =+ !node.source.id || !context.recordOwners || context.recordOwners.get(node.source.id) === nodeId+ const importedGuid = node.source.id && ownsSource ? parseGuidOrNull(node.source.id) : null// When source.id maps to a GUID value that is already assigned to a// different node (e.g. two nodes from different canvases with the samediff --git a/packages/fig/src/node-change/export/node.ts b/packages/fig/src/node-change/export/node.tsindex 663d40d21..9b1e8a438 100644--- a/packages/fig/src/node-change/export/node.ts+++ b/packages/fig/src/node-change/export/node.ts@@ -64,6 +64,8 @@ function exportOrderKey(node: SceneNode,childIndex: number): string {+ const decided = context.orderKeys?.get(node.id)+ if (decided) return decidedconst parentId = node.parentIdif (!parentId) return context.fractionalPosition(childIndex)const parent = context.graph.getNode(parentId)@@ -900,6 +902,14 @@ function nodeToSerialize(context: SceneNodeToKiwiContext, source: SceneNode): Sc}}+/** An instance writes its contents on its own record; a caller may write a record alone. */+function exportedChildren(node: SceneNode, context: SceneNodeToKiwiContext): SceneNode[] {+ if (node.type === 'INSTANCE' || context.writeChildren === false) return []+ return context.graph+ .getChildren(node.id)+ .filter((child) => !child.internalOnly && child.sharedStyleType === null)+}+export function sceneNodeToKiwiWithContext(source: SceneNode,parentGuid: GUID,@@ -986,12 +996,7 @@ export function sceneNodeToKiwiWithContext(}const result: KiwiNodeChange[] = [nc]- const children =- node.type === 'INSTANCE'- ? []- : context.graph- .getChildren(node.id)- .filter((child) => !child.internalOnly && child.sharedStyleType === null)+ const children = exportedChildren(node, context)for (let i = 0; i < children.length; i++) {result.push(...context.sceneNodeToKiwi(children[i], guid, i, localIdCounter, context))}diff --git a/packages/fig/src/node-change/serialize.ts b/packages/fig/src/node-change/serialize.tsindex 632669a0a..cd500b9f9 100644--- a/packages/fig/src/node-change/serialize.ts+++ b/packages/fig/src/node-change/serialize.ts@@ -449,6 +449,9 @@ export interface SceneNodeToKiwiOptions {slotContentRecords?: KiwiNodeChange[]/** Plugin data a node is written with instead of its own, by node ID. */pluginDataOverrides?: ReadonlyMap<string, PluginDataEntry[]>+ recordOwners?: ReadonlyMap<string, string>+ orderKeys?: ReadonlyMap<string, string>+ writeChildren?: boolean}export function sceneNodeToKiwi(@@ -472,7 +475,10 @@ export function sceneNodeToKiwi(modeIdToGuid,propertyIdToGuid = new Map<string, GUID>(),slotContentRecords,- pluginDataOverrides+ pluginDataOverrides,+ recordOwners,+ orderKeys,+ writeChildren} = options// Raw paints retain library asset refs; effects use this map because their// Kiwi schema accepts only GUID-backed aliases.@@ -492,6 +498,9 @@ export function sceneNodeToKiwi(propertyIdToGuid,slotContentRecords,pluginDataOverrides,+ recordOwners,+ orderKeys,+ writeChildren,fractionalPosition,mapToFigmaType,fillToKiwiPaint,diff --git a/packages/fig/src/record-patch.ts b/packages/fig/src/record-patch.tsnew file mode 100644index 000000000..545cb8885--- /dev/null+++ b/packages/fig/src/record-patch.ts@@ -0,0 +1,525 @@+import { inflateSync } from 'fflate'++import type { GUID, NodeChange } from '@open-pencil/kiwi/fig/codec'+import { guidToString } from '@open-pencil/kiwi/fig/guid'+import {+ ByteBuffer,+ compileSchema,+ createSchemaSkipper,+ decodeBinarySchema,+ type SchemaSkipper+} from '@open-pencil/kiwi/schema-runtime'+import { orderKeyBetween } from '@open-pencil/scene-graph/order-keys'++import { readFigArchiveParts, writeFigArchive, type FigImageEntry } from './archive'+import { symbolDataOf, symbolOverridesOf } from './instance-overrides/types'++/**+ * What an edited document changes in the archive it was opened from: the records to write, by+ * GUID, and the records to drop. Records nobody touched stay as the archive saved them.+ */+export interface FigRecordPatch {+ /** Written in place of the archive's record with the same GUID, or added after the rest. */+ records: NodeChange[]+ /** Blobs the records add; their indices continue after the archive's. */+ blobs: Uint8Array[]+ /** Dropped together with every record below them. */+ removed: GUID[]+ /** The records hold every variable and collection, so the archive's are dropped. */+ replaceVariables: boolean+}++/** Archive facts a patch is built against: where new blob indices and new GUIDs start. */+export interface FigArchiveRecordInfo {+ blobCount: number+ /** Above every local ID the archive uses with session 0 or 1, the sessions new GUIDs take. */+ nextLocalId: number+ imageHashes: string[]+ /** The document record's GUID, as `session:local`. */+ documentId: string | null+}++export interface FigPatchedArchiveInput {+ thumbnailPNG: Uint8Array+ metaJSON: string+ /** Images the archive does not hold yet. */+ images: FigImageEntry[]+}++const VARIABLE_TYPES = new Set(['VARIABLE', 'VARIABLE_SET'])++function key(guid: GUID | undefined): string | undefined {+ return guid ? guidToString(guid) : undefined+}++function isGuidLike(value: object): value is GUID {+ return (+ 'sessionID' in value &&+ 'localID' in value &&+ typeof value.sessionID === 'number' &&+ typeof value.localID === 'number'+ )+}++/** Every GUID an archive embeds, record identities and references alike. */+function highestLocalId(records: readonly NodeChange[]): number {+ let highest = 0+ const visit = (value: unknown): void => {+ if (!value || typeof value !== 'object' || ArrayBuffer.isView(value)) return+ if (Array.isArray(value)) {+ for (const item of value) visit(item)+ return+ }+ if (isGuidLike(value) && (value.sessionID === 0 || value.sessionID === 1))+ highest = Math.max(highest, value.localID)+ for (const item of Object.values(value)) visit(item)+ }+ visit(records)+ return highest+}++export function figArchiveRecordInfo(archive: {+ records: readonly NodeChange[]+ blobCount: number+ imageHashes: readonly string[]+}): FigArchiveRecordInfo {+ return {+ blobCount: archive.blobCount,+ nextLocalId: highestLocalId(archive.records) + 1,+ imageHashes: [...archive.imageHashes],+ documentId: key(archive.records.find((record) => record.type === 'DOCUMENT')?.guid) ?? null+ }+}++/**+ * A record as merging sees it: its identity, place and type, with its contents either the bytes+ * the archive holds or a record to encode.+ */+interface RecordEntry<Value> {+ id: string | undefined+ parent: string | undefined+ type: string | undefined+ parentIndex: NodeChange['parentIndex']+ value: Value+}++function entryOf(record: NodeChange): RecordEntry<NodeChange> {+ return {+ id: key(record.guid),+ parent: key(record.parentIndex?.guid),+ type: record.type,+ parentIndex: record.parentIndex,+ value: record+ }+}++const isVariable = (entry: { type: string | undefined }) =>+ !!entry.type && VARIABLE_TYPES.has(entry.type)++/** The removed records and those below them, except records the patch writes elsewhere. */+function removedRecords(+ records: ReadonlyArray<RecordEntry<unknown>>,+ roots: readonly GUID[],+ written: ReadonlySet<string>+): Set<string> {+ const children = new Map<string, string[]>()+ for (const record of records) {+ if (!record.id || !record.parent) continue+ const siblings = children.get(record.parent) ?? []+ siblings.push(record.id)+ children.set(record.parent, siblings)+ }+ const removed = new Set<string>()+ const pending = roots.map(guidToString)+ for (let id = pending.pop(); id !== undefined; id = pending.pop()) {+ if (removed.has(id) || written.has(id)) continue+ removed.add(id)+ for (const child of children.get(id) ?? []) pending.push(child)+ }+ return removed+}++/** Writes records in order, holding back one whose new parent has not been written yet. */+function createRecordWriter<Value>(+ written: ReadonlyMap<string, unknown>,+ archiveIds: ReadonlySet<string>+) {+ const result: Array<RecordEntry<Value>> = []+ const emitted = new Set<string>()+ const waiting = new Map<string, Array<RecordEntry<Value>>>()+ const isPending = (parent: string | undefined): parent is string =>+ !!parent && !emitted.has(parent) && written.has(parent) && !archiveIds.has(parent)+ const emit = (entry: RecordEntry<Value>): void => {+ if (isPending(entry.parent)) {+ waiting.set(entry.parent, [...(waiting.get(entry.parent) ?? []), entry])+ return+ }+ result.push(entry)+ if (!entry.id) return+ emitted.add(entry.id)+ const children = waiting.get(entry.id) ?? []+ waiting.delete(entry.id)+ for (const child of children) emit(child)+ }+ return {+ emit,+ emitted: (id: string) => emitted.has(id),+ finish(): Array<RecordEntry<Value>> {+ for (const children of waiting.values()) for (const child of children) result.push(child)+ return result+ }+ }+}++/** The last position under each parent among records that keep the position they have. */+function lastPositions(+ entries: ReadonlyArray<RecordEntry<unknown>>,+ archivePositions: ReadonlyMap<string, NodeChange['parentIndex']>+): Map<string, string> {+ const last = new Map<string, string>()+ for (const entry of entries) {+ const position = entry.parentIndex?.position+ const moves = isVariable(entry) && !(entry.id && archivePositions.has(entry.id))+ if (!entry.parent || !position || moves) continue+ if (position > (last.get(entry.parent) ?? '')) last.set(entry.parent, position)+ }+ return last+}++/**+ * Variables return to the places they had; new ones go after their canvas's last child, so no+ * position is shared with the styles and variables the archive keeps there.+ */+function placeVariableRecords(+ entries: ReadonlyArray<RecordEntry<unknown>>,+ archivePositions: ReadonlyMap<string, NodeChange['parentIndex']>+): void {+ const last = lastPositions(entries, archivePositions)+ for (const entry of entries) {+ if (!isVariable(entry) || entry.value instanceof Uint8Array) continue+ const record = entry.value as NodeChange+ const kept = entry.id ? archivePositions.get(entry.id) : undefined+ if (kept) record.parentIndex = structuredClone(kept)+ else if (entry.parent && record.parentIndex) {+ const position = orderKeyBetween(last.get(entry.parent) ?? null, null)+ record.parentIndex = { ...record.parentIndex, position }+ last.set(entry.parent, position)+ }+ }+}++/**+ * Merge a patch into the archive's records. Untouched records keep their place; a written+ * record takes its archive record's, and one whose parent is new follows that parent.+ */+function mergeRecords<Value>(+ archive: ReadonlyArray<RecordEntry<Value>>,+ patch: Pick<FigRecordPatch, 'records' | 'removed' | 'replaceVariables'>+): Array<RecordEntry<Value | NodeChange>> {+ const written = new Map<string, RecordEntry<NodeChange>>()+ for (const record of patch.records) {+ const entry = entryOf(record)+ if (entry.id) written.set(entry.id, entry)+ }+ const removed = removedRecords(archive, patch.removed, new Set(written.keys()))+ const archiveIds = new Set(archive.flatMap((entry) => entry.id ?? []))+ const archivePositions = new Map<string, NodeChange['parentIndex']>()+ const writer = createRecordWriter<Value | NodeChange>(written, archiveIds)++ for (const entry of archive) {+ if (entry.id && removed.has(entry.id)) continue+ const replacement = entry.id ? written.get(entry.id) : undefined+ if (patch.replaceVariables && isVariable(entry)) {+ // A variable the records still hold takes its old place; one they dropped is gone.+ if (entry.id) archivePositions.set(entry.id, entry.parentIndex)+ if (replacement) writer.emit(replacement)+ continue+ }+ writer.emit(replacement ?? entry)+ }+ for (const entry of written.values())+ if (entry.id && !writer.emitted(entry.id) && !archiveIds.has(entry.id)) writer.emit(entry)+ const result = writer.finish()+ if (patch.replaceVariables) placeVariableRecords(result, archivePositions)+ return result+}++/** Merge a patch into decoded archive records. */+export function patchFigRecords(+ archive: readonly NodeChange[],+ patch: Pick<FigRecordPatch, 'records' | 'removed' | 'replaceVariables'>+): NodeChange[] {+ return mergeRecords(archive.map(entryOf), patch).map((entry) => entry.value)+}++type KiwiCodec = ReturnType<typeof compileSchema>+type RecordHead = Omit<RecordEntry<Uint8Array>, 'value'> & {+ guid?: GUID+ /** Read only when asked for: what an instance shows, for finding where components are used. */+ symbolData?: unknown+}+type EncodedPart =+ | { kind: 'bytes'; bytes: Uint8Array }+ | { kind: 'records' | 'blobs'; fieldId: number; type: string }++/** The encoded message's fields in order, with its records and blobs located by byte range. */+interface ScannedMessage {+ parts: EncodedPart[]+ records: Array<RecordHead & { value: Uint8Array }>+ blobs: Uint8Array[]+ hasBlobs: boolean+}++function fieldType(type: string | null): string {+ if (type === null) throw new Error('Kiwi field without a type')+ return type+}++function decoderOf(codec: KiwiCodec, type: string): (bb: ByteBuffer) => unknown {+ const decoder = codec[`decode${type}`]+ if (typeof decoder !== 'function') throw new Error(`No Kiwi decoder for ${type}`)+ return decoder as (bb: ByteBuffer) => unknown+}++/** Reads a record's identity, place and type, and moves past everything else in it. */+function scanRecord(+ codec: KiwiCodec,+ skipper: SchemaSkipper,+ type: string,+ bb: ByteBuffer,+ readSymbolData: boolean+): RecordHead {+ const fields = skipper.fieldsById(type)+ const head: RecordHead = {+ id: undefined,+ parent: undefined,+ type: undefined,+ parentIndex: undefined+ }+ for (let id = bb.readVarUint(); id !== 0; id = bb.readVarUint()) {+ const field = fields.get(id)+ if (!field) throw new Error('Attempted to parse invalid message')+ const name = field.isArray ? '' : field.name+ if (name === 'guid') {+ head.guid = decoderOf(codec, fieldType(field.type))(bb) as GUID+ head.id = key(head.guid)+ } else if (name === 'parentIndex') {+ const parentIndex = decoderOf(codec, fieldType(field.type))(bb) as NodeChange['parentIndex']+ head.parentIndex = parentIndex+ head.parent = key(parentIndex?.guid)+ } else if (name === 'type') {+ const names = codec[fieldType(field.type)] as Record<number, string> | undefined+ head.type = names?.[bb.readVarUint()]+ } else if (name === 'symbolData' && readSymbolData)+ head.symbolData = decoderOf(codec, fieldType(field.type))(bb)+ else skipper.skipField(field, bb)+ }+ return head+}++function scanMessage(+ codec: KiwiCodec,+ skipper: SchemaSkipper,+ data: Uint8Array,+ readSymbolData = false+): ScannedMessage {+ const fields = skipper.fieldsById('Message')+ const bb = new ByteBuffer(data)+ const scanned: ScannedMessage = { parts: [], records: [], blobs: [], hasBlobs: false }+ for (;;) {+ const start = bb.offset+ const id = bb.readVarUint()+ if (id === 0) break+ const field = fields.get(id)+ if (!field) throw new Error('Attempted to parse invalid message')+ const type = fieldType(field.type)+ const isRecords = field.name === 'nodeChanges' && field.isArray+ const isBlobs = field.name === 'blobs' && field.isArray+ if (!isRecords && !isBlobs) {+ skipper.skipField(field, bb)+ scanned.parts.push({ kind: 'bytes', bytes: data.subarray(start, bb.offset) })+ continue+ }+ for (let count = bb.readVarUint(); count > 0; count--) {+ const begin = bb.offset+ if (isRecords) {+ const head = scanRecord(codec, skipper, type, bb, readSymbolData)+ scanned.records.push({ ...head, value: data.subarray(begin, bb.offset) })+ } else {+ skipper.skipValue(type, bb)+ scanned.blobs.push(data.subarray(begin, bb.offset))+ }+ }+ scanned.hasBlobs ||= isBlobs+ scanned.parts.push({ kind: isRecords ? 'records' : 'blobs', fieldId: id, type })+ }+ if (scanned.records.length === 0) throw new Error('No nodes found in .fig file')+ return scanned+}++function archiveCodec(schemaDeflated: Uint8Array) {+ const schema = decodeBinarySchema(new ByteBuffer(inflateSync(schemaDeflated)))+ return { codec: compileSchema(schema), skipper: createSchemaSkipper(schema) }+}++function varUint(value: number): Uint8Array {+ const bb = new ByteBuffer()+ bb.writeVarUint(value)+ return bb.toUint8Array().slice()+}++function encoded(codec: KiwiCodec, type: string, value: unknown): Uint8Array {+ const encoder = codec[`encode${type}`]+ if (typeof encoder !== 'function') throw new Error(`No Kiwi encoder for ${type}`)+ const bytes = (encoder as (message: unknown) => Uint8Array | undefined)(value)+ if (!bytes) throw new Error(`Encoding ${type} produced nothing`)+ return bytes.slice()+}++function concat(chunks: readonly Uint8Array[]): Uint8Array {+ let length = 0+ for (const chunk of chunks) length += chunk.byteLength+ const result = new Uint8Array(length)+ let offset = 0+ for (const chunk of chunks) {+ result.set(chunk, offset)+ offset += chunk.byteLength+ }+ return result+}++/**+ * The archive's message with the patch applied. Untouched records and blobs are copied as the+ * bytes they are and only written records are encoded, so the cost follows the edit.+ */+export function patchFigMessage(+ schemaDeflated: Uint8Array,+ dataRaw: Uint8Array,+ patch: FigRecordPatch+): Uint8Array {+ const { codec, skipper } = archiveCodec(schemaDeflated)+ const scanned = scanMessage(codec, skipper, dataRaw)+ const merged = mergeRecords(scanned.records, patch)+ const blobs = (type: string) => [+ ...scanned.blobs,+ ...patch.blobs.map((bytes) => encoded(codec, type, { bytes }))+ ]+ // Appended one by one: a document's records are far more than a call can take as arguments.+ const chunks: Uint8Array[] = []+ const appendField = (fieldId: number, values: readonly Uint8Array[]) => {+ chunks.push(varUint(fieldId), varUint(values.length))+ for (const value of values) chunks.push(value)+ }+ for (const part of scanned.parts) {+ if (part.kind === 'bytes') chunks.push(part.bytes)+ else if (part.kind === 'blobs') appendField(part.fieldId, blobs(part.type))+ else+ appendField(+ part.fieldId,+ merged.map((entry) =>+ entry.value instanceof Uint8Array ? entry.value : encoded(codec, part.type, entry.value)+ )+ )+ }+ if (!scanned.hasBlobs && patch.blobs.length > 0) {+ const field = skipper.definition('Message').fields.find((entry) => entry.name === 'blobs')+ if (!field) throw new Error('The archive schema has no blobs')+ appendField(field.value, blobs(fieldType(field.type)))+ }+ chunks.push(varUint(0))+ return concat(chunks)+}++/** Write the archive with the patch applied, in the archive's own schema. */+export function patchFigArchive(+ bytes: ArrayBuffer,+ patch: FigRecordPatch,+ input: FigPatchedArchiveInput+): Uint8Array {+ const parts = readFigArchiveParts(bytes)+ const kiwiData = patchFigMessage(parts.schemaDeflated, parts.dataRaw, patch)+ const archiveImages = new Set(parts.images.map(([hash]) => hash))+ return writeFigArchive({+ schemaDeflated: parts.schemaDeflated,+ kiwiData,+ thumbnailPNG: input.thumbnailPNG,+ metaJSON: input.metaJSON,+ images: [+ ...parts.images.map(([name, data]) => ({ name: `images/${name}`, data })),+ ...input.images.filter((image) => !archiveImages.has(image.name.replace(/^images\//, '')))+ ],+ figKiwiVersion: parts.figKiwiVersion+ })+}++/**+ * The pages whose instances use these components, directly or through a component that nests+ * one of them. Their records carry overrides and sizes derived from the components.+ */+export function figComponentUsePages(+ records: readonly NodeChange[],+ componentIds: readonly string[]+): string[] {+ const byId = new Map<string, NodeChange>()+ const instancesOf = new Map<string, string[]>()+ const use = (componentId: string | undefined, instanceId: string) => {+ if (!componentId) return+ const instances = instancesOf.get(componentId) ?? []+ instances.push(instanceId)+ instancesOf.set(componentId, instances)+ }+ for (const record of records) {+ const id = key(record.guid)+ if (!id) continue+ byId.set(id, record)+ if (record.type !== 'INSTANCE') continue+ use(key(symbolDataOf(record)?.symbolID), id)+ for (const override of symbolOverridesOf(record)) use(key(override.overriddenSymbolID), id)+ }+ const nearest = (id: string, type: string): string | undefined => {+ const seen = new Set<string>()+ for (let current = byId.get(id); current;) {+ const currentId = key(current.guid)+ if (!currentId || seen.has(currentId)) return undefined+ seen.add(currentId)+ if (current.type === type) return currentId+ const parent = key(current.parentIndex?.guid)+ current = parent ? byId.get(parent) : undefined+ }+ return undefined+ }+ const pages = new Set<string>()+ const visited = new Set<string>()+ const pending = [...componentIds]+ for (let componentId = pending.pop(); componentId !== undefined; componentId = pending.pop()) {+ if (visited.has(componentId)) continue+ visited.add(componentId)+ for (const instanceId of instancesOf.get(componentId) ?? []) {+ const page = nearest(instanceId, 'CANVAS')+ if (page) pages.add(page)+ const owner = nearest(instanceId, 'SYMBOL')+ if (owner) pending.push(owner)+ }+ }+ return [...pages]+}++/** `figComponentUsePages` over an encoded archive, reading only what it needs of each record. */+export function figArchiveComponentUsePages(+ bytes: ArrayBuffer,+ componentIds: readonly string[]+): string[] {+ const parts = readFigArchiveParts(bytes)+ const { codec, skipper } = archiveCodec(parts.schemaDeflated)+ const scanned = scanMessage(codec, skipper, parts.dataRaw, true)+ const records = scanned.records.map(+ (entry) =>+ ({+ guid: entry.guid,+ parentIndex: entry.parentIndex,+ type: entry.type,+ symbolData: entry.symbolData+ }) as NodeChange+ )+ return figComponentUsePages(records, componentIds)+}diff --git a/packages/fig/tests/record-patch.test.ts b/packages/fig/tests/record-patch.test.tsnew file mode 100644index 000000000..4bc36f82f--- /dev/null+++ b/packages/fig/tests/record-patch.test.ts@@ -0,0 +1,149 @@+import { beforeAll, describe, expect, test } from 'bun:test'++import { guid } from '#fig-tests/helpers/guid'+import { figComponentUsePages, patchFigMessage, patchFigRecords } from '#fig/record-patch'+import { deflateSync } from 'fflate'++import {+ getCompiledSchema,+ getSchemaBytes,+ initCodec,+ type NodeChange+} from '@open-pencil/kiwi/fig/codec'++const record = (localID: number, parent: number | null, extra: Partial<NodeChange> = {}) =>+ ({+ guid: guid(localID),+ ...(parent === null ? {} : { parentIndex: { guid: guid(parent), position: `${localID}` } }),+ type: 'FRAME',+ ...extra+ }) as NodeChange++const ids = (records: readonly NodeChange[]) => records.map((change) => change.guid?.localID)++describe('patching archive records', () => {+ const archive = [+ record(0, null, { type: 'DOCUMENT' }),+ record(1, 0, { type: 'CANVAS' }),+ record(2, 1),+ record(3, 2),+ record(4, 2),+ record(5, 1)+ ]++ test('keeps untouched records and writes a changed one in its place', () => {+ const renamed = record(3, 2, { name: 'Renamed' })+ const result = patchFigRecords(archive, {+ records: [renamed],+ removed: [],+ replaceVariables: false+ })+ expect(ids(result)).toEqual([0, 1, 2, 3, 4, 5])+ expect(result[3]).toBe(renamed)+ expect(result[4]).toBe(archive[4])+ })++ test('removes a record with those below it, but not one the patch moved out', () => {+ const moved = record(4, 1)+ const result = patchFigRecords(archive, {+ records: [moved],+ removed: [guid(2)],+ replaceVariables: false+ })+ expect(ids(result)).toEqual([0, 1, 4, 5])+ })++ test('writes a record moved into a new layer after that layer', () => {+ const added = record(9, 1)+ const moved = record(5, 9)+ const result = patchFigRecords(archive, {+ records: [moved, added],+ removed: [],+ replaceVariables: false+ })+ expect(ids(result)).toEqual([0, 1, 2, 3, 4, 9, 5])+ })++ test('replaces variables, keeping the places of those that remain', () => {+ const variables = [+ record(6, 1, { type: 'VARIABLE_SET' }),+ record(7, 6, { type: 'VARIABLE' }),+ record(8, 6, { type: 'VARIABLE' })+ ]+ const result = patchFigRecords([...archive, ...variables], {+ records: [+ record(7, 6, { type: 'VARIABLE', name: 'Kept' }),+ record(9, 6, { type: 'VARIABLE' })+ ],+ removed: [],+ replaceVariables: true+ })+ expect(ids(result)).toEqual([0, 1, 2, 3, 4, 5, 7, 9])+ expect(result.find((change) => change.guid?.localID === 7)?.parentIndex?.position).toBe('7')+ const added = result.find((change) => change.guid?.localID === 9)?.parentIndex?.position ?? ''+ expect(added > '7').toBe(true)+ })+})++describe('pages using components', () => {+ test('follows instances through the components that nest them', () => {+ const records = [+ record(1, 0, { type: 'CANVAS' }),+ record(2, 0, { type: 'CANVAS' }),+ record(3, 0, { type: 'CANVAS' }),+ record(10, 1, { type: 'SYMBOL' }),+ record(11, 1, { type: 'SYMBOL' }),+ record(12, 11, { type: 'INSTANCE', symbolData: { symbolID: guid(10) } }),+ record(13, 2, { type: 'INSTANCE', symbolData: { symbolID: guid(11) } }),+ record(14, 3, { type: 'FRAME' })+ ]+ expect(figComponentUsePages(records, ['1:10']).toSorted()).toEqual(['1:1', '1:2'])+ })+})++describe('splicing an encoded message', () => {+ beforeAll(() => initCodec())++ test('decodes as merging the decoded records would, copying untouched records as bytes', () => {+ const codec = getCompiledSchema()+ const archive = [+ record(0, null, { type: 'DOCUMENT', name: 'Document' }),+ record(1, 0, { type: 'CANVAS', name: 'Page' }),+ record(2, 1, { name: 'Frame', opacity: 0.5 }),+ record(3, 2, { name: 'Kept' }),+ record(4, 2, { name: 'Renamed' })+ ]+ const message = {+ type: 'NODE_CHANGES',+ sessionID: 0,+ ackID: 0,+ nodeChanges: archive,+ blobs: [{ bytes: new Uint8Array([1, 2]) }]+ }+ const data = codec.encodeMessage(message)+ const patch = {+ records: [record(4, 2, { name: 'Renamed again' }), record(9, 2, { name: 'Added' })],+ blobs: [new Uint8Array([3])],+ removed: [guid(3)],+ replaceVariables: false+ }+ const spliced = codec.decodeMessage(+ patchFigMessage(deflateSync(getSchemaBytes()), data, patch)+ ) as typeof message+ const merged = codec.decodeMessage(+ codec.encodeMessage({+ ...message,+ nodeChanges: patchFigRecords(archive, patch),+ blobs: [...message.blobs, { bytes: new Uint8Array([3]) }]+ })+ )+ expect(spliced).toEqual(merged as typeof message)+ expect(spliced.nodeChanges.map((change) => change.name)).toEqual([+ 'Document',+ 'Page',+ 'Frame',+ 'Renamed again',+ 'Added'+ ])+ })+})diff --git a/packages/kiwi/src/schema-runtime/index.ts b/packages/kiwi/src/schema-runtime/index.tsindex baa71d9aa..18cdf264a 100644--- a/packages/kiwi/src/schema-runtime/index.ts+++ b/packages/kiwi/src/schema-runtime/index.ts@@ -3,6 +3,7 @@ export { ByteBuffer } from './bb'export { compileSchemaRuntime as compileSchema } from './interpreter'export { decodeBinarySchema, encodeBinarySchema } from './binary'export { parseSchema } from './parser'+export { createSchemaSkipper, type SchemaSkipper } from './skip'export {validateSchema,expectFieldNumber,diff --git a/packages/kiwi/src/schema-runtime/skip.ts b/packages/kiwi/src/schema-runtime/skip.tsnew file mode 100644index 000000000..23649f86e--- /dev/null+++ b/packages/kiwi/src/schema-runtime/skip.ts@@ -0,0 +1,95 @@+import type { ByteBuffer } from './bb'+import type { Definition, Field, Schema } from './schema'+import { error, quote } from './util'++/**+ * Moves a buffer past encoded values without building them, so a reader can find where each+ * value lies in a large message and copy the bytes of the ones it does not change.+ */+export interface SchemaSkipper {+ definition(name: string): Definition+ /** A definition's fields by their IDs, built once per definition. */+ fieldsById(name: string): ReadonlyMap<number, Field>+ /** Past one value of a field, its array included. */+ skipField(field: Field, bb: ByteBuffer): void+ /** Past one value of a type: a native type, an enum, a struct or a message. */+ skipValue(type: string, bb: ByteBuffer): void+}++export function createSchemaSkipper(schema: Schema): SchemaSkipper {+ const definitions = new Map(schema.definitions.map((definition) => [definition.name, definition]))+ const fieldsById = new Map<string, Map<number, Field>>()+ const definition = (name: string): Definition => {+ const found = definitions.get(name)+ if (!found) error('Invalid type ' + quote(name), 0, 0)+ return found+ }+ const messageFields = (found: Definition): Map<number, Field> => {+ let fields = fieldsById.get(found.name)+ if (!fields) {+ fields = new Map(found.fields.map((field) => [field.value, field]))+ fieldsById.set(found.name, fields)+ }+ return fields+ }++ const skipValue = (type: string, bb: ByteBuffer): void => {+ switch (type) {+ case 'bool':+ case 'byte':+ bb.readByte()+ return+ case 'int':+ case 'uint':+ bb.readVarUint()+ return+ case 'float':+ bb.readVarFloat()+ return+ case 'string':+ bb.skipString()+ return+ case 'int64':+ case 'uint64':+ bb.readVarUint64()+ return+ }+ const found = definition(type)+ if (found.kind === 'ENUM') {+ bb.readVarUint()+ return+ }+ if (found.kind === 'STRUCT') {+ for (const field of found.fields) skipField(field, bb)+ return+ }+ const fields = messageFields(found)+ for (let id = bb.readVarUint(); id !== 0; id = bb.readVarUint()) {+ const field = fields.get(id)+ if (!field) throw new Error('Attempted to parse invalid message')+ skipField(field, bb)+ }+ }++ const skipField = (field: Field, bb: ByteBuffer): void => {+ const type = field.type+ if (type === null)+ error('Invalid type null for field ' + quote(field.name), field.line, field.column)+ if (!field.isArray) {+ skipValue(type, bb)+ return+ }+ if (type === 'byte') {+ bb.skipByteArray()+ return+ }+ for (let length = bb.readVarUint(); length > 0; length--) skipValue(type, bb)+ }++ return {+ definition,+ fieldsById: (name) => messageFields(definition(name)),+ skipField,+ skipValue+ }+}diff --git a/packages/scene-graph/src/index.ts b/packages/scene-graph/src/index.tsindex bc3dbf51c..60e8a476d 100644--- a/packages/scene-graph/src/index.ts+++ b/packages/scene-graph/src/index.ts@@ -175,6 +175,7 @@ export class SceneGraph {private sourceMetadataPreservationDepth = 0private importedStateApplicationDepth = 0private layoutMutationDepth = 0+ private derivedLayoutDepth = 0positionPreviewVersion = 0instanceIndex = new Map<string, Set<string>>()@@ -561,6 +562,23 @@ export class SceneGraph {return this.importedStateApplicationDepth > 0}+ /**+ * Lays out what the document already holds: a page as it loads, or again once its fonts+ * arrive. What this writes derives from the document; an edit's own layout is not this.+ */+ applyDerivedLayoutDuring(fn: () => void): void {+ this.derivedLayoutDepth+++ try {+ fn()+ } finally {+ this.derivedLayoutDepth--+ }+ }++ get isApplyingDerivedLayout(): boolean {+ return this.derivedLayoutDepth > 0+ }+withLayoutMutations(fn: () => void): void {this.layoutMutationDepth++try {diff --git a/src/app/tabs/index.ts b/src/app/tabs/index.tsindex 9ee91e4b2..449f98801 100644--- a/src/app/tabs/index.ts+++ b/src/app/tabs/index.ts@@ -218,11 +218,13 @@ function reusableTabStore(): { store: EditorStore; created: boolean } {async function readFigForTab(file: File, signal?: AbortSignal): Promise<SceneGraph> {const imported = await readFigDocument(file, signal)const firstPageId = imported.getPages()[0]?.id- if (firstPageId) computeAllLayouts(imported, firstPageId)+ const layOut = (pageId: string) =>+ imported.applyDerivedLayoutDuring(() => computeAllLayouts(imported, pageId))+ if (firstPageId) layOut(firstPageId)const coverPageId = findFigThumbnailPageId(imported.getPages())if (coverPageId && coverPageId !== firstPageId) {populateFigPage(imported, coverPageId)- computeAllLayouts(imported, coverPageId)+ layOut(coverPageId)}return imported}