Skip to content

Build your personal knowledge base with Trilium Notes

feat(mobile): save the database backup onto the device

Elian Doran0b4fadb45854
diff --git a/.claude/skills/developing-capacitor-mobile/SKILL.md b/.claude/skills/developing-capacitor-mobile/SKILL.md
index 26419975c2..a33c6a1491 100644
--- a/.claude/skills/developing-capacitor-mobile/SKILL.md
+++ b/.claude/skills/developing-capacitor-mobile/SKILL.md
@@ -83,8 +83,15 @@ sheet.
`capacitor.config.json` (iOS only builds what is listed), and `cap update android` to regenerate the
tracked `capacitor.settings.gradle` / `app/capacitor.build.gradle`. iOS's `CapApp-SPM/Package.swift`
is gitignored and regenerated by CI.
-- **Still unfixed: the database backup download** (`downloadDatabase` in `local-bridge.ts`) navigates a
- hidden iframe to `/local-backup-download` and hits the same wall.
+- **The database backup takes the same last step, from a different source.** `saveDatabase()` in
+ `local-bridge.ts` consumes the worker's `BACKUP_STREAM` channel in the page rather than relaying the
+ port to the service worker, so no SW is involved (which is also why it works on iOS), and
+ `setBackupPinging` stays off — that keepalive exists only for a stream the SW is holding open.
+ Two things differ from a download: it goes to `Directory.Documents` under `Trilium/`, not the cache,
+ and the folder is **not** swept first, which would delete the previous backup. A dismissed share
+ sheet is `done`, not `cancelled`: the file is complete before the sheet opens.
+- **`rechunk()` is what makes any source safe to write.** A producer picks its chunk sizes for its own
+ reasons — a response body by packet, the backup by database page — and neither is 3-aligned.
## Native shells
diff --git a/apps/client/src/services/backup_download.spec.ts b/apps/client/src/services/backup_download.spec.ts
index 72f97d0978..a35b0810dd 100644
--- a/apps/client/src/services/backup_download.spec.ts
+++ b/apps/client/src/services/backup_download.spec.ts
@@ -1,6 +1,6 @@
-import { describe, expect, it } from "vitest";
+import { afterEach, describe, expect, it, vi } from "vitest";
-import { backupFileName } from "./backup_download";
+import { backupFileName, isBackupDownloadSupported, startBackupDownload } from "./backup_download";
// What the suggested name itself looks like is settled in commons, which every platform shares;
// this is about the file that name becomes.
@@ -28,3 +28,46 @@ describe("naming a backup file", () => {
expect(backupFileName("com1")).toMatch(dated);
});
});
+
+describe("taking the backup", () => {
+ const realWindow = window as unknown as { standaloneApi?: unknown };
+
+ afterEach(() => {
+ delete realWindow.standaloneApi;
+ });
+
+ it("hands the backup to the device where one saves it, and to the browser otherwise", async () => {
+ const downloadDatabase = vi.fn(async () => ({ status: "done" as const }));
+ const saveDatabase = vi.fn(async () => ({
+ status: "done" as const,
+ location: "/documents/Trilium/b.tnbackup"
+ }));
+ const onProgress = vi.fn();
+
+ realWindow.standaloneApi = { backup: { downloadDatabase } };
+ expect(await startBackupDownload("b.tnbackup", "pw", onProgress)).toEqual({ status: "done" });
+ expect(downloadDatabase).toHaveBeenCalledWith("b.tnbackup", "pw", onProgress);
+
+ // The mobile shell has no download manager, so it writes the file itself and says where.
+ realWindow.standaloneApi = { backup: { downloadDatabase, saveDatabase } };
+ expect(await startBackupDownload("b.tnbackup", "pw", onProgress)).toEqual({
+ status: "done",
+ location: "/documents/Trilium/b.tnbackup"
+ });
+ expect(saveDatabase).toHaveBeenCalledWith("b.tnbackup", "pw", onProgress);
+ expect(downloadDatabase).toHaveBeenCalledTimes(1);
+ });
+
+ it("passes no passphrase at all for an empty one, and fails where nothing backs up this way", async () => {
+ const downloadDatabase = vi.fn(async () => ({ status: "done" as const }));
+ realWindow.standaloneApi = { backup: { downloadDatabase } };
+
+ await startBackupDownload("b.tnbackup", "");
+ // An empty string would be a passphrase, and would lock the container with it.
+ expect(downloadDatabase).toHaveBeenCalledWith("b.tnbackup", undefined, undefined);
+
+ delete realWindow.standaloneApi;
+ expect(isBackupDownloadSupported()).toBe(false);
+ expect(await startBackupDownload("b.tnbackup")).toMatchObject({ status: "failed" });
+ });
+});
diff --git a/apps/client/src/services/backup_download.ts b/apps/client/src/services/backup_download.ts
index 536d255750..f4ed973bd7 100644
--- a/apps/client/src/services/backup_download.ts
+++ b/apps/client/src/services/backup_download.ts
@@ -32,10 +32,13 @@ export function backupFileName(name: string): string {
}
/**
- * Starts the download and resolves with how it ended, which is when the stream behind it has been
+ * Starts the backup and resolves with how it ended, which is when the stream behind it has been
* fully produced — the closest thing to "finished" the application can see, with the browser's
* own download UI carrying the transfer itself.
*
+ * Inside the mobile shell there is no download manager to carry it, so the backup is written onto
+ * the device and offered to the share sheet instead; the result then carries where it went.
+ *
* @param passphrase encrypts the container. Omitted or empty leaves it unencrypted, which is still
* a container: one format, one extension, whatever the user chose.
*/
@@ -44,10 +47,11 @@ export async function startBackupDownload(
passphrase?: string,
onProgress?: (sentBytes: number, totalBytes: number) => void
): Promise<StandaloneDownloadResult> {
- const api = window.standaloneApi;
- if (!api) {
+ const backup = window.standaloneApi?.backup;
+ if (!backup) {
return { status: "failed", message: "This platform does not back up by download." };
}
- return await api.backup.downloadDatabase(fileName, passphrase || undefined, onProgress);
+ const take = backup.saveDatabase ?? backup.downloadDatabase;
+ return await take.call(backup, fileName, passphrase || undefined, onProgress);
}
diff --git a/apps/client/src/setup_backup.spec.tsx b/apps/client/src/setup_backup.spec.tsx
index e620431567..9b393d7217 100644
--- a/apps/client/src/setup_backup.spec.tsx
+++ b/apps/client/src/setup_backup.spec.tsx
@@ -11,7 +11,7 @@ const mocks = vi.hoisted(() => ({
_fileName: string,
_passphrase?: string,
_onProgress?: (sentBytes: number, totalBytes: number) => void
- ) => ({ status: "done" }) as { status: string; message?: string }
+ ) => ({ status: "done" }) as { status: string; message?: string; location?: string }
)
}));
@@ -340,6 +340,23 @@ describe("backing up from the setup screen", () => {
expect(button("setup.backup-finish")?.disabled).toBe(false);
});
+ it("names where the file went on a platform that puts it somewhere it can name", async () => {
+ // The mobile shell writes the backup itself rather than handing it to a download manager,
+ // so it is the only thing that knows where the user should go looking.
+ mocks.startBackupDownload.mockResolvedValue({
+ status: "done",
+ location: "/documents/Trilium/b.tnbackup"
+ });
+ await reachDownload();
+
+ button("setup.backup-download")?.click();
+ await settle();
+
+ const outcome = container.querySelector(".backup-download-outcome")?.textContent;
+ expect(outcome).toContain("setup.backup-saved-to");
+ expect(outcome).not.toContain("setup.backup-downloaded");
+ });
+
it("counts the download as it goes, since the browser's own progress is out of sight on a phone", async () => {
let report: ((sent: number, total: number) => void) | undefined;
mocks.startBackupDownload.mockImplementation((_name, _passphrase, onProgress) => {
diff --git a/apps/client/src/setup_backup.tsx b/apps/client/src/setup_backup.tsx
index f7eadcb846..d0d49874ca 100644
--- a/apps/client/src/setup_backup.tsx
+++ b/apps/client/src/setup_backup.tsx
@@ -42,6 +42,8 @@ export interface BackupDownload {
error?: string;
/** How far it has got, once the first bytes have gone. */
progress?: { sentBytes: number; totalBytes: number };
+ /** Where the file was written, on the platforms that put it somewhere they can name. */
+ location?: string;
start: () => void;
}
@@ -58,16 +60,19 @@ export function useBackupDownload(settings: SetupBackupSettings): BackupDownload
const [ state, setState ] = useState<BackupDownload["state"]>("idle");
const [ error, setError ] = useState<string>();
const [ progress, setProgress ] = useState<BackupDownload["progress"]>();
+ const [ location, setLocation ] = useState<string>();
return {
fileName,
state,
error,
progress,
+ location,
start: () => {
setState("running");
setError(undefined);
setProgress(undefined);
+ setLocation(undefined);
const onProgress = (sentBytes: number, totalBytes: number) =>
setProgress({ sentBytes, totalBytes });
@@ -75,6 +80,7 @@ export function useBackupDownload(settings: SetupBackupSettings): BackupDownload
void startBackupDownload(fileName, settings.passphrase, onProgress).then((result) => {
setState(result.status === "done" ? "done" : "failed");
setError(result.status === "done" ? undefined : result.message);
+ setLocation(result.location);
});
}
};
@@ -231,7 +237,9 @@ export function BackupDownloadPanel({ download }: { download: BackupDownload })
it, and the button is what the user does next either way. */}
{download.state === "done" && (
<Admonition type="note" className="backup-download-outcome">
- {t("setup.backup-downloaded")}
+ {download.location
+ ? t("setup.backup-saved-to", { location: download.location })
+ : t("setup.backup-downloaded")}
</Admonition>
)}
{download.state === "failed" && (
diff --git a/apps/client/src/translations/en/entry.json b/apps/client/src/translations/en/entry.json
index 0e5225a80f..c77246d833 100644
--- a/apps/client/src/translations/en/entry.json
+++ b/apps/client/src/translations/en/entry.json
@@ -159,6 +159,7 @@
"restore-stage-validating": "Checking the database",
"restore-stage-swapping": "Putting it in place",
"restore-stage-migrating": "Opening the database",
- "restore-do-not-close": "Do not close this window."
+ "restore-do-not-close": "Do not close this window.",
+ "backup-saved-to": "The backup has been saved to {{- location}}. Press \"Finish\" to return to the application."
}
}
diff --git a/apps/standalone/src/local-bridge.spec.ts b/apps/standalone/src/local-bridge.spec.ts
index a6e2b99944..a3aeb60353 100644
--- a/apps/standalone/src/local-bridge.spec.ts
+++ b/apps/standalone/src/local-bridge.spec.ts
@@ -2,15 +2,48 @@ import { afterEach, describe, expect, it, vi } from "vitest";
// A controllable stand-in for the bundled local-server-worker. vi.hoisted lets the
// (hoisted) vi.mock factory share the instance registry with the test body.
-const { workerInstances, leadership } = vi.hoisted(() => ({
+const { workerInstances, leadership, saved } = vi.hoisted(() => ({
workerInstances: [] as MockWorker[],
// Most of this suite exercises the leader, which is the tab that owns the
// worker. Followers take a different path — see the leadership describe.
- leadership: { isLeader: true }
+ leadership: { isLeader: true },
+ // What saveDatabase() handed the device, and what the device said back.
+ saved: {
+ fileName: "",
+ chunks: [] as Uint8Array[],
+ target: null as unknown,
+ result: { status: "saved", location: "/documents/Trilium/db.tnbackup" } as {
+ status: string;
+ location?: string;
+ message?: string;
+ }
+ }
}));
vi.mock("./leader_election.js", () => ({ isLeader: () => leadership.isLeader }));
+// The Capacitor plugins are not there under happy-dom, and what matters on this side is which
+// target the backup asked for and what reached it — see capacitor_download.spec.ts for the writing.
+vi.mock("./services/capacitor_download.js", () => ({
+ BACKUP_TARGET: { directory: "DOCUMENTS", folder: "Trilium", sweep: false },
+ saveChunksToDevice: async (
+ fileName: string, chunks: AsyncIterable<Uint8Array>, target: unknown
+ ) => {
+ saved.fileName = fileName;
+ saved.target = target;
+ saved.chunks = [];
+ try {
+ for await (const chunk of chunks) {
+ saved.chunks.push(chunk);
+ }
+ } catch (e) {
+ // The real one reports a broken source rather than rejecting; see its own spec.
+ return { fileName, status: "failed", message: (e as Error).message };
+ }
+ return { fileName, ...saved.result };
+ }
+}));
+
class MockWorker {
postMessage = vi.fn();
terminate = vi.fn();
@@ -33,6 +66,7 @@ async function freshBridge(withServiceWorker = true): Promise<LocalBridge> {
vi.resetModules();
workerInstances.length = 0;
leadership.isLeader = true;
+ saved.result = { status: "saved", location: "/documents/Trilium/db.tnbackup" };
swHandler = undefined;
if (withServiceWorker) {
Object.defineProperty(navigator, "serviceWorker", {
@@ -430,7 +464,7 @@ describe("backup download keepalive", () => {
vi.useFakeTimers();
const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue(new Response(null));
const bridge = await freshBridge();
- bridge.startLocalServerWorker();
+ void bridge.downloadDatabase("Backup.tnbackup");
lastWorker().onmessage?.({ data: { type: "BACKUP_STREAM_ACTIVE", active: true } });
await vi.advanceTimersByTimeAsync(25_000);
@@ -443,6 +477,21 @@ describe("backup download keepalive", () => {
vi.useRealTimers();
});
+
+ it("does not ping for a stream the page consumes itself", async () => {
+ vi.useFakeTimers();
+ const fetchSpy = vi.spyOn(globalThis, "fetch").mockResolvedValue(new Response(null));
+ const bridge = await freshBridge();
+ bridge.startLocalServerWorker();
+
+ // saveDatabase() holds the stream open in the page, so there is no service worker waiting
+ // on it — and on iOS there is no service worker at all.
+ lastWorker().onmessage?.({ data: { type: "BACKUP_STREAM_ACTIVE", active: true } });
+ await vi.advanceTimersByTimeAsync(60_000);
+
+ expect(fetchSpy).not.toHaveBeenCalled();
+ vi.useRealTimers();
+ });
});
describe("downloadDatabase", () => {
@@ -493,6 +542,92 @@ describe("downloadDatabase", () => {
});
});
+describe("saveDatabase", () => {
+ /** What the bridge asked the worker to stream, including the port to play it down. */
+ function backupMessage(worker: MockWorker): { port?: MessagePort; passphrase?: string } {
+ return worker.postMessage.mock.calls
+ .map(([ message ]) => message as { type: string; port?: MessagePort; passphrase?: string })
+ .find((message) => message.type === "BACKUP_STREAM") ?? {};
+ }
+
+ /** The port the bridge handed the worker, played from the worker's side. */
+ function backupPort(worker: MockWorker): MessagePort {
+ const port = backupMessage(worker).port;
+ if (!port) {
+ throw new Error("no BACKUP_STREAM sent");
+ }
+ return port;
+ }
+
+ /** Answers the bridge's pulls with `chunks`, then ends — what the local worker does. */
+ function serveStream(port: MessagePort, chunks: Uint8Array[]) {
+ let next = 0;
+ port.onmessage = (event) => {
+ const message = event.data as { type?: string };
+ if (message?.type !== "pull") {
+ return;
+ }
+ if (next < chunks.length) {
+ const chunk = chunks[next++];
+ port.postMessage({ type: "chunk", data: chunk.buffer.slice(0) });
+ } else {
+ port.postMessage({ type: "end" });
+ }
+ };
+ port.postMessage({ type: "begin", byteSize: chunks.reduce((sum, c) => sum + c.length, 0) });
+ }
+
+ it("streams the backup to the device and reports where it went", async () => {
+ const bridge = await freshBridge();
+ const progress: number[] = [];
+ const pending = bridge.saveDatabase("db.tnbackup", "hunter2", (sent) => progress.push(sent));
+
+ // The dynamic import of the save module resolves before the stream is asked for.
+ await vi.waitFor(() => backupPort(lastWorker()));
+ serveStream(backupPort(lastWorker()), [ Uint8Array.from([ 1, 2, 3 ]), Uint8Array.from([ 4, 5 ]) ]);
+ lastWorker().onmessage?.({
+ data: { type: "BACKUP_STREAM_PROGRESS", sentBytes: 3, totalBytes: 5 }
+ });
+
+ expect(await pending).toEqual({ status: "done", location: "/documents/Trilium/db.tnbackup" });
+ expect(saved.chunks).toEqual([ Uint8Array.from([ 1, 2, 3 ]), Uint8Array.from([ 4, 5 ]) ]);
+ expect(saved.fileName).toBe("db.tnbackup");
+ // Documents, not the share sheet's scratch directory: a backup outlives the sheet.
+ expect(saved.target).toMatchObject({ directory: "DOCUMENTS", sweep: false });
+ // The passphrase rides the worker message rather than any URL.
+ expect(backupMessage(lastWorker()).passphrase).toBe("hunter2");
+ expect(progress).toEqual([ 3 ]);
+ });
+
+ it("is done, not cancelled, when the user dismisses the share sheet", async () => {
+ const bridge = await freshBridge();
+ saved.result = { status: "cancelled", location: "/documents/Trilium/db.tnbackup" };
+ const pending = bridge.saveDatabase("db.tnbackup");
+
+ await vi.waitFor(() => backupPort(lastWorker()));
+ serveStream(backupPort(lastWorker()), [ Uint8Array.from([ 1 ]) ]);
+
+ // The file was written before the sheet opened, so there is a backup either way.
+ expect(await pending).toEqual({ status: "done", location: "/documents/Trilium/db.tnbackup" });
+ });
+
+ it("fails when the stream never begins, and refuses outright in a follower tab", async () => {
+ const bridge = await freshBridge();
+ const pending = bridge.saveDatabase("db.tnbackup");
+
+ await vi.waitFor(() => backupPort(lastWorker()));
+ backupPort(lastWorker()).postMessage({ type: "error", message: "The database is not ready yet." });
+
+ expect(await pending).toMatchObject({
+ status: "failed",
+ message: "The database is not ready yet."
+ });
+
+ leadership.isLeader = false;
+ expect(await bridge.saveDatabase("db.tnbackup")).toMatchObject({ status: "failed" });
+ });
+});
+
describe("restoreBackup", () => {
/** The message the bridge sent to the worker to start a restore. */
function restoreMessage(worker: MockWorker) {
diff --git a/apps/standalone/src/local-bridge.ts b/apps/standalone/src/local-bridge.ts
index 83a5e52a92..45fa743a1c 100644
--- a/apps/standalone/src/local-bridge.ts
+++ b/apps/standalone/src/local-bridge.ts
@@ -209,6 +209,116 @@ function settlePendingDownload(result: StandaloneDownloadResult): void {
}
}
+/** Where a save in flight wants its progress, which arrives on the worker's own channel. */
+let saveProgress: ((sentBytes: number, totalBytes: number) => void) | null = null;
+
+/**
+ * Writes the same backup onto the device, for the mobile shell.
+ *
+ * The service worker drops out of the path entirely: it exists in {@link downloadDatabase} only to
+ * turn the stream into a `Response` the browser's download manager will take, and the WebView has
+ * no download manager to take it. Here the page consumes the stream itself — the same pull-driven
+ * protocol, the same container, the same passphrase — and writes it to the app's documents
+ * directory, so it also works on iOS, where a service worker cannot be registered at all.
+ *
+ * A dismissed share sheet is still a backup: the file is complete before the sheet opens, so that
+ * outcome is `done` with the path it went to, not `cancelled`.
+ */
+export async function saveDatabase(
+ fileName: string,
+ passphrase?: string,
+ onProgress?: (sentBytes: number, totalBytes: number) => void
+): Promise<StandaloneDownloadResult> {
+ if (!isLeader()) {
+ return {
+ status: "failed",
+ message: "A backup can only be taken from the tab that owns the database. "
+ + "Close the other Trilium tabs and try again."
+ };
+ }
+
+ const { saveChunksToDevice, BACKUP_TARGET } = await import("./services/capacitor_download.js");
+
+ saveProgress = onProgress ?? null;
+ try {
+ const result = await saveChunksToDevice(fileName, backupChunks(passphrase), BACKUP_TARGET);
+ if (result.status === "failed") {
+ return { status: "failed", message: result.message };
+ }
+ return { status: "done", location: result.location };
+ } finally {
+ saveProgress = null;
+ }
+}
+
+/**
+ * The backup as a stream of bytes, pulled straight off the local worker.
+ *
+ * The worker answers one `chunk` per `pull` and ends with `end`, which is what carries backpressure
+ * back to the database reads: chunks are asked for as fast as they can be written and no faster. A
+ * few pulls are kept outstanding so the worker reads the next pages while the current chunk is
+ * still crossing the plugin bridge, which is by far the slower of the two.
+ */
+async function* backupChunks(passphrase: string | undefined): AsyncGenerator<Uint8Array> {
+ const worker = startLocalServerWorker();
+ const channel = new MessageChannel();
+ const port = channel.port1;
+
+ const arrived: unknown[] = [];
+ let waiting: ((message: unknown) => void) | null = null;
+ port.onmessage = (event) => {
+ if (waiting) {
+ const resolve = waiting;
+ waiting = null;
+ resolve(event.data);
+ } else {
+ arrived.push(event.data);
+ }
+ };
+ const next = (): Promise<unknown> => arrived.length
+ ? Promise.resolve(arrived.shift())
+ : new Promise((resolve) => { waiting = resolve; });
+
+ worker.postMessage({ type: "BACKUP_STREAM", port: channel.port2, passphrase }, [ channel.port2 ]);
+
+ try {
+ const begin = await next() as { type?: string; message?: string } | null;
+ if (begin?.type !== "begin") {
+ throw new Error(begin?.message ?? "The backup stream did not start.");
+ }
+
+ let outstanding = 0;
+ for (;;) {
+ while (outstanding < MAX_BACKUP_CHUNKS_IN_FLIGHT) {
+ outstanding++;
+ port.postMessage({ type: "pull" });
+ }
+
+ const message = await next() as { type?: string; data?: ArrayBuffer; message?: string } | null;
+ outstanding--;
+
+ if (message?.type === "end") {
+ return;
+ }
+ if (message?.type !== "chunk" || !message.data) {
+ throw new Error(message?.message ?? "The backup stream failed.");
+ }
+ yield new Uint8Array(message.data);
+ }
+ } finally {
+ // Ends the stream for a consumer that stopped early — a failed write, say — rather than
+ // leaving the worker reading pages nothing will collect.
+ port.postMessage({ type: "cancel" });
+ port.close();
+ }
+}
+
+/**
+ * How many chunks may be in flight at once. The worker reads ahead this far while the plugin
+ * bridge is busy with the previous one; every chunk beyond that is memory held for nothing.
+ */
+const MAX_BACKUP_CHUNKS_IN_FLIGHT = 4;
+
export function isLocalApiRequest(url: URL): boolean {
return LOCAL_API_PREFIXES.some(p => url.pathname.startsWith(p));
}
@@ -362,11 +472,14 @@ export function startLocalServerWorker() {
// download UI behind the notification shade.
if (msg?.type === "BACKUP_STREAM_PROGRESS") {
pendingDownload?.onProgress?.(Number(msg.sentBytes), Number(msg.totalBytes));
+ saveProgress?.(Number(msg.sentBytes), Number(msg.totalBytes));
return;
}
if (msg?.type === "BACKUP_STREAM_ACTIVE") {
- setBackupPinging(msg.active === true);
+ // Only a stream the service worker is holding open needs it kept alive; `saveDatabase`
+ // consumes its own, and on iOS the ping would be a request to a worker that is not there.
+ setBackupPinging(msg.active === true && pendingDownload !== null);
if (msg.active === true) {
if (pendingDownload) {
clearTimeout(pendingDownload.startTimer);
diff --git a/apps/standalone/src/main.ts b/apps/standalone/src/main.ts
index dec7feb4f8..fd35a6378d 100644
--- a/apps/standalone/src/main.ts
+++ b/apps/standalone/src/main.ts
@@ -6,7 +6,7 @@ import {
import { showErrorOverlay } from "./error-overlay.js";
import { installIosInterceptors } from "./ios-interceptors.js";
import { claimLeadership } from "./leader_election.js";
-import { announceLeadership, attachServiceWorkerBridge, downloadDatabase, registerNativeHttpHandler, restoreBackup, startLocalServerWorker } from "./local-bridge.js";
+import { announceLeadership, attachServiceWorkerBridge, downloadDatabase, registerNativeHttpHandler, restoreBackup, saveDatabase, startLocalServerWorker } from "./local-bridge.js";
/**
* What a cold standalone start passes through, drawn as the splash's progress bar. Weights are
@@ -99,9 +99,11 @@ async function bootstrap() {
registerNativeHttpHandler(capacitorHttpHandler);
// The shell's WebView drops a download the moment the response says `attachment`,
- // so the client routes downloads through the share sheet instead.
+ // so the client routes downloads through the share sheet instead. The backup takes the
+ // same route, off its own stream rather than a response.
const { saveUrlToDevice } = await import("./services/capacitor_download.js");
standaloneApi.save = { saveUrl: saveUrlToDevice };
+ standaloneApi.backup.saveDatabase = saveDatabase;
}
// 1) Start the local worker ASAP (so /bootstrap is fast) — but only in
diff --git a/apps/standalone/src/services/capacitor_download.spec.ts b/apps/standalone/src/services/capacitor_download.spec.ts
index 579ca0abb1..b14a948a11 100644
--- a/apps/standalone/src/services/capacitor_download.spec.ts
+++ b/apps/standalone/src/services/capacitor_download.spec.ts
@@ -1,6 +1,6 @@
import { afterEach, describe, expect, it, vi } from "vitest";
-import { fileNameOf, saveUrlToDevice } from "./capacitor_download.js";
+import { BACKUP_TARGET, fileNameOf, saveChunksToDevice, saveUrlToDevice } from "./capacitor_download.js";
interface CapacitorWindow {
Capacitor?: unknown;
@@ -44,7 +44,11 @@ function installCapacitor({ share }: { share?: ReturnType<typeof vi.fn> } = {})
/** The bytes handed to the plugin, decoded back out of the base64 each write carried. */
function writtenBytes(written: { data: string }[]): Uint8Array {
const binary = written.map((write) => atob(write.data)).join("");
- return Uint8Array.from(binary, (char) => char.charCodeAt(0));
+ const bytes = new Uint8Array(binary.length);
+ for (let i = 0; i < binary.length; i++) {
+ bytes[i] = binary.charCodeAt(i);
+ }
+ return bytes;
}
/** A response whose body streams in `chunkSize` pieces, as a real network body would. */
@@ -141,7 +145,12 @@ describe("capacitor download", () => {
const result = await saveUrlToDevice("api/branches/b1/export/subtree/html/t1");
- expect(result).toEqual({ status: "saved", fileName: "Export.zip" });
+ expect(result).toEqual({
+ status: "saved",
+ fileName: "Export.zip",
+ // The `file://` scheme is stripped: this is shown to a person, not opened.
+ location: "/cache/trilium-downloads/Export.zip"
+ });
expect(removed).toEqual(["trilium-downloads"]);
expect(written.map((write) => write.append)).toEqual([false, true]);
expect(written.every((write) => write.path === "trilium-downloads/Export.zip")).toBe(true);
@@ -162,12 +171,12 @@ describe("capacitor download", () => {
headers: { "content-disposition": `attachment; filename="a.bin"` },
buffer: bytes
})));
- expect(await saveUrlToDevice("api/x")).toEqual({ status: "saved", fileName: "a.bin" });
+ expect(await saveUrlToDevice("api/x")).toMatchObject({ status: "saved", fileName: "a.bin" });
expect(writtenBytes(written)).toEqual(bytes);
written.length = 0;
vi.stubGlobal("fetch", vi.fn(async () => fakeResponse({})));
- expect(await saveUrlToDevice("api/empty.bin")).toEqual({ status: "saved", fileName: "empty.bin" });
+ expect(await saveUrlToDevice("api/empty.bin")).toMatchObject({ status: "saved", fileName: "empty.bin" });
expect(written).toEqual([
expect.objectContaining({ data: "", append: false })
]);
@@ -176,10 +185,16 @@ describe("capacitor download", () => {
it("reports a cancelled share sheet as cancelled, and any other share error as a failure", async () => {
installCapacitor({ share: vi.fn(async () => { throw new Error("Share canceled"); }) });
vi.stubGlobal("fetch", vi.fn(async () => fakeResponse({ buffer: Uint8Array.from([1]) })));
- expect(await saveUrlToDevice("api/a.bin")).toEqual({ status: "cancelled", fileName: "a.bin" });
+ // Still a file on disk, which is what `location` says and what makes a dismissed sheet
+ // survivable for a backup.
+ expect(await saveUrlToDevice("api/a.bin")).toEqual({
+ status: "cancelled",
+ fileName: "a.bin",
+ location: "/cache/trilium-downloads/a.bin"
+ });
installCapacitor({ share: vi.fn(async () => { throw new Error("No app can open this"); }) });
- expect(await saveUrlToDevice("api/a.bin")).toEqual({
+ expect(await saveUrlToDevice("api/a.bin")).toMatchObject({
status: "failed",
fileName: "a.bin",
message: "No app can open this"
@@ -206,4 +221,64 @@ describe("capacitor download", () => {
});
});
});
+
+ describe("saveChunksToDevice", () => {
+ /** Emits `bytes` in pieces of `size`, which is how an arbitrary producer feeds the writer. */
+ async function* inPieces(bytes: Uint8Array, size: number) {
+ for (let offset = 0; offset < bytes.length; offset += size) {
+ yield bytes.slice(offset, offset + size);
+ }
+ }
+
+ it("realigns a source whose chunks are not a multiple of 3 bytes", async () => {
+ const { written } = installCapacitor();
+ // 4001 is coprime with 3, so every incoming piece straddles a base64 group. Writing
+ // these through unaligned would pad mid-file and corrupt everything after the first
+ // piece, which is exactly what the database backup's page-sized chunks would do. The
+ // total clears one 768 KiB write, so there is a boundary to get wrong.
+ const bytes = Uint8Array.from({ length: 800 * 1024 }, (_, i) => (i * 7) % 251);
+
+ const result = await saveChunksToDevice("db.tnbackup", inPieces(bytes, 4001), BACKUP_TARGET);
+
+ expect(result.status).toBe("saved");
+ expect(writtenBytes(written)).toEqual(bytes);
+ // Every write but the last is a whole chunk, so every write but the last is 3-aligned.
+ expect(written.slice(0, -1).every((write) => atob(write.data).length % 3 === 0)).toBe(true);
+ });
+
+ it("writes a backup into documents and leaves the previous ones alone", async () => {
+ const { written, removed, sharePlugin } = installCapacitor();
+
+ const result = await saveChunksToDevice(
+ "db.tnbackup", inPieces(Uint8Array.from([1, 2, 3]), 3), BACKUP_TARGET
+ );
+
+ // Sweeping the folder before each write is right for the share sheet's scratch space
+ // and would destroy the previous backup here.
+ expect(removed).toEqual([]);
+ expect(written).toEqual([
+ expect.objectContaining({ path: "Trilium/db.tnbackup", directory: "DOCUMENTS" })
+ ]);
+ expect(result.location).toBe("/cache/Trilium/db.tnbackup");
+ expect(sharePlugin.share).toHaveBeenCalledWith({
+ title: "db.tnbackup",
+ files: ["file:///cache/Trilium/db.tnbackup"]
+ });
+ });
+
+ it("fails without sharing when the source throws part-way through", async () => {
+ const { sharePlugin } = installCapacitor();
+ async function* breaks() {
+ yield Uint8Array.from([1, 2, 3]);
+ throw new Error("The backup stream failed.");
+ }
+
+ expect(await saveChunksToDevice("db.tnbackup", breaks(), BACKUP_TARGET)).toEqual({
+ status: "failed",
+ fileName: "db.tnbackup",
+ message: "The backup stream failed."
+ });
+ expect(sharePlugin.share).not.toHaveBeenCalled();
+ });
+ });
});
diff --git a/apps/standalone/src/services/capacitor_download.ts b/apps/standalone/src/services/capacitor_download.ts
index 34f1be3e08..666e341d9a 100644
--- a/apps/standalone/src/services/capacitor_download.ts
+++ b/apps/standalone/src/services/capacitor_download.ts
@@ -10,13 +10,45 @@ import type { StandaloneSaveResult } from "@triliumnext/commons";
* them. The page therefore does the whole job: fetch (still routed to the worker by the service
* worker on Android and by the iOS interceptors on `capacitor://`), write to the app's cache
* directory, hand the file to the system share sheet.
+ *
+ * A database backup takes the same last step through {@link saveChunksToDevice}, reading its bytes
+ * straight off the local worker rather than out of a response.
*/
-/** Where a saved file waits between being written and being handed to the share sheet. */
-const DOWNLOAD_DIR = "trilium-downloads";
+/** Where a kind of saved file is written, and what happens to what is already there. */
+export interface SaveTarget {
+ /** `Directory`'s value, which the Filesystem plugin takes as a string over the bridge. */
+ directory: "CACHE" | "DOCUMENTS";
+ /** The folder inside it, so saved files do not litter the directory's root. */
+ folder: string;
+ /**
+ * Whether the folder is emptied before the write. True for scratch space the share sheet reads
+ * from once; false where the folder holds the user's own files.
+ */
+ sweep: boolean;
+}
+
+/**
+ * A download: transient. The share sheet is what the user does with it, and the copy left behind is
+ * rubbish, cleared before the next one. It cannot be cleared *after* the sheet closes, because the
+ * receiving app reads the shared URI on its own schedule.
+ */
+export const DOWNLOAD_TARGET: SaveTarget = {
+ directory: "CACHE",
+ folder: "trilium-downloads",
+ sweep: true
+};
-/** `Directory.Cache` — the Filesystem plugin takes the enum's string value over the bridge. */
-const CACHE = "CACHE";
+/**
+ * A backup: the opposite. It has to survive the share sheet being dismissed and the phone running
+ * low on space, neither of which a cache directory does, and sweeping the folder would destroy the
+ * previous backup. Same-day repeats overwrite, which is what the date-based default name means.
+ */
+export const BACKUP_TARGET: SaveTarget = {
+ directory: "DOCUMENTS",
+ folder: "Trilium",
+ sweep: false
+};
/**
* Bytes per write. A multiple of 3, so each chunk encodes independently: base64 pads any group
@@ -37,8 +69,31 @@ export async function saveUrlToDevice(url: string): Promise<StandaloneSaveResult
}
fileName = fileNameOf(response, url);
- const uri = await writeToCache(fileName, response);
- return await shareFile(uri, fileName);
+ return await saveChunksToDevice(fileName, bodyChunks(response), DOWNLOAD_TARGET);
+ } catch (e) {
+ return {
+ status: "failed",
+ fileName,
+ message: e instanceof Error ? e.message : String(e)
+ };
+ }
+}
+
+/**
+ * Writes a stream of bytes into `target` and offers the file to the share sheet.
+ *
+ * The file is complete before the sheet opens, and `location` is set as soon as it is on disk, so
+ * a `cancelled` result still names a file that exists — which is what makes this usable for a
+ * backup, where dismissing the sheet must not mean the backup never happened.
+ */
+export async function saveChunksToDevice(
+ fileName: string,
+ chunks: AsyncIterable<Uint8Array>,
+ target: SaveTarget
+): Promise<StandaloneSaveResult> {
+ try {
+ const uri = await writeChunks(fileName, chunks, target);
+ return { ...await shareFile(uri, fileName), location: displayPath(uri) };
} catch (e) {
return {
status: "failed",
@@ -85,37 +140,53 @@ function getPlugin<T>(name: string): T {
return plugin as T;
}
-/** Writes the response into the cache directory a chunk at a time and returns the file's URI. */
-async function writeToCache(fileName: string, response: Response): Promise<string> {
+/** Writes the chunks into `target` one at a time and returns the finished file's URI. */
+async function writeChunks(
+ fileName: string,
+ chunks: AsyncIterable<Uint8Array>,
+ { directory, folder, sweep }: SaveTarget
+): Promise<string> {
const filesystem = getPlugin<FilesystemPlugin>("Filesystem");
- const path = `${DOWNLOAD_DIR}/${fileName}`;
+ const path = `${folder}/${fileName}`;
- // What the previous save left behind. The share sheet hands the receiving app a URI it reads
- // on its own schedule, so a shared file can only be removed before the next save, never after
- // `share()` resolves.
- await filesystem
- .rmdir({ path: DOWNLOAD_DIR, directory: CACHE, recursive: true })
- .catch(() => undefined);
+ if (sweep) {
+ await filesystem
+ .rmdir({ path: folder, directory, recursive: true })
+ .catch(() => undefined);
+ }
let started = false;
- for await (const chunk of chunksOf(response, CHUNK_BYTES)) {
+ for await (const chunk of rechunk(chunks, CHUNK_BYTES)) {
const data = toBase64(chunk);
if (started) {
- await filesystem.appendFile({ path, data, directory: CACHE });
+ await filesystem.appendFile({ path, data, directory });
} else {
- await filesystem.writeFile({ path, data, directory: CACHE, recursive: true });
+ await filesystem.writeFile({ path, data, directory, recursive: true });
started = true;
}
}
if (!started) {
- await filesystem.writeFile({ path, data: "", directory: CACHE, recursive: true });
+ await filesystem.writeFile({ path, data: "", directory, recursive: true });
}
- const { uri } = await filesystem.getUri({ path, directory: CACHE });
+ const { uri } = await filesystem.getUri({ path, directory });
return uri;
}
+/**
+ * The file's URI as somewhere a person can be told to look. `getUri` answers with a `file://` URL,
+ * percent-encoded, which is a path with two layers of machinery on top of it.
+ */
+function displayPath(uri: string): string {
+ const path = uri.replace(/^file:\/\//, "");
+ try {
+ return decodeURIComponent(path);
+ } catch {
+ return path;
+ }
+}
+
async function shareFile(uri: string, fileName: string): Promise<StandaloneSaveResult> {
try {
await getPlugin<SharePlugin>("Share").share({ title: fileName, files: [uri] });
@@ -130,12 +201,12 @@ async function shareFile(uri: string, fileName: string): Promise<StandaloneSaveR
}
/**
- * Reads the body in pieces of `size` bytes, with whatever is left over emitted last.
+ * Reads the body as it arrives.
*
* A response with no `body` — happy-dom under the specs, and a WebView old enough to lack
* streaming — is read whole instead.
*/
-async function* chunksOf(response: Response, size: number): AsyncGenerator<Uint8Array> {
+async function* bodyChunks(response: Response): AsyncGenerator<Uint8Array> {
const reader = response.body?.getReader();
if (!reader) {
const whole = new Uint8Array(await response.arrayBuffer());
@@ -145,29 +216,73 @@ async function* chunksOf(response: Response, size: number): AsyncGenerator<Uint8
return;
}
- let pending = new Uint8Array(0);
for (;;) {
const { done, value } = await reader.read();
if (value?.length) {
- const merged = new Uint8Array(pending.length + value.length);
- merged.set(pending, 0);
- merged.set(value, pending.length);
- pending = merged;
-
- while (pending.length >= size) {
- const chunk = pending.slice(0, size);
- pending = pending.slice(size);
- yield chunk;
- }
+ yield value;
}
if (done) {
- break;
+ return;
+ }
+ }
+}
+
+/**
+ * Regroups a stream into pieces of `size` bytes, with whatever is left over emitted last.
+ *
+ * This is what keeps every write but the last a multiple of 3 bytes, so each encodes to base64
+ * independently. It belongs here rather than in the callers because a source picks its chunk sizes
+ * for its own reasons — a response body by packet, the backup stream by database page — and none
+ * of them is obliged to know what the writer needs.
+ */
+async function* rechunk(chunks: AsyncIterable<Uint8Array>, size: number): AsyncGenerator<Uint8Array> {
+ // Held as they arrive and joined only when a whole chunk is due. Merging each arrival into one
+ // growing buffer instead would copy everything received so far on every arrival, which a
+ // database backup pays for by the gigabyte.
+ const parts: Uint8Array[] = [];
+ let pending = 0;
+
+ for await (const value of chunks) {
+ if (!value.length) {
+ continue;
}
+ parts.push(value);
+ pending += value.length;
+
+ while (pending >= size) {
+ yield take(parts, size);
+ pending -= size;
+ }
+ }
+
+ if (pending) {
+ yield take(parts, pending);
}
+}
+
+/** Removes the first `size` bytes of `parts`, leaving the tail of a part it splits at the front. */
+function take(parts: Uint8Array[], size: number): Uint8Array {
+ const out = new Uint8Array(size);
+ let filled = 0;
- if (pending.length) {
- yield pending;
+ while (filled < size) {
+ const part = parts[0];
+ if (!part) {
+ return out.subarray(0, filled);
+ }
+
+ const taken = Math.min(part.length, size - filled);
+ out.set(part.subarray(0, taken), filled);
+ filled += taken;
+
+ if (taken === part.length) {
+ parts.shift();
+ } else {
+ parts[0] = part.subarray(taken);
+ }
}
+
+ return out;
}
/**
diff --git a/packages/commons/src/lib/standalone_api_interface.ts b/packages/commons/src/lib/standalone_api_interface.ts
index ffb38b8be3..fb9b459b80 100644
--- a/packages/commons/src/lib/standalone_api_interface.ts
+++ b/packages/commons/src/lib/standalone_api_interface.ts
@@ -47,6 +47,12 @@ export interface StandaloneDownloadResult {
status: "done" | "cancelled" | "failed";
/** What stopped it, when `status` is `failed`. */
message?: string;
+ /**
+ * Where the file was written, when it went somewhere nameable. A browser download lands
+ * wherever the browser puts it and reports nothing back; a save onto a device has a path,
+ * which the screen shows because the user otherwise has no way to find it again.
+ */
+ location?: string;
}
export interface StandaloneBackupApi {
@@ -70,6 +76,22 @@ export interface StandaloneBackupApi {
passphrase?: string,
onProgress?: (sentBytes: number, totalBytes: number) => void
): Promise<StandaloneDownloadResult>;
+
+ /**
+ * Writes the same backup onto the device instead, for the mobile shell, whose WebView has no
+ * download manager to hand it to.
+ *
+ * The bytes take the same pull-driven path off the database, so the container, the passphrase
+ * and the progress mean exactly what they do above; only the far end differs. The file lands
+ * in the app's documents directory and the share sheet then offers to put it somewhere that
+ * outlives the device — but the file is already written by then, so a dismissed sheet is still
+ * a backup, and the result carries where it went.
+ */
+ saveDatabase?(
+ fileName: string,
+ passphrase?: string,
+ onProgress?: (sentBytes: number, totalBytes: number) => void
+ ): Promise<StandaloneDownloadResult>;
}
/** How saving a download onto the device ended. */
@@ -79,6 +101,8 @@ export interface StandaloneSaveResult {
fileName?: string;
/** What stopped it, when `status` is `failed`. */
message?: string;
+ /** Where the file was written, present once it is on disk whatever the share sheet then did. */
+ location?: string;
}
export interface StandaloneSaveApi {