Skip to content

Build your personal knowledge base with Trilium Notes

feat(clipper): save a long full-page screenshot as several images

Elian Dorana45b4b1ffeda
diff --git a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html
index 4713fa6508..8f1577f8e5 100644
--- a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html
+++ b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html
@@ -100,8 +100,10 @@
once, at the top of the image, and headings that stick while scrolling
appear where they are in the page. Other elements that stay in place while
scrolling (such as cookie banners and chat buttons) are left out, while
- a background that fills the whole window repeats on every screen. A very
- long page is cut off.</p>
+ a background that fills the whole window repeats on every screen. A long
+ page is saved as several images, one below the other in the note. A page
+ longer than 60,000 pixels (30,000 on a display scaled to 200%) is cut off
+ there, and the message after saving says so.</p>
</li>
<li>
<p>If the current page was already clipped, a notice at the top offers to
diff --git a/apps/web-clipper/entrypoints/background/index.spec.ts b/apps/web-clipper/entrypoints/background/index.spec.ts
index 2ffd30c36c..aa3e72941c 100644
--- a/apps/web-clipper/entrypoints/background/index.spec.ts
+++ b/apps/web-clipper/entrypoints/background/index.spec.ts
@@ -17,10 +17,19 @@ const { facade, TriliumError } = vi.hoisted(() => ({
}
}));
-const stitchScreenshots = vi.hoisted(() => vi.fn());
+const backgroundStitcher = vi.hoisted(() => ({ created: vi.fn(), add: vi.fn(), finish: vi.fn() }));
vi.mock("@/screenshot", async (importOriginal) => ({
...await importOriginal<typeof import("@/screenshot")>(),
- stitchScreenshots
+ ScreenshotStitcher: class {
+
+ add = backgroundStitcher.add;
+ finish = backgroundStitcher.finish;
+
+ constructor(request: unknown) {
+ backgroundStitcher.created(request);
+ }
+
+ }
}));
vi.mock("./trilium_server_facade", () => ({
@@ -500,17 +509,29 @@ describe("background", () => {
describe("full page screenshots", () => {
const captureVisibleTab = vi.fn(async () => PNG);
let tabActive: boolean;
+ /** What the offscreen document was asked to stitch, and the images it answers with. */
+ let offscreen: { request?: unknown, slices: { y: number }[], parts: string[] };
beforeEach(() => {
vi.useFakeTimers({ toFake: [ "setTimeout", "Date" ] });
tabActive = true;
+ offscreen = { slices: [], parts: [ CROPPED_PNG ] };
Object.assign(fakeBrowser.tabs, {
get: async () => ({ active: tabActive }),
captureVisibleTab
});
- runtimeSendMessage.mockImplementation(async (message) => (
- (message as { type?: string }).type === "STITCH_IMAGES" ? { dataUrl: CROPPED_PNG } : PNG
- ));
+ runtimeSendMessage.mockImplementation(async (message) => {
+ const { type, request, slice, index = 0 } = message as {
+ type?: string, request?: unknown, slice: { y: number }, index?: number
+ };
+ switch (type) {
+ case "STITCH_BEGIN": offscreen.request = request; return {};
+ case "STITCH_ADD": offscreen.slices.push(slice); return {};
+ case "STITCH_FINISH": return { count: offscreen.parts.length };
+ case "STITCH_PART": return { dataUrl: offscreen.parts[index] };
+ default: return PNG;
+ }
+ });
});
afterEach(() => {
@@ -536,13 +557,9 @@ describe("background", () => {
.map((message) => (message.name === "trilium-scroll-to" ? message.y : message.step));
}
- function stitchRequest() {
- const call = runtimeSendMessage.mock.calls.find(([ message ]) =>
- (message as { type?: string }).type === "STITCH_IMAGES");
- return (call?.[0] as { request?: unknown } | undefined)?.request;
- }
+ const sliceTops = () => offscreen.slices.map((slice) => slice.y);
- it("captures the page a viewport at a time, then stitches the captures offscreen", async () => {
+ it("captures the page a viewport at a time, stitching each capture offscreen", async () => {
simulatePage({ startY: 300, viewportHeight: 500, pageHeight: 1200 });
const captureTimes: number[] = [];
captureVisibleTab.mockImplementation(async () => {
@@ -553,31 +570,45 @@ describe("background", () => {
await run(onCommand("saveFullPageScreenshot"));
expect(pageSteps()).toEqual([ undefined, 0, "first", 0, "rest", 500, 1000, "restore", 300 ]);
- expect(stitchRequest()).toEqual({
- slices: [ { dataUrl: PNG, y: 0 }, { dataUrl: PNG, y: 500 }, { dataUrl: PNG, y: 700 } ],
- viewportHeight: 500,
- pageHeight: 1200
- });
+ expect(offscreen.request).toEqual({ viewportHeight: 500, pageHeight: 1200 });
+ expect(offscreen.slices).toEqual([ { dataUrl: PNG, y: 0 }, { dataUrl: PNG, y: 500 }, { dataUrl: PNG, y: 700 } ]);
const intervals = captureTimes.slice(1).map((time, index) => time - (captureTimes[index] ?? 0));
expect(intervals.every((interval) => interval >= 550)).toBe(true);
expect(lastPayload()).toMatchObject({
+ content: expect.stringMatching(/^<img src="[A-Za-z0-9]{20}">$/),
images: [ { src: "inline.png", dataUrl: CROPPED_PNG } ],
pageUrl: PAGE_URL
});
expect(await lastToast()).toMatchObject({ message: "Screenshot has been saved to Trilium." });
});
- it("stops where the page stops scrolling, and at the height a screenshot can hold", async () => {
+ it("saves a page stitched into several images as one image after another", async () => {
+ simulatePage({ viewportHeight: 500, pageHeight: 500 });
+ offscreen.parts = [ PNG, CROPPED_PNG ];
+
+ await run(sendRuntimeMessage({ name: "save-full-page-screenshot" }));
+
+ const { content, images } = lastPayload() as { content: string, images: { imageId: string, dataUrl: string }[] };
+ expect(images.map((image) => image.dataUrl)).toEqual([ PNG, CROPPED_PNG ]);
+ expect(content).toBe(images.map((image) => `<img src="${image.imageId}">`).join(""));
+ });
+
+ it("stops where the page stops scrolling, and says when a page is too long to capture in full", async () => {
simulatePage({ viewportHeight: 500, pageHeight: 5000, scrollLimit: 1000 });
await run(onContextMenuClicked({ menuItemId: "trilium-save-full-page-screenshot", pageUrl: MENU_URL }));
- expect(stitchRequest()).toMatchObject({ slices: [ { y: 0 }, { y: 500 } ] });
+ expect(sliceTops()).toEqual([ 0, 500 ]);
expect(lastPayload()).toMatchObject({ pageUrl: MENU_URL });
+ expect(await lastToast()).toMatchObject({ message: "Screenshot has been saved to Trilium." });
vi.useFakeTimers({ toFake: [ "setTimeout", "Date" ] });
- runtimeSendMessage.mockClear();
+ offscreen.slices = [];
simulatePage({ viewportHeight: 2000, pageHeight: 100000, devicePixelRatio: 4 });
await run(sendRuntimeMessage({ name: "save-full-page-screenshot" }));
- expect(stitchRequest()).toMatchObject({ slices: [ { y: 0 }, { y: 2000 }, { y: 4000 } ] });
+ expect(offscreen.request).toEqual({ viewportHeight: 2000, pageHeight: 15000 });
+ expect(sliceTops()).toEqual([ 0, 2000, 4000, 6000, 8000, 10000, 12000, 14000 ]);
+ expect(await lastToast()).toMatchObject({
+ message: "Screenshot has been saved to Trilium, but the page was too long to capture in full."
+ });
});
it("captures the page as it is laid out once its scrollbars are hidden", async () => {
@@ -585,11 +616,8 @@ describe("background", () => {
await run(onCommand("saveFullPageScreenshot"));
- expect(stitchRequest()).toEqual({
- slices: [ { dataUrl: PNG, y: 0 }, { dataUrl: PNG, y: 515 }, { dataUrl: PNG, y: 585 } ],
- viewportHeight: 515,
- pageHeight: 1100
- });
+ expect(offscreen.request).toEqual({ viewportHeight: 515, pageHeight: 1100 });
+ expect(sliceTops()).toEqual([ 0, 515, 585 ]);
});
it("stops and scrolls back when another tab is opened, even if scrolling back fails", async () => {
@@ -630,17 +658,14 @@ describe("background", () => {
it("stitches in the background page under Manifest V2", async () => {
vi.stubGlobal("__MANIFEST_VERSION__", 2);
- stitchScreenshots.mockResolvedValue(CROPPED_PNG);
+ backgroundStitcher.finish.mockReturnValue([ CROPPED_PNG ]);
simulatePage({ viewportHeight: 500, pageHeight: 500 });
await run(sendRuntimeMessage({ name: "save-full-page-screenshot" }));
expect(runtimeSendMessage).not.toHaveBeenCalled();
- expect(stitchScreenshots).toHaveBeenCalledWith({
- slices: [ { dataUrl: PNG, y: 0 } ],
- viewportHeight: 500,
- pageHeight: 500
- });
+ expect(backgroundStitcher.created).toHaveBeenCalledWith({ viewportHeight: 500, pageHeight: 500 });
+ expect(backgroundStitcher.add).toHaveBeenCalledWith({ dataUrl: PNG, y: 0 });
expect(lastPayload()).toMatchObject({ images: [ { dataUrl: CROPPED_PNG } ] });
});
});
diff --git a/apps/web-clipper/entrypoints/background/index.ts b/apps/web-clipper/entrypoints/background/index.ts
index 2daab7a180..5484b2ddd8 100644
--- a/apps/web-clipper/entrypoints/background/index.ts
+++ b/apps/web-clipper/entrypoints/background/index.ts
@@ -1,5 +1,5 @@
import {
- MAX_SCREENSHOT_HEIGHT, type ScreenshotSlice, stitchScreenshots, type StitchRequest
+ MAX_SCREENSHOT_HEIGHT, type ScreenshotSlice, ScreenshotStitcher, type StitchMessage, type StitchRequest
} from "@/screenshot";
import { extractPage, randomString, type Rect } from "@/utils";
@@ -40,6 +40,12 @@ type ScrollPosition = {
devicePixelRatio: number;
};
+/** Draws the captures of a full-page screenshot into images, wherever the canvas is. */
+interface Stitcher {
+ add(slice: ScreenshotSlice): Promise<void>;
+ finish(): Promise<string[]> | string[];
+}
+
/** Chrome allows two `captureVisibleTab()` calls a second, and fails a third. */
const CAPTURE_INTERVAL_MS = 550;
/** How long a scrolled page gets to draw what came into view before it is captured. */
@@ -73,7 +79,7 @@ export default defineBackground(() => {
}
case "saveFullPageScreenshot": {
const activeTab = await getActiveTab();
- await saveScreenshot(await takeFullPageScreenshot(), activeTab.url);
+ await saveFullPageScreenshot(activeTab.url);
break;
}
default:
@@ -156,7 +162,8 @@ export default defineBackground(() => {
/**
* Scrolls the active tab from the top, one viewport at a time, captures each, and stitches the
- * captures into one image. The tab is scrolled back to where it was afterwards.
+ * captures into images of the page from the top. The tab is scrolled back to where it was
+ * afterwards. `cutOff` says the page was longer than a screenshot holds.
*/
async function takeFullPageScreenshot() {
const { id: tabId, windowId } = await getActiveTab();
@@ -165,24 +172,17 @@ export default defineBackground(() => {
}
const start = await scrollTab(tabId);
- // Measured again once the page is prepared: without its scrollbars, it can lay out differently.
- let page = start;
- const slices: ScreenshotSlice[] = [];
- let lastCaptureAt = 0;
-
try {
- let y = 0;
- while (y < Math.min(page.pageHeight, MAX_SCREENSHOT_HEIGHT / page.devicePixelRatio)) {
- let position = await scrollTab(tabId, start.scrollX, y);
- const previous = slices.at(-1);
- // The page stopped scrolling short of the height it reported.
- if (previous && position.scrollY <= previous.y) break;
- if (!previous) {
- await setFloatingElements(tabId, "first");
- position = await scrollTab(tabId, start.scrollX, y);
- page = position;
- }
-
+ await scrollTab(tabId, start.scrollX, 0);
+ await setFloatingElements(tabId, "first");
+ // Measured once the page is prepared: without its scrollbars, it can lay out differently.
+ const page = await scrollTab(tabId, start.scrollX, 0);
+ const height = Math.min(page.pageHeight, MAX_SCREENSHOT_HEIGHT / page.devicePixelRatio);
+ const stitcher = await createStitcher({ viewportHeight: page.viewportHeight, pageHeight: height });
+
+ let position = page;
+ let lastCaptureAt = 0;
+ for (;;) {
await delay(Math.max(SCROLL_SETTLE_MS, lastCaptureAt + CAPTURE_INTERVAL_MS - Date.now()));
const tab = await browser.tabs.get(tabId);
if (!tab.active) {
@@ -191,18 +191,31 @@ export default defineBackground(() => {
lastCaptureAt = Date.now();
const dataUrl = await browser.tabs.captureVisibleTab(windowId, { format: "png" });
- slices.push({ dataUrl, y: position.scrollY });
- if (!previous) {
+ await stitcher.add({ dataUrl, y: position.scrollY });
+ if (position === page) {
await setFloatingElements(tabId, "rest");
}
- y = position.scrollY + page.viewportHeight;
+
+ const y = position.scrollY + page.viewportHeight;
+ if (y >= height) break;
+ const next = await scrollTab(tabId, start.scrollX, y);
+ // The page stopped scrolling short of the height it reported.
+ if (next.scrollY <= position.scrollY) break;
+ position = next;
}
+
+ return { parts: await stitcher.finish(), cutOff: height < page.pageHeight };
} finally {
await setFloatingElements(tabId, "restore").catch(() => undefined);
await scrollTab(tabId, start.scrollX, start.scrollY).catch(() => undefined);
}
+ }
- return await stitch({ slices, viewportHeight: page.viewportHeight, pageHeight: page.pageHeight });
+ async function saveFullPageScreenshot(pageUrl: string | null | undefined) {
+ const { parts, cutOff } = await takeFullPageScreenshot();
+ await saveScreenshot(parts, pageUrl, cutOff
+ ? "Screenshot has been saved to Trilium, but the page was too long to capture in full."
+ : undefined);
}
/** Scrolls the tab when given a position, and answers where it is scrolled to. */
@@ -224,17 +237,38 @@ export default defineBackground(() => {
await browser.tabs.sendMessage(tabId, message);
}
- async function stitch(request: StitchRequest) {
+ /**
+ * Stitches in the background page under Manifest V2, and in the offscreen document under V3,
+ * where the service worker has no canvas.
+ */
+ async function createStitcher(request: StitchRequest): Promise<Stitcher> {
if (import.meta.env.MANIFEST_VERSION !== 3) {
- return await stitchScreenshots(request);
+ return new ScreenshotStitcher(request);
}
await ensureOffscreenDocument();
- const { dataUrl, error } = await browser.runtime.sendMessage({ type: "STITCH_IMAGES", request });
- if (error) {
- throw new Error(error);
+ await askOffscreen({ type: "STITCH_BEGIN", request });
+ return {
+ add: async (slice) => {
+ await askOffscreen({ type: "STITCH_ADD", slice });
+ },
+ finish: async () => {
+ const { count } = await askOffscreen({ type: "STITCH_FINISH" });
+ const parts: string[] = [];
+ for (let index = 0; index < count; index++) {
+ parts.push((await askOffscreen({ type: "STITCH_PART", index })).dataUrl);
+ }
+ return parts;
+ }
+ };
+ }
+
+ async function askOffscreen(message: StitchMessage) {
+ const response = await browser.runtime.sendMessage(message);
+ if (response.error) {
+ throw new Error(response.error);
}
- return dataUrl as string;
+ return response;
}
async function getActiveTab() {
@@ -381,14 +415,15 @@ export default defineBackground(() => {
toast(savedMessage("Selection", failedImages), resp.noteId);
}
- async function getImagePayloadFromSrc(src: string, pageUrl: string | null | undefined) {
- const imageId = randomString(20);
+ /** A clipping of the images, one after another. */
+ async function getImagePayload(srcs: string[], pageUrl: string | null | undefined) {
+ const images = srcs.map((src) => ({ imageId: randomString(20), src }));
const activeTab = await getActiveTab();
const { description } = await readPageMetadata();
const payload = {
title: activeTab.title,
- content: `<img src="${imageId}">`,
- images: [ { imageId, src } ],
+ content: images.map(({ imageId }) => `<img src="${imageId}">`).join(""),
+ images,
pageUrl,
...(description && { labels: { pageDescription: description } })
};
@@ -417,19 +452,23 @@ export default defineBackground(() => {
});
if (!rect) return;
- await saveScreenshot(await takeCroppedScreenshot(rect, devicePixelRatio), pageUrl);
+ await saveScreenshot([ await takeCroppedScreenshot(rect, devicePixelRatio) ], pageUrl);
}
- async function saveScreenshot(src: string, pageUrl: string | null | undefined) {
- const { payload } = await getImagePayloadFromSrc(src, pageUrl);
+ async function saveScreenshot(
+ srcs: string[],
+ pageUrl: string | null | undefined,
+ message = "Screenshot has been saved to Trilium."
+ ) {
+ const { payload } = await getImagePayload(srcs, pageUrl);
const resp = await triliumServerFacade.callService("POST", "clippings", payload);
- toast("Screenshot has been saved to Trilium.", resp.noteId);
+ toast(message, resp.noteId);
}
async function saveImage(srcUrl: string, pageUrl: string | null | undefined) {
- const { payload, failedImages } = await getImagePayloadFromSrc(srcUrl, pageUrl);
+ const { payload, failedImages } = await getImagePayload([ srcUrl ], pageUrl);
const resp = await triliumServerFacade.callService("POST", "clippings", payload);
@@ -547,10 +586,10 @@ export default defineBackground(() => {
await saveCroppedScreenshot(info.pageUrl);
}
else if (info.menuItemId === 'trilium-save-whole-screenshot') {
- await saveScreenshot(await takeWholeScreenshot(), info.pageUrl);
+ await saveScreenshot([ await takeWholeScreenshot() ], info.pageUrl);
}
else if (info.menuItemId === 'trilium-save-full-page-screenshot') {
- await saveScreenshot(await takeFullPageScreenshot(), info.pageUrl);
+ await saveFullPageScreenshot(info.pageUrl);
}
else if (info.menuItemId === 'trilium-save-image') {
if (!info.srcUrl) return;
@@ -614,12 +653,12 @@ export default defineBackground(() => {
else if (request.name === 'save-whole-screenshot') {
const activeTab = await getActiveTab();
- return await saveScreenshot(await takeWholeScreenshot(), activeTab.url);
+ return await saveScreenshot([ await takeWholeScreenshot() ], activeTab.url);
}
else if (request.name === 'save-full-page-screenshot') {
const activeTab = await getActiveTab();
- return await saveScreenshot(await takeFullPageScreenshot(), activeTab.url);
+ return await saveFullPageScreenshot(activeTab.url);
}
else if (request.name === 'save-whole-page') {
return await saveWholePage(request.page);
diff --git a/apps/web-clipper/entrypoints/offscreen/index.spec.ts b/apps/web-clipper/entrypoints/offscreen/index.spec.ts
index 8e7c7c64b3..dda8bb902f 100644
--- a/apps/web-clipper/entrypoints/offscreen/index.spec.ts
+++ b/apps/web-clipper/entrypoints/offscreen/index.spec.ts
@@ -1,8 +1,19 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { fakeBrowser } from "wxt/testing/fake-browser";
-const stitchScreenshots = vi.hoisted(() => vi.fn());
-vi.mock("@/screenshot", () => ({ stitchScreenshots }));
+const stitcher = vi.hoisted(() => ({ created: vi.fn(), add: vi.fn(), finish: vi.fn() }));
+vi.mock("@/screenshot", () => ({
+ ScreenshotStitcher: class {
+
+ add = stitcher.add;
+ finish = stitcher.finish;
+
+ constructor(request: unknown) {
+ stitcher.created(request);
+ }
+
+ }
+}));
import "./index";
@@ -56,14 +67,29 @@ describe("offscreen document", () => {
await expect(sendMessage(emptyCrop)).resolves.toBe(CROPPED_PNG);
});
- it("stitches screenshots, answering with the image or why it failed", async () => {
- const request = { slices: [ { dataUrl: "a", y: 0 } ], viewportHeight: 10, pageHeight: 10 };
- stitchScreenshots.mockResolvedValueOnce(CROPPED_PNG);
- await expect(sendMessage({ type: "STITCH_IMAGES", request })).resolves.toEqual({ dataUrl: CROPPED_PNG });
- expect(stitchScreenshots).toHaveBeenCalledWith(request);
+ it("stitches a screenshot one message at a time, handing out the images one by one", async () => {
+ const request = { viewportHeight: 10, pageHeight: 30 };
+ const slice = { dataUrl: "a", y: 0 };
+ stitcher.finish.mockReturnValue([ "part-0", CROPPED_PNG ]);
+
+ await expect(sendMessage({ type: "STITCH_BEGIN", request })).resolves.toEqual({});
+ expect(stitcher.created).toHaveBeenCalledWith(request);
+ await expect(sendMessage({ type: "STITCH_ADD", slice })).resolves.toEqual({});
+ expect(stitcher.add).toHaveBeenCalledWith(slice);
+ await expect(sendMessage({ type: "STITCH_FINISH" })).resolves.toEqual({ count: 2 });
+ await expect(sendMessage({ type: "STITCH_PART", index: 1 })).resolves.toEqual({ dataUrl: CROPPED_PNG });
+ });
+
+ it("answers why stitching failed, also when no screenshot was begun", async () => {
+ await sendMessage({ type: "STITCH_BEGIN", request: { viewportHeight: 10, pageHeight: 10 } });
+ stitcher.add.mockRejectedValueOnce(new Error("Too tall"));
+ await expect(sendMessage({ type: "STITCH_ADD", slice: { dataUrl: "a", y: 0 } }))
+ .resolves.toEqual({ error: "Too tall" });
- stitchScreenshots.mockRejectedValueOnce(new Error("Too tall"));
- await expect(sendMessage({ type: "STITCH_IMAGES", request })).resolves.toEqual({ error: "Too tall" });
+ stitcher.finish.mockReturnValue([]);
+ await sendMessage({ type: "STITCH_FINISH" });
+ await expect(sendMessage({ type: "STITCH_ADD", slice: { dataUrl: "a", y: 0 } }))
+ .resolves.toEqual({ error: "No screenshot is being stitched." });
});
it("ignores other messages", async () => {
diff --git a/apps/web-clipper/entrypoints/offscreen/index.ts b/apps/web-clipper/entrypoints/offscreen/index.ts
index 6889821e21..43f2877758 100644
--- a/apps/web-clipper/entrypoints/offscreen/index.ts
+++ b/apps/web-clipper/entrypoints/offscreen/index.ts
@@ -1,20 +1,49 @@
-import { stitchScreenshots } from "@/screenshot";
+import { ScreenshotStitcher, type StitchMessage } from "@/screenshot";
+
+let stitcher: ScreenshotStitcher | undefined;
+let stitchedParts: string[] = [];
browser.runtime.onMessage.addListener((message, _sender, sendResponse) => {
if (message.type === 'CROP_IMAGE') {
cropImage(message.cropRect, message.dataUrl).then(sendResponse);
return true; // Keep channel open for async response
}
- if (message.type === 'STITCH_IMAGES') {
+ if (typeof message.type === "string" && message.type.startsWith("STITCH_")) {
// A failure answers its message, which the background script throws again.
- stitchScreenshots(message.request).then(
- (dataUrl) => sendResponse({ dataUrl }),
- (e: Error) => sendResponse({ error: e.message })
- );
+ stitch(message).then(sendResponse, (e: Error) => sendResponse({ error: e.message }));
return true;
}
});
+/**
+ * Stitches a full-page screenshot one message at a time: Chrome refuses a message over 64 MiB,
+ * which the captures of a long page, or all of its images, would exceed.
+ */
+async function stitch(message: StitchMessage) {
+ switch (message.type) {
+ case "STITCH_BEGIN":
+ stitcher = new ScreenshotStitcher(message.request);
+ stitchedParts = [];
+ return {};
+ case "STITCH_ADD":
+ await currentStitcher().add(message.slice);
+ return {};
+ case "STITCH_FINISH":
+ stitchedParts = currentStitcher().finish();
+ stitcher = undefined;
+ return { count: stitchedParts.length };
+ case "STITCH_PART":
+ return { dataUrl: stitchedParts[message.index] };
+ }
+}
+
+function currentStitcher() {
+ if (!stitcher) {
+ throw new Error("No screenshot is being stitched.");
+ }
+ return stitcher;
+}
+
function cropImage(newArea: { x: number, y: number, width: number, height: number }, dataUrl: string) {
return new Promise<string>((resolve) => {
const img = new Image();
diff --git a/apps/web-clipper/floating_elements.spec.ts b/apps/web-clipper/floating_elements.spec.ts
index 8c0c01ead3..ab50022a46 100644
--- a/apps/web-clipper/floating_elements.spec.ts
+++ b/apps/web-clipper/floating_elements.spec.ts
@@ -97,11 +97,11 @@ describe("setFloatingElements", () => {
expect(styleOf("th")).toBe("top: 10px; color: red");
});
- it("restores the page on its own when it is not asked to within a minute", () => {
+ it("restores the page on its own when it is not asked to within three minutes", () => {
vi.useFakeTimers();
setFloatingElements("first");
- vi.advanceTimersByTime(59_999);
+ vi.advanceTimersByTime(179_999);
expect(styleOf(".cookies")).toBe("display: none !important;");
vi.advanceTimersByTime(1);
diff --git a/apps/web-clipper/floating_elements.ts b/apps/web-clipper/floating_elements.ts
index 2375671854..0e8a39a5b0 100644
--- a/apps/web-clipper/floating_elements.ts
+++ b/apps/web-clipper/floating_elements.ts
@@ -11,7 +11,7 @@ export type FloatingElementsStep = "first" | "rest" | "restore";
/** A fixed element covering this much of the viewport's width and height is a backdrop and stays. */
const BACKDROP_COVERAGE = 0.9;
/** How long changes are kept when the background script does not ask to restore them. */
-const RESTORE_TIMEOUT_MS = 60_000;
+const RESTORE_TIMEOUT_MS = 180_000;
const PINNED_STICKY: [ string, string ][] = [
[ "position", "relative" ],
diff --git a/apps/web-clipper/screenshot.spec.ts b/apps/web-clipper/screenshot.spec.ts
index 13d732e2e7..e2d9cfafa3 100644
--- a/apps/web-clipper/screenshot.spec.ts
+++ b/apps/web-clipper/screenshot.spec.ts
@@ -1,56 +1,91 @@
import { afterEach, describe, expect, it, vi } from "vitest";
-import { MAX_SCREENSHOT_HEIGHT, stitchScreenshots } from "./screenshot";
+import { MAX_PART_HEIGHT, ScreenshotStitcher } from "./screenshot";
-const STITCHED = "data:image/png;base64,U1RJVENI";
-
-describe("stitchScreenshots", () => {
+describe("ScreenshotStitcher", () => {
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
it("draws each capture where it was taken, scaled to device pixels", async () => {
- const images = stubImages({ naturalWidth: 1600, naturalHeight: 1000 });
- const { canvas, drawImage } = stubCanvas();
-
- const result = await stitchScreenshots({
- slices: [ { dataUrl: "first", y: 0 }, { dataUrl: "second", y: 500 }, { dataUrl: "last", y: 700 } ],
- viewportHeight: 500,
- pageHeight: 1200
- });
-
- expect(result).toBe(STITCHED);
- expect(images.map((image) => image.src)).toEqual([ "first", "second", "last" ]);
- expect([ canvas().width, canvas().height ]).toEqual([ 1600, 2400 ]);
- expect(drawImage.mock.calls).toEqual([
- [ images[0], 0, 0 ],
- [ images[1], 0, 1000 ],
- [ images[2], 0, 1400 ]
+ stubImages({ naturalWidth: 1600, naturalHeight: 1000 });
+ const canvas = stubCanvas();
+ const stitcher = new ScreenshotStitcher({ viewportHeight: 500, pageHeight: 1200 });
+
+ for (const y of [ 0, 500, 700 ]) {
+ await stitcher.add({ dataUrl: `slice-${y}`, y });
+ }
+
+ expect(stitcher.finish()).toEqual([ "part-0" ]);
+ expect(canvas.sizes()).toEqual([ [ 1600, 2400 ] ]);
+ expect(canvas.events).toEqual([
+ "draw 0: slice-0 at 0", "draw 0: slice-500 at 1000", "draw 0: slice-700 at 1400", "encode 0"
+ ]);
+ });
+
+ it("splits a long page into images, encoding each before the next is drawn", async () => {
+ stubImages({ naturalWidth: 900, naturalHeight: 3000 });
+ const canvas = stubCanvas();
+ const stitcher = new ScreenshotStitcher({ viewportHeight: 1000, pageHeight: 6000 });
+
+ for (const y of [ 0, 1000, 2000, 3000, 4000, 5000 ]) {
+ await stitcher.add({ dataUrl: `slice-${y}`, y });
+ }
+
+ expect(stitcher.finish()).toEqual([ "part-0", "part-1", "part-2" ]);
+ expect(canvas.sizes()).toEqual([ [ 900, MAX_PART_HEIGHT ], [ 900, MAX_PART_HEIGHT ], [ 900, 1616 ] ]);
+ expect(canvas.events).toEqual([
+ "draw 0: slice-0 at 0", "draw 0: slice-1000 at 3000", "draw 0: slice-2000 at 6000",
+ "encode 0",
+ "draw 1: slice-2000 at -2192", "draw 1: slice-3000 at 808", "draw 1: slice-4000 at 3808",
+ "draw 1: slice-5000 at 6808",
+ "encode 1",
+ "draw 2: slice-5000 at -1384",
+ "encode 2"
]);
});
- it("keeps the canvas within the height a browser can draw", async () => {
- stubImages({ naturalWidth: 800, naturalHeight: 4000 });
- const { canvas } = stubCanvas();
+ it("leaves out the part of the last capture that reaches back into an image already encoded", async () => {
+ stubImages({ naturalWidth: 100, naturalHeight: 1500 });
+ const canvas = stubCanvas();
+ const stitcher = new ScreenshotStitcher({ viewportHeight: 1500, pageHeight: 9400 });
- await stitchScreenshots({ slices: [ { dataUrl: "a", y: 0 } ], viewportHeight: 1000, pageHeight: 10000 });
+ for (const y of [ 0, 1500, 3000, 4500, 6000, 7500, 7900 ]) {
+ await stitcher.add({ dataUrl: `slice-${y}`, y });
+ }
+ stitcher.finish();
+
+ expect(canvas.events.slice(-5)).toEqual([
+ "draw 0: slice-7500 at 7500", "encode 0", "draw 1: slice-7500 at -692", "draw 1: slice-7900 at -292",
+ "encode 1"
+ ]);
+ });
+
+ it("ignores what a capture shows below the height it was given", async () => {
+ stubImages({ naturalWidth: 100, naturalHeight: 1000 });
+ const canvas = stubCanvas();
+ const stitcher = new ScreenshotStitcher({ viewportHeight: 1000, pageHeight: MAX_PART_HEIGHT });
- expect(canvas().height).toBe(MAX_SCREENSHOT_HEIGHT);
+ await stitcher.add({ dataUrl: "slice", y: 7500 });
+
+ expect(stitcher.finish()).toEqual([ "part-0" ]);
+ expect(canvas.sizes()).toEqual([ [ 100, MAX_PART_HEIGHT ] ]);
});
it("fails without captures, without a canvas, or on a capture that cannot be read", async () => {
- const request = { slices: [ { dataUrl: "a", y: 0 } ], viewportHeight: 100, pageHeight: 100 };
+ const request = { viewportHeight: 100, pageHeight: 100 };
stubImages({ naturalWidth: 100, naturalHeight: 100 });
- await expect(stitchScreenshots({ ...request, slices: [] }))
- .rejects.toThrow("There is no screenshot to stitch.");
+ expect(() => new ScreenshotStitcher(request).finish()).toThrow("There is no screenshot to stitch.");
vi.spyOn(HTMLCanvasElement.prototype, "getContext").mockReturnValue(null);
- await expect(stitchScreenshots(request)).rejects.toThrow("The screenshot could not be drawn.");
+ await expect(new ScreenshotStitcher(request).add({ dataUrl: "a", y: 0 }))
+ .rejects.toThrow("The screenshot could not be drawn.");
stubImages({ naturalWidth: 100, naturalHeight: 100, fail: true });
- await expect(stitchScreenshots(request)).rejects.toThrow("A part of the screenshot could not be read.");
+ await expect(new ScreenshotStitcher(request).add({ dataUrl: "a", y: 0 }))
+ .rejects.toThrow("A part of the screenshot could not be read.");
});
});
@@ -58,7 +93,6 @@ describe("stitchScreenshots", () => {
function stubImages({ naturalWidth, naturalHeight, fail = false }: {
naturalWidth: number, naturalHeight: number, fail?: boolean
}) {
- const images: { src?: string }[] = [];
vi.stubGlobal("Image", class {
naturalWidth = naturalWidth;
@@ -66,23 +100,28 @@ function stubImages({ naturalWidth, naturalHeight, fail = false }: {
onload?: () => void;
onerror?: () => void;
- constructor() {
- images.push(this);
- }
-
set src(value: string) {
Object.defineProperty(this, "src", { value });
setTimeout(() => (fail ? this.onerror?.() : this.onload?.()));
}
});
- return images;
}
+/** Records, in order, what is drawn on which canvas and when each is encoded. */
function stubCanvas() {
- const drawImage = vi.fn();
- const getContext = vi.spyOn(HTMLCanvasElement.prototype, "getContext")
- .mockReturnValue({ drawImage } as unknown as CanvasRenderingContext2D);
- vi.spyOn(HTMLCanvasElement.prototype, "toDataURL").mockReturnValue(STITCHED);
- return { drawImage, canvas: () => getContext.mock.contexts[0] as HTMLCanvasElement };
+ const canvases: HTMLCanvasElement[] = [];
+ const events: string[] = [];
+ vi.spyOn(HTMLCanvasElement.prototype, "getContext").mockImplementation(function (this: HTMLCanvasElement) {
+ const index = canvases.push(this) - 1;
+ return {
+ drawImage: (image: { src: string }, _x: number, y: number) => events.push(`draw ${index}: ${image.src} at ${y}`)
+ } as unknown as CanvasRenderingContext2D;
+ });
+ vi.spyOn(HTMLCanvasElement.prototype, "toDataURL").mockImplementation(function (this: HTMLCanvasElement) {
+ const index = canvases.indexOf(this);
+ events.push(`encode ${index}`);
+ return `part-${index}`;
+ });
+ return { events, sizes: () => canvases.map((canvas) => [ canvas.width, canvas.height ]) };
}
diff --git a/apps/web-clipper/screenshot.ts b/apps/web-clipper/screenshot.ts
index 91d3812a54..6f6d43a6db 100644
--- a/apps/web-clipper/screenshot.ts
+++ b/apps/web-clipper/screenshot.ts
@@ -5,51 +5,93 @@ export interface ScreenshotSlice {
}
export interface StitchRequest {
- slices: ScreenshotSlice[];
/** The height of the page's viewport, in CSS pixels. */
viewportHeight: number;
- /** The height of the whole page, in CSS pixels. */
+ /** The height of the page to stitch, in CSS pixels. */
pageHeight: number;
}
+/** How the background script has the offscreen document stitch a screenshot, one step a message. */
+export type StitchMessage =
+ | { type: "STITCH_BEGIN", request: StitchRequest }
+ | { type: "STITCH_ADD", slice: ScreenshotSlice }
+ | { type: "STITCH_FINISH" }
+ | { type: "STITCH_PART", index: number };
+
/**
- * The tallest image the stitched screenshot can be, in device pixels. Browsers refuse to draw a
- * canvas much taller than this, and Chrome's limit is the lowest.
+ * The tallest image of a screenshot, in device pixels; a longer page is saved as several images.
+ * Small enough for one image, as a PNG data URL, to fit in one extension message (64 MiB in Chrome).
*/
-export const MAX_SCREENSHOT_HEIGHT = 16384;
+export const MAX_PART_HEIGHT = 8192;
/**
- * Draws the captures of a scrolled page onto one canvas, each at the position it was taken at, and
- * returns the result as a PNG data URL. The last capture overlaps the one before it when the page
- * height is not a multiple of the viewport height, and is drawn over it.
+ * The longest page a screenshot captures, in device pixels. It bounds the memory and the time a
+ * capture takes: about a second per two viewports.
*/
-export async function stitchScreenshots({ slices, viewportHeight, pageHeight }: StitchRequest) {
- const loaded = await Promise.all(slices.map(async (slice) => ({
- image: await loadImage(slice.dataUrl),
- y: slice.y
- })));
- const first = loaded[0]?.image;
- if (!first) {
- throw new Error("There is no screenshot to stitch.");
+export const MAX_SCREENSHOT_HEIGHT = 60_000;
+
+/**
+ * Draws the captures of a scrolled page, given in order from the top, into images of at most
+ * {@link MAX_PART_HEIGHT}. Each image is encoded as soon as the next one begins, so only one is
+ * held as pixels.
+ */
+export class ScreenshotStitcher {
+
+ private scale = 0;
+ private width = 0;
+ private height = 0;
+ private part: { index: number, canvas: HTMLCanvasElement, context: CanvasRenderingContext2D } | undefined;
+ private readonly parts: string[] = [];
+
+ constructor(private readonly request: StitchRequest) {}
+
+ async add({ dataUrl, y }: ScreenshotSlice) {
+ const image = await loadImage(dataUrl);
+ if (!this.scale) {
+ // The captures are in device pixels, which include the page zoom and the display's scaling.
+ this.scale = image.naturalHeight / this.request.viewportHeight;
+ this.width = image.naturalWidth;
+ this.height = Math.round(this.request.pageHeight * this.scale);
+ }
+
+ const top = Math.round(y * this.scale);
+ const bottom = Math.min(top + image.naturalHeight, this.height);
+ for (let index = Math.floor(top / MAX_PART_HEIGHT); index * MAX_PART_HEIGHT < bottom; index++) {
+ // The last capture overlaps the one before it, which already drew any part it reaches back into.
+ if (this.part && index < this.part.index) continue;
+ this.openPart(index).drawImage(image, 0, top - index * MAX_PART_HEIGHT);
+ }
}
- // The captures are in device pixels, which include the page zoom and the display's scaling.
- const scale = first.naturalHeight / viewportHeight;
+ /** Returns the images as PNG data URLs, from the top of the page. */
+ finish() {
+ if (!this.part) {
+ throw new Error("There is no screenshot to stitch.");
+ }
+ this.closePart(this.part);
+ return this.parts;
+ }
- const canvas = document.createElement("canvas");
- canvas.width = first.naturalWidth;
- canvas.height = Math.min(Math.round(pageHeight * scale), MAX_SCREENSHOT_HEIGHT);
+ private openPart(index: number) {
+ if (this.part?.index === index) return this.part.context;
+ if (this.part) this.closePart(this.part);
- const context = canvas.getContext("2d");
- if (!context) {
- throw new Error("The screenshot could not be drawn.");
+ const canvas = document.createElement("canvas");
+ canvas.width = this.width;
+ canvas.height = Math.min(MAX_PART_HEIGHT, this.height - index * MAX_PART_HEIGHT);
+ const context = canvas.getContext("2d");
+ if (!context) {
+ throw new Error("The screenshot could not be drawn.");
+ }
+ this.part = { index, canvas, context };
+ return context;
}
- for (const { image, y } of loaded) {
- context.drawImage(image, 0, Math.round(y * scale));
+ private closePart(part: { canvas: HTMLCanvasElement }) {
+ this.parts.push(part.canvas.toDataURL("image/png"));
+ this.part = undefined;
}
- return canvas.toDataURL("image/png");
}
function loadImage(src: string) {
diff --git a/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md b/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md
index 22c1fb7d1b..0fbaf4db96 100644
--- a/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md
+++ b/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md
@@ -40,7 +40,7 @@ Clicking the extension's button in the browser toolbar opens its popup:
The popup opens on the selection if there is one, otherwise on _Page_, showing the article, or the whole page when the page has no article. On a page that shows nothing to save, it opens on _Bookmark_ and says so. The keyboard shortcuts and the context menu still save right away, without a preview: saving the whole page that way saves the article, or the whole page when the page has no article.
* Pages the extension cannot access (the browser's own pages, the extension stores, and pages that were open before the extension was installed until they are reloaded) cannot be saved or captured, so the popup says so in place of the preview, and only _Tabs_ remains available there.
* Below the preview, a row of buttons offers the other actions: _Crop_ (a screenshot of an area you select), _Screenshot_ (of the visible part of the page), _Full page_ (a screenshot of the whole page) and _Tabs_ (saves the links of every tab in the current window as a list of <a class="reference-link" href="../Note%20Types/Text/Link%20Previews.md">Link Previews</a>, or of plain links when _Autogenerate link previews_ is turned off). Hovering over a button shows its full name and, if it has one, its keyboard shortcut.
-* _Full page_ scrolls through the page, takes a screenshot of each part and joins them into one image, then scrolls back to where the page was. The scrollbars are hidden while it does, so they do not appear in the image. Stay on the tab until it is saved: switching to another tab stops the screenshot. A header that stays at the top of the window appears once, at the top of the image, and headings that stick while scrolling appear where they are in the page. Other elements that stay in place while scrolling (such as cookie banners and chat buttons) are left out, while a background that fills the whole window repeats on every screen. A very long page is cut off.
+* _Full page_ scrolls through the page, takes a screenshot of each part and joins them into one image, then scrolls back to where the page was. The scrollbars are hidden while it does, so they do not appear in the image. Stay on the tab until it is saved: switching to another tab stops the screenshot. A header that stays at the top of the window appears once, at the top of the image, and headings that stick while scrolling appear where they are in the page. Other elements that stay in place while scrolling (such as cookie banners and chat buttons) are left out, while a background that fills the whole window repeats on every screen. A long page is saved as several images, one below the other in the note. A page longer than 60,000 pixels (30,000 on a display scaled to 200%) is cut off there, and the message after saving says so.
* If the current page was already clipped, a notice at the top offers to open the note in Trilium.
* If Trilium cannot be found, the popup explains why instead of showing the actions, with buttons to look for Trilium again and to open the options.
* The bottom of the popup shows whether the extension is connected to Trilium, as a colored dot (green when connected, amber when the versions are not compatible, red when Trilium was not found) next to a description of the connection, followed by the buttons for the extension's options and for this help page. A button to check the connection again appears next to them when hovering over the bar, and stays visible while Trilium cannot be reached or is not compatible.