Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
feat(clipper): show floating panels once in a full-page screenshot
Elian Doranbc7ce046e403
diff --git a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.htmlindex 8b7067667b..991621a0d1 100644--- a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html+++ b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html@@ -66,7 +66,9 @@when the page has none, and is saved in the <code>#pageDescription</code> labelof the note. Clippings saved with the keyboard shortcuts or the contextmenu, screenshots and images included, get the description as the page- gives it. Each clipping that brings a description updates the label.</p>+ gives it. Each clipping that brings a description updates the label. The+ description, the author and the dates are shown by clicking the <em>Web clip</em> badge+ next to the title of the note (see <a class="reference-link" href="#root/_help_B5dihXRDksTk">Note badges</a>).</p><p>The popup opens on the selection if there is one, otherwise on <em>Page</em>,showing the article, or the whole page when the page has no article. Ona page that shows nothing to save, it opens on <em>Bookmark</em> and says@@ -93,9 +95,12 @@<p><em>Full page</em> scrolls through the page, takes a screenshot of eachpart and joins them into one image, then scrolls back to where the pagewas. Stay on the tab until it is saved: switching to another tab stops- the screenshot. Elements that stay in place while the page scrolls (such- as a fixed header) appear in every part of the image, and a very long page- is cut off.</p>+ the screenshot. A header that stays at the top of the window appears once,+ at the top of the image, and headings that stick while scrolling appear+ where they are in the page. Other elements that stay in place while scrolling+ (such as cookie banners and chat buttons) are left out, while a background+ that fills the whole window repeats on every screen. A very long page is+ cut off.</p></li><li><p>If the current page was already clipped, a notice at the top offers todiff --git a/apps/web-clipper/entrypoints/background/index.spec.ts b/apps/web-clipper/entrypoints/background/index.spec.tsindex 17bb122d59..d528353468 100644--- a/apps/web-clipper/entrypoints/background/index.spec.ts+++ b/apps/web-clipper/entrypoints/background/index.spec.ts@@ -525,12 +525,15 @@ describe("background", () => {vi.useRealTimers();}- /** The positions the page was asked to scroll to; `undefined` only asks where it is. */- function scrolls() {+ /**+ * What the page was asked to do, in order: the positions it was scrolled to (`undefined`+ * only asks where it is) and the steps of changing its floating elements.+ */+ function pageSteps() {return tabsSendMessage.mock.calls- .map(([ , message ]) => message as { name: string, y?: number })- .filter((message) => message.name === "trilium-scroll-to")- .map((message) => message.y);+ .map(([ , message ]) => message as { name: string, y?: number, step?: string })+ .filter((message) => [ "trilium-scroll-to", "trilium-set-floating-elements" ].includes(message.name))+ .map((message) => (message.name === "trilium-scroll-to" ? message.y : message.step));}function stitchRequest() {@@ -549,7 +552,7 @@ describe("background", () => {await run(onCommand("saveFullPageScreenshot"));- expect(scrolls()).toEqual([ undefined, 0, 500, 1000, 300 ]);+ expect(pageSteps()).toEqual([ undefined, 0, "first", "rest", 500, 1000, "restore", 300 ]);expect(stitchRequest()).toEqual({slices: [ { dataUrl: PNG, y: 0 }, { dataUrl: PNG, y: 500 }, { dataUrl: PNG, y: 700 } ],viewportHeight: 500,@@ -581,7 +584,8 @@ describe("background", () => {simulatePage({ startY: 300, viewportHeight: 500, pageHeight: 1200 });const scrollPage = tabMessageHandler;tabMessageHandler = (message) => {- if ((message as { y?: number }).y === 300) throw new Error("The tab is gone.");+ const { y, step } = message as { y?: number, step?: string };+ if (y === 300 || step === "restore") throw new Error("The tab is gone.");return scrollPage(message);};tabActive = false;@@ -589,7 +593,7 @@ describe("background", () => {await run(sendRuntimeMessage({ name: "save-full-page-screenshot" }));expect(captureVisibleTab).not.toHaveBeenCalled();- expect(scrolls()).toEqual([ undefined, 0, 300 ]);+ expect(pageSteps()).toEqual([ undefined, 0, "first", "restore", 300 ]);expect(await lastToast()).toMatchObject({message: "Saving to Trilium failed: The screenshot was stopped because another tab was opened."});diff --git a/apps/web-clipper/entrypoints/background/index.ts b/apps/web-clipper/entrypoints/background/index.tsindex 5065c08f43..6500edfd57 100644--- a/apps/web-clipper/entrypoints/background/index.ts+++ b/apps/web-clipper/entrypoints/background/index.ts@@ -17,6 +17,10 @@ type BackgroundMessage = {name: "trilium-get-rectangle-for-screenshot";} | {name: "trilium-get-page-metadata";+} | {+ /** Changes the page's fixed and sticky elements for a full-page screenshot, or restores them. */+ name: "trilium-set-floating-elements";+ step: "first" | "rest" | "restore";} | {/** Scrolls the page to `x`, `y` when given, and answers with a {@link ScrollPosition}. */name: "trilium-scroll-to";@@ -169,6 +173,9 @@ export default defineBackground(() => {const previous = slices.at(-1);// The page stopped scrolling short of the height it reported.if (previous && position.scrollY <= previous.y) break;+ if (!previous) {+ await setFloatingElements(tabId, "first");+ }await delay(Math.max(SCROLL_SETTLE_MS, lastCaptureAt + CAPTURE_INTERVAL_MS - Date.now()));const tab = await browser.tabs.get(tabId);@@ -179,9 +186,13 @@ export default defineBackground(() => {lastCaptureAt = Date.now();const dataUrl = await browser.tabs.captureVisibleTab(windowId, { format: "png" });slices.push({ dataUrl, y: position.scrollY });+ if (!previous) {+ await setFloatingElements(tabId, "rest");+ }y = position.scrollY + start.viewportHeight;}} finally {+ await setFloatingElements(tabId, "restore").catch(() => undefined);await scrollTab(tabId, start.scrollX, start.scrollY).catch(() => undefined);}@@ -198,6 +209,15 @@ export default defineBackground(() => {return position;}+ /**+ * Asks the page to change or restore its fixed and sticky elements. A content script from an+ * earlier version of the extension does not know the message, and the page stays as it is.+ */+ async function setFloatingElements(tabId: number, step: "first" | "rest" | "restore") {+ const message: BackgroundMessage = { name: "trilium-set-floating-elements", step };+ await browser.tabs.sendMessage(tabId, message);+ }+async function stitch(request: StitchRequest) {if (import.meta.env.MANIFEST_VERSION !== 3) {return await stitchScreenshots(request);diff --git a/apps/web-clipper/entrypoints/content/index.spec.ts b/apps/web-clipper/entrypoints/content/index.spec.tsindex 91ab7f6a20..0cbb7d86b5 100644--- a/apps/web-clipper/entrypoints/content/index.spec.ts+++ b/apps/web-clipper/entrypoints/content/index.spec.ts@@ -3,6 +3,8 @@ import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "viteimport { fakeBrowser } from "wxt/testing/fake-browser";vi.mock("@/lib/toast", () => ({}));+const setFloatingElements = vi.hoisted(() => vi.fn());+vi.mock("@/floating_elements.js", () => ({ setFloatingElements }));import contentScript from "./index";@@ -260,6 +262,14 @@ describe("content script", () => {expect(scrollTo).toHaveBeenCalledWith({ left: 5, top: 240, behavior: "instant" });});+ it("changes or restores the floating elements for a full-page screenshot", async () => {+ expect(await send({ name: "trilium-set-floating-elements", step: "rest" })).toBe(true);+ expect(setFloatingElements).toHaveBeenCalledWith("rest");++ expect(await send({ name: "trilium-set-floating-elements" })).toBeUndefined();+ expect(setFloatingElements).toHaveBeenCalledOnce();+ });+it("answers unknown messages with nothing", async () => {expect(await send({ name: "unknown" })).toBeUndefined();expect(console.error).toHaveBeenCalledOnce();diff --git a/apps/web-clipper/entrypoints/content/index.ts b/apps/web-clipper/entrypoints/content/index.tsindex 76766780e9..f41e1c9092 100644--- a/apps/web-clipper/entrypoints/content/index.ts+++ b/apps/web-clipper/entrypoints/content/index.ts@@ -1,3 +1,4 @@+import { type FloatingElementsStep, setFloatingElements } from "@/floating_elements.js";import {createLink, descriptionLabels, getImages, getPageDescription, getPageLocationOrigin,makeLinksAbsolute, type Rect@@ -142,7 +143,8 @@ export default defineContentScript({}async function prepareMessageResponse(message: {- name: string, noteId?: string, message?: string, tabIds?: string[], x?: number, y?: number+ name: string, noteId?: string, message?: string, tabIds?: string[], x?: number, y?: number,+ step?: FloatingElementsStep}) {console.info(`Message: ${ message.name}`);@@ -226,6 +228,10 @@ export default defineContentScript({devicePixelRatio: window.devicePixelRatio};}+ else if (message.name === "trilium-set-floating-elements" && message.step) {+ setFloatingElements(message.step);+ return true;+ }else if (message.name === "trilium-get-page-metadata") {return { description: getPageDescription() };}diff --git a/apps/web-clipper/floating_elements.spec.ts b/apps/web-clipper/floating_elements.spec.tsnew file mode 100644index 0000000000..a91b339002--- /dev/null+++ b/apps/web-clipper/floating_elements.spec.ts@@ -0,0 +1,114 @@+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";++import { setFloatingElements } from "./floating_elements";++describe("setFloatingElements", () => {+ beforeEach(() => {+ document.head.innerHTML = `<style>+ .header, .cookies, .backdrop, .gone { position: fixed; }+ .gone { display: none; }+ th { position: sticky; }+ </style>`;+ document.body.innerHTML = `+ <div class="header"></div>+ <div class="cookies" style="display: flex"></div>+ <div class="backdrop"></div>+ <div class="gone"></div>+ <table><tr><th style="top: 10px; color: red"></th></tr></table>+ <p>Text</p>`;+ place(".header", { top: 0, width: 1024, height: 60 });+ place(".cookies", { top: 700, width: 1024, height: 68 });+ place(".backdrop", { top: 0, width: 1024, height: 768 });+ });++ afterEach(() => {+ setFloatingElements("restore");+ vi.useRealTimers();+ vi.restoreAllMocks();+ });++ it("pins sticky elements, hides fixed ones below the top, then the top ones after the first capture", () => {+ setFloatingElements("first");++ expect(styleOf("th")).toBe("top: auto !important; color: red; position: relative !important; "+ + "right: auto !important; bottom: auto !important; left: auto !important;");+ expect(styleOf(".cookies")).toBe("display: none !important;");+ expect(styleOf(".header")).toBeNull();+ expect(styleOf(".backdrop")).toBeNull();+ expect(styleOf(".gone")).toBeNull();+ expect(styleOf("p")).toBeNull();++ setFloatingElements("rest");+ expect(styleOf(".header")).toBe("display: none !important;");+ expect(styleOf(".backdrop")).toBeNull();+ expect(styleOf(".gone")).toBeNull();+ });++ it("puts back every inline style as it was", () => {+ const before = [ ...document.body.querySelectorAll("*") ].map((element) => element.getAttribute("style"));++ setFloatingElements("first");+ setFloatingElements("rest");+ setFloatingElements("rest");+ setFloatingElements("restore");++ expect([ ...document.body.querySelectorAll("*") ].map((element) => element.getAttribute("style")))+ .toEqual(before);+ });++ it("restores values marked important, and keeps what the page changed in the meantime", () => {+ const heading = element("th");+ heading.setAttribute("style", "position: sticky !important; top: 10px");++ setFloatingElements("first");+ heading.style.setProperty("color", "blue");+ element(".header").style.setProperty("opacity", "0.5");+ setFloatingElements("rest");+ setFloatingElements("restore");++ expect(heading.style.getPropertyValue("position")).toBe("sticky");+ expect(heading.style.getPropertyPriority("position")).toBe("important");+ expect(heading.style.getPropertyValue("top")).toBe("10px");+ expect(heading.style.getPropertyValue("color")).toBe("blue");+ expect(styleOf(".header")).toBe("opacity: 0.5;");+ });++ it("starts from the page's own styles when the previous screenshot left changes behind", () => {+ setFloatingElements("first");+ setFloatingElements("rest");+ setFloatingElements("first");+ expect(styleOf(".header")).toBeNull();++ setFloatingElements("restore");+ expect(styleOf(".cookies")).toBe("display: flex");+ expect(styleOf("th")).toBe("top: 10px; color: red");+ });++ it("restores the page on its own when it is not asked to within a minute", () => {+ vi.useFakeTimers();+ setFloatingElements("first");++ vi.advanceTimersByTime(59_999);+ expect(styleOf(".cookies")).toBe("display: none !important;");++ vi.advanceTimersByTime(1);+ expect(styleOf(".cookies")).toBe("display: flex");+ expect(styleOf("th")).toBe("top: 10px; color: red");+ });+});++function element(selector: string) {+ const found = document.querySelector<HTMLElement>(selector);+ if (!found) throw new Error(`No ${selector} in the fixture.`);+ return found;+}++function styleOf(selector: string) {+ return element(selector).getAttribute("style");+}++/** Gives an element the box it would have in a browser; happy-dom lays nothing out. */+function place(selector: string, { top, width, height }: { top: number, width: number, height: number }) {+ vi.spyOn(element(selector), "getBoundingClientRect")+ .mockReturnValue({ top, width, height } as DOMRect);+}diff --git a/apps/web-clipper/floating_elements.ts b/apps/web-clipper/floating_elements.tsnew file mode 100644index 0000000000..31e3f3f095--- /dev/null+++ b/apps/web-clipper/floating_elements.ts@@ -0,0 +1,122 @@+/**+ * The steps of a full-page screenshot that change the page:+ * - `first`, before the first capture, pins sticky elements where they are in the page and hides+ * the fixed elements that are not at the top of the viewport;+ * - `rest`, after the first capture, hides the fixed elements at the top too, so a header appears+ * once, at the top of the screenshot;+ * - `restore` puts back every inline style the other steps changed.+ */+export type FloatingElementsStep = "first" | "rest" | "restore";++/** A fixed element covering this much of the viewport's width and height is a backdrop and stays. */+const BACKDROP_COVERAGE = 0.9;+/** How long changes are kept when the background script does not ask to restore them. */+const RESTORE_TIMEOUT_MS = 60_000;++const PINNED_STICKY: [ string, string ][] = [+ [ "position", "relative" ],+ [ "top", "auto" ],+ [ "right", "auto" ],+ [ "bottom", "auto" ],+ [ "left", "auto" ]+];+const HIDDEN: [ string, string ][] = [ [ "display", "none" ] ];++interface SavedStyle {+ /** The element's `style` attribute before any change, or null when it had none. */+ attribute: string | null;+ /** `style.cssText` before any change, which is how the browser writes `attribute` back. */+ cssText: string;+ properties: Map<string, { value: string, priority: string }>;+}++const saved = new Map<HTMLElement, SavedStyle>();+let topFixedElements: HTMLElement[] = [];+let restoreTimer: ReturnType<typeof setTimeout> | undefined;++export function setFloatingElements(step: FloatingElementsStep) {+ if (step === "first") {+ restoreFloatingElements();+ prepareFirstCapture();+ restoreTimer = setTimeout(restoreFloatingElements, RESTORE_TIMEOUT_MS);+ } else if (step === "rest") {+ for (const element of topFixedElements) {+ override(element, HIDDEN);+ }+ topFixedElements = [];+ } else {+ restoreFloatingElements();+ }+}++function prepareFirstCapture() {+ for (const element of document.body.querySelectorAll<HTMLElement>("*")) {+ const style = getComputedStyle(element);+ if (style.display === "none") continue;++ if (style.position === "sticky") {+ override(element, PINNED_STICKY);+ } else if (style.position === "fixed" && !isBackdrop(element)) {+ if (element.getBoundingClientRect().top < window.innerHeight / 2) {+ topFixedElements.push(element);+ } else {+ override(element, HIDDEN);+ }+ }+ }+}++function isBackdrop(element: HTMLElement) {+ const { width, height } = element.getBoundingClientRect();+ return width >= window.innerWidth * BACKDROP_COVERAGE+ && height >= window.innerHeight * BACKDROP_COVERAGE;+}++/**+ * Sets inline `!important` properties, first saving what the element had. Each element is changed+ * by one step only, so what is saved is always the page's own style.+ */+function override(element: HTMLElement, properties: [ string, string ][]) {+ const original: SavedStyle = {+ attribute: element.getAttribute("style"),+ cssText: element.style.cssText,+ properties: new Map()+ };+ for (const [ name, value ] of properties) {+ original.properties.set(name, {+ value: element.style.getPropertyValue(name),+ priority: element.style.getPropertyPriority(name)+ });+ element.style.setProperty(name, value, "important");+ }+ saved.set(element, original);+}++/**+ * Puts back each property that was changed, leaving the others as the page has them now. When the+ * page changed nothing else, the `style` attribute is also put back as the page wrote it, or+ * removed if it had none.+ */+function restoreFloatingElements() {+ clearTimeout(restoreTimer);+ restoreTimer = undefined;++ for (const [ element, original ] of saved) {+ for (const [ name, { value, priority } ] of original.properties) {+ if (value) {+ element.style.setProperty(name, value, priority);+ } else {+ element.style.removeProperty(name);+ }+ }+ if (element.style.cssText !== original.cssText) continue;+ if (original.attribute === null) {+ element.removeAttribute("style");+ } else {+ element.setAttribute("style", original.attribute);+ }+ }++ saved.clear();+ topFixedElements = [];+}diff --git a/apps/web-clipper/vitest.config.mts b/apps/web-clipper/vitest.config.mtsindex 7cd4ed13a9..7256c1999b 100644--- a/apps/web-clipper/vitest.config.mts+++ b/apps/web-clipper/vitest.config.mts@@ -19,7 +19,7 @@ export default defineConfig({coverage: {reportsDirectory: join(import.meta.dirname, "test-output/vitest/coverage"),provider: "v8" as const,- include: [ "entrypoints/**/*.{ts,tsx}", "utils.ts" ],+ include: [ "entrypoints/**/*.{ts,tsx}", "utils.ts", "screenshot.ts", "floating_elements.ts" ],exclude: [ "**/*.{test,spec}.{ts,tsx}", "**/*.d.ts" ],// Repo-root-relative `SF:` paths, so Codecov does not match a bare `utils.ts` or// `entrypoints/…/index.ts` to another project of the monorepo.diff --git a/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md b/docs/User Guide/User Guide/Installation & Setup/Web Clipper.mdindex 54a2498e38..0c45e63236 100644--- a/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md+++ b/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md@@ -35,12 +35,12 @@ Clicking the extension's button in the browser toolbar opens its popup:* _Page_: _Save page to Trilium_ saves the page's text and images. A smaller switch at the top of the preview chooses how much of the page to keep: _Article_ keeps only the page's main text, without its menus, sidebars and footers, and _Whole page_ keeps everything the page shows. _Whole page_ is for pages that are not an article (such as a gallery or a page of cards), where _Article_ can leave out part of the content; the preview shows what each one keeps. _Article_ is unavailable on a page where Trilium finds no article. The author the page names shows under the description, marked by the <span class="tn-icon bx bx-user"></span> icon. It can be edited or cleared before saving, or written in when the page names none, and is saved in the `#pageAuthor` label of the note. When the page states when it was published, that date shows under the author, marked by the <span class="tn-icon bx bx-calendar"></span> icon, and is saved in the `#publishedDate` label of the note. When it states when it was last modified, that date shows below, marked by the <span class="tn-icon bx bx-history"></span> icon, and is saved in the `#modifiedDate` label.* _Bookmark_, always: a text box for a note about the page replaces the preview, and _Save bookmark_ (or <kbd>Ctrl</kbd>+<kbd>Enter</kbd>) saves a link to the page with that note. Left empty, the title stays the page's title.- The description, marked by the <span class="tn-icon bx bx-align-left"></span> icon, comes from the page's `description` meta tag, or the one it gives social sites. It can be edited or cleared before saving, or written in when the page has none, and is saved in the `#pageDescription` label of the note. Clippings saved with the keyboard shortcuts or the context menu, screenshots and images included, get the description as the page gives it. Each clipping that brings a description updates the label.+ The description, marked by the <span class="tn-icon bx bx-align-left"></span> icon, comes from the page's `description` meta tag, or the one it gives social sites. It can be edited or cleared before saving, or written in when the page has none, and is saved in the `#pageDescription` label of the note. Clippings saved with the keyboard shortcuts or the context menu, screenshots and images included, get the description as the page gives it. Each clipping that brings a description updates the label. The description, the author and the dates are shown by clicking the _Web clip_ badge next to the title of the note (see <a class="reference-link" href="../Basic%20Concepts%20and%20Features/Notes/Note%20badges.md">Note badges</a>).The popup opens on the selection if there is one, otherwise on _Page_, showing the article, or the whole page when the page has no article. On a page that shows nothing to save, it opens on _Bookmark_ and says so. The keyboard shortcuts and the context menu still save right away, without a preview: saving the whole page that way saves the article, or the whole page when the page has no article.* Pages the extension cannot access (the browser's own pages, the extension stores, and pages that were open before the extension was installed until they are reloaded) cannot be saved or captured, so the popup says so in place of the preview, and only _Tabs_ remains available there.* Below the preview, a row of buttons offers the other actions: _Crop_ (a screenshot of an area you select), _Screenshot_ (of the visible part of the page), _Full page_ (a screenshot of the whole page) and _Tabs_ (saves the links of every tab in the current window as a list of <a class="reference-link" href="../Note%20Types/Text/Link%20Previews.md">Link Previews</a>, or of plain links when _Autogenerate link previews_ is turned off). Hovering over a button shows its full name and, if it has one, its keyboard shortcut.-* _Full page_ scrolls through the page, takes a screenshot of each part and joins them into one image, then scrolls back to where the page was. Stay on the tab until it is saved: switching to another tab stops the screenshot. Elements that stay in place while the page scrolls (such as a fixed header) appear in every part of the image, and a very long page is cut off.+* _Full page_ scrolls through the page, takes a screenshot of each part and joins them into one image, then scrolls back to where the page was. Stay on the tab until it is saved: switching to another tab stops the screenshot. A header that stays at the top of the window appears once, at the top of the image, and headings that stick while scrolling appear where they are in the page. Other elements that stay in place while scrolling (such as cookie banners and chat buttons) are left out, while a background that fills the whole window repeats on every screen. A very long page is cut off.* If the current page was already clipped, a notice at the top offers to open the note in Trilium.* If Trilium cannot be found, the popup explains why instead of showing the actions, with buttons to look for Trilium again and to open the options.* The bottom of the popup shows whether the extension is connected to Trilium, as a colored dot (green when connected, amber when the versions are not compatible, red when Trilium was not found) next to a description of the connection, followed by the buttons for the extension's options and for this help page. A button to check the connection again appears next to them when hovering over the bar, and stays visible while Trilium cannot be reached or is not compatible.