Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
perf(clipper): read the whole page only when it is needed
Elian Doran442f2a07ca1b
diff --git a/.claude/skills/developing-web-clipper/SKILL.md b/.claude/skills/developing-web-clipper/SKILL.mdindex 2b36acef5f..22cd8bf007 100644--- a/.claude/skills/developing-web-clipper/SKILL.md+++ b/.claude/skills/developing-web-clipper/SKILL.md@@ -18,7 +18,7 @@ repository.| `entrypoints/background/index.ts` | service worker (MV3) / background page (MV2) | context menus, keyboard commands, every save, image download, toasts || `entrypoints/background/trilium_server_facade.ts` | background | finds Trilium, authenticates, `callService()` || `entrypoints/content/index.ts` | every page (`<all_urls>`) | selection, page description, crop overlay, toast UI |-| `entrypoints/readable-page/index.ts` | the tab, injected on demand | Defuddle extraction of the page, as an article and as a whole; its `main()` value is the `executeScript()` result |+| `entrypoints/readable-page/index.ts` | the tab, injected on demand | registers `triliumReadPage()`: Defuddle extraction of the page, as an article or as a whole || `entrypoints/popup/main.tsx` | the toolbar popup | Preact `Popup`: capture buttons, link-with-note form, connection status || `entrypoints/options/main.tsx` | the options page | Preact `Options`: desktop port, server login || `entrypoints/offscreen/` | MV3 only (Chrome) | crops screenshots on a canvas, since an MV3 service worker has no DOM |@@ -38,11 +38,14 @@ Everything goes through `browser.runtime.sendMessage` / `browser.tabs.sendMessag(`message`, `noteId`, `tabIds`).- **background → offscreen:** `{ type: "CROP_IMAGE" }`, the one message keyed by `type`.-The readable page is not a message: the popup and the background call `extractPage(tabId)`-(`utils.ts`), which injects `/readable-page.js` with `browser.scripting.executeScript()` (the-`scripting` permission, in both builds), so Defuddle stays out of the content script that loads-on every page. Specs mock it with `Object.assign(fakeBrowser, { scripting: { executeScript } })`,-answering `[ { result: { article, wholePage } } ]` (`ExtractedPage`).+The readable page is not a message: the popup and the background call+`extractPage(tabId, variant)` (`utils.ts`), which injects `/readable-page.js` with+`browser.scripting.executeScript()` (the `scripting` permission, in both builds), so Defuddle stays+out of the content script that loads on every page. A file injection takes no arguments, so the+script only registers `globalThis.triliumReadPage()` and a second `executeScript({ func, args })`+calls it in the same isolated world. Specs mock it with+`Object.assign(fakeBrowser, { scripting: { executeScript } })`, answering `[]` to the file and+`[ { result: page } ]` to the call, by `args[0]`.Every user action in the background runs inside `showFailures()`, which turns an exception into atoast. A new action belongs inside it too, or it fails silently. A toast cannot show on browser@@ -120,16 +123,20 @@ against older versions of the other.** Prefer changes that need nothing new frommade with `cloneNode()` does not have. `url` makes links and images absolute.- Its HTML is parsed with `DOMParser`, a document that loads no image and runs no code, before`getImages()` swaps the sources for placeholders.- - `readPage()` parses twice: the article, and the whole page (`WHOLE_PAGE_OPTIONS`:+ - A page reads two ways: `"article"`, and `"wholePage"` (`WHOLE_PAGE_OPTIONS`:`contentSelector: "body"` with scoring and the clutter selectors off, hidden-element removaland standardization kept). Defuddle drops a grid of cards with little text under each headingas "related articles" (`isCardGrid()`) and any `.footer`, which on a gallery or test page is the- content (#9611). The popup offers both (`Article` / `Whole page`); the shortcut and the menu- save the article, or the whole page when there is none.- - A page without an article comes back as `<body></body>`, which `readPage()` turns into- `article: undefined`; only a page whose whole page is empty too returns `undefined` (the- popup's bookmark mode). Defuddle's own retries relax its rules on a very short page ("possible- index page"), so a spec fixture of a card grid needs a paragraph of text to stay above that.+ content (#9611). Each reading is one parse, made only when needed: the popup and the shortcut+ read the article, and the whole page only when there is no article — or, in the popup, when+ the user switches to `Whole page` (`laterReadings` in `PagePreview`).+ - A reading that shows nothing (an article-less page comes back as `<body></body>`) is+ `undefined`; the popup stores it as `null`, keeping `undefined` for "not read yet". A page+ whose two readings both show nothing opens on Bookmark. Defuddle's own retries relax its rules+ on a very short page ("possible index page"), so a spec fixture of a card grid needs a+ paragraph of text to stay above that.+ - v8 can record a negative count for the code after `x ?? await y()`, which fails the branch+ gate with every case tested; write such a fallback as consecutive `if`s.- Use `parse()`, never `parseAsync()`: the async extractors fetch third-party APIs.- **happy-dom drops lazy images** (`data-src` behind a `data:image/svg+xml` placeholder) thatChromium and linkedom promote to `src`; test such Defuddle behavior in a real browser, notdiff --git a/apps/web-clipper/entrypoints/background/index.spec.ts b/apps/web-clipper/entrypoints/background/index.spec.tsindex b069ab1576..93f7995c88 100644--- a/apps/web-clipper/entrypoints/background/index.spec.ts+++ b/apps/web-clipper/entrypoints/background/index.spec.ts@@ -56,9 +56,9 @@ let tabs: Tab[];let tabMessageHandler: (message: { name: string }) => unknown;const tabsSendMessage = vi.fn((_tabId: number, message: { name: string }) => Promise.resolve(tabMessageHandler(message)));-/** What `readable-page.js` returns in the active tab; nothing to save unless a test says so. */-let extractedPage: unknown;-const executeScript = vi.fn(async () => [ { result: extractedPage } ]);+/** How `readable-page.js` reads the active tab, by variant; nothing to save unless a test says so. */+let pageReadings: Record<string, unknown>;+const executeScript = vi.fn(async ({ args }: { args?: [ string ] }) => args ? [ { result: pageReadings[args[0]] } ] : []);const contextMenusCreate = vi.fn();const contextMenusRemoveAll = vi.fn(async () => {});const getContexts = vi.fn(async (): Promise<unknown[]> => []);@@ -76,7 +76,7 @@ describe("background", () => {fakeBrowser.reset();tabs = [ ACTIVE_TAB ];tabMessageHandler = () => undefined;- extractedPage = undefined;+ pageReadings = {};facade.callService.mockResolvedValue({ noteId: "saved" });vi.stubGlobal("fetch", fetchMock);vi.spyOn(console, "log").mockImplementation(() => {});@@ -283,14 +283,15 @@ describe("background", () => {describe("saving the whole page", () => {it("saves the article from the shortcut, the menu and the popup", async () => {const page = { title: "Article", content: "<p>Body</p>", images: [], clipType: "page" };- extractedPage = { article: page, wholePage: { ...page, content: "<nav>Menu</nav><p>Body</p>" } };+ pageReadings = { article: page, wholePage: { ...page, content: "<nav>Menu</nav><p>Body</p>" } };await onCommand("saveWholePage");await onContextMenuClicked({ menuItemId: "trilium-save-page" });await sendRuntimeMessage({ name: "save-whole-page" });- expect(executeScript).toHaveBeenCalledTimes(3);+ expect(executeScript).toHaveBeenCalledTimes(6);expect(executeScript).toHaveBeenCalledWith({ target: { tabId: 7 }, files: [ "/readable-page.js" ] });+ expect(executeScript).not.toHaveBeenCalledWith(expect.objectContaining({ args: [ "wholePage" ] }));expect(facade.callService).toHaveBeenCalledTimes(3);expect(facade.callService).toHaveBeenLastCalledWith("POST", "notes", page);expect(await lastToast()).toMatchObject({ noteId: "saved" });@@ -303,7 +304,7 @@ describe("background", () => {it("saves the whole page from the shortcut when the page has no article", async () => {const wholePage = { title: "Gallery", content: "<h2>Card</h2>", images: [], clipType: "page" };- extractedPage = { wholePage };+ pageReadings = { wholePage };await onCommand("saveWholePage");@@ -318,7 +319,7 @@ describe("background", () => {message: "Saving to Trilium failed: Trilium found nothing to save on this page."});- extractedPage = { wholePage: { title: "Article", content: "<p>Body</p>", images: [] } };+ pageReadings = { wholePage: { title: "Article", content: "<p>Body</p>", images: [] } };tabs = [ { title: "Tab without id" } ];await onCommand("saveWholePage");@@ -330,7 +331,7 @@ describe("background", () => {message: "Saving to Trilium failed: Cannot access a chrome:// URL"});- expect(executeScript).toHaveBeenCalledTimes(2);+ expect(executeScript).toHaveBeenCalledTimes(5);expect(facade.callService).not.toHaveBeenCalled();});diff --git a/apps/web-clipper/entrypoints/background/index.ts b/apps/web-clipper/entrypoints/background/index.tsindex 29bc141612..fd2f42c541 100644--- a/apps/web-clipper/entrypoints/background/index.ts+++ b/apps/web-clipper/entrypoints/background/index.ts@@ -347,11 +347,14 @@ export default defineBackground(() => {async function readActivePage() {const activeTab = await getActiveTab();- const page = activeTab.id === undefined ? undefined : await extractPage(activeTab.id);- if (!page) {- throw new Error("Trilium found nothing to save on this page.");- }- return page.article ?? page.wholePage;+ const tabId = activeTab.id;+ if (tabId !== undefined) {+ const article = await extractPage(tabId, "article");+ if (article) return article;+ const wholePage = await extractPage(tabId, "wholePage");+ if (wholePage) return wholePage;+ }+ throw new Error("Trilium found nothing to save on this page.");}async function saveLinkWithNote(title: string, content: string, description?: string) {diff --git a/apps/web-clipper/entrypoints/popup/main.tsx b/apps/web-clipper/entrypoints/popup/main.tsxindex 909a91cc9c..57b85537c5 100644--- a/apps/web-clipper/entrypoints/popup/main.tsx+++ b/apps/web-clipper/entrypoints/popup/main.tsx@@ -24,7 +24,7 @@ import type { ComponentChildren } from "preact";import { Fragment, render } from "preact";import { useEffect, useRef, useState } from "preact/hooks";-import { type ExtractedPage, extractPage, type ReadablePage } from "@/utils";+import { extractPage, type PageVariant, type ReadablePage } from "@/utils";import type { TriliumSearchNoteStatus, TriliumSearchStatus } from "../background/trilium_server_facade";@@ -52,8 +52,11 @@ interface SelectedContent {labels?: Record<string, string>;}-/** The current page: its article and its whole page, or why there is nothing to save. */-type PageState = ExtractedPage | "unreadable" | "inaccessible";+/** The tab, and each reading of its page: `undefined` until it is read, `null` when it shows nothing. */+type PageReadings = { tabId: number } & Record<PageVariant, ReadablePage | null | undefined>;++/** The current page: its readings, or why there is nothing to save. */+type PageState = PageReadings | "unreadable" | "inaccessible";/** What the active tab offers to save. */interface TabContent {@@ -73,9 +76,6 @@ interface TabContent {*/type ClipMode = "selection" | "page" | "note";-/** Which reading of the page the `page` mode saves. */-type PageVariant = keyof ExtractedPage;-/** What the user has chosen and typed in the card, kept by `Popup` so a reconnection does not lose it. */interface Draft {mode?: ClipMode;@@ -130,6 +130,12 @@ const MODE_LABELS: Record<ClipMode, string> = { selection: "Selection", page: "Pconst VARIANT_LABELS: Record<PageVariant, string> = { article: "Article", wholePage: "Whole page" };+/** Why a reading of the page is unavailable. */+const VARIANT_UNAVAILABLE: Record<PageVariant, string> = {+ article: "Trilium found no article on this page.",+ wholePage: "Trilium found nothing to save on this page."+};+/** The save button of each mode, with the command whose shortcut it shows, or its own shortcut. */const SAVE_BUTTONS: Record<ClipMode, { icon: IconDefinition, label: string, command: string, shortcut?: string }> = {selection: { icon: Highlight, label: "Save selection", command: "saveSelection" },@@ -347,20 +353,33 @@ function PagePreview({ tab, disabled, shortcuts, draft, onDraftChange }: {}) {const { mode: chosenMode, variant: chosenVariant, title, description, author, text } = draft;const textAreaRef = useRef<HTMLTextAreaElement>(null);-- const page = typeof tab?.page === "object" ? tab.page : undefined;- const variant = chosenVariant && page?.[chosenVariant] ? chosenVariant : page?.article ? "article" : "wholePage";- const pageClip = page?.[variant];+ /** The readings of the page made after the popup opened, when the user switched to them. */+ const [ laterReadings, setLaterReadings ] = useState<Partial<PageReadings>>({});++ const page = typeof tab?.page === "object" ? { ...tab.page, ...laterReadings } : undefined;+ const variant = chosenVariant && page?.[chosenVariant] !== null ? chosenVariant : page?.article ? "article" : "wholePage";+ const pageClip = page?.[variant] ?? undefined;+ /** Either reading: both carry the same title and labels. */+ const pageReading = page?.article ?? page?.wholePage ?? undefined;const selection = tab?.selection ?? undefined;- const modes = tab ? availableModes(pageClip, selection) : [];+ const modes = tab ? availableModes(!!page, selection) : [];const mode = chosenMode && modes.includes(chosenMode) ? chosenMode : modes[0];const clip = mode === "selection" ? selection : mode === "page" ? pageClip : undefined;- const shownTitle = title ?? (pageClip ?? selection)?.title ?? tab?.title ?? "";+ const shownTitle = title ?? (pageReading ?? selection)?.title ?? tab?.title ?? "";const shownDescription = description ?? tab?.description ?? "";const savedDescription = shownDescription.replace(/\s+/g, " ").trim();- const shownAuthor = author ?? pageClip?.labels.pageAuthor ?? "";+ const shownAuthor = author ?? pageReading?.labels.pageAuthor ?? "";const savedAuthor = shownAuthor.replace(/\s+/g, " ").trim();+ const tabId = page?.tabId;+ const unreadVariant = mode === "page" && page?.[variant] === undefined ? variant : undefined;+ useEffect(() => {+ if (tabId === undefined || !unreadVariant) return;+ void readVariant(tabId, unreadVariant).then((reading) => {+ setLaterReadings((readings) => ({ ...readings, [unreadVariant]: reading }));+ });+ }, [ tabId, unreadVariant ]);+useEffect(() => {if (mode === "note") textAreaRef.current?.focus();}, [ mode ]);@@ -404,7 +423,7 @@ function PagePreview({ tab, disabled, shortcuts, draft, onDraftChange }: {}const pageUrl = mode === "note" ? tab?.url : clip?.pageUrl;- const pageLabels = mode === "page" ? pageClip?.labels : undefined;+ const pageLabels = mode === "page" ? pageReading?.labels : undefined;const published = pageLabels?.publishedDate;const modified = pageLabels?.modifiedDate;const button = SAVE_BUTTONS[mode ?? "page"];@@ -521,8 +540,8 @@ function PagePreview({ tab, disabled, shortcuts, draft, onDraftChange }: {<buttonkey={option}aria-pressed={option === variant}- disabled={!page[option]}- title={page[option] ? undefined : "Trilium found no article on this page."}+ disabled={page[option] === null}+ title={page[option] === null ? VARIANT_UNAVAILABLE[option] : undefined}onClick={() => onDraftChange({ ...draft, variant: option })}>{VARIANT_LABELS[option]}@@ -539,12 +558,12 @@ function PagePreview({ tab, disabled, shortcuts, draft, onDraftChange }: {onLoad={(e) => fitToContent(e.currentTarget)}/>)}- {!mode && <div className="page-preview-placeholder">Reading the page…</div>}+ {(!mode || (mode === "page" && !clip)) && <div className="page-preview-placeholder">Reading the page…</div>}</div><buttonclassName="btn btn-primary primary-action"- disabled={disabled || !mode}+ disabled={disabled || !onSave}title={disabled ? DISCONNECTED_TITLE : undefined}onClick={onSave}>@@ -580,10 +599,10 @@ function fitToContent(frame: HTMLIFrameElement) {}/** The modes the page offers, the default first: the selection, then the page, then a note. */-function availableModes(page: ReadablePage | undefined, selection: SelectedContent | undefined): ClipMode[] {+function availableModes(hasPage: boolean, selection: SelectedContent | undefined): ClipMode[] {const modes: ClipMode[] = [];if (selection) modes.push("selection");- if (page) modes.push("page");+ if (hasPage) modes.push("page");modes.push("note");return modes;}@@ -651,17 +670,28 @@ async function readActiveTab(): Promise<TabContent> {}/**- * `extractPage()` answers `undefined` when the page shows nothing, and fails where the extension- * cannot run scripts.+ * Reads the article, and the whole page only when there is no article: the user can ask for it+ * later. `extractPage()` fails where the extension cannot run scripts.*/async function readPage(tabId: number): Promise<PageState> {try {- return await extractPage(tabId) ?? "unreadable";+ const article = await extractPage(tabId, "article") ?? null;+ const wholePage = article ? undefined : await extractPage(tabId, "wholePage") ?? null;+ return article || wholePage ? { tabId, article, wholePage } : "unreadable";} catch {return "inaccessible";}}+/** The page as `variant` reads it, or `null` when that reading shows nothing or fails. */+async function readVariant(tabId: number, variant: PageVariant) {+ try {+ return await extractPage(tabId, variant) ?? null;+ } catch {+ return null;+ }+}+/*** The selection on the page, or `null` when there is none. A click on the page leaves a collapsed* selection, which the content script still answers with, empty.diff --git a/apps/web-clipper/entrypoints/popup/popup.spec.tsx b/apps/web-clipper/entrypoints/popup/popup.spec.tsxindex 43cf4e6cba..51f9e7bde8 100644--- a/apps/web-clipper/entrypoints/popup/popup.spec.tsx+++ b/apps/web-clipper/entrypoints/popup/popup.spec.tsx@@ -28,8 +28,10 @@ const PAGE = {labels: { pageAuthor: "Jane Doe", publishedDate: "2024-05-01", modifiedDate: "2024-06-12" }};const WHOLE_PAGE = { ...PAGE, content: `<nav>Menu</nav><p>Body</p><img src="i1">` };-/** What `readable-page.js` returns in the tab, as `browser.scripting.executeScript()` answers. */-const executeScript = vi.fn(async (): Promise<unknown> => extracted(PAGE));+/** How `readable-page.js` reads the tab, by variant, as `browser.scripting.executeScript()` answers. */+let pageReadings: Record<string, unknown> = {};+const executeScript = vi.fn(async ({ args }: { args?: [ string ] }): Promise<unknown> =>+ args ? [ { result: pageReadings[args[0]] } ] : []);const tabsQuery = vi.fn(async (): Promise<{ id?: number, title?: string, url?: string }[]> => [ { id: 7 } ]);const SELECTION = {title: "Page title",@@ -56,6 +58,7 @@ describe("popup", () => {fakeBrowser.reset();selection = undefined;metadata = {};+ pageReadings = { article: PAGE, wholePage: WHOLE_PAGE };Object.assign(fakeBrowser.runtime, { sendMessage, openOptionsPage });Object.assign(fakeBrowser.commands, { getAll: getAllCommands });Object.assign(fakeBrowser.tabs, { query: tabsQuery, sendMessage: tabsSendMessage });@@ -235,13 +238,30 @@ describe("popup", () => {expect(button("Save page to Trilium")?.disabled).toBe(false);});- it("switches the page between its article and the whole page, and keeps the choice", async () => {+ it("reads the whole page when the user switches to it, once, and keeps the choice", async () => {expect(variants()).toEqual([ [ "Article", "true", false ], [ "Whole page", "false", false ] ]);expect(container.querySelector(".page-variant")?.getAttribute("aria-label")).toBe("How to read the page");-+ expect(wholePageReads()).toBe(0);++ let finishReading: () => void = () => {};+ executeScript+ .mockResolvedValueOnce([])+ .mockReturnValueOnce(new Promise((resolve) => {+ finishReading = () => resolve([ { result: WHOLE_PAGE } ]);+ }));await click("Whole page");expect(variants()).toEqual([ [ "Article", "false", false ], [ "Whole page", "true", false ] ]);+ expect(placeholder()).toBe("Reading the page…");+ expect(button("Save page to Trilium")?.disabled).toBe(true);++ await act(() => finishReading());+ await flush();expect(frameDocument()).toBe(previewDocument(WHOLE_PAGE));+ await click("Article");+ expect(frameDocument()).toBe(previewDocument(PAGE));+ await click("Whole page");+ expect(frameDocument()).toBe(previewDocument(WHOLE_PAGE));+ expect(wholePageReads()).toBe(1);await click("Bookmark");expect(container.querySelector(".page-variant")).toBeNull();@@ -249,6 +269,7 @@ describe("popup", () => {await receive({ name: "trilium-search-status", triliumSearch: { status: "found-desktop", port: 37840, url: "" } });await flush();await click("Page");+ await flush();expect(variants()).toEqual([ [ "Article", "false", false ], [ "Whole page", "true", false ] ]);await type(".page-title", "Gallery");@@ -259,8 +280,23 @@ describe("popup", () => {expect(frameDocument()).toBe(previewDocument(PAGE));});+ it("goes back to the article when the whole page shows nothing or cannot be read", async () => {+ pageReadings = { article: PAGE };+ await click("Whole page");+ await flush();+ expect(variants()).toEqual([ [ "Article", "true", false ], [ "Whole page", "false", true ] ]);+ expect(button("Whole page")?.title).toBe("Trilium found nothing to save on this page.");+ expect(frameDocument()).toBe(previewDocument(PAGE));++ await rerender();+ executeScript.mockRejectedValueOnce(new Error("The tab was closed."));+ await click("Whole page");+ await flush();+ expect(variants()).toEqual([ [ "Article", "true", false ], [ "Whole page", "false", true ] ]);+ });+it("opens a page without an article on the whole page", async () => {- executeScript.mockResolvedValueOnce(extracted(undefined));+ pageReadings = { wholePage: WHOLE_PAGE };await rerender();expect(modes()).toEqual([ [ "Page", "true" ], [ "Bookmark", "false" ] ]);expect(variants()).toEqual([ [ "Article", "false", true ], [ "Whole page", "true", false ] ]);@@ -284,7 +320,7 @@ describe("popup", () => {});it("leaves out the dates of a page that states none", async () => {- executeScript.mockResolvedValueOnce(extracted({ ...PAGE, labels: {} }));+ pageReadings = { article: { ...PAGE, labels: {} }, wholePage: WHOLE_PAGE };await rerender();expect(container.querySelector(".page-meta")?.textContent).toBe("example.com");expect(container.querySelectorAll(".page-fields > .page-field")).toHaveLength(2);@@ -302,13 +338,13 @@ describe("popup", () => {expect(cardParts()).toEqual([ "page-body", "btn btn-primary primary-action" ]);expect(button("Save page to Trilium")?.disabled).toBe(true);- resolvePage(extracted(PAGE));+ resolvePage([]);await flush();expect(container.querySelector(".page-body iframe")).not.toBeNull();});it("opens a page with nothing to save on a bookmark, and keeps every other action", async () => {- executeScript.mockResolvedValueOnce([ { result: null } ]);+ pageReadings = {};await rerender();expect(container.querySelector(".no-items")).toBeNull();expect(container.querySelector(".clip-mode")).toBeNull();@@ -334,7 +370,7 @@ describe("popup", () => {tabsQuery.mockResolvedValueOnce([ {} ]);await rerender();expect(unavailable()).toContain("The extension cannot access this page.");- expect(executeScript).toHaveBeenCalledTimes(2);+ expect(executeScript).toHaveBeenCalledTimes(3);expect(tabsSendMessage).toHaveBeenCalledTimes(4);});@@ -365,7 +401,7 @@ describe("popup", () => {it("saves the selection of a page with nothing else to save, and ignores an empty one", async () => {selection = SELECTION;- executeScript.mockResolvedValueOnce([ { result: null } ]);+ pageReadings = {};await rerender();expect(container.querySelector(".no-items")).toBeNull();expect(modes()).toEqual([ [ "Selection", "true" ], [ "Bookmark", "false" ] ]);@@ -375,6 +411,7 @@ describe("popup", () => {expect(sendMessage).toHaveBeenLastCalledWith({ name: "save-selection", selection: SELECTION });selection = { ...SELECTION, content: " <p> </p>", images: [] };+ pageReadings = { article: PAGE, wholePage: WHOLE_PAGE };await rerender();expect(modes()).toEqual([ [ "Page", "true" ], [ "Bookmark", "false" ] ]);expect(frameDocument()).toBe(previewDocument(PAGE));@@ -531,7 +568,7 @@ describe("popup", () => {expect(author()).toBeNull();selection = undefined;- executeScript.mockResolvedValueOnce(extracted({ ...PAGE, labels: {} }));+ pageReadings = { article: { ...PAGE, labels: {} }, wholePage: WHOLE_PAGE };await rerender();expect(author()?.value).toBe("");await type(".page-author", "Written by hand");@@ -838,16 +875,17 @@ function modes() {.map((option) => [ option.textContent, option.getAttribute("aria-pressed") ]);}+/** How many times the popup read the whole page. */+function wholePageReads() {+ return executeScript.mock.calls.filter(([ injection ]) => injection.args?.[0] === "wholePage").length;+}+/** The page variant switch's options, each with whether it is the current one and whether it is disabled. */function variants() {return [ ...container.querySelectorAll<HTMLButtonElement>(".page-body > .page-variant button") ].map((option) => [ option.textContent, option.getAttribute("aria-pressed"), option.disabled ]);}-/** What `executeScript()` answers for a page with `article`, whose whole page is `WHOLE_PAGE`. */-function extracted(article: object | undefined) {- return [ { result: { article, wholePage: WHOLE_PAGE } } ];-}function frameDocument() {const frame = container.querySelector(".page-body iframe.page-content");diff --git a/apps/web-clipper/entrypoints/readable-page/index.spec.ts b/apps/web-clipper/entrypoints/readable-page/index.spec.tsindex 0e5856654c..558bb4f193 100644--- a/apps/web-clipper/entrypoints/readable-page/index.spec.ts+++ b/apps/web-clipper/entrypoints/readable-page/index.spec.ts@@ -1,7 +1,7 @@import type { Window as HappyDOMWindow } from "happy-dom";import { beforeEach, describe, expect, it } from "vitest";-import type { ExtractedPage } from "@/utils";+import type { PageVariant } from "@/utils";import readablePage from "./index";@@ -37,7 +37,7 @@ describe("readable page", () => {</article>`;const pageBefore = document.documentElement.outerHTML;- const page = readPage()?.article;+ const page = readPage("article");expect(page).toMatchObject({title: "Article title",@@ -74,12 +74,12 @@ describe("readable page", () => {document.head.innerHTML = `<title>Post title</title>${structuredData("2025-12-24T08:00:00Z")}<meta property="article:published_time" content="">`;document.body.innerHTML = article;- expect(readPage()?.article?.labels).toEqual({ publishedDate: "2025-12-24" });+ expect(readPage("article")?.labels).toEqual({ publishedDate: "2025-12-24" });document.head.innerHTML = `<title>Post title</title>${structuredData("next week")}<meta property="article:modified_time" content="not a date">`;document.body.innerHTML = article;- expect(readPage()?.article?.labels).toEqual({});+ expect(readPage("article")?.labels).toEqual({});});it("keeps the day the page states, whatever its offset from UTC", () => {@@ -88,25 +88,25 @@ describe("readable page", () => {<meta property="article:published_time" content="2022-02-09T00:30:00+01:00"><meta property="article:modified_time" content="2022-02-10T23:30:00-05:00">`;document.body.innerHTML = `<article><p>${PARAGRAPH.repeat(5)}</p></article>`;- expect(readPage()?.article?.labels).toEqual({ publishedDate: "2022-02-09", modifiedDate: "2022-02-10" });+ expect(readPage("article")?.labels).toEqual({ publishedDate: "2022-02-09", modifiedDate: "2022-02-10" });document.head.innerHTML = "<title>Post title</title>";document.body.innerHTML = `<article><p>${PARAGRAPH.repeat(5)}</p><p>Published <time>February 8, 2022</time></p></article>`;- expect(readPage()?.article?.labels).toEqual({});+ expect(readPage("article")?.labels).toEqual({});});it("keeps an article that is only a picture, and the whole page of a page without one", () => {document.body.innerHTML = `<article><img src="photo.jpg" width="800" height="600"></article>`;- expect(readPage()?.article?.images.map(({ src }) => src)).toEqual([ "https://example.com/blog/photo.jpg" ]);+ expect(readPage("article")?.images.map(({ src }) => src)).toEqual([ "https://example.com/blog/photo.jpg" ]);document.body.innerHTML = "<nav><a href=\"/\">Home</a> <a href=\"/archive\">Archive</a></nav>";- const page = readPage();- expect(page?.article).toBeUndefined();- expect(page?.wholePage.content).toContain("href=\"https://example.com/archive\"");+ expect(readPage("article")).toBeUndefined();+ expect(readPage("wholePage")?.content).toContain("href=\"https://example.com/archive\"");document.body.innerHTML = "<div> </div>";- expect(readPage()).toBeUndefined();+ expect(readPage("article")).toBeUndefined();+ expect(readPage("wholePage")).toBeUndefined();});it("keeps every card of a card grid in the whole page, without hidden elements", () => {@@ -132,7 +132,8 @@ describe("readable page", () => {<div style="display: none">Cookie banner</div><script>window.tracked = true;</script>`;- const { article, wholePage } = readPage() ?? {};+ const article = readPage("article");+ const wholePage = readPage("wholePage");expect(article?.images.length).toBeLessThan(4);expect(article?.content).not.toContain("What to check");@@ -151,6 +152,7 @@ describe("readable page", () => {});});-function readPage() {- return readablePage.main() as ExtractedPage | undefined;+function readPage(variant: PageVariant) {+ readablePage.main();+ return globalThis.triliumReadPage(variant);}diff --git a/apps/web-clipper/entrypoints/readable-page/index.ts b/apps/web-clipper/entrypoints/readable-page/index.tsindex 4a97d27d74..15bdd39db3 100644--- a/apps/web-clipper/entrypoints/readable-page/index.ts+++ b/apps/web-clipper/entrypoints/readable-page/index.ts@@ -1,13 +1,15 @@import Defuddle from "defuddle";-import { descriptionLabels, type ExtractedPage, getImages, getPageLocationOrigin, type ReadablePage } from "@/utils";+import { descriptionLabels, getImages, getPageLocationOrigin, type PageVariant, type ReadablePage } from "@/utils";/**- * Reads the page with Defuddle, as an article and as a whole. `extractPage()` injects it into the- * tab only when a page is read, so Defuddle does not load with the content script on every page. The- * value `main()` returns is the result of `executeScript()`.+ * Registers `triliumReadPage()` in the tab, which reads the page with Defuddle, as an article or as a+ * whole. `extractPage()` injects it into the tab only when a page is read, so Defuddle does not load+ * with the content script on every page, then calls it with the reading it needs.*/-export default defineUnlistedScript(readPage);+export default defineUnlistedScript(() => {+ globalThis.triliumReadPage = readPage;+});/*** Keeps all of `<body>`: Defuddle still removes hidden elements, makes links absolute and converts@@ -22,35 +24,28 @@ const WHOLE_PAGE_OPTIONS = {removeSmallImages: false};-function readPage(): ExtractedPage | undefined {+/** The page as `variant` reads it, or `undefined` when that reading shows nothing. */+function readPage(variant: PageVariant): ReadablePage | undefined {// `parse()` works on a copy of the document and reads styles and image sizes from the page,// so it is given the live document.- const article = new Defuddle(document, { url: location.href }).parse();- const wholePage = new Defuddle(document, { url: location.href, ...WHOLE_PAGE_OPTIONS }).parse();+ const options = variant === "wholePage" ? WHOLE_PAGE_OPTIONS : {};+ const result = new Defuddle(document, { url: location.href, ...options }).parse();- const page = {- title: article.title,- pageUrl: getPageLocationOrigin() + location.pathname + location.search,- clipType: "page" as const,- labels: pageLabels(article)- };- const whole = withContent(page, wholePage.content);- if (!whole) {- return undefined;- }- return { article: withContent(page, article.content), wholePage: whole };-}--/** The clip of `html`, its images replaced by placeholders, or `undefined` when it shows nothing. */-function withContent(page: Omit<ReadablePage, "content" | "images">, html: string): ReadablePage | undefined {// A document of its own loads none of the page's images and runs none of its code.- const body = new DOMParser().parseFromString(html, "text/html").body;+ const body = new DOMParser().parseFromString(result.content, "text/html").body;if (!body.textContent.trim() && !body.querySelector("img")) {return undefined;}const images = getImages(body);- return { ...page, content: body.innerHTML, images };+ return {+ title: result.title,+ content: body.innerHTML,+ images,+ pageUrl: getPageLocationOrigin() + location.pathname + location.search,+ clipType: "page",+ labels: pageLabels(result)+ };}function pageLabels(article: { author: string, published: string }) {diff --git a/apps/web-clipper/utils.spec.ts b/apps/web-clipper/utils.spec.tsindex 882e869459..3842c796e3 100644--- a/apps/web-clipper/utils.spec.ts+++ b/apps/web-clipper/utils.spec.ts@@ -44,20 +44,29 @@ describe("createLink", () => {describe("extractPage", () => {afterEach(() => {Reflect.deleteProperty(fakeBrowser, "scripting");+ Reflect.deleteProperty(globalThis, "triliumReadPage");});- it("runs the readable-page script in the tab and returns its result", async () => {+ it("injects the readable-page script, then reads the page as asked", async () => {const page = { title: "Article", content: "<p>Body</p>" };- const executeScript = vi.fn()- .mockResolvedValueOnce([ { result: page } ])- .mockResolvedValueOnce([ { result: null } ])- .mockResolvedValueOnce([]);+ const readPage = vi.fn((variant: string) => variant === "article" ? page : undefined);+ const executeScript = vi.fn(async ({ func, args }: { func?: (variant: string) => unknown, args?: [ string ] }) => {+ if (!func || !args) return [];+ Object.assign(globalThis, { triliumReadPage: readPage });+ return [ { result: func(...args) } ];+ });Object.assign(fakeBrowser, { scripting: { executeScript } });- expect(await extractPage(7)).toBe(page);- expect(executeScript).toHaveBeenCalledWith({ target: { tabId: 7 }, files: [ "/readable-page.js" ] });- expect(await extractPage(7)).toBeUndefined();- expect(await extractPage(7)).toBeUndefined();+ expect(await extractPage(7, "article")).toBe(page);+ expect(executeScript.mock.calls.map(([ injection ]) => injection)).toEqual([+ { target: { tabId: 7 }, files: [ "/readable-page.js" ] },+ { target: { tabId: 7 }, func: expect.any(Function), args: [ "article" ] }+ ]);+ expect(await extractPage(7, "wholePage")).toBeUndefined();+ expect(readPage.mock.calls).toEqual([ [ "article" ], [ "wholePage" ] ]);++ executeScript.mockResolvedValue([]);+ expect(await extractPage(7, "article")).toBeUndefined();});});diff --git a/apps/web-clipper/utils.ts b/apps/web-clipper/utils.tsindex d07f29b848..a0ad71d661 100644--- a/apps/web-clipper/utils.ts+++ b/apps/web-clipper/utils.ts@@ -13,22 +13,27 @@ export interface ReadablePage {labels: Record<string, string>;}-/** The two ways to clip a page: its article, when Defuddle finds one, and all of it. */-export interface ExtractedPage {- article?: ReadablePage;- wholePage: ReadablePage;+/** The two ways to read a page: its article, as Defuddle finds it, and all of it. */+export type PageVariant = "article" | "wholePage";++declare global {+ /** Registered in the tab's isolated world by `entrypoints/readable-page/index.ts`. */+ var triliumReadPage: (variant: PageVariant) => ReadablePage | undefined;}/**- * Runs `entrypoints/readable-page/index.ts` in the tab and returns the page it read, or `undefined`- * when the page shows nothing. Throws where the extension cannot run scripts, such as browser pages.+ * Injects `entrypoints/readable-page/index.ts` into the tab and reads the page as `variant`, or+ * returns `undefined` when that reading shows nothing. Throws where the extension cannot run+ * scripts, such as browser pages.*/-export async function extractPage(tabId: number): Promise<ExtractedPage | undefined> {+export async function extractPage(tabId: number, variant: PageVariant): Promise<ReadablePage | undefined> {+ await browser.scripting.executeScript({ target: { tabId }, files: [ "/readable-page.js" ] });const [ injection ] = await browser.scripting.executeScript({target: { tabId },- files: [ "/readable-page.js" ]+ func: (variant: PageVariant) => globalThis.triliumReadPage(variant),+ args: [ variant ]});- return (injection?.result ?? undefined) as ExtractedPage | undefined;+ return (injection?.result ?? undefined) as ReadablePage | undefined;}export function randomString(len: number) {