Build your personal knowledge base with Trilium Notes
TypeScript electronelectron-appknowledge-baseknowledge-graphknowledge-managementknowledge-management-graphlocal-firstnote-managmentnote-takernote-takingnotebooknotesnotes-apppersonal-knowledge-basepersonal-wikiscriptableself-hostedself-hostingwiki
fix(web-clipper): report images that cannot be downloaded
Elian Doran3f4b1fab9cae
diff --git a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.htmlindex fb2ead793d..29ec84b990 100644--- a/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html+++ b/apps/server/src/assets/doc_notes/en/User Guide/User Guide/Installation & Setup/Web Clipper.html@@ -80,6 +80,15 @@<li>Trilium rejected the request: the message includes the reason Triliumgave.</li></ul>+<p>The extension downloads the images of a clipping itself, so that they+ are stored in Trilium together with the note. If some of them cannot be+ downloaded (for example because the website refuses the request), the clipping+ is still saved, and the notification says how many images are missing.+ Those images keep their address on the original website: Trilium tries+ to download them once more if <em>Download images automatically</em> is enabled+ in <a class="reference-link" href="#root/_help_4TIF1oA4VQRO">Options</a> → <em>Media</em>,+ and otherwise the note shows them from the website for as long as it serves+ them.</p><p>The notification is shown on the page itself, so it does not appear onpages where extensions cannot run (such as the browser's own settings pagesand the extension stores).</p>diff --git a/apps/web-clipper/entrypoints/background/index.spec.ts b/apps/web-clipper/entrypoints/background/index.spec.tsindex e0d7a3828c..82865164c8 100644--- a/apps/web-clipper/entrypoints/background/index.spec.ts+++ b/apps/web-clipper/entrypoints/background/index.spec.ts@@ -121,12 +121,10 @@ describe("background", () => {describe("saving the selection", () => {it("inlines the images, saves a clipping and shows a toast", async () => {- fetchMock.mockRejectedValueOnce(new Error("blocked"));tabMessageHandler = () => ({title: "Page",- content: "<p>Selected</p>",+ content: `<img src="i2"><img src="i3"><img src="i4">`,images: [- { imageId: "i1", src: "https://example.com/missing.png" },{ imageId: "i2", src: "https://example.com/a.png" },{ imageId: "i3", src: "data:image/jpeg;base64,/9j/4AAQ" },{ imageId: "i4", src: "data:image/;base64,AAAA" }@@ -138,9 +136,8 @@ describe("background", () => {expect(tabsSendMessage).toHaveBeenCalledWith(7, { name: "trilium-save-selection" });expect(facade.callService).toHaveBeenCalledWith("POST", "clippings", {title: "Page",- content: "<p>Selected</p>",+ content: `<img src="i2"><img src="i3"><img src="i4">`,images: [- { imageId: "i1", src: "https://example.com/missing.png" },{imageId: "i2",src: "https://example.com/a.png",@@ -158,9 +155,48 @@ describe("background", () => {}]});- expect(console.error).toHaveBeenCalledOnce();- expect(await lastToast())- .toMatchObject({ name: "toast", noteId: "saved", tabIds: null });+ expect(await lastToast()).toMatchObject({+ name: "toast",+ message: "Selection has been saved to Trilium.",+ noteId: "saved",+ tabIds: null+ });+ });++ it("links the images it cannot download to their website and says so", async () => {+ fetchMock+ .mockRejectedValueOnce(new Error("blocked"))+ .mockResolvedValueOnce(new Response("Forbidden", { status: 403 }))+ .mockResolvedValueOnce(new Response("<html>", { headers: { "Content-Type": "text/html" } }));+ tabMessageHandler = () => ({+ title: "Page",+ content: `<img src="i1"><img src="i2"><img src="i3"><img src="i4">`,+ images: [+ { imageId: "i1", src: "https://example.com/blocked.png" },+ { imageId: "i2", src: "https://example.com/forbidden.png?a=1&b=\"2\"" },+ { imageId: "i3", src: "https://example.com/page.html" },+ { imageId: "i4", src: "https://example.com/ok.png" }+ ]+ });++ await onContextMenuClicked({ menuItemId: "trilium-save-selection" });++ expect(lastPayload()).toEqual({+ title: "Page",+ content: `<img src="https://example.com/blocked.png">`+ + `<img src="https://example.com/forbidden.png?a=1&b="2"">`+ + `<img src="https://example.com/page.html"><img src="i4">`,+ images: [ {+ imageId: "i4",+ src: "https://example.com/ok.png",+ dataUrl: expect.stringMatching(PNG_DATA_URL)+ } ]+ });+ expect(console.error).toHaveBeenCalledTimes(3);+ expect(await lastToast()).toMatchObject({+ message: "Selection has been saved to Trilium, but 3 images could not be downloaded.",+ noteId: "saved"+ });});it("is bound to the keyboard shortcut and shows why saving failed", async () => {@@ -362,6 +398,21 @@ describe("background", () => {});expect(await lastToast()).toMatchObject({ noteId: "saved" });+ fetchMock.mockResolvedValueOnce(new Response("Not found", { status: 404 }));+ await onContextMenuClicked({+ menuItemId: "trilium-save-image",+ srcUrl: "https://example.com/gone.png",+ pageUrl: PAGE_URL+ });+ expect(lastPayload()).toMatchObject({+ content: `<img src="https://example.com/gone.png">`,+ images: []+ });+ expect(await lastToast()).toMatchObject({+ message: "Image has been saved to Trilium, but 1 image could not be downloaded.",+ noteId: "saved"+ });+facade.callService.mockRejectedValue(TRILIUM_FAILURE);tabsSendMessage.mockClear();await onContextMenuClicked({diff --git a/apps/web-clipper/entrypoints/background/index.ts b/apps/web-clipper/entrypoints/background/index.tsindex 58706e6a37..7b7e51f61d 100644--- a/apps/web-clipper/entrypoints/background/index.ts+++ b/apps/web-clipper/entrypoints/background/index.ts@@ -16,6 +16,9 @@ type BackgroundMessage = {name: "trilium-save-page";};+/** An image of a clipping; `content` references it by `imageId` until Trilium stores it. */+type ClippedImage = { imageId: string, src: string, dataUrl?: string };+/** `linkText` is only available on Firefox. */type ContextMenuInfo = Browser.contextMenus.OnClickData & { linkText?: string };@@ -195,61 +198,86 @@ export default defineBackground(() => {async function fetchImage(url: string) {const resp = await fetch(url);+ if (!resp.ok) {+ throw new Error(`HTTP ${resp.status}`);+ }+const blob = await resp.blob();+ if (!blob.type.startsWith("image/")) {+ throw new Error(`Not an image: '${blob.type}'`);+ }++ const dataUrl = await blob2base64(blob);+ if (!dataUrl) {+ throw new Error("The image could not be read.");+ }- return await blob2base64(blob);+ return dataUrl;}- async function postProcessImage(image: { src: string, dataUrl?: string | null }) {+ /** Inlines the image as a data URL, and returns whether it could be downloaded. */+ async function postProcessImage(image: ClippedImage) {if (image.src.startsWith("data:image/")) {image.dataUrl = image.src;const mimeSubtype = image.src.match(/data:image\/(.*?);/)?.[1];- if (!mimeSubtype) return;+ if (!mimeSubtype) return true;image.src = `inline.${mimeSubtype}`; // this should extract file type - png/jpg+ return true;}- else {- try {- image.dataUrl = await fetchImage(image.src);- } catch (e) {- console.error(`Cannot fetch image from ${image.src}`, e);- }++ try {+ image.dataUrl = await fetchImage(image.src);+ return true;+ } catch (e) {+ console.error(`Cannot fetch image from ${image.src}`, e);+ return false;}}- async function postProcessImages(resp: { images?: { src: string, dataUrl?: string }[] }) {- if (resp.images) {- for (const image of resp.images) {- await postProcessImage(image);+ /**+ * Inlines the payload's images, and returns how many could not be downloaded. Those go back to+ * their original URL in the content, which Trilium downloads itself or shows from the website.+ */+ async function postProcessImages(payload: { content?: string, images?: ClippedImage[] }) {+ if (!payload.images) return 0;++ const downloaded: ClippedImage[] = [];+ for (const image of payload.images) {+ if (await postProcessImage(image)) {+ downloaded.push(image);+ } else {+ payload.content = payload.content?.replaceAll(image.imageId, escapeAttribute(image.src));}}++ const failedCount = payload.images.length - downloaded.length;+ payload.images = downloaded;+ return failedCount;}async function saveSelection() {const payload = await requestFromPage({name: 'trilium-save-selection'});- await postProcessImages(payload);+ const failedImages = await postProcessImages(payload);const resp = await triliumServerFacade.callService('POST', 'clippings', payload);- toast("Selection has been saved to Trilium.", resp.noteId);+ toast(savedMessage("Selection", failedImages), resp.noteId);}async function getImagePayloadFromSrc(src: string, pageUrl: string | null | undefined) {- const image = {- imageId: randomString(20),- src- };-- await postProcessImage(image);-+ const imageId = randomString(20);const activeTab = await getActiveTab();-- return {+ const payload = {title: activeTab.title,- content: `<img src="${image.imageId}">`,- images: [image],+ content: `<img src="${imageId}">`,+ images: [ { imageId, src } ],pageUrl};++ const failedImages = await postProcessImages(payload);++ return { payload, failedImages };}async function saveCroppedScreenshot(pageUrl: string | null | undefined) {@@ -260,7 +288,7 @@ export default defineBackground(() => {const src = await takeCroppedScreenshot(rect, devicePixelRatio);- const payload = await getImagePayloadFromSrc(src, pageUrl);+ const { payload } = await getImagePayloadFromSrc(src, pageUrl);const resp = await triliumServerFacade.callService("POST", "clippings", payload);@@ -270,7 +298,7 @@ export default defineBackground(() => {async function saveWholeScreenshot(pageUrl: string | null | undefined) {const src = await takeWholeScreenshot();- const payload = await getImagePayloadFromSrc(src, pageUrl);+ const { payload } = await getImagePayloadFromSrc(src, pageUrl);const resp = await triliumServerFacade.callService("POST", "clippings", payload);@@ -278,21 +306,21 @@ export default defineBackground(() => {}async function saveImage(srcUrl: string, pageUrl: string | null | undefined) {- const payload = await getImagePayloadFromSrc(srcUrl, pageUrl);+ const { payload, failedImages } = await getImagePayloadFromSrc(srcUrl, pageUrl);const resp = await triliumServerFacade.callService("POST", "clippings", payload);- toast("Image has been saved to Trilium.", resp.noteId);+ toast(savedMessage("Image", failedImages), resp.noteId);}async function saveWholePage() {const payload = await requestFromPage({name: 'trilium-save-page'});- await postProcessImages(payload);+ const failedImages = await postProcessImages(payload);const resp = await triliumServerFacade.callService('POST', 'notes', payload);- toast("Page has been saved to Trilium.", resp.noteId);+ toast(savedMessage("Page", failedImages), resp.noteId);}async function saveLinkWithNote(title: string, content: string) {@@ -451,6 +479,20 @@ export default defineBackground(() => {}));});+function savedMessage(subject: string, failedImages: number) {+ if (!failedImages) return `${subject} has been saved to Trilium.`;+ const images = failedImages === 1 ? "1 image" : `${failedImages} images`;+ return `${subject} has been saved to Trilium, but ${images} could not be downloaded.`;+}++function escapeAttribute(value: string) {+ return value+ .replaceAll("&", "&")+ .replaceAll("\"", """)+ .replaceAll("<", "<")+ .replaceAll(">", ">");+}+function failureMessage(error: unknown) {if (error instanceof TriliumError) return error.message;const details = error instanceof Error ? error.message : "";diff --git a/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md b/docs/User Guide/User Guide/Installation & Setup/Web Clipper.mdindex 8b2e3dea16..8e873f79b2 100644--- a/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md+++ b/docs/User Guide/User Guide/Installation & Setup/Web Clipper.md@@ -65,6 +65,8 @@ After each clipping, a notification in the corner of the page confirms that it w* The versions of Trilium and the extension are not compatible: update the one the message names.* Trilium rejected the request: the message includes the reason Trilium gave.+The extension downloads the images of a clipping itself, so that they are stored in Trilium together with the note. If some of them cannot be downloaded (for example because the website refuses the request), the clipping is still saved, and the notification says how many images are missing. Those images keep their address on the original website: Trilium tries to download them once more if _Download images automatically_ is enabled in <a class="reference-link" href="../Basic%20Concepts%20and%20Features/UI%20Elements/Options.md">Options</a> → _Media_, and otherwise the note shows them from the website for as long as it serves them.+The notification is shown on the page itself, so it does not appear on pages where extensions cannot run (such as the browser's own settings pages and the extension stores).## Testing development versions