Skip to content

YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context

eval: paste note images through the real note editor

Manas Raghuwanshi9c6459780896
diff --git a/evals/coding-agent/README.md b/evals/coding-agent/README.md
index ae9c42947..8ec1dafc1 100644
--- a/evals/coding-agent/README.md
+++ b/evals/coding-agent/README.md
@@ -1277,8 +1277,8 @@ a full browser exploit defense or proof of execution isolation.
### Remote Markdown images
`app-markdown-remote-images` renders the actual chat MarkdownBlock (both roles),
-the shared MemoizedReactMarkdown and the meeting note editor built from
-`createMeetingNoteEditorExtensions`, with a synthetic native media port.
+the shared MemoizedReactMarkdown and the meeting note editor (`NoteEditor` and
+`createMeetingNoteEditorExtensions`), with a synthetic native media port.
Sixty-four outcomes check that remote, protocol-relative, relative,
reference-style, raw `<img>` and `<picture><source srcset>` images, and remote or
network-share media, create no element that loads them and never reach the media
@@ -1286,23 +1286,26 @@ reader, while their alt text, link or code stays visible. Network-share media
includes a home-relative path that resolves to a share and local-looking strings
from which the reader would take a share path; images whose source is removed for
safety keep their alt text. In a meeting note, remote images an AI summary wrote
-or a later update brings do not load, keep their alt text and stay in the note's
-Markdown, also when the note is copied and pasted back, while embedded data:
-images still show. Local absolute, file-URL, Windows and home-relative files still
-render through the native reader. The parent fails fifty-seven outcomes and
-preserves seven; the first, markdown-only fix fails sixteen; the second fails only
-the copy outcome; the fix and current source pass all.
+or a later update brings (also in place of an embedded image) do not load, keep
+their alt text and stay in the note's Markdown, also when the note is copied
+through the editor's clipboard handling and pasted into a note, while embedded
+data: images still show and keep their alt text and size when pasted. Local
+absolute, file-URL, Windows and home-relative files still render through the
+native reader. The parent fails fifty-seven outcomes and preserves seven; the
+first, markdown-only fix fails sixteen; the second and third fail the note copy
+and later-update outcomes; the fix and current source pass all.
Run `bun test evals/coding-agent/calibrate-markdown-remote-images.test.js`.
-Controls include parent/reference/current, the first two fixes, an equivalent
+Controls include parent/reference/current, the first three fixes, an equivalent
alt-text element, a restored remote `<img>` fallback, re-allowed picture/source,
an extension-only media check, a media check that allows a second leading
separator, network shares treated as local, blanket image removal, a note editor
-that renders any source or deletes remote images from the note, unused correct
+that renders any source, deletes remote images from the note, takes over pastes
+of embedded images or reuses an image view for a different source, unused correct
source and missing-source setup failure. Webview CSP, Mermaid diagrams (jsdom
cannot lay them out; unit tests and browser checks cover the no-network frame),
-pasting into notes, native file policy and execution isolation are outside this
-evidence.
+pasting from other apps, native file policy and execution isolation are outside
+this evidence.
`ai-gateway-trusted-runner-admission` runs the actual HTTP entrypoint, authentication,
Free-plan gates, chat handler and provider adapter with synthetic external ports.
diff --git a/evals/coding-agent/calibrate-markdown-remote-images.test.js b/evals/coding-agent/calibrate-markdown-remote-images.test.js
index afe0c1b98..71ae89a67 100644
--- a/evals/coding-agent/calibrate-markdown-remote-images.test.js
+++ b/evals/coding-agent/calibrate-markdown-remote-images.test.js
@@ -13,6 +13,10 @@ const notes = `${app}/components/meeting-notes/note-editor.tsx`;
const PREVIOUS_FIX = '7ee2e6ef078200f9781a98c39347d7c8f48c7663';
// The second fix: notes blocked remote images, but a copied one pasted back as text.
const SECOND_FIX = '359ac0c2512578a972ba07c09e895725d5021b6c';
+// The third fix: a copied blocked image pasted back, but pasting part of a note
+// renamed and resized its embedded images, and a later update could keep
+// showing an old picture where a blocked image now was.
+const THIRD_FIX = 'bbf239eaa5626876e48b30e8cc04b1e4c1187c1e';
const root = mkdtempSync(join(tmpdir(), 'markdown-remote-images-calibration-')), archives = new Map();
const receipts = process.env.SCREENPIPE_EVAL_CALIBRATION_RECEIPTS;
afterAll(() => rmSync(root, {recursive: true, force: true}));
@@ -45,7 +49,9 @@ function passes(r) { expect(r.status).toBe(0); expect(r.stdout).toContain('64 pa
function fails(r) { expect(r.status).toBe(1); expect(r.stderr).toMatch(BEHAVIOR_FAILURE); expect(r.stdout+r.stderr).not.toMatch(/Unhandled|Uncaught|Failed to resolve import|Failed to load url|Cannot find module/); }
test('parent fails fifty-seven remote outcomes and preserves seven local files', () => { const r = grade('parent', item.base_ref); fails(r); expect(r.stdout).toContain('57 failed | 7 passed'); }, 120000);
test('previous markdown-only fix fails media-path, alt-text and note outcomes', () => { const r = grade('previous-fix', PREVIOUS_FIX); fails(r); expect(r.stdout).toContain('16 failed | 48 passed'); }, 120000);
-test('second fix fails only the note copy outcome', () => { const r = grade('second-fix', SECOND_FIX); fails(r); expect(r.stdout).toContain('1 failed | 63 passed'); expect(r.stderr).toContain('copied and pasted back'); }, 120000);
+const NOTE_OUTCOMES = ['copied and pasted into a note', 'arrives with a later update'];
+test('second fix fails the note copy and later-update outcomes', () => { const r = grade('second-fix', SECOND_FIX); fails(r); expect(r.stdout).toContain('2 failed | 62 passed'); for (const name of NOTE_OUTCOMES) expect(r.stderr).toContain(name); }, 120000);
+test('third fix fails the note copy and later-update outcomes', () => { const r = grade('third-fix', THIRD_FIX); fails(r); expect(r.stdout).toContain('2 failed | 62 passed'); for (const name of NOTE_OUTCOMES) expect(r.stderr).toContain(name); }, 120000);
test('historical reference passes every outcome', () => passes(grade('reference')), 120000);
test('current caller passes every outcome', () => passes(grade('current', 'HEAD')), 120000);
test('equivalent alt-text element passes', () => passes(grade('equivalent', item.oracle_ref, cwd => replace(cwd, markdown, 'return <ImageAltText alt={alt} />;', 'return alt ? <em>{alt}</em> : null;'))), 120000);
@@ -58,6 +64,8 @@ test('network-share paths treated as local fail', () => fails(grade('network-sha
test('blanket image removal fails preserved local files', () => fails(grade('no-images', item.oracle_ref, cwd => replace(cwd, markdown, ' img({ src, alt }) {\n if (src && isLocalMediaPath(src)) {', ' img({ src, alt }) {\n if (src || !src) return null;\n if (src && isLocalMediaPath(src)) {'))), 120000);
test('note editor that renders any image source fails', () => fails(grade('note-any-image', item.oracle_ref, cwd => replace(cwd, notes, 'return typeof src === "string" && src.startsWith("data:image/");', 'return typeof src === "string";'))), 120000);
test('note editor that deletes remote images from the note fails', () => fails(grade('note-drops-images', item.oracle_ref, cwd => replace(cwd, notes, ' renderHTML(props) {', ' parseHTML() {\n return [{ tag: \'img[src^="data:"]\' }];\n },\n\n renderHTML(props) {'))), 120000);
+test('note editor that takes over pastes of embedded images fails', () => fails(grade('note-paste-takeover', item.oracle_ref, cwd => replace(cwd, notes, 'htmlImageSources.every(isEmbeddedImageSource)', 'htmlImageSources.length === 0'))), 120000);
+test('note image view reused for a different source fails', () => fails(grade('note-stale-view', item.oracle_ref, cwd => replace(cwd, notes, 'node.attrs.src === props.node.attrs.src && Boolean(parentUpdate?.(node, ...rest))', 'Boolean(parentUpdate?.(node, ...rest))'))), 120000);
test('unused correct renderer cannot hide broken active caller', () => fails(grade('unused', item.oracle_ref, cwd => {
writeFileSync(join(cwd, `${app}/unused-correct-markdown.tsx`), readFileSync(join(cwd, markdown)));
writeFileSync(join(cwd, markdown), execFileSync('git', ['show', `${item.base_ref}:${markdown}`], {cwd: repo}));
diff --git a/evals/coding-agent/cases.json b/evals/coding-agent/cases.json
index 713527aff..b846fad86 100644
--- a/evals/coding-agent/cases.json
+++ b/evals/coding-agent/cases.json
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"suite": "screenpipe-app-coding-agent-regressions",
- "dataset_version": "2026-10-01-remote-images-media-notes-copy",
+ "dataset_version": "2026-10-02-remote-images-note-paste",
"suite_type": "regression",
"owner": "screenpipe engineering",
"policy": {
@@ -4078,10 +4078,10 @@
"name": "Render only local images and media from untrusted Markdown and meeting notes",
"source": {
"kind": "git_regression",
- "fix_commit": "bbf239eaa5626876e48b30e8cc04b1e4c1187c1e"
+ "fix_commit": "72c1f9685ce74bc2f696c8373cb568ca0392ae67"
},
"base_ref": "7274f29b6b1bb940332b4e912b9cfcb20d591d3d",
- "oracle_ref": "bbf239eaa5626876e48b30e8cc04b1e4c1187c1e",
+ "oracle_ref": "72c1f9685ce74bc2f696c8373cb568ca0392ae67",
"oracle_paths": [
"apps/screenpipe-app-tauri/components/markdown.tsx",
"apps/screenpipe-app-tauri/components/markdown/image-alt-text.tsx",
@@ -4109,7 +4109,7 @@
"privacy",
"preserved-behavior"
],
- "prompt": "Rendered Markdown and meeting notes make the application load remote resources the moment they render. Chat replies, summaries and AI meeting notes are built from captured outside content, so steered text can put private data in an image URL that leaks without any click. Repair the chat MarkdownBlock for both user and assistant messages, the shared MemoizedReactMarkdown renderer, and the meeting note editor built from createMeetingNoteEditorExtensions. Only local files may display as images or media in Markdown: absolute paths, file URLs and Windows drive paths for images, plus home-relative paths for media, must still be read through the supplied native media port and displayed. No other image source (http, https, protocol-relative, relative, reference-style, raw HTML img, or picture/source srcset) may produce an element that loads it; keep its alt text visible instead, including when the source is removed for safety. Media images, links and code that name a remote URL or a network-share path must never reach the native media reader, including a home-relative path that resolves to a share or a local-looking string from which the reader would take a share path; keep their alt text, link or code visible. In a meeting note, a remote image written by the AI or arriving with a later update must not load: keep its alt text visible, keep images embedded as data: URLs displaying, and keep the note's Markdown, including the remote image reference, so viewing, saving, or copying and pasting the note loses nothing. Keep the surrounding text. Keep the public component, editor extension and native port contracts compatible. No particular sanitizer, helper or internal implementation is required. This task checks actual React/Markdown and editor rendering with synthetic native ports; it does not establish webview CSP, Mermaid diagrams, pasting into notes, native file access policy or network isolation.",
+ "prompt": "Rendered Markdown and meeting notes make the application load remote resources the moment they render. Chat replies, summaries and AI meeting notes are built from captured outside content, so steered text can put private data in an image URL that leaks without any click. Repair the chat MarkdownBlock for both user and assistant messages, the shared MemoizedReactMarkdown renderer, and the meeting note editor (NoteEditor, built from createMeetingNoteEditorExtensions). Only local files may display as images or media in Markdown: absolute paths, file URLs and Windows drive paths for images, plus home-relative paths for media, must still be read through the supplied native media port and displayed. No other image source (http, https, protocol-relative, relative, reference-style, raw HTML img, or picture/source srcset) may produce an element that loads it; keep its alt text visible instead, including when the source is removed for safety. Media images, links and code that name a remote URL or a network-share path must never reach the native media reader, including a home-relative path that resolves to a share or a local-looking string from which the reader would take a share path; keep their alt text, link or code visible. In a meeting note, a remote image written by the AI or arriving with a later update, including in place of an embedded image, must not load: keep its alt text visible, keep images embedded as data: URLs displaying, and keep the note's Markdown, including the remote image reference, so viewing, saving, or copying a note and pasting it into a note loses nothing; pasted embedded images keep their alt text and size. Keep the surrounding text. Keep the public component, editor extension and native port contracts compatible. No particular sanitizer, helper or internal implementation is required. This task checks actual React/Markdown and editor rendering with synthetic native ports; it does not establish webview CSP, Mermaid diagrams, pasting from other apps, native file access policy or network isolation.",
"agent_timeout_seconds": 900,
"grader_dependency_links": [
{
diff --git a/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt b/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt
index a22055b40..8f8d822ef 100644
--- a/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt
+++ b/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt
@@ -1,12 +1,12 @@
// screenpipe — AI that knows everything you've seen, said, or heard
// https://screenpipe.com
-import { act, render, screen, waitFor } from "@testing-library/react";
-import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { act, fireEvent, render, screen, waitFor } from "@testing-library/react";
+import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
import { Editor } from "@tiptap/core";
import { MarkdownBlock } from "./markdown-block";
import { MemoizedReactMarkdown } from "@/components/markdown";
-import { createMeetingNoteEditorExtensions } from "@/components/meeting-notes/note-editor";
+import { createMeetingNoteEditorExtensions, NoteEditor } from "@/components/meeting-notes/note-editor";
const GIF = "R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7";
@@ -39,6 +39,8 @@ vi.mock("@/lib/hooks/use-timeline-store", () => ({
useTimelineStore: { getState: () => ({ setPendingNavigation: vi.fn() }) },
}));
vi.mock("@/lib/notifications/actions", () => ({ routeNotificationDeeplink: vi.fn(async () => undefined) }));
+// The note's formatting menus play no part in what the note shows.
+vi.mock("@/components/meeting-notes/editor-menus", () => ({ FormatToolbar: () => null, SlashCommandMenu: () => null }));
vi.hoisted(() => { globalThis.fetch = (() => { throw new Error("Eval network denied"); }) as typeof fetch; });
@@ -158,6 +160,17 @@ describe("meeting note editor", () => {
const EMBEDDED = `data:image/gif;base64,${GIF}`;
let editor: Editor | null = null;
afterEach(() => { editor?.destroy(); editor = null; document.body.innerHTML = ""; });
+ // jsdom has no layout; the note scrolls its caret into view after edits.
+ beforeAll(() => {
+ Range.prototype.getClientRects = () => [] as unknown as DOMRectList;
+ Range.prototype.getBoundingClientRect = () => ({ x: 0, y: 0, top: 0, left: 0, right: 0, bottom: 0, width: 0, height: 0 }) as DOMRect;
+ vi.spyOn(window, "scrollBy").mockImplementation(() => {});
+ });
+ afterAll(() => {
+ delete (Range.prototype as Partial<Range>).getClientRects;
+ delete (Range.prototype as Partial<Range>).getBoundingClientRect;
+ vi.mocked(window.scrollBy).mockRestore();
+ });
function mountNote(markdown: string) {
const element = document.createElement("div");
@@ -165,6 +178,22 @@ describe("meeting note editor", () => {
editor = new Editor({ element, extensions: createMeetingNoteEditorExtensions(""), content: markdown });
return element;
}
+ // The note editor component as the meeting view uses it.
+ async function renderNoteEditor(value: string) {
+ const saved = vi.fn();
+ const { container, rerender } = render(<NoteEditor value={value} onChange={saved} />);
+ const editable = await waitFor(() => {
+ const element = container.querySelector<HTMLElement>("[contenteditable]");
+ if (!element) throw new Error("note editor did not mount");
+ return element;
+ });
+ return {
+ container,
+ editable,
+ lastSaved: () => saved.mock.calls.at(-1)?.[0] as string | undefined,
+ update: (next: string) => rerender(<NoteEditor value={next} onChange={saved} />),
+ };
+ }
const remoteImages = (element: HTMLElement) =>
fetchingElements(element).filter((html) => !html.includes(EMBEDDED));
const markdownOf = () => (editor!.storage as unknown as { markdown: { getMarkdown(): string } }).markdown.getMarkdown();
@@ -194,33 +223,48 @@ describe("meeting note editor", () => {
expect(markdownOf()).toContain("https://example.com/raw.png?d=secret");
});
- it("keeps an image an AI summary wrote when the note is copied and pasted back", async () => {
- const element = mountNote([
+ it("keeps every image when a note is copied and pasted into a note", async () => {
+ mountNote([
"before",
"",
- '<img src="https://example.com/chart.png?d=secret" alt="sales chart" width="40" height="30" />',
+ "![sales chart](https://example.com/chart.png?d=secret)",
"",
- "after",
+ `<img src="${EMBEDDED}" alt="whiteboard" width="40" height="30" />`,
].join("\n"));
- const view = editor!.view;
- // What the editor puts on the clipboard, then a cut and a paste.
- const { dom } = view.serializeForClipboard(view.state.doc.slice(0, view.state.doc.content.size));
- editor!.commands.clearContent();
- expect(markdownOf()).not.toContain("chart.png");
- view.pasteHTML(dom.innerHTML, new Event("paste") as ClipboardEvent);
+ // A copy through the editor's own clipboard handling.
+ const clipboard = new Map<string, string>();
+ editor!.commands.selectAll();
+ const copy = new Event("copy", { bubbles: true, cancelable: true });
+ Object.defineProperty(copy, "clipboardData", {
+ value: { clearData: () => clipboard.clear(), setData: (type: string, data: string) => clipboard.set(type, data), getData: (type: string) => clipboard.get(type) ?? "" },
+ });
+ editor!.view.dom.dispatchEvent(copy);
+ expect(clipboard.get("text/html")).toBeTruthy();
+
+ // Pasted into the note editor as a browser delivers a paste.
+ const note = await renderNoteEditor("");
+ const paste = new Event("paste", { bubbles: true, cancelable: true });
+ Object.defineProperty(paste, "clipboardData", {
+ value: { files: [], items: [], types: [...clipboard.keys()], getData: (type: string) => clipboard.get(type) ?? "" },
+ });
+ fireEvent(note.editable, paste);
+ await waitFor(() => expect(note.lastSaved()).toContain("https://example.com/chart.png?d=secret"));
await settle();
- expect(element).toHaveTextContent("sales chart");
- expect(element).toHaveTextContent("after");
- expect(remoteImages(element)).toEqual([]);
- expect(markdownOf()).toContain("https://example.com/chart.png?d=secret");
+ expect(note.container).toHaveTextContent("before");
+ expect(note.container).toHaveTextContent("sales chart");
+ expect(remoteImages(note.container)).toEqual([]);
+ // The embedded image keeps its alt text and size.
+ expect(note.container.querySelector(`img[src="${EMBEDDED}"]`)).toHaveAttribute("alt", "whiteboard");
+ expect(note.lastSaved()).toContain('width="40"');
});
it("does not load a remote image that arrives with a later update", async () => {
- const element = mountNote("notes");
- editor!.commands.setContent("before\n\n![sales chart](https://example.com/late.png?d=secret)\n\nafter");
+ const note = await renderNoteEditor(`before\n\n![pasted screenshot](${EMBEDDED})\n\nafter`);
+ note.update("before\n\n![sales chart](https://example.com/late.png?d=secret)\n\nafter");
await settle();
- expect(element).toHaveTextContent("sales chart");
- expect(element).toHaveTextContent("after");
- expect(remoteImages(element)).toEqual([]);
+ expect(note.container).toHaveTextContent("sales chart");
+ expect(note.container).toHaveTextContent("after");
+ // The earlier embedded image is gone too; nothing may show in its place.
+ expect(fetchingElements(note.container)).toEqual([]);
});
});