Skip to content

YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context

eval: cover copying a blocked note image

Manas Raghuwanshi343ee1c493bd
diff --git a/evals/coding-agent/README.md b/evals/coding-agent/README.md
index 149e9e955..ae9c42947 100644
--- a/evals/coding-agent/README.md
+++ b/evals/coding-agent/README.md
@@ -1278,24 +1278,25 @@ a full browser exploit defense or proof of execution isolation.
`app-markdown-remote-images` renders the actual chat MarkdownBlock (both roles),
the shared MemoizedReactMarkdown and the meeting note editor built from
-`createMeetingNoteEditorExtensions`, with a synthetic native media port. Sixty-three
-outcomes check that remote, protocol-relative, relative, reference-style, raw
-`<img>` and `<picture><source srcset>` images, and remote or network-share media,
-create no element that loads them and never reach the media reader, while their
-alt text, link or code stays visible. Network-share media includes a home-relative
-path that resolves to a share and local-looking strings from which the reader
-would take a share path; images whose source is removed for safety keep their alt
-text. In a meeting note, remote images an AI summary wrote or a later update
-brings do not load, keep their alt text and stay in the note's Markdown, while
-embedded data: images still show. Local absolute, file-URL, Windows and
-home-relative files still render through the native reader. The parent fails
-fifty-six outcomes and preserves seven; the first, markdown-only fix fails
-fifteen; the fix and current source pass all.
+`createMeetingNoteEditorExtensions`, with a synthetic native media port.
+Sixty-four outcomes check that remote, protocol-relative, relative,
+reference-style, raw `<img>` and `<picture><source srcset>` images, and remote or
+network-share media, create no element that loads them and never reach the media
+reader, while their alt text, link or code stays visible. Network-share media
+includes a home-relative path that resolves to a share and local-looking strings
+from which the reader would take a share path; images whose source is removed for
+safety keep their alt text. In a meeting note, remote images an AI summary wrote
+or a later update brings do not load, keep their alt text and stay in the note's
+Markdown, also when the note is copied and pasted back, while embedded data:
+images still show. Local absolute, file-URL, Windows and home-relative files still
+render through the native reader. The parent fails fifty-seven outcomes and
+preserves seven; the first, markdown-only fix fails sixteen; the second fails only
+the copy outcome; the fix and current source pass all.
Run `bun test evals/coding-agent/calibrate-markdown-remote-images.test.js`.
-Controls include parent/reference/current, the first fix, an equivalent alt-text
-element, a restored remote `<img>` fallback, re-allowed picture/source, an
-extension-only media check, a media check that allows a second leading
+Controls include parent/reference/current, the first two fixes, an equivalent
+alt-text element, a restored remote `<img>` fallback, re-allowed picture/source,
+an extension-only media check, a media check that allows a second leading
separator, network shares treated as local, blanket image removal, a note editor
that renders any source or deletes remote images from the note, unused correct
source and missing-source setup failure. Webview CSP, Mermaid diagrams (jsdom
diff --git a/evals/coding-agent/calibrate-markdown-remote-images.test.js b/evals/coding-agent/calibrate-markdown-remote-images.test.js
index a1efe03e4..afe0c1b98 100644
--- a/evals/coding-agent/calibrate-markdown-remote-images.test.js
+++ b/evals/coding-agent/calibrate-markdown-remote-images.test.js
@@ -11,6 +11,8 @@ const markdown = `${app}/components/markdown.tsx`, block = `${app}/components/ch
const notes = `${app}/components/meeting-notes/note-editor.tsx`;
// The first fix: markdown images only, with a raw-string media gate.
const PREVIOUS_FIX = '7ee2e6ef078200f9781a98c39347d7c8f48c7663';
+// The second fix: notes blocked remote images, but a copied one pasted back as text.
+const SECOND_FIX = '359ac0c2512578a972ba07c09e895725d5021b6c';
const root = mkdtempSync(join(tmpdir(), 'markdown-remote-images-calibration-')), archives = new Map();
const receipts = process.env.SCREENPIPE_EVAL_CALIBRATION_RECEIPTS;
afterAll(() => rmSync(root, {recursive: true, force: true}));
@@ -39,10 +41,11 @@ function grade(name, ref = item.oracle_ref, mutate = () => {}) {
}
// Behavior failures surface as chai AssertionErrors or jest-dom matcher errors.
const BEHAVIOR_FAILURE = /AssertionError|Error: expect\(/;
-function passes(r) { expect(r.status).toBe(0); expect(r.stdout).toContain('63 passed'); expect(r.stdout+r.stderr).not.toMatch(/Unhandled|Uncaught/); }
+function passes(r) { expect(r.status).toBe(0); expect(r.stdout).toContain('64 passed'); expect(r.stdout+r.stderr).not.toMatch(/Unhandled|Uncaught/); }
function fails(r) { expect(r.status).toBe(1); expect(r.stderr).toMatch(BEHAVIOR_FAILURE); expect(r.stdout+r.stderr).not.toMatch(/Unhandled|Uncaught|Failed to resolve import|Failed to load url|Cannot find module/); }
-test('parent fails fifty-six remote outcomes and preserves seven local files', () => { const r = grade('parent', item.base_ref); fails(r); expect(r.stdout).toContain('56 failed | 7 passed'); }, 120000);
-test('previous markdown-only fix fails media-path, alt-text and note outcomes', () => { const r = grade('previous-fix', PREVIOUS_FIX); fails(r); expect(r.stdout).toContain('15 failed | 48 passed'); }, 120000);
+test('parent fails fifty-seven remote outcomes and preserves seven local files', () => { const r = grade('parent', item.base_ref); fails(r); expect(r.stdout).toContain('57 failed | 7 passed'); }, 120000);
+test('previous markdown-only fix fails media-path, alt-text and note outcomes', () => { const r = grade('previous-fix', PREVIOUS_FIX); fails(r); expect(r.stdout).toContain('16 failed | 48 passed'); }, 120000);
+test('second fix fails only the note copy outcome', () => { const r = grade('second-fix', SECOND_FIX); fails(r); expect(r.stdout).toContain('1 failed | 63 passed'); expect(r.stderr).toContain('copied and pasted back'); }, 120000);
test('historical reference passes every outcome', () => passes(grade('reference')), 120000);
test('current caller passes every outcome', () => passes(grade('current', 'HEAD')), 120000);
test('equivalent alt-text element passes', () => passes(grade('equivalent', item.oracle_ref, cwd => replace(cwd, markdown, 'return <ImageAltText alt={alt} />;', 'return alt ? <em>{alt}</em> : null;'))), 120000);
diff --git a/evals/coding-agent/cases.json b/evals/coding-agent/cases.json
index 1911023a4..713527aff 100644
--- a/evals/coding-agent/cases.json
+++ b/evals/coding-agent/cases.json
@@ -1,7 +1,7 @@
{
"schema_version": 1,
"suite": "screenpipe-app-coding-agent-regressions",
- "dataset_version": "2026-10-01-remote-images-media-notes",
+ "dataset_version": "2026-10-01-remote-images-media-notes-copy",
"suite_type": "regression",
"owner": "screenpipe engineering",
"policy": {
@@ -4078,10 +4078,10 @@
"name": "Render only local images and media from untrusted Markdown and meeting notes",
"source": {
"kind": "git_regression",
- "fix_commit": "359ac0c2512578a972ba07c09e895725d5021b6c"
+ "fix_commit": "bbf239eaa5626876e48b30e8cc04b1e4c1187c1e"
},
"base_ref": "7274f29b6b1bb940332b4e912b9cfcb20d591d3d",
- "oracle_ref": "359ac0c2512578a972ba07c09e895725d5021b6c",
+ "oracle_ref": "bbf239eaa5626876e48b30e8cc04b1e4c1187c1e",
"oracle_paths": [
"apps/screenpipe-app-tauri/components/markdown.tsx",
"apps/screenpipe-app-tauri/components/markdown/image-alt-text.tsx",
@@ -4109,7 +4109,7 @@
"privacy",
"preserved-behavior"
],
- "prompt": "Rendered Markdown and meeting notes make the application load remote resources the moment they render. Chat replies, summaries and AI meeting notes are built from captured outside content, so steered text can put private data in an image URL that leaks without any click. Repair the chat MarkdownBlock for both user and assistant messages, the shared MemoizedReactMarkdown renderer, and the meeting note editor built from createMeetingNoteEditorExtensions. Only local files may display as images or media in Markdown: absolute paths, file URLs and Windows drive paths for images, plus home-relative paths for media, must still be read through the supplied native media port and displayed. No other image source (http, https, protocol-relative, relative, reference-style, raw HTML img, or picture/source srcset) may produce an element that loads it; keep its alt text visible instead, including when the source is removed for safety. Media images, links and code that name a remote URL or a network-share path must never reach the native media reader, including a home-relative path that resolves to a share or a local-looking string from which the reader would take a share path; keep their alt text, link or code visible. In a meeting note, a remote image written by the AI or arriving with a later update must not load: keep its alt text visible, keep images embedded as data: URLs displaying, and keep the note's Markdown, including the remote image reference, so viewing or saving loses nothing. Keep the surrounding text. Keep the public component, editor extension and native port contracts compatible. No particular sanitizer, helper or internal implementation is required. This task checks actual React/Markdown and editor rendering with synthetic native ports; it does not establish webview CSP, Mermaid diagrams, pasting into notes, native file access policy or network isolation.",
+ "prompt": "Rendered Markdown and meeting notes make the application load remote resources the moment they render. Chat replies, summaries and AI meeting notes are built from captured outside content, so steered text can put private data in an image URL that leaks without any click. Repair the chat MarkdownBlock for both user and assistant messages, the shared MemoizedReactMarkdown renderer, and the meeting note editor built from createMeetingNoteEditorExtensions. Only local files may display as images or media in Markdown: absolute paths, file URLs and Windows drive paths for images, plus home-relative paths for media, must still be read through the supplied native media port and displayed. No other image source (http, https, protocol-relative, relative, reference-style, raw HTML img, or picture/source srcset) may produce an element that loads it; keep its alt text visible instead, including when the source is removed for safety. Media images, links and code that name a remote URL or a network-share path must never reach the native media reader, including a home-relative path that resolves to a share or a local-looking string from which the reader would take a share path; keep their alt text, link or code visible. In a meeting note, a remote image written by the AI or arriving with a later update must not load: keep its alt text visible, keep images embedded as data: URLs displaying, and keep the note's Markdown, including the remote image reference, so viewing, saving, or copying and pasting the note loses nothing. Keep the surrounding text. Keep the public component, editor extension and native port contracts compatible. No particular sanitizer, helper or internal implementation is required. This task checks actual React/Markdown and editor rendering with synthetic native ports; it does not establish webview CSP, Mermaid diagrams, pasting into notes, native file access policy or network isolation.",
"agent_timeout_seconds": 900,
"grader_dependency_links": [
{
diff --git a/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt b/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt
index dbe1a60c7..a22055b40 100644
--- a/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt
+++ b/evals/coding-agent/graders/markdown-remote-image-outcomes.fixture.tsx.txt
@@ -194,6 +194,27 @@ describe("meeting note editor", () => {
expect(markdownOf()).toContain("https://example.com/raw.png?d=secret");
});
+ it("keeps an image an AI summary wrote when the note is copied and pasted back", async () => {
+ const element = mountNote([
+ "before",
+ "",
+ '<img src="https://example.com/chart.png?d=secret" alt="sales chart" width="40" height="30" />',
+ "",
+ "after",
+ ].join("\n"));
+ const view = editor!.view;
+ // What the editor puts on the clipboard, then a cut and a paste.
+ const { dom } = view.serializeForClipboard(view.state.doc.slice(0, view.state.doc.content.size));
+ editor!.commands.clearContent();
+ expect(markdownOf()).not.toContain("chart.png");
+ view.pasteHTML(dom.innerHTML, new Event("paste") as ClipboardEvent);
+ await settle();
+ expect(element).toHaveTextContent("sales chart");
+ expect(element).toHaveTextContent("after");
+ expect(remoteImages(element)).toEqual([]);
+ expect(markdownOf()).toContain("https://example.com/chart.png?d=secret");
+ });
+
it("does not load a remote image that arrives with a later update", async () => {
const element = mountNote("notes");
editor!.commands.setContent("before\n\n![sales chart](https://example.com/late.png?d=secret)\n\nafter");