~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
"""Parser-posture contracts for the PDF render path.
Two defenses of the report-rendering pipeline currently hold only by
architecture, and nothing in the test tree pins them:
1. **Pillow's decompression-bomb guard stays armed.** WeasyPrint decodes
images referenced from report HTML; the only thing standing between a
hostile ``<img>`` and a multi-gigapixel decode is Pillow's default
``MAX_IMAGE_PIXELS``. The classic operational "fix" for a user's
oversized-image complaint — ``Image.MAX_IMAGE_PIXELS = None`` — would
silently reopen decode bombs; no test would notice.
2. **Inline XML DTDs in report bodies are inert.** A report body can
carry an SVG with a DOCTYPE and entity declarations. Two layers keep
them inert, and each is pinned by its own test. The markdown-to-HTML
step escapes raw HTML in report text, so such a block is handed to
the renderer as text, not markup. Beneath that, the HTML5 parsing
pipeline does not process XML DTDs, so a block that does reach it as
markup neither expands its entities nor resolves an external one;
that test injects the block at the HTML layer, past the escaping, so
it exercises the parser whatever the markdown step does. Removing the
escaping fails ``test_the_hostile_block_is_escaped_not_passed_through``;
a swap to an XML-based renderer that honoured the DTD would fail
``test_dtd_entities_neither_expand_nor_read_files``.
These contracts pin today's posture so either regression goes red.
"""
from __future__ import annotations
import io
import re
import struct
import zlib
from pathlib import Path
from unittest.mock import patch
import pytest
pytest.importorskip("weasyprint")
from local_deep_research.web.services import pdf_service # noqa: E402
def _extract_pdf_text(pdf_bytes: bytes) -> str:
from pypdf import PdfReader
reader = PdfReader(io.BytesIO(pdf_bytes))
return "\n".join(page.extract_text() or "" for page in reader.pages)
def _bomb_png(dim: int = 40_000) -> bytes:
"""A PNG declaring ``dim x dim`` 1-bit pixels with trivial data."""
def chunk(tag: bytes, data: bytes) -> bytes:
return (
struct.pack(">I", len(data))
+ tag
+ data
+ struct.pack(">I", zlib.crc32(tag + data) & 0xFFFFFFFF)
)
ihdr = struct.pack(">IIBBBBB", dim, dim, 1, 0, 0, 0, 0)
raw = b"\x00" + b"\x00" * 16 # partial first row; decoders size from IHDR
return (
b"\x89PNG\r\n\x1a\n"
+ chunk(b"IHDR", ihdr)
+ chunk(b"IDAT", zlib.compress(raw))
+ chunk(b"IEND", b"")
)
class TestPillowBombGuardStaysArmed:
"""Nothing in the codebase may defuse Pillow's decode-bomb guard."""
def test_no_assignment_disables_max_image_pixels(self):
src_root = (
Path(pdf_service.__file__).resolve().parents[3]
/ "local_deep_research"
)
offenders: list[str] = []
# (?!=) so equality READS (== comparisons) are not flagged.
pattern = re.compile(r"MAX_IMAGE_PIXELS\s*=(?!=)")
for module_path in sorted(src_root.rglob("*.py")):
source = module_path.read_text(encoding="utf-8")
if pattern.search(source):
offenders.append(str(module_path.relative_to(src_root)))
assert offenders == [], (
f"code assigns MAX_IMAGE_PIXELS (the decode-bomb guard): {offenders}"
)
def test_the_guard_fires_in_the_render_process(self):
"""Importing the pdf pipeline leaves the guard armed, and a
bomb-shaped image is refused — not decoded."""
from PIL import Image
with pytest.raises(Image.DecompressionBombError):
Image.open(io.BytesIO(_bomb_png())).load()
class TestInlineDTDsAreInert:
"""Entity declarations in report bodies must not expand or resolve."""
HOSTILE_BODY = """<?xml version="1.0"?>
<!DOCTYPE r [
<!ENTITY leaf "Z0">
<!ENTITY branch "&leaf;&leaf;&leaf;&leaf;&leaf;&leaf;&leaf;&leaf;">
<!ENTITY tree "&branch;&branch;&branch;&branch;&branch;&branch;&branch;&branch;">
<!ENTITY xxe SYSTEM "file://{sentinel_path}">
]>
<svg xmlns="http://www.w3.org/2000/svg"><text>&tree;&xxe;</text></svg>
Normal report body text with the ARRIVAL-CONTROL marker.
"""
def test_dtd_entities_neither_expand_nor_read_files(self, tmp_path):
"""The parser layer on its own. Report markdown escapes this block,
so it is injected at the HTML layer instead (by stubbing
``_markdown_to_html``) and reaches WeasyPrint's HTML5 parser as
markup, independent of the markdown step."""
sentinel = tmp_path / "sentinel.txt"
sentinel.write_text("SENTINEL-FILE-CONTENT-9QW", encoding="utf-8")
# The block placed in the body verbatim, as markup.
html_doc = (
"<!DOCTYPE html><html><head><meta charset='utf-8'></head><body>"
+ self.HOSTILE_BODY.format(sentinel_path=sentinel)
+ "</body></html>"
)
with patch.object(
pdf_service.PDFService,
"_markdown_to_html",
lambda self, content, title=None, metadata=None: html_doc,
):
pdf_bytes = pdf_service.get_pdf_service().markdown_to_pdf(
"# Posture probe\n", title="Posture probe"
)
assert pdf_bytes, "render must complete on a DTD-bearing body"
text = _extract_pdf_text(pdf_bytes)
# Arrival control: the benign marker made it into the render,
# so the absence assertions below are not vacuous.
assert "ARRIVAL-CONTROL" in text, text[:400]
# And the declarations reached the parser as markup: as escaped text
# they would render, "<!ENTITY" lines and all.
assert "ENTITY" not in text, text[:400]
# No billion-laughs expansion (declaration text contains
# "&leaf;", never the literal "Z0Z0Z0").
assert "Z0Z0Z0" not in text, "internal entity expanded in render"
# No external-entity file read.
assert "SENTINEL-FILE-CONTENT-9QW" not in text, (
"external entity resolved during render"
)
def test_the_hostile_block_is_escaped_not_passed_through(self, tmp_path):
"""The escaping layer on its own: the markdown-to-HTML step escapes
the raw SVG/DTD block, so it is handed to the renderer as text rather
than markup. The benign text after the block is checked too, so the
conversion is known to have produced the body."""
sentinel = tmp_path / "sentinel.txt"
sentinel.write_text("x", encoding="utf-8")
html = pdf_service.get_pdf_service()._markdown_to_html(
self.HOSTILE_BODY.format(sentinel_path=sentinel),
title="Escaping probe",
)
# Escaped, not passed through as markup.
assert "<svg" not in html, "raw SVG markup must not reach the renderer"
assert "<svg" in html, "the SVG block must arrive as escaped text"
# The benign report text still renders.
assert "ARRIVAL-CONTROL" in html