~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
💥 Breaking Changes
SearchResultsCollector.add_results()now returnstuple[int, list[dict]]instead ofint. Callers that didstart = collector.add_results(results)must migrate tostart, indexed = collector.add_results(results)and passindexedto_format_results()so deduped citation indices ([N]) stay in sync with the collector. The oldint-only return is no longer available; this is a breaking change for external programmatic users ofSearchResultsCollector. Internal call sites (web_search,specialized_search,fetch_content) have been updated. (#5381)
🔒 Security
/library/document/...URL resolution now requires the path segment to be a canonical document id (32/64-hex or UUID), rejecting guessable filenames at the regex instead of relying solely on a downstream database miss. (#5485)Benchmark progress is no longer delivered to the wrong user's browser. Socket.IO subscriptions were held in a process-global map keyed by the research id alone, but
BenchmarkRun.idis a per-user autoincrement integer — so two users who each own a benchmark run with the same id (both have run 1) were filed under the same subscriber set, and each user's browser received the other'sresearch_progress_<id>events: live queries, partial results, rate-limit warnings. Subscriptions are now keyed by(username, run id)for numeric benchmark ids, matching the same fix applied to the benchmark run registry. Ordinary research is unaffected — those ids are UUIDs and cannot collide. (#5520)Benchmark runs are now isolated per user in the in-process run registry. That registry was keyed by the benchmark run id — a per-user autoincrement integer, so two different users can each own a run with the same id (both have run 1). Because the id was trusted on its own, one signed-in user could cancel another user's in-flight benchmark (a denial of service), trigger a read/persist of another user's in-memory results, or observe their run state; and two users running a benchmark with the same id at the same time would overwrite each other's entry, crossing their results and — via the run's stored password — their credentials. The registry is now keyed by
(username, run id), so every operation only ever reaches the caller's own run and colliding per-user ids can no longer collide.Sanitize credential-shaped content in relevance-filter parser diagnostics before persisting or streaming the response preview.
Starting a follow-up research now verifies you own the parent research.
POST /api/followup/startprepared and spawned a follow-up without checking thatparent_research_idbelonged to the caller (unlike/api/followup/prepare, which returns 404). Research ids are per-user, so this could not read another user's research data (it lives in a separate encrypted database), but it let a user start a research thread referencing a parent that isn't theirs.startnow applies the same 404 ownership check asprepare.Terminating a research run now verifies you own it.
cancel_research(reached fromPOST /research/api/terminate/<id>) drove the process-global termination registry — keyed by research id alone and shared across users — before any ownership check, so a signed-in user who knew another user's research id could terminate that user's in-progress research. Ownership is now confirmed against the caller's own database before any global termination state is touched.The
LDR_SECURITY_ALLOW_NAT64operator opt-in (for IPv6-only deployments that reach IPv4 via DNS64/NAT64) no longer widens the private-IP policy, and now decodes NAT64-wrapped addresses at the correct RFC 6052 §2.2 byte positions. A NAT64-wrapped destination is evaluated exactly as a direct connection to its embedded IPv4 would be: a wrapped RFC1918 or loopback address is blocked unless the matching private-IP / localhost allowance is also enabled, the six always-blocked cloud-metadata literals stay blocked unconditionally, and link-local wraps are blocked wherever a direct connection to the embedded address would be — including in the RFC 8215/48local-use form, where the embedded IPv4 lives at different byte positions than the trailing 32 bits (previously a crafted/48address could hide a metadata target behind a public-looking suffix). Note that the general fetch path callsvalidate_url, which does not setblock_link_local, so a NAT64-wrapped link-local address outside those six literals (for example a wrap of Scaleway's169.254.42.42) is reachable onceallow_private_ipsis enabled — exactly as a direct connection to it already is. That is a pre-existing limitation of the private-IP opt-in, not something this change widens; the point of this fix is that enabling NAT64 grants nothing beyond it. NAT64-wrapped public addresses — the opt-in's intended use — remain reachable. Closes a gap where enabling NAT64 could reach internal or metadata IPv4 targets through the general fetch path.The notification service URL is now redacted on the settings API read paths.
notifications.service_url(apprise-style) embeds credentials — e.g.mailto://user:pass@host,discord://webhook_id/token, Slack/ntfy tokens — and was returned in plaintext by the settings API (single, bulk, and full GET, and the save-all echo), unlike API keys which are masked. It is now treated as a secret and returned as[REDACTED]on read (an unconfigured/empty URL stays readable; whitespace-only counts as unconfigured), so the credential no longer reaches the browser at all and cannot be picked up by anything that records a settings response — devtools, a saved HAR, or a logging proxy. Clearing the URL from the UI now stores the empty value instead of silently retaining the old URL, and Test Notification uses the stored URL when the field contains the redaction sentinel. The app already masked it in notification call-site logs; the settings debug logger now redacts it too.notifications.service_urlis the first sensitive setting that is not a password input, so it is the first one whose control renders its value into an editable field. The settings UI now renders any redacted control blank with a "Saved — type to replace, or press Enter while empty to clear" placeholder, the way password inputs already do, and the write routes reject a submitted value that merely contains[REDACTED]with a 400 instead of storing it. Without both, a stale tab produced[REDACTED],discord://webhook/tok— not an exact sentinel match, so the existing no-op did not fire, and the saved list madeNotificationManagerrefuse every URL in it, silently stopping all notifications. The reverse order,discord://webhook/tok,[REDACTED], passed URL validation and appended the sentinel to the webhook token. An exact sentinel keeps its existing idempotent no-op behaviour on update; on the create/recreate path, where there is no stored value to preserve, it is rejected too.Because Test Notification can now fall back to the stored URL, it accepts an empty body and becomes a zero-argument send trigger, so that path gets its own rate-limit bucket; testing a URL supplied in the request stays unlimited. The endpoint also treats a whitespace-only URL as unconfigured, matching the notification manager, instead of handing Apprise literal whitespace.
🐛 Bug Fixes
Fixed search result collection in
LangGraphAgentStrategyto deduplicate URLs so repeated search hits across queries reuse their existing citation index instead of assigning duplicate numbers. Also fixes RAG chunk URI fragments for library search hits to use correct metadata keys (source_id/document_id) and adds chunk-targeted HTML anchors (id="chunk-{{ chunk.index }}") so citations scroll to the cited chunk.Further hardened the chunk-anchor contract: chunk-URL fragments are now validated centrally in
src/local_deep_research/utilities/chunk_anchor.pyand reused byCollectionSearchEngine,LibraryRAGSearchEngine, andSearchResultsCollector.add_results. A malformedchunk_index/chunk_id(UUID, boolean, negative, etc.) cannot leak into a citation; a missing/rejecteddoc_idno longer appends#chunk-...to an unrelated route; and any producer-supplied unvalidated#chunk-...fragment is stripped rather than trusted. RAG producers also sanitisedoc_id/source_id(including the primaryr.source_idpath) before URL construction.SearchResultsCollectornow allocates citation indices past the highest seeded index so sparse seeds are not overwritten; the_format_resultsfallback warning routes the URL throughredact_url_for_logso credentials never appear raw in logs; and the constructor skips legacy non-dict seeded entries. (#5381)Collection uploads now correctly report each file's outcome when a single request batch contains two files with identical bytes: dropped intra-batch duplicates are listed under a new "Skipped — duplicate of another file in this upload" bucket using the uploaded filename, instead of being mis-reported under the first occurrence's filename as "Skipped — already in collection". In addition, uploads matching existing library documents now report the newly uploaded filename rather than the stored document's canonical filename. Multi-file uploads also isolate failures per file via savepoints instead of rolling back the whole request, ensuring that a subsequent failing file no longer discards earlier successful uploads. (#5495)
Skip SearXNG rate-limit locks that are in use when the URL tracker evicts at capacity, so a held lock is not replaced by a fresh one that ignores the configured delay. (#5517)
Enforce the settings lock (
app.lock_settings) on the delete, import and reset-to-defaults endpoints, which previously ignored it: deleting or bulk-importing settings while locked now returns 403, and reset-to-defaults can no longer clear the lock as a side effect. Startup paths that load newly shipped defaults are unaffected. (#5531)Telegram notifications now use Apprise's canonical
tgram://scheme; the unregisteredtelegram://alias is rejected. Apprise diagnostic log output that could contain notification credentials is suppressed. Multi-target URLs such astgram://...?to=id1,id2are no longer split into partial notifications, andtelegram://URLs now get a dedicated migration error pointing at thetgram://form. (#5576)import_settings()no longer accepts values that are invalid under the current defaults schema. Both the DB-backedSettingsManagerandInMemorySettingsManagervalidate every file-supplied value for a known setting key against that key's current-defaults constraints (select options, numeric min/max, checkbox type) and skip the entry — with a warning — when it does not match, so importing a pre-upgrade export can no longer resurrect stale field shapes (e.g. a free-textdefault_params.time_periodvalue from before the field gained its options list). Keys not in the current defaults keep their previous behavior, dynamically populated dropdowns (llm.provider,llm.model,search.tool) stay exempt, values equal to the current default always pass, and values retained from existing storage (the version-bump reconciliation path and environment-locked settings) are deliberately not re-validated. (#5589) (#5589)Token metrics now persist the total the provider reported instead of recomputing it as prompt + completion, so the metrics dashboard and the in-session counter no longer disagree about the same response. A provider that reports usage without a total is now counted as prompt + completion rather than as zero tokens. (#5623)
Clarified the help text and tooltips on the embedding settings page so they match which chunk-size, chunk-overlap, and text-separator settings each text splitter type actually uses. Previously, the descriptions implied every splitter honored every setting — for example, the chunk-size field was described as "characters per chunk" even though the
tokenandsentencesplitters interpret it as tokens, and thesemanticsplitter silently forces chunk overlap to 0. (#5660)The SearXNG rate limiter now keeps each URL's lock and its last-request time in one tracked entry, so a URL evicted while one of its requests is still in flight can no longer leave its timestamp behind.
MAX_TRACKED_URLSremains an eviction target rather than a hard bound, so the tracker can still grow past it while every tracked lock is held. (#5669)Fixed three downstream effects of the chunk-anchor citation URLs introduced in #5381.
fetch_contentcan read library documents again: the RAG engines now cite/library/document/<id>/chunks#chunk-<n>, but the fetch tool's library resolver only matched an optional/pdfsuffix and never stripped the fragment, so every such citation fell through to the egress gate and was rejected asunsupported_scheme. The resolver now accepts thechunksroute and drops the fragment before matching.SearchResultsCollector.sourcesno longer loses deduplicated URLs afterreset().reset()clears_sourcesper report subsection while the dedup maps deliberately persist, so a subsection that re-cited an earlier section's hits reported no sources at all — an emptysourceslist from the MCP server, andsources_count: 0in news cards.Library citations collapse to one bibliography entry per document again.
canonical_url_keystripped fragments only from absolute URLs, so each cited chunk of a document became a separate## Sourcesline. Library routes now key per document —/library/document/<id>, its/pdfview and its/chunks#chunk-<n>view are one source — which also collapses the two entries a document used to get when some of its chunks carriedchunk_indexmetadata and others fell back to/pdf. The renderedURL:line keeps a#chunk-<n>anchor, preferring an anchored view over the anchor-less/pdfone regardless of which was registered first, so collapsing the list does not cost the scroll-to-chunk behaviour the anchor exists for. Note the entry carries one URL for the whole group: it anchors at the first cited chunk, so a document cited across several chunks scrolls to the first of them rather than to each citation's own chunk. Per-citation anchors would need a different Sources layout and are not attempted here. Query strings are not part of a library document's identity —/library/document/<id>?v=2keys with/library/document/<id>— which no current producer emits.Citations of one library document via the absolute
https://library.document/<id>alias — the form the agent sometimes emits and the fetch resolver accepts — now share a bibliography entry with that document's other citations instead of rendering as a separate, anchor-less external-looking link. Only the exact<id>[/pdf|/chunks]shape normalizes: look-alike hosts, deeper paths, queries and traversal are rejected, so a crafted alias cannot merge into a real document's entry and replace its link.Library/RAG citations no longer point at the wrong document when a chunk's denormalised metadata disagrees with its
DocumentChunkforeign key. The authoritative document id is now carried at the top level of each search result and preferred over metadata wherever the citation URL is built or rebuilt.format_links_to_markdownno longer crashes the whole bibliography when a citation index contains a non-ASCII digit character ("¹".isdigit()is true whileint("¹")raises).Bibliography exports (BibTeX/Quarto and LaTeX) keep library documents that are cited from several chunks. Collapsing those citations onto one Sources line produced a
[1, 2, 3]index group that the exporters' single-index patterns skipped, silently dropping the source from the exported bibliography.sources_countnow agrees with the rendered bibliography. The bibliography groups sources withcanonical_url_key, butSearchResultsCollectorstill keyed its dedup map and itssourceslist on the raw link, so the three views of one library document (/library/document/<id>,/<id>/pdf,/<id>/chunks#chunk-<n>, plus the absolutehttps://library.document/<id>alias) produced three citation indices and asources_countof 3 for a report whose## Sourcesblock renders one line — the count the MCP server returns and the news impact score reads. The collector now dedups on the same canonical key while recording the display URL, anchor included. This also means a search hit and a laterfetch_contentof the same document share one citation index no matter which spelling of the URL the agent typed back.A subsection that only re-fetches previously cited URLs renders its
## Sourcesblock again.find_or_add_resultrecorded the source on all three branches but appended to_resultsonly when it allocated a new citation, so afterreset()such a subsection reported N sources, rendered none, and tripped the #4969 "citation pass skipped" warning.Library-route parsing is now one code path for the relative route, its
/lib/document/<id>abbreviation and the absolute alias, so all three enforce the same validation. Previously only the alias validated its document id:/library/document/7/../../../admin/wipe#chunk-1keyed as document 7, inherited that document's title and citation numbers, and replaced its link. The/lib/spelling and percent-encoded ids (a%2Dbvsa-b) now key with the document they denote instead of fanning it across extra entries, and the displayed route is always the registered/library/document/<id>one. The bare[<uuid>]alias the agent sometimes pastes intofetch_contentis re-cited as the resolved document's canonical route instead of verbatim, closing the third spelling that used to get its own (dead-linking) entry. A route carrying a control character is now refused outright rather than having its document-id segment truncated: truncating discarded the payload but keyed the crafted URL as the real/library/document/<id>it was prefixed with, merging it into that document's bibliography entry and lending it the document's title and citation number. The payload is still neutralised where it is rendered.An external URL that merely contains
/library/document/—https://evil.example/library/document/7/chunks— is no longer relabelled as a local library document. The eligibility test is anchored at the start of the URL instead of being a substring match.Chunk indices and document ids are validated against ASCII character classes with a magnitude bound, so non-ASCII digits (
٧,𝟝), whitespace-padded values, absurd magnitudes (10**30,1e30) and fullwidth document ids can no longer reach a URL fragment or path. The bibliography's "prefer an anchored URL" rule now looks for#chunk-specifically, so an empty or unrelated fragment can no longer lock out the real chunk anchor.sourcesin the strategy result keeps first-seen order. It was passed throughlist(set(...)), which deduped nothing (the collector already guarantees uniqueness) and randomized the ordering differently on every process.Note: RIS export does not yet handle merged index lines, and mis-attributes rather than merely skipping them — a record can carry one source's title with another source's URL. It needs a different fix from the other exporters and is tracked in #5687. (#5685)
Fixed
SearchResultsCollector(LangGraph agent strategy) discarding a source's later excerpts. Citations were deduplicated on the canonical URL alone, so a URL reached by several queries kept only the first occurrence's snippet — the agent read three excerpts and the bibliography,research_resourcesand the APIsourcespayload retained one. The dedup key is now the pair(canonical URL, snippet): a repeat with the same snippet still collapses onto its existing[N](the #5381 behaviour), while a repeat with a genuinely different snippet becomes its own entry with its own[N]. Each entry owns exactly one citation index, soCitationFormatter's index → URL map stays 1:1. Snippets are compared after normalising whitespace, case and trailing ellipsis/punctuation, over ablake2bdigest of the whole passage — never a prefix, which cannot tell "one passage truncated twice" from "two passages that open alike". There is deliberately no cap on how many excerpts one source may contribute: folding a distinct excerpt onto an existing citation would show the model text under a number that resolves to different text, and the count is already bounded by the number of queries run.all_links_of_systemtherefore holds one entry per distinct excerpt rather than one per source, and every "N sources" number now counts distinct canonical URLs via the newsearch_utilities.count_distinct_sources— the agent step heartbeat,_finalize's accumulated-sources milestone and its two citation-skip warnings, and the detailed report'sinitial_sourcesmetadata.initial_sourcesnow counts distinct sources rather than raw entries for every strategy, not only the LangGraph agent:report_generatorreadsall_links_of_systemwhatever produced it, andsource_based,focused_iterationandtopic_organizationhave no cross-call dedup at all, so a run that revisits a URL acrossanalyze_topic()calls will report a lower — and now correct — number than before (e.g. 4 raw entries reported as 3 sources).format_links_to_markdownstill renders one## Sourcesline per source, now listing every citation number that source owns ([1, 4] Title), and it groups on the same field the collector keys on (linkfirst, via the newsource_url_field) so a result carrying divergenturlandlinkcannot split one source across two lines. (#5894)The news feed no longer fails with a generic "Database error" when a stored source row's URL is not a string; such rows are now skipped instead of aborting the whole feed.
📝 Other Changes
- Docs now state concretely that the private engine-URL allowlist requires v1.10.5 or newer, replacing the pre-release "newer than v1.10.4" phrasing, and add the same caveat to the two
docker runrecipes (FAQ, installation guide) that recommended the variable with no version note. (#5647) - Docs and
.env.templatenow state clearly that LDR itself never loads a.envfile — it reads only the environment of the server process — while pointing at the file-based mechanisms that do work (docker run --env-file, Composeenv_file:, systemdEnvironmentFile=), with a new per-deployment persistence table in the environment-configuration guide and a note on thecrawl4aidependency's ownload_dotenv()call. (#5649) - The remediation text shown when a research run fails on a missing API key no longer implies a
.envfile is an option — LDR itself reads only the server process's environment. To keep values in a file, hand the file to whatever starts LDR (docker run --env-file, Composeenv_file:, systemdEnvironmentFile=). (#5661) - All production directory creation now routes through
security.create_directory(audited chokepoint with opt-in path containment), enforced by a pre-commit/CI check that forbids rawmkdir/os.makedirs. - Clarified the
VectorStore.apply()contract:dedupde-duplicates ids within one call and never means "skip an add whose id is already live" — an already-live id must be replaced, because SQLite recycles AUTOINCREMENT ids on ROLLBACK and keeping the stale vector makes search rehydrate the wrong document's text. - Removed a maintainer's local filesystem path from the test-review completion report.
- The missing-inline-citations warnings added for #4969 are now more precise: the iteration-limit failure sentinel no longer triggers the skip warning (an agent failure is not a citation gap), lenticular-bracket citations (
【N】) count as markers, the zero-marker warning is suppressed when the citation handler itself failed (its own warning already fires, including new ones for non-dict handler results and for dicts missing both thecontentandresponsekeys — each naming the returned type or missing keys and the query), and the log wording states facts without referencing internal issue numbers. - Unblock the release gate: pin the rebuilt
python:3.14.7-slimbase-image digest (2026-08-25 rebuild) and add Grype suppressions for the 2026-08-18 CPython security batch (CVE-2026-19672 tarfile, CVE-2026-15806 urllib HTTPPasswordMgr, CVE-2026-17084 stringprep). The fixes ship first in Python 3.14.8, which has no release or Docker image yet, and none of the three stdlib APIs is reachable insrc/.