~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
🔒 Security
- Explicit
max_resultsandtime_periodoverrides are now validated, retain precedence over queued values, and invalid persisted queued values are safely rejected instead of triggering transient retries. (#5148) - Environment-locked settings can no longer be overwritten or deleted through programmatic settings mutations. Scheduler, RAG, and model-discovery paths now also fail closed (rather than falling back to defaults) when the effective settings snapshot cannot be built, so a corrupted or missing snapshot never downgrades egress protection.
- Host classification now treats a hostname as local only when every resolved address is private, and fails closed on resolution errors. This closes a mixed-DNS SSRF bypass where a single private answer among public ones could mark an otherwise-public host as trusted-local.
- Reading a single environment-overridden secret setting (e.g. an API key
supplied via an
LDR_*variable) through the settings API no longer returns the value in plaintext — it is now redacted like every other settings response. - Scrub credential-shaped data from residual web-search exception logs and engine-loading error messages.
- The
unprotectedegress scope is now disabled by default and requires the server operator to setLDR_POLICY_ALLOW_UNPROTECTED_EGRESS=true. Migration0027rewrites legacy stored and queuedunprotectedselections toadaptive, so a later operator opt-in can't silently reactivate them. Environment policy overrides are reapplied when queued work is dispatched. While the gate is off, newunprotectedwrites (settings save, per-research override) are rejected, and any residual or tampered queued value is coerced toadaptiveat dispatch with apolicy_auditwarning. Operator-enabledunprotectedremains supported, and the hard SSRF and cloud-metadata blocks apply in every scope.
✨ New Features
- A collection can now hold multiple RAG indexes for the same embedding model when they differ in chunking or vector-index configuration, so changing the chunk size or index type no longer overwrites an existing index. A database migration relaxes the previous per-collection+model uniqueness accordingly.
- Add Atlas Cloud as a built-in OpenAI-compatible LLM provider.
- The Search Engine dropdown on the New Research form and the news-subscription form is scope-aware: under
Private onlyorPublic onlyEgress Scopes, engines that would be refused at submit time (scope_mismatch_*,unclassified) are rendered as disabled entries witharia-disabled="true"and a one-line reason label (Blocked: not a local source under Private only, etc.), so the user no longer has to click Start Research to discover the mismatch. UnderAdaptive,Strict, andUnprotectedmodes, all primary search engines remain selectable. Switching egress scope automatically reconciles an invalid visible selection to an allowed option (searxngpreferred /libraryfallback). Disabled entries are skipped in keyboard navigation, blocked on click, and reachable by Tab for screen readers. The dropdown serializes fetches and dropdown callbacks (onSelectfires beforeupdateHiddenField) so engine changes carry the correct primary, and rapid changes avoid fetch races via anAbortControllerand per-call generation tokens. The backend precheck (PR #4300) and inline-error UX remain the second backstop for direct API callers and auto-expansion paths. - The Search Engine dropdown on the New Research form now also greys out collections whose “Available to the research agent” flag is off when the LangGraph research agent is the selected strategy — the flag is exclusive to the LangGraph agent, so other strategies (source-based, focused-iteration, topic-organization) keep the collection selectable. The dropdown re-evaluates the disabled set on every strategy switch without a server re-fetch, and the run is refused at submit time as a second backstop for direct API callers (
reason: collection_agent_disabled,field: strategy). The collection form's help text now spells out the LangGraph-only scope so the flag isn't mistaken for a privacy / egress control. Superseded (see #7873): a later change removes both mechanisms — agent-hidden collections are no longer greyed out under LangGraph and can be selected explicitly as a primary engine for an individual run, and there is no longer a submit-timecollection_agent_disabledrefusal; the collection help text now explains the distinction between automatic discovery and primary selection.
🐛 Bug Fixes
- Correct source-based strategy diagnostics to read the canonical questions-per-iteration setting. (#4515)
- Handle list-form LLM content blocks when candidate explorers generate follow-up search queries. (#4615)
- Skip malformed benchmark result payloads without rolling back the rest of the pending persistence batch. (#4861)
- Return a mapping for settings namespaces with exactly one child, matching snapshot-backed reads while preserving scalar exact-key reads. (#4948)
- Keep restore audit bookends from reducing the retained pool of ordinary note versions. (#4964)
- The LangGraph agent's
research_subtopictool no longer silently drops inputs beyond the documented "2-5" limit: the cap is lowered from 8 to 5 to match the lead prompt, and any overflow is now surfaced via a logger warning, atruncated_fromkey in progress metadata, and a model-facing note naming the dropped subtopics so the lead agent can re-issue or avoid citing them. (#5013) - Semantic document chunking now enforces the configured character
chunk_sizebefore comparison embeddings and on final outputs, preventing unbounded semantic inputs or groups from bypassing the document chunk limit. Invalid non-positive OpenAI embedding batch sizes now fall back to the safe application default of 5 instead of LangChain's 1,000-input default. (#5178) - Group CRITICAL and FATAL research logs under Errors, and SUCCESS logs under Milestones, while preserving their original severity labels and styling. (#5231)
- Fixed a
DocumentBlobquery attribute error in the research library'shas_pdf()check that could raise when probing whether a document's PDF was available, so PDF availability now resolves correctly across the database and filesystem backends. (#5263) - Detailed-mode reports no longer stack a duplicate subsection title under the framework's
## i.j Nameheading, and no longer keep per-subsection## Sources/## References/### Selected Bibliography…blocks that the model sometimes emits; prompts now forbid those patterns andIntegratedReportGeneratorstrips a leading restated heading (including sibling bleed, Roman/bold prefixes, and curly quotes), a mirrored italic purpose line, and embedded bibliography blocks before appending each subsection's content. (#5294) - Report whether a skipped LangGraph citation pass preserved existing inline citation markers instead of always warning that the answer has none. (#5297)
- The LangGraph research progress summary now shows human-readable collection names instead of raw UUIDs, and no longer prints a trailing ellipsis when every available tool is already listed. (#5332)
- LangGraph subagent timeout accounting now includes worker cleanup, so stalled cleanup can no longer hang research past the configured deadline. (#5336)
- Prevent library delete confirmations from issuing duplicate requests, and make concurrent duplicate document deletions return not found without repeating success logs or RAG cleanup. (#5337)
- Apply the same generous 5-second Ollama model-list timeout to embedding-model discovery as to LLM-model discovery. The embedding provider still used a 2→3-second timeout, so on a cold Ollama the embedding-model dropdown could come up empty for the same reason the language-model dropdown did. Both providers now share a single
OLLAMA_MODEL_LIST_TIMEOUTconstant so the value can't drift apart again. - Cancelling collection indexing now returns an error when the cancellation status cannot be persisted, instead of reporting success while the task remains active.
- Citation source splitting now strips the internal appended-sources sentinel from collection names, preventing crafted names from shifting persisted report boundaries; its boundary-provenance helper return values are now documented explicitly.
- Concurrent page fetches now reuse one citation index for the same URL, and deep acyclic citation chains report a distinct depth-limit error instead of being mislabeled as circular.
- Disable the Collections background-index toggle with an explanation while the document scheduler is turned off.
- Elasticsearch availability checks now support IPv6 hosts, discard expired probe-cache entries, and cannot break successful engine construction when custom availability methods raise.
- Exclude log-export HEAD pre-flight checks from the per-user GET download quota.
- Existing databases now receive the composite
app_logs(research_id, timestamp, id)index used by ordered log reads and streaming exports. - Fixed Stop requests being ignored for the entire source-gathering phase of a topic-organization research.
TopicOrganizationStrategynever passed its progress callback on to the source-gathering strategy it delegates to, so all of the delegate's termination checks were silent no-ops and a click on Stop was only honoured once the delegate had completely finished (often minutes later). The callback is now propagated to both the main source-gathering delegate and the per-iteration refinement strategies. - Google Programmable Search now shares its minimum request-spacing state across engine instances using the same API key and CSE ID, so research-agent tool calls cannot bypass throttling by constructing a fresh engine per request.
- Honor per-call OpenAI-compatible endpoint URL overrides in research and benchmark paths.
- Keep the background-sweep toggle ON when a successful enable response has an empty or unparseable body.
- Per-user encrypted databases now follow a runtime
LDR_DATA_DIRoverride even when the database manager was imported earlier, preventing files from leaking into the platform-default application-data directory. - Prevent duplicate log rows from appearing while the log panel switches between research sessions.
- Queueing or cancelling research no longer risks a corrupted database session: if a commit fails inside
QueueManager.add_to_queue/remove_from_queue, the transaction is now rolled back before the error propagates, so a subsequent operation on that session doesn't fail with a masking error. Completes the rollback coverage of #2055 for the queue manager (thequeue_serviceandprocessor_v2halves landed earlier in #2414, #3481, #3506, #3521). - RAG configuration changes now update saved settings and the selected index atomically, preventing searches from observing mismatched configuration state.
- Reject invalid Notes API control values and stale similar-passage note IDs instead of executing unintended operations.
- Resolve non-canonical local-library references emitted by research agents instead of sending them to the egress-denial path.
- Return HTTP 404 from the history log-count endpoint when the research does not exist.
- Return a clear validation error when collection creation receives a non-string collection type.
- The Werkzeug client-disconnect log filter now suppresses
write() before start_responseerrors only when the record also identifies anAssertionError, so unrelated server errors containing the same substring remain visible.
📝 Other Changes
- Compute collection document totals and indexed totals in one grouped query instead of lazy-loading document links per collection. (#4856)
- Clarified in Settings UI descriptions that
search.final_max_resultsandsearch.cross_engine_max_resultsapply to pipeline/source-based strategies, while autonomous agent strategies (langgraph-agent) manage context autonomously. (#5380) - CI:
test_settings_routes_coverage.pynow runs in the parallel pytest batch again; the dedicated serial Docker-tests step (a #4397 workaround) is removed now that the queue-processor (#5055) and news-scheduler daemons are disabled under pytest.