~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
fix(citations): stop a streamed subsection that repeats its own headings (#6457)
Lukase87159fcdf29
diff --git a/changelog.d/6457.bugfix.md b/changelog.d/6457.bugfix.mdnew file mode 100644index 0000000000..f3690fd94d--- /dev/null+++ b/changelog.d/6457.bugfix.md@@ -0,0 +1,14 @@+A streamed subsection that falls into a degenerate loop is now abandoned+instead of running until the context window clips it. A local model that+misses its EOS token can attend back to what it just wrote and reproduce the+same section cycle 30 or more times, and the whole repeated block landed in the+report (#6452). The stream stops once the model has stopped producing new+headings and is replaying ones it already emitted; everything written before+the loop is kept, and the abort is logged so the model, not the truncation, is+what an operator sees.++A repeated heading on its own is not the signal. A comparative report repeats+subsection titles by design, once per item, so the guard also requires a run of+headings the model has already emitted, which any previously-unseen heading+resets. Headings inside a fenced code block are not counted at all, since+`# comment` opens most shell and Python snippets.diff --git a/src/local_deep_research/citation_handlers/base_citation_handler.py b/src/local_deep_research/citation_handlers/base_citation_handler.pyindex 767ad3839c..3601953a95 100644--- a/src/local_deep_research/citation_handlers/base_citation_handler.py+++ b/src/local_deep_research/citation_handlers/base_citation_handler.py@@ -3,7 +3,7 @@ Base class for all citation handlers."""from abc import ABC, abstractmethod-from typing import Any, Callable, Dict, List, Optional, Union+from typing import Any, Callable, Dict, List, Optional, Sequence, Unionfrom langchain_core.documents import Documentfrom loguru import logger@@ -12,6 +12,107 @@ from ..utilities.type_utils import unwrap_settingfrom ..utilities.json_utils import _coerce_content_blocks, get_llm_response_text+class _HeadingRepetitionGuard:+ """Detects a model re-emitting its own headings instead of stopping.++ A local model that misses its EOS token can attend back to the start of+ what it just wrote and reproduce the same section cycle until the context+ window hard-clips — 30+ iterations of identical subheadings, tables and+ prose, all of which then land in the report (#6452). Nothing was watching+ the stream, so the only bound was the context window.++ The signal is a markdown heading line repeating verbatim. It is chosen+ over n-gram or block similarity because it is O(1) per line, needs no+ buffer of the text so far, and is nearly impossible to trip by accident:+ a legitimate subsection does not emit one identical ``### Heading`` four+ times. Prose repetition without repeated headings is NOT caught — the+ guard is a bound on the reported failure, not a general degeneration+ detector, and a rule loose enough to catch every loop would truncate real+ documents.++ Headings are compared after stripping trailing whitespace and leading+ ``#``/space, so ``## Costs`` and ``### Costs`` count together: the loop+ reproduces its own text, and a level change between cycles is still the+ same cycle.++ A repeated heading alone is not the signal, because a legitimate+ comparative report repeats them by design: ``## Study A`` /+ ``### Methodology`` / ``### Results``, then ``## Study B`` and the same+ two subsections again. A four-item report reaches four+ ``### Methodology`` with nothing degenerate happening, and truncating it+ is exactly the silent quality loss this guard exists to prevent.++ What separates the two is whether the document is still PRODUCING. A+ comparative report emits a heading nobody has seen before at every item+ (``## Study B``); a loop has stopped emitting new ones entirely and only+ replays what it already wrote. So the abort needs both a heading at+ ``LIMIT`` and a run of ``CONSECUTIVE_KNOWN`` headings in a row that the+ model has already emitted, and any previously-unseen heading resets that+ run. Both are O(1) per line and need no buffer of the text so far.++ That also bounds a loop repeating one heading and nothing else: every+ occurrence after the first is a known heading, so the run climbs and the+ abort follows a cycle later than for a two-heading loop.++ Lines inside a fenced code block are not headings. ``# comment`` is the+ first line of half the bash and Python snippets there are, and a technical+ report quoting four similar config blocks would otherwise trip the guard.++ Chunk boundaries fall anywhere, so a partial last line is held back until+ its newline arrives rather than being counted as its own heading.+ """++ #: Occurrences of one heading before it can end the stream. Three is+ #: reachable by a document that legitimately revisits a title (a summary+ #: repeating a section name); four is not, and a loop reaches it in two+ #: cycles.+ LIMIT = 4++ #: Headings in a row that the model has already emitted. A comparative+ #: report breaks this run at every item with a heading nobody has seen+ #: before; a loop cannot, because it has stopped producing new ones.+ CONSECUTIVE_KNOWN = 4++ def __init__(self) -> None:+ self._counts: Dict[str, int] = {}+ self._pending = ""+ self._in_fence = False+ self._consecutive_known = 0++ def feed(self, texts: Sequence[str]) -> Optional[str]:+ """Consume newly streamed text; return the repeated heading, or None."""+ for text in texts:+ self._pending += text+ *lines, self._pending = self._pending.split("\n")+ for line in lines:+ repeated = self._count(line)+ if repeated is not None:+ return repeated+ return None++ def _count(self, line: str) -> Optional[str]:+ stripped = line.strip()+ if stripped.startswith("```"):+ self._in_fence = not self._in_fence+ return None+ if self._in_fence or not stripped.startswith("#"):+ return None+ heading = stripped.lstrip("#").strip()+ if not heading:+ return None+ seen = self._counts.get(heading, 0) + 1+ self._counts[heading] = seen+ if seen == 1:+ self._consecutive_known = 0+ return None+ self._consecutive_known += 1+ if seen < self.LIMIT:+ return None+ if self._consecutive_known < self.CONSECUTIVE_KNOWN:+ return None+ return heading++class BaseCitationHandler(ABC):"""Abstract base class for citation handlers."""@@ -25,6 +126,10 @@ class BaseCitationHandler(ABC):"""Set a callback that receives each streamed LLM token."""self.stream_callback = callback+ def _new_repetition_guard(self) -> "_HeadingRepetitionGuard":+ """The streaming degeneration guard, as a seam a subclass can widen."""+ return _HeadingRepetitionGuard()+def _handle_chunk(self, chunk: Any, chunks: List[str]) -> None:"""Normalize one streamed chunk, record it and push it to the callback.@@ -53,6 +158,18 @@ class BaseCitationHandler(ABC):"stream_callback failed") # Non-critical: don't break synthesis+ def _log_repetition_abort(self, heading: str, chunks: List[str]) -> None:+ """One line a reader can act on: what repeated, and how much was kept."""+ logger.warning(+ "Stream aborted after a degenerate repetition loop: the heading "+ "{!r} was emitted {} times. Keeping the {} chunk(s) produced so "+ "far; the model was re-emitting its own output instead of "+ "stopping.",+ heading,+ _HeadingRepetitionGuard.LIMIT,+ len(chunks),+ )+def _join_chunks(self, chunks: List[str]) -> str:"""Normalize the joined chunks exactly like the invoke() path does.@@ -101,9 +218,15 @@ class BaseCitationHandler(ABC):"""if self.stream_callback and hasattr(self.llm, "stream"):chunks: List[str] = []+ guard = self._new_repetition_guard()try:for chunk in self.llm.stream(prompt):+ before = len(chunks)self._handle_chunk(chunk, chunks)+ repeated = guard.feed(chunks[before:])+ if repeated is not None:+ self._log_repetition_abort(repeated, chunks)+ breakreturn self._join_chunks(chunks)except Exception:partial = self._partial_after_stream_failure(chunks)@@ -129,9 +252,15 @@ class BaseCitationHandler(ABC):"""if self.stream_callback and hasattr(self.llm, "astream"):chunks: List[str] = []+ guard = self._new_repetition_guard()try:async for chunk in self.llm.astream(prompt):+ before = len(chunks)self._handle_chunk(chunk, chunks)+ repeated = guard.feed(chunks[before:])+ if repeated is not None:+ self._log_repetition_abort(repeated, chunks)+ breakreturn self._join_chunks(chunks)except Exception:partial = self._partial_after_stream_failure(chunks)diff --git a/tests/citation_handlers/test_async_cores.py b/tests/citation_handlers/test_async_cores.pyindex cef345650c..ccdbe6dab8 100644--- a/tests/citation_handlers/test_async_cores.py+++ b/tests/citation_handlers/test_async_cores.py@@ -299,3 +299,49 @@ class TestStreamContentBlocks:assert received == ["partial ", "answer"]llm.invoke.assert_not_called()llm.ainvoke.assert_not_called()+++class TestAsyncStreamRepetitionGuard:+ """The async path carries the same guard as the sync one (#6452).++ Worth its own cell: the two streaming loops are separate code, and a fix+ applied to one of them looks complete from the other's tests.+ """++ _CYCLE = [+ "### High-Speed Intercity Express Routing\n",+ "Prose about routing and station placement.\n",+ "### Electrification Infrastructure\n",+ "Prose about grid interconnect standards.\n",+ ]++ @pytest.mark.asyncio+ async def test_a_repeating_cycle_stops_the_async_stream(self):+ loop = self._CYCLE * 30+ llm = _dual_api_llm(chunks=loop)+ handler = StandardCitationHandler(llm=llm)+ handler.set_stream_callback(Mock())++ text = await handler._invoke_with_streaming_async("prompt")++ assert text.count("High-Speed Intercity Express Routing") <= 4+ assert len(text) < len("".join(loop)) / 2+ # The usable first cycle survives; only the loop is dropped.+ assert "Prose about routing and station placement." in text++ @pytest.mark.asyncio+ async def test_an_ordinary_async_document_streams_whole(self):+ chunks = [+ "# Report\n",+ "## Costs\n",+ "Body.\n",+ "## Summary\n",+ "Body.\n",+ ]+ llm = _dual_api_llm(chunks=chunks)+ handler = StandardCitationHandler(llm=llm)+ handler.set_stream_callback(Mock())++ text = await handler._invoke_with_streaming_async("prompt")++ assert text == "".join(chunks).strip()diff --git a/tests/citation_handlers/test_streaming.py b/tests/citation_handlers/test_streaming.pyindex 066e5f06a0..7b2e3fde6b 100644--- a/tests/citation_handlers/test_streaming.py+++ b/tests/citation_handlers/test_streaming.py@@ -634,3 +634,209 @@ class TestStreamingIntegrationWithCitationHandler:assert handler._handler.stream_callback == callback, (f"Handler type '{handler_type}' should support streaming")+++class TestStreamRepetitionGuard:+ """A local model that misses its EOS token re-emits its own sections until+ the context window hard-clips, and the whole ~40k-token repeated block+ lands in the report (#6452). Nothing watched the stream, so the context+ window was the only bound.+ """++ def _handler(self, chunks):+ from local_deep_research.citation_handlers.standard_citation_handler import (+ StandardCitationHandler,+ )++ llm = MockLLMWithStreaming(chunks=chunks)+ handler = StandardCitationHandler(llm=llm)+ handler.set_stream_callback(Mock())+ return handler, llm++ def _loop(self, cycles):+ """The reported shape: one subsection's headings, re-emitted verbatim."""+ cycle = [+ "### High-Speed Intercity Express Routing\n",+ "Prose about routing and station placement.\n",+ "### Electrification Infrastructure\n",+ "Prose about grid interconnect standards.\n",+ ]+ return cycle * cycles++ def test_a_repeating_cycle_stops_the_stream(self):+ handler, llm = self._handler(self._loop(30))++ text = handler._invoke_with_streaming("prompt")++ # Four occurrences is the limit, so the fourth one ends it: what+ # survives is far short of the 30 cycles the model wanted to emit.+ assert text.count("High-Speed Intercity Express Routing") <= 4+ assert len(text) < len("".join(self._loop(30))) / 2++ def test_what_was_already_produced_is_kept(self):+ """A truncated subsection beats 40k tokens of repetition, and beats+ discarding the usable first cycle along with the loop."""+ handler, _llm = self._handler(self._loop(30))++ text = handler._invoke_with_streaming("prompt")++ assert "Prose about routing and station placement." in text+ assert "### Electrification Infrastructure" in text++ def test_an_ordinary_document_streams_whole(self):+ """The guard must not truncate real output. Distinct headings, a+ heading repeated twice (a summary naming a section), and prose that+ repeats verbatim all have to survive."""+ chunks = [+ "# Report\n",+ "## Costs\n",+ "Identical boilerplate sentence.\n",+ "## Integration\n",+ "Identical boilerplate sentence.\n",+ "## Summary\n",+ "Revisiting Costs and Integration.\n",+ "## Costs\n",+ "Identical boilerplate sentence.\n",+ ]+ handler, _llm = self._handler(chunks)++ text = handler._invoke_with_streaming("prompt")++ # The join is normalized exactly as the invoke() path is, which strips+ # the trailing newline; nothing else may be missing.+ assert text == "".join(chunks).strip()++ def test_a_heading_split_across_chunks_still_counts(self):+ """Chunk boundaries fall anywhere; a guard that counted each fragment+ as its own line would miss every real loop."""+ handler, _llm = self._handler(["### Cos", "ts\n### Ris", "ks\n"] * 8)++ text = handler._invoke_with_streaming("prompt")++ assert text.count("### Costs") <= 4++ def test_a_split_heading_is_one_heading_not_two_fragments(self):+ """The shape above trips either way — on ``Cos`` if fragments are+ counted separately, on ``Costs`` if the line is reassembled — so the+ NAME is what tells the two apart. Counting fragments would also mean a+ loop whose cycle happens to split differently each time is never seen.+ """+ from local_deep_research.citation_handlers.base_citation_handler import (+ _HeadingRepetitionGuard,+ )++ guard = _HeadingRepetitionGuard()+ results = [+ guard.feed(["### Cos", "ts\n### Ris", "ks\n"]) for _ in range(4)+ ]++ assert results[:3] == [None, None, None]+ assert results[3] == "Costs"++ def test_an_unterminated_final_line_is_not_a_heading(self):+ """The last fragment of a stream has no newline yet. Counting it would+ make the guard's verdict depend on where the provider happened to cut+ the last chunk."""+ from local_deep_research.citation_handlers.base_citation_handler import (+ _HeadingRepetitionGuard,+ )++ guard = _HeadingRepetitionGuard()++ assert [guard.feed(["### Costs"]) for _ in range(10)] == [None] * 10++ def test_the_level_of_a_repeated_heading_does_not_matter(self):+ """The loop reproduces its own text; a level change between cycles is+ still the same cycle."""+ handler, _llm = self._handler(+ [+ "# Costs\n### Risks\n",+ "## Costs\n#### Risks\n",+ "### Costs\n# Risks\n",+ "#### Costs\n##### Risks\n",+ "##### Costs\n## Risks\n",+ ]+ )++ text = handler._invoke_with_streaming("prompt")++ assert "##### Costs" not in text++ def test_a_parallel_structure_report_is_not_truncated(self):+ """The false positive a count-one-heading rule has. "Compare N+ papers" emits one subsection title per item with distinct siblings+ between them, so a four-item report reaches four ``### Methodology``+ with nothing degenerate happening, and a guard that counted that would+ abandon the report on its last item.++ A loop reproduces a CYCLE: two or more headings reach the limit+ together. That is the discriminator, and it costs nothing extra.+ """+ chunks = []+ for item in ("Alpha", "Beta", "Gamma", "Delta", "Epsilon", "Zeta"):+ chunks.append(f"## Study {item}\n")+ chunks.append("### Methodology\n")+ chunks.append(f"We ran {item} on the public split.\n")+ chunks.append("### Results\n")+ chunks.append(f"{item} scored well.\n")+ handler, _llm = self._handler(chunks)++ text = handler._invoke_with_streaming("prompt")++ assert "Study Zeta" in text+ assert text.count("### Methodology") == 6++ def test_a_repeated_heading_inside_a_code_fence_is_not_a_heading(self):+ """``# comment`` opens half the bash and Python snippets there are, and+ ``### Fake heading`` inside a fence is text. A technical report+ quoting four similar config blocks would otherwise be truncated.+ """+ chunks = []+ for index in range(6):+ chunks.append(f"Config {index}:\n")+ chunks.append("```bash\n")+ chunks.append("# Set the region\n")+ chunks.append("# Set the bucket\n")+ chunks.append("```\n")+ handler, _llm = self._handler(chunks)++ text = handler._invoke_with_streaming("prompt")++ assert "Config 5:" in text+ assert text.count("# Set the region") == 6++ def test_a_fence_does_not_hide_a_loop_that_follows_it(self):+ """Accept control for the fence tracking: the toggle has to close, or+ one stray fence in the prose would switch the guard off for the rest+ of the stream.+ """+ chunks = ["```bash\n# inside\n```\n"]+ chunks.extend(self._loop(30))+ handler, _llm = self._handler(chunks)++ text = handler._invoke_with_streaming("prompt")++ assert text.count("High-Speed Intercity Express Routing") <= 4+ assert len(text) < len("".join(self._loop(30))) / 2++ def test_the_abort_is_logged(self, monkeypatch):+ """A silently truncated subsection is worse than a long one: the log+ line is how an operator learns the model, not the guard, is the+ problem."""+ from local_deep_research.citation_handlers import base_citation_handler++ warnings = []+ monkeypatch.setattr(+ base_citation_handler.logger,+ "warning",+ lambda msg, *a, **k: warnings.append((msg, a)),+ )++ handler, _llm = self._handler(self._loop(30))+ handler._invoke_with_streaming("prompt")++ assert any("degenerate repetition loop" in msg for msg, _a in warnings)+ assert any(+ "High-Speed Intercity Express Routing" in str(a)+ for _msg, a in warnings+ )