~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
fix(security): render the token-metrics warning context as keys, not values (#6690)
LearningCircuita6c0d33ba631
diff --git a/changelog.d/+token-metrics-context-keys.security.md b/changelog.d/+token-metrics-context-keys.security.mdnew file mode 100644index 0000000000..af7f11c02d--- /dev/null+++ b/changelog.d/+token-metrics-context-keys.security.md@@ -0,0 +1 @@+The token-metrics warning for a missing username on background-thread saves now renders the research context's **keys** instead of dumping the whole dict. The context is the search context in production flows, which carries `user_password` and a settings snapshot that can hold engine credentials, and no logger in this codebase redacts values inside a log message — so nothing would have scrubbed the old line regardless of which logger it went through. No current caller builds a password-bearing context without `username`, so this hardens a latent leak before it becomes a real one; the diagnostic still names the context keys so operators can see why the save failed.diff --git a/src/local_deep_research/metrics/token_counter.py b/src/local_deep_research/metrics/token_counter.pyindex 0436ce27dd..6aa9f0031e 100644--- a/src/local_deep_research/metrics/token_counter.py+++ b/src/local_deep_research/metrics/token_counter.py@@ -772,10 +772,37 @@ class TokenCountingCallback(BaseCallbackHandler):)if not username:+ # Render the context KEYS only, never the values: this dict+ # is the search context in production flows, which carries+ # user_password ("for metrics tracking", see+ # api/research_functions.py) and a settings_snapshot that can+ # hold engine credentials. No logger in this codebase redacts+ # values inside a log message — the secure_logging.logger+ # wrapper only gates exception() tracebacks behind diagnose+ # mode (every other call, including .warning(), still+ # forwards straight to loguru unmodified), and the global+ # loguru patcher only strips control characters. A value+ # must never be interpolated into a log message unless it+ # has already been passed through redact_secrets()/+ # scrub_error() at the call site, so we log key names only.+ # Those names are fixed literals the code assigns (username,+ # user_password, settings_snapshot, research_query, ...; see+ # research_service.py's shared_research_context and+ # chat/context.py's build_research_context), never+ # caller-supplied data, which is why logging them here is+ # safe despite the "do not log context keys" caution at+ # research_service.py:1186 — that guards against logging+ # arbitrary values reachable under those keys, not the key+ # names themselves.+ context_keys = (+ sorted(map(str, self.research_context))+ if self.research_context+ else []+ )logger.warning(f"Cannot save token metrics - no username in research context. "f"Token usage: prompt={prompt_tokens}, completion={completion_tokens}, "- f"Research context: {self.research_context}"+ f"Research context keys: {context_keys}")returndiff --git a/tests/security/test_token_counter_context_redaction.py b/tests/security/test_token_counter_context_redaction.pynew file mode 100644index 0000000000..3b8cefb2c0--- /dev/null+++ b/tests/security/test_token_counter_context_redaction.py@@ -0,0 +1,82 @@+"""TokenCounter's no-username warning must not dump the raw research context.++``TokenCountingCallback`` carries the full ``research_context`` dict — in+production flows that dict is the search context, which includes+``user_password`` (``api/research_functions.py`` puts it there "for metrics+tracking") and a ``settings_snapshot`` that can carry engine credentials.+When a metrics save runs on a background thread and the context has no+``username`` key, ``token_counter._save_to_db`` logs the ENTIRE dict via a+f-string on a raw ``loguru`` logger (``from loguru import logger``). No+logger in this codebase redacts values inside a log message —+``security.secure_logging``'s wrapper only gates ``exception()``+tracebacks behind diagnose mode and forwards every other call (including+a ``.warning()`` like this one) straight to loguru unmodified — so nothing+would have scrubbed this line even had it gone through that wrapper:++ Cannot save token metrics - no username in research context.+ ... Research context: {self.research_context}++No current caller builds a password-bearing context without ``username``, so+this is a latent hazard rather than a demonstrated leak — but the sink is one+forgotten ``username`` key away from writing the plaintext password (and the+snapshot's secrets) into server logs. The pins here keep the diagnostic while+forcing value-free rendering of the context.+"""++import threading++from local_deep_research.metrics.token_counter import TokenCounter++_SECRET = "SuperSecretPassword-7f3a"+++def _run_background_save(research_context):+ """Drive _save_to_db on a non-main thread; return when it finishes."""+ callback = TokenCounter().create_callback(+ "redaction-probe-1", research_context+ )++ def worker():+ callback._save_to_db(5, 7)++ thread = threading.Thread(target=worker, name="metrics-worker")+ thread.start()+ thread.join()+++def test_no_username_warning_does_not_log_context_values(loguru_caplog):+ """A password-bearing, username-less context must never have its values+ written to the log — the warning may fire, but rendered without values.+ RED on main: the f-string dumps the whole dict verbatim."""+ context = {+ "user_password": _SECRET,+ "research_query": "benign query",+ }+ with loguru_caplog.at_level("DEBUG"):+ _run_background_save(context)++ assert _SECRET not in loguru_caplog.text, (+ "the no-username metrics warning logged the raw research context, "+ "including the plaintext user_password"+ )+++def test_no_username_warning_still_fires_with_the_diagnostic(loguru_caplog):+ """The fix must not silence the diagnostic itself: the warning still+ fires and still says which context keys were present (so operators can+ see *why* the save failed) — just never the values."""+ context = {+ "user_password": _SECRET,+ "research_query": "benign query",+ }+ with loguru_caplog.at_level("DEBUG"):+ _run_background_save(context)++ warning_text = loguru_caplog.text+ assert "Cannot save token metrics" in warning_text, (+ "the missing-username diagnostic disappeared entirely"+ )+ assert "user_password" in warning_text, (+ "the diagnostic no longer names the context keys"+ )+ assert _SECRET not in warning_text