Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Python aicode-complexitycode-healthcode-intelligencecode-qualitydead-codedeveloper-toolsdocumentationgit-analyticsmcpopen-sourcepythonrefactoringstatic-analysistechnical-debt
fix(answer): grade confidence on whether the top hit covers the question (#3716)
Swati Ahuja18893b373422
diff --git a/README.md b/README.mdindex 3ff55d7fe..e8a9c1f69 100644--- a/README.md+++ b/README.md@@ -598,7 +598,7 @@ knows how much to trust what it just read.| Tool | What only this tool answers ||---|---|| `get_overview()` | Architecture summary, module map, entry points, git health. The first call on an unfamiliar codebase. |-| `get_answer(question)` | Hybrid retrieval (full-text plus vector), graph expansion and one cited answer with a calibrated `retrieval_quality`. Search, read and reason in a single round-trip. |+| `get_answer(question)` | Hybrid retrieval (full-text plus vector), graph expansion and one cited answer with a single calibrated `confidence`. Search, read and reason in a single round-trip. || `get_context(targets, include?)` | Triage card for files, modules or symbols: summary, signatures, hotspot flag, governing decisions, symbol ids. `include` opens callers, callees, ownership and metrics. Batch many targets in one call. || `get_symbol("file.py::Name")` | One indexed symbol's source with exact line bounds. || `search_codebase(query)` | Hybrid search over code and docs, by symbol, path or concept. |diff --git a/docs/agent/MCP_TOOLS.md b/docs/agent/MCP_TOOLS.mdindex 1ff10f915..1da9a132f 100644--- a/docs/agent/MCP_TOOLS.md+++ b/docs/agent/MCP_TOOLS.md@@ -94,7 +94,7 @@ Every tool returns a JSON object with a `_meta` envelope. Most fields appear onl| `omitted` | Refs to content cut for size. Restore with `repowise expand <ref>` or `get_symbol("repowise#<ref>")`. || `response_budget` | The character ceiling that applied and the size delivered. Present when the budget cut something. |-`get_answer` adds a top-level `confidence` (rates the prose) and `retrieval_quality` (rates the evidence). Diagnostics such as `timing_ms` and `contract_version` ride only on `get_overview`; `REPOWISE_MCP_DEBUG_META=1` restores them everywhere. Full envelope: [MCP_RESPONSE_FIELDS.md](../reference/MCP_RESPONSE_FIELDS.md#the-_meta-envelope).+`get_answer` adds a top-level `confidence`, its one trust signal, graded from the retrieval and from whether the top hit covers what the question asks; `include=["diagnostics"]` adds `retrieval_quality`, the rating of the retrieval alone. Diagnostics such as `timing_ms` and `contract_version` ride only on `get_overview`; `REPOWISE_MCP_DEBUG_META=1` restores them everywhere. Full envelope: [MCP_RESPONSE_FIELDS.md](../reference/MCP_RESPONSE_FIELDS.md#the-_meta-envelope).### Reversible truncation: `_meta.omitted`@@ -147,7 +147,7 @@ Answers a how, where or why question in one call: it runs hybrid retrieval over| `repo` | string | default repo | Workspace repo alias. `"all"` is not supported || `include` | list[string] | none | `["evidence"]` returns the full evidence projection with a larger budget |-**Key return fields:** `answer`, `confidence` (`high` / `medium` / `low`, rates the prose), `retrieval_quality` (`high` / `partial` / `weak`, rates the evidence), `citations`, `symbol_bodies` (live bodies of symbols the answer names), `retrieval`, `best_guesses` and `fallback_targets` (on low confidence; a low `best_guesses` row carries `functions`, `lines` and `size_bytes` instead of a page excerpt), `candidate_files` (ranked file paths the citations do not already name: up to 3 at `high`, 5 otherwise), `episodes` (dated facts bearing on the question), `degraded` (synthesis could not run), `_meta.scope_hint` (areas the answer did not touch).+**Key return fields:** `answer`, `confidence` (`high` / `medium` / `low`), `citations`, `symbol_bodies` (live bodies of symbols the answer names), `retrieval`, `best_guesses` and `fallback_targets` (on low confidence; a low `best_guesses` row carries `functions`, `lines` and `size_bytes` instead of a page excerpt), `candidate_files` (ranked file paths the citations do not already name: up to 3 at `high`, 5 otherwise), `episodes` (dated facts bearing on the question), `degraded` (synthesis could not run), `_meta.scope_hint` (areas the answer did not touch).A `high` answer can be cited directly. On `low`, read the rows the reply names, then `candidate_files`, before searching again. Without an LLM provider the tool still answers from retrieval, marked `degraded`. When retrieval is also `weak`, the reply serves `best_guesses` and `candidate_files` with one guidance line in `answer`, and no `citations`.diff --git a/docs/reference/MCP_RESPONSE_FIELDS.md b/docs/reference/MCP_RESPONSE_FIELDS.mdindex c80bc1787..cf72f1cd8 100644--- a/docs/reference/MCP_RESPONSE_FIELDS.md+++ b/docs/reference/MCP_RESPONSE_FIELDS.md@@ -124,15 +124,15 @@ Every suggested follow-up (`get_health`'s `fix_first.lead.next_call`, the refact| Field | Meaning ||-------|---------|| `answer` | The synthesised answer, with file and symbol citations |-| `confidence` | `high`, `medium` or `low`: rates the prose. `high` is content-grounded and can be cited directly. A why-answer whose named mechanism is absent from the retrieved source is downgraded to `medium` |-| `retrieval_quality` | `high`, `partial` or `weak`: rates the evidence under the prose |+| `confidence` | `high`, `medium` or `low`: the one trust signal. `high` is content-grounded, on a top hit that covers what the question asks, and can be cited directly. A why-answer whose named mechanism is absent from the retrieved source is downgraded to `medium` |+| `retrieval_quality` | Only under `include=["diagnostics"]`. `high`, `partial` or `weak`: rates the retrieval alone, and is `weak` when the top hit leaves out most of what the question asks || `symbol_bodies` | Live bodies of the symbols the answer names. Read these before calling `get_symbol` || `retrieval` | Evidence rows (summary, snippet, key symbols). Shrinks as confidence rises || `candidate_files` | Ranked file paths retrieval resolved, minus those already in `citations`: up to 3 at `high`, 5 at `medium`, `low` or `degraded`. Navigation, not evidence. Served at every confidence; absent when retrieval resolved no file || `candidates` | The top 5 of those files as `{path, lines?, defines?}` rows. Only with `include=["evidence"]` || `best_guesses`, `fallback_targets` | On low confidence: where to look, each with a one-line reason. At `low` or `degraded` a `best_guesses` row drops its page `excerpt` and carries `functions` (up to 3 `{name, line}` symbols relevant to the question) and the live file's `lines` and `size_bytes`; `include=["evidence"]` brings the excerpt back || `episodes` | A dated fact recorded about this checkout that bears on the question; `still_true` says how current it is |-| `degraded` | Synthesis could not run (no provider, or the call failed). The answer is assembled from retrieval and mined rationale with no LLM, and `confidence` is graded from the retrieval: `medium` unless `retrieval_quality` is `weak`, never `high` |+| `degraded` | Synthesis could not run (no provider, or the call failed). The answer is assembled from retrieval and mined rationale with no LLM, and `confidence` is graded from the retrieval: `medium` on a dominant top hit that covers the question, else `low`, never `high` |How much evidence rides along depends on the grade. `high` keeps one quote, and one symbol body when the answer is not already grounded. `medium` keeps one body, two quotes and the top two evidence rows. `low` keeps two bodies and the top three rows. `include=["evidence"]` skips this trimming.diff --git a/docs/reference/TELEMETRY.md b/docs/reference/TELEMETRY.mdindex 82c4da65f..c0a369228 100644--- a/docs/reference/TELEMETRY.md+++ b/docs/reference/TELEMETRY.md@@ -74,7 +74,7 @@ response, **never the question, query, results, paths, or repo/symbol names**:| `properties.status` | `ok` / `error` | Tool error rate || `properties.duration_ms` | `820` | Tool latency in the wild || `properties.confidence` | `high` / `medium` / `low` | Answer confidence distribution |-| `properties.retrieval_quality` | `strong` / `weak` | Retrieval quality distribution |+| `properties.retrieval_quality` | `high` / `partial` / `weak` | Retrieval quality distribution; sent only when the call asked for `include=["diagnostics"]` || `properties.results_bucket` | `1-3`, `4-10` | Result count for searches (a range) || `properties.index_behind` | `true` / `false` | How often served content is stale || `properties.embedder_degraded` | `true` / `false` | How often search runs on mock vectors |diff --git a/packages/cli/src/repowise/cli/commands/ask_cmd.py b/packages/cli/src/repowise/cli/commands/ask_cmd.pyindex 027007a2d..e205536d9 100644--- a/packages/cli/src/repowise/cli/commands/ask_cmd.py+++ b/packages/cli/src/repowise/cli/commands/ask_cmd.py@@ -31,7 +31,7 @@ def project(payload: dict, question: str) -> dict:================== ===========================================kept answer, confidence, grounding,- retrieval_quality, citations, quotes,+ citations, quotes,fallback_targets, candidate_files,best_guesses (without theirexcerpts), next_action_hint, note,@@ -52,7 +52,6 @@ def project(payload: dict, question: str) -> dict:"question": question,"answer": payload.get("answer", ""),"confidence": payload.get("confidence", ""),- "retrieval_quality": payload.get("retrieval_quality", ""),"citations": list(payload.get("citations") or []),}quotes = payload.get("quotes") or []@@ -181,13 +180,11 @@ def _render(projected: dict) -> None:console.print("[yellow]No answer synthesised for that question.[/yellow]")confidence = projected.get("confidence") or "?"- quality = projected.get("retrieval_quality") or "?"colour = {"high": "green", "medium": "yellow"}.get(confidence, "red")grounding = projected.get("grounding")console.print(- f"\n[{colour}]confidence: {confidence}[/{colour}] [dim]retrieval: {quality}"- + (f" grounding: {grounding}" if grounding else "")- + "[/dim]"+ f"\n[{colour}]confidence: {confidence}[/{colour}]"+ + (f" [dim]grounding: {grounding}[/dim]" if grounding else ""))if projected.get("note"):# The value-grounding gate ("the numbers may be synthesised, read Xdiff --git a/packages/core/src/repowise/core/generation/editor_files/tool_table.py b/packages/core/src/repowise/core/generation/editor_files/tool_table.pyindex e347840eb..907d552ad 100644--- a/packages/core/src/repowise/core/generation/editor_files/tool_table.py+++ b/packages/core/src/repowise/core/generation/editor_files/tool_table.py@@ -65,11 +65,10 @@ TOOL_TABLE_ROWS: dict[str, tuple[str, str]] = {# question with `confidence: "low"`, since confidence rates prose that# was never synthesised, and this row told the agent to distrust the# payload on the strength of it, so it re-searched after every call.- # `retrieval_quality` is the field that rates what such a payload does- # carry. Reworded, not lengthened: 186 chars against the 179 it replaced.+ # `confidence` there is graded from the evidence such a payload does carry."First call for any how/where/why question. "+ _KEYED_ONLY["get_answer"]- + "`degraded` means judge by `retrieval_quality`. `symbol_bodies` has live bodies.",+ + "`degraded` means no prose; `confidence` rates the evidence. `symbol_bodies` has live bodies.",),"get_context": ("get_context(targets=[...])",diff --git a/packages/server/src/repowise/server/mcp_server/_code_rationale.py b/packages/server/src/repowise/server/mcp_server/_code_rationale.pyindex 05364f974..91ff11c05 100644--- a/packages/server/src/repowise/server/mcp_server/_code_rationale.py+++ b/packages/server/src/repowise/server/mcp_server/_code_rationale.py@@ -206,6 +206,17 @@ def _keep(return len(matched) >= 2+# A mined row may stand in for the answer only on this many distinct matches+# (question terms or numbers). A marker plus one shared word clears `_keep`+# ("FIXME ... unused by cobra" for a question naming cobra) but answers nothing.+_LEAD_MIN_MATCHES = 2+++def can_lead_answer(row: dict) -> bool:+ """Whether a mined rationale row matches enough of the question to be the answer."""+ return len(set(row.get("matched_terms") or ())) >= _LEAD_MIN_MATCHES++def _read_text(repo_root: Path, file_path: str) -> str | None:"""Read a repo file's live text, refusing paths outside the root."""try:diff --git a/packages/server/src/repowise/server/mcp_server/_meta.py b/packages/server/src/repowise/server/mcp_server/_meta.pyindex 95fc20a2c..c7887b259 100644--- a/packages/server/src/repowise/server/mcp_server/_meta.py+++ b/packages/server/src/repowise/server/mcp_server/_meta.py@@ -938,7 +938,7 @@ def answer_hint(doubt its files on every question is what made a keyless agent callsearch_codebase after every get_answer: one question, two calls, measured on11 of 26 questions whose rank-1 file was right. So say which half is missing- and let `retrieval_quality` rate the other half.+ and let `confidence` rate the other half."""if degraded:if retrieval_quality == "weak":@@ -953,11 +953,11 @@ def answer_hint(if has_bodies:return ("Synthesis is what is missing here, not retrieval. Answer from "- "symbol_bodies; retrieval_quality rates what was served."+ "symbol_bodies.")return (- "Synthesis is what is missing here, not retrieval. retrieval_quality "- "rates the ranked hits; start from the first one."+ "Synthesis is what is missing here, not retrieval. Start from the "+ "first ranked hit.")if confidence == "low":return (diff --git a/packages/server/src/repowise/server/mcp_server/_retrieval_rank.py b/packages/server/src/repowise/server/mcp_server/_retrieval_rank.pyindex a67832af3..186b9647e 100644--- a/packages/server/src/repowise/server/mcp_server/_retrieval_rank.py+++ b/packages/server/src/repowise/server/mcp_server/_retrieval_rank.py@@ -23,6 +23,12 @@ _INFLECTIONS = {}+def window_term_weight(df: int, n: int) -> float:+ """Weight of a term that *df* of *n* retrieved hits carry: 1 when every hit+ carries it, more the fewer do, so the word that tells hits apart leads."""+ return 1.0 + math.log((n + 1) / (df + 1))++def _ranking_terms(text: str, *, max_terms: int = 8) -> set[str]:"""Content terms with a tiny retrieval-only inflection normalisation."""return {_INFLECTIONS.get(term, term) for term in content_terms(text, max_terms=max_terms)}@@ -103,8 +109,7 @@ def rerank_by_context_coverage(term: sum(1 for tokens in tokens_by_hit if term in tokens) for term in terms}weights = {- term: 1.0 + math.log((len(hits) + 1) / (document_frequency[term] + 1))- for term in terms+ term: window_term_weight(document_frequency[term], len(hits)) for term in terms}total_weight = sum(weights.values())diff --git a/packages/server/src/repowise/server/mcp_server/tool_answer/answer.py b/packages/server/src/repowise/server/mcp_server/tool_answer/answer.pyindex a287295b6..7994a8c0d 100644--- a/packages/server/src/repowise/server/mcp_server/tool_answer/answer.py+++ b/packages/server/src/repowise/server/mcp_server/tool_answer/answer.py@@ -138,6 +138,7 @@ from repowise.server.mcp_server.tool_answer.confidence import (_is_value_question,_retrieval_quality,dominance_reason,+ uncovered_terms,)from repowise.server.mcp_server.tool_answer.config import ( # noqa: F401_GATED_EXCERPT_CHARS,@@ -388,9 +389,9 @@ async def get_answer(whose named mechanism is absent from the retrieved source is downgradedto medium (the rationale may be conflated). Low confidence returnsbest_guesses with one-line justifications instead of an empty answer.- retrieval_quality separately rates the retrieval that fed synthesis; when- it reads "weak" beside confidence=high, the note says what the confidence- rests on instead of the ranking, and that is the claim to trust.+ confidence is the one trust signal: it also asks whether the top hit+ covers what was asked. retrieval_quality, the rating of the retrieval+ alone, is served only under ``include=["diagnostics"]``.When the answer names a function/method/class, ``symbol_bodies`` carriesits full live body — read that instead of a follow-up get_symbol.``episodes``, when present, is a dated fact recorded about this checkout@@ -405,7 +406,8 @@ async def get_answer(question: developer question.scope: optional path-prefix filter (e.g. "src/pkg/").repo: usually omitted.- include: optional ``["evidence"]`` expanded projection.+ include: optional ``["evidence"]`` expanded projection;+ ``["diagnostics"]`` adds ``retrieval_quality``."""if repo == "all":return _unsupported_repo_all("get_answer")@@ -534,7 +536,7 @@ async def get_answer(ctx,repository,t0,- _retrieval_quality(hits, agreement_dominant),+ _retrieval_quality(hits, agreement_dominant, question),)if union_payload is not None:return _with_candidates(@@ -708,6 +710,8 @@ async def get_answer(_gather_body_candidates(hits, answer_text), repo_root)+ # One coverage measurement for the grade and the retrieval rating.+ uncovered = uncovered_terms(question, hits)grade = _grade_answer(question=question,question_ids=question_ids,@@ -717,9 +721,10 @@ async def get_answer(symbol_bodies=symbol_bodies,served_named_body=served_named_body,dominance=dominance,+ uncovered=uncovered,)confidence = grade.confidence- retrieval_quality = _retrieval_quality(hits, agreement_dominant)+ retrieval_quality = _retrieval_quality(hits, agreement_dominant, question, uncovered)payload = await build_synthesized_payload(question=question,@@ -830,7 +835,8 @@ async def get_answer(question: Developer question.scope: Optional repository-relative path prefix.repo: Usually omitted; a workspace alias when needed.- include: Optional ``["evidence"]`` expanded projection.+ include: Optional ``["evidence"]`` expanded projection;+ ``["diagnostics"]`` adds ``retrieval_quality``."""return await _projected_get_answer(question=question,diff --git a/packages/server/src/repowise/server/mcp_server/tool_answer/confidence.py b/packages/server/src/repowise/server/mcp_server/tool_answer/confidence.pyindex 57c9139c5..ad752e9de 100644--- a/packages/server/src/repowise/server/mcp_server/tool_answer/confidence.py+++ b/packages/server/src/repowise/server/mcp_server/tool_answer/confidence.py@@ -1,9 +1,10 @@"""Confidence and retrieval_quality grading for get_answer.-Two ratings that answer different questions. ``confidence`` says how much to-trust the synthesised text; ``retrieval_quality`` says how good the retrieval-that fed it was. The agent reads the first to decide whether to re-read the-source, the second to decide whether to search again.+``confidence`` is the one trust signal an agent reads: how much to trust the+answer, from how the retrieval went and whether its top hit covers what was+asked. ``retrieval_quality`` rates the retrieval alone; it feeds the degraded+grade and is served only under ``include=["diagnostics"]``, so two ratings of+one answer never disagree in front of the agent.:func:`_grade_answer` runs the gate cascade — one starting grade from retrievaldominance, then a run of gates that can only demote it — and the predicates each@@ -22,9 +23,11 @@ from repowise.server.mcp_server._answer_context import (is_why_question as _is_why_question,)from repowise.server.mcp_server._meta import _normalize_target_path+from repowise.server.mcp_server._retrieval_rank import window_term_weightfrom repowise.server.mcp_server.tool_answer.config import (_AGREEMENT_RANK_GAP,_AGREEMENT_TOP_RANK_MAX,+ _ANSWER_COVERAGE_FLOOR,_CLAIM_SUPPORT_GATE_ENV,_DOMINANCE_ABS_GAP,_DOMINANCE_ABS_SCORE_FLOOR,@@ -40,6 +43,12 @@ from repowise.server.mcp_server.tool_answer.config import (_flag_on,_opt_in,)+from repowise.server.mcp_server.tool_answer.hydration import (+ _stem,+ _stem_hit,+ _text_stems,+ _text_words,+)from repowise.server.mcp_server.tool_answer.symbols import is_symbol_lookup_question@@ -482,15 +491,67 @@ def is_dominant(hits: list[dict], *, agreement_dominant: bool = False) -> bool:return dominance_reason(hits, agreement_dominant=agreement_dominant) is not None-def _retrieval_quality(hits: list[dict], agreement_dominant: bool) -> str:+# Kept apart from `_query_terms.STOPWORDS` on purpose: these are dropped for+# coverage only, retrieval still searches them.+# Words that frame what is asked ("how does X work", "where is X implemented")+# rather than name it. Code rarely says them, so a hit that answers the question+# would otherwise read as missing its rarest word.+_QUESTION_FRAME_STEMS = frozenset(+ _stem(w)+ for w in (+ "work", "works", "working", "implement", "implemented", "implements",+ "implementation", "handle", "handled", "handles", "handling", "happen",+ "happens", "use", "used", "uses", "using", "way", "done", "code", "logic",+ )+)+++def uncovered_terms(question: str, hits: list[dict]) -> list[str]:+ """The question's words the top hit leaves out, when they outweigh the rest.++ Dominance compares hits with each other, so a page about something else+ still dominates a worse page about something else. This asks whether the+ hit mentions what was asked, in the text the answer was built from. Each+ word weighs by how few retrieved hits carry it: one every hit shares (the+ project's own name) counts little, the one naming the topic counts most.+ Returns ``[]`` when the hit covers at least :data:`_ANSWER_COVERAGE_FLOOR`+ of that weight.+ """+ words = {+ t: w for t, w in _text_words(question).items() if t not in _QUESTION_FRAME_STEMS+ }+ if not words or not hits:+ return []+ served = [_text_stems(_retrieval_corpus([h], include_paths=True)) for h in hits]+ weight = {+ t: window_term_weight(sum(_stem_hit(t, s) for s in served), len(served)) for t in words+ }+ missing = [t for t in words if not _stem_hit(t, served[0])]+ left_out = sum(weight[t] for t in missing)+ if left_out <= (1.0 - _ANSWER_COVERAGE_FLOOR) * sum(weight.values()):+ return []+ return [words[t] for t in missing]+++def _retrieval_quality(+ hits: list[dict],+ agreement_dominant: bool,+ question: str = "",+ uncovered: list[str] | None = None,+) -> str:"""Rate the retrieval, independently of the text it fed.Shared with the degraded path so "high" means the same to keyless and keyed- callers, and :func:`_degraded_confidence` grades from it. "weak" means- exactly "not dominant" per :func:`is_dominant`.+ callers, and :func:`_degraded_confidence` grades from it. "weak" means not+ dominant per :func:`is_dominant`, or a top hit that leaves out most of what+ *question* asks (:func:`uncovered_terms`; pass *uncovered* when already computed)."""top_score = _confidence_score(hits[0]) if hits else 0.0dominant_grade = is_dominant(hits, agreement_dominant=agreement_dominant)+ if uncovered is None:+ uncovered = uncovered_terms(question, hits) if dominant_grade else []+ if dominant_grade and uncovered:+ return "weak"if dominant_grade and top_score >= _HIGH_CONFIDENCE_SCORE_FLOOR:return "high"return "partial" if dominant_grade else "weak"@@ -606,6 +667,8 @@ class _Grade(NamedTuple):#: "symbol_body" may tell the agent not to re-read.high_reason: str | Noneungrounded_values: list[str]+ #: Question words the top hit leaves out, when they outweigh what it covers.+ uncovered: list[str]frame_unsupported: list[str]exclusivity_over_truncated: boolwithheld_implicated: list[str]@@ -623,6 +686,7 @@ def _grade_answer(symbol_bodies: list[dict],served_named_body: bool,dominance: str | None,+ uncovered: list[str] | None = None,) -> _Grade:"""Grade the synthesised answer through the gate cascade, in order.@@ -632,6 +696,7 @@ def _grade_answer(``dominance`` is :func:`dominance_reason`'s verdict, computed once by thecaller and used for the starting grade, the ceiling, and the note's reason.+ ``uncovered`` is :func:`uncovered_terms`, when the caller already has it."""dominant = dominance is not None_ratio = _top_two_score_ratio(hits)@@ -752,6 +817,15 @@ def _grade_answer(if lookup_body_truncated:confidence = "medium"+ # Coverage gate, last so a more specific gate writes the note: dominance+ # says the top hit beat the others, not that it is about the question. A+ # served body does not exempt it: a question can name a symbol in passing.+ gap: list[str] = []+ if confidence == "high":+ gap = uncovered_terms(question, hits) if uncovered is None else uncovered+ if gap:+ confidence = "medium"+# Non-dominant ceiling: with no single dominant page the answer must not# read "high" (cite without verifying) unless it EARNED high. Keyed on# `earn_high`, not `earned_body`, so the grounding opt-in really restores@@ -772,6 +846,7 @@ def _grade_answer(ratio=_ratio,top_score=_top_score,ungrounded_values=ungrounded_values,+ uncovered=gap,frame_unsupported=frame_unsupported,exclusivity_over_truncated=exclusivity_over_truncated,withheld_implicated=withheld_implicated,diff --git a/packages/server/src/repowise/server/mcp_server/tool_answer/config.py b/packages/server/src/repowise/server/mcp_server/tool_answer/config.pyindex d298e6b89..1a77795fc 100644--- a/packages/server/src/repowise/server/mcp_server/tool_answer/config.py+++ b/packages/server/src/repowise/server/mcp_server/tool_answer/config.py@@ -270,6 +270,14 @@ _DOMAIN_PENALTY = 0.5# weak retrieval, so it grades "medium". Useful wiki hits routinely score >1.5._HIGH_CONFIDENCE_SCORE_FLOOR = 1.5+# Share of the question's words, weighted by rarity across the retrieved hits,+# the answering hit must mention before confidence may rest on it. Dominance+# only compares hits with each other, so an off-topic page beating a worse one+# still dominates. Chosen on one repo's questions, where it kept every top hit+# that answered and dropped most off-topic and wrong-file ones, then checked on+# repos in other languages.+_ANSWER_COVERAGE_FLOOR = 0.5+# Schema version stamped on every cached payload. Bump whenever the synthesised# response shape or the meaning of a field (including `confidence` and `note`)# changes: a cached row at a lower version is a miss and re-synthesises, since@@ -281,7 +289,8 @@ _HIGH_CONFIDENCE_SCORE_FLOOR = 1.5# 19: rows carry ``_candidate_file_facts``; low answers serve slim best_guesses rows.# 20: caller questions carry ``graph_callers`` and synthesise with them.# 21: impact questions carry ``graph_neighbors``, the users of the lead file.-_ANSWER_SCHEMA_VERSION = 21+# 22: ``confidence`` also asks whether the top hit covers the question's words.+_ANSWER_SCHEMA_VERSION = 22# Above this a whole-file Read costs more than the answer, so the low-confidence# hint names a ranged Read or a skeleton instead.diff --git a/packages/server/src/repowise/server/mcp_server/tool_answer/degraded.py b/packages/server/src/repowise/server/mcp_server/tool_answer/degraded.pyindex d80acb999..1f94e510c 100644--- a/packages/server/src/repowise/server/mcp_server/tool_answer/degraded.py+++ b/packages/server/src/repowise/server/mcp_server/tool_answer/degraded.py@@ -9,6 +9,7 @@ from __future__ import annotationsimport time+from repowise.server.mcp_server._code_rationale import can_lead_answerfrom repowise.server.mcp_server._meta import _normalize_target_pathfrom repowise.server.mcp_server._meta import answer_hint as _answer_hintfrom repowise.server.mcp_server._meta import build_meta as _build_meta@@ -119,7 +120,7 @@ async def _degraded_payload(signals. ``confidence`` is graded by :func:`_degraded_confidence` from``retrieval_quality`` and whether retrieval backs the file ``answer`` leads with."""- retrieval_quality = _retrieval_quality(hits, agreement_dominant)+ retrieval_quality = _retrieval_quality(hits, agreement_dominant, question)repo_root = _repo_root(ctx)symbol_bodies, _served_named_body = _build_symbol_bodies(_gather_body_candidates(hits, "", anchor_names=question_ids or set()),@@ -158,7 +159,7 @@ async def _degraded_payload(elif best_guesses:lead = best_guesses[0]["file"]else:- lead = code_rationale[0].get("path") if code_rationale else None+ lead = next((r.get("path") for r in code_rationale[:1] if can_lead_answer(r)), None)confidence = _degraded_confidence(reason,retrieval_quality,diff --git a/packages/server/src/repowise/server/mcp_server/tool_answer/hydration.py b/packages/server/src/repowise/server/mcp_server/tool_answer/hydration.pyindex fad1fb133..b83e5055d 100644--- a/packages/server/src/repowise/server/mcp_server/tool_answer/hydration.py+++ b/packages/server/src/repowise/server/mcp_server/tool_answer/hydration.py@@ -50,13 +50,20 @@ def _stem(token: str) -> str:return token+def _text_words(text: str) -> dict[str, str]:+ """Stemmed content tokens of *text*, hump- and separator-split, each mapped+ to the first word it came from (for quoting a stem back to a reader)."""+ words: dict[str, str] = {}+ for tok in re.split(r"[^A-Za-z0-9]+", split_humps(text)):+ low = tok.lower()+ if len(tok) >= 3 and low not in _STOPWORDS:+ words.setdefault(_stem(low), low)+ return words++def _text_stems(text: str) -> set[str]:"""Stemmed content tokens of *text*, hump- and separator-split."""- return {- _stem(tok.lower())- for tok in re.split(r"[^A-Za-z0-9]+", split_humps(text))- if len(tok) >= 3 and tok.lower() not in _STOPWORDS- }+ return set(_text_words(text))def _stem_hit(term: str, tokens: set[str]) -> bool:diff --git a/packages/server/src/repowise/server/mcp_server/tool_answer/payload.py b/packages/server/src/repowise/server/mcp_server/tool_answer/payload.pyindex 345b3b2c6..2c1024b31 100644--- a/packages/server/src/repowise/server/mcp_server/tool_answer/payload.py+++ b/packages/server/src/repowise/server/mcp_server/tool_answer/payload.py@@ -347,7 +347,9 @@ def build_value_payload(The verbatim assignment line (read live by the hydrator) IS the answer.Not cached: extraction is cheap and must reflect the current source.Only a constant the question names exactly is ``high``: one whose name the- question only spells out may not be the one meant.+ question only spells out may not be the one meant. The coverage gate does+ not apply: an exact name is the match, and the words around it ("default",+ "value") are the question's frame, so it would only demote right answers."""top_score_fp = hits[0].get("score", 0.0) if hits else 0.0answer_text = extraction["answer"]@@ -702,6 +704,12 @@ async def _graded_payload(payload["next_action_hint"] = (f"call get_symbol id='{cut['continuation']}' for the rest of {cut['name']}")+ elif grade.uncovered:+ payload["note"] = (+ f"Coverage gate: the top source does not mention {', '.join(grade.uncovered[:4])}, "+ "so it may answer a different question. Held at medium; confirm against "+ "the next evidence row or candidate_files."+ )elif confidence == "high":# "No hedging" is not cited as evidence: the prompt forbids hedging, so# reading its absence back as a signal would be circular.diff --git a/packages/server/src/repowise/server/mcp_server/tool_answer/projection.py b/packages/server/src/repowise/server/mcp_server/tool_answer/projection.pyindex abeebd1bc..38db94b89 100644--- a/packages/server/src/repowise/server/mcp_server/tool_answer/projection.py+++ b/packages/server/src/repowise/server/mcp_server/tool_answer/projection.py@@ -9,6 +9,7 @@ from functools import wrapsfrom pathlib import Pathfrom typing import Any+from repowise.server.mcp_server._code_rationale import can_lead_answerfrom repowise.server.mcp_server.tool_answer.config import (_CANDIDATE_FILES_HIGH,_CANDIDATE_FILES_MAX,@@ -28,6 +29,11 @@ _COLLECTIONS = ("fallback_targets",)+# ``evidence`` restores the expanded projection; ``diagnostics`` adds+# ``retrieval_quality``, which feeds ``confidence`` and is otherwise not served+# so the agent reads one trust signal.+_INCLUDES = frozenset({"evidence", "diagnostics"})+def _path(row: Any) -> str | None:if isinstance(row, str):@@ -243,9 +249,14 @@ def _same_file(a: str | None, b: str | None) -> bool:def _lead_rationale(payload: dict[str, Any]) -> dict[str, Any] | None:"""The rationale row ``answer`` may lead with: one from the top-ranked file.- A row from any other file would name a file retrieval did not put first.+ A row from any other file would name a file retrieval did not put first,+ and one matching a single question word is not an answer (`can_lead_answer`)."""- rows = [row for row in payload.get("code_rationale") or [] if isinstance(row, dict)]+ rows = [+ row+ for row in payload.get("code_rationale") or []+ if isinstance(row, dict) and can_lead_answer(row)+ ]if not rows:return Noneguess = _first_guess(payload)@@ -734,7 +745,9 @@ def _project(payload, totals, scope=scope, repo=repo, expanded=expanded, ranked=ranked,bodies_cut=bodies_cut,)- unknown = sorted(set(include or []) - {"evidence"})+ if "diagnostics" not in set(include or []):+ payload.pop("retrieval_quality", None)+ unknown = sorted(set(include or []) - _INCLUDES)if unknown:payload.setdefault("_meta", {})["ignored_arguments"] = {"include": unknown}return payload, slimmed, rankeddiff --git a/plugins/claude-code/skills/codebase-exploration/SKILL.md b/plugins/claude-code/skills/codebase-exploration/SKILL.mdindex 961b00161..9c8e40bcf 100644--- a/plugins/claude-code/skills/codebase-exploration/SKILL.md+++ b/plugins/claude-code/skills/codebase-exploration/SKILL.md@@ -20,7 +20,7 @@ plain file reads don't, usually in one round-trip instead of many.| You want… | Call ||---|---|| First orientation in an unfamiliar repo | `get_overview()` — architecture summary, key modules, entry points, git health, knowledge map. Skip it once you have the map. |-| A direct answer to "how/where/why does X work" | `get_answer(question="…")` — synthesised answer with citations + a `retrieval_quality` signal. Collapses the search → read → reason loop. |+| A direct answer to "how/where/why does X work" | `get_answer(question="…")`: synthesised answer with citations + one `confidence` trust signal. Collapses the search → read → reason loop. || Find a symbol, file, or fuzzy concept | `search_codebase(query="…")` — hybrid search. `mode="auto"` routes an identifier to indexed symbol hits (`symbol_id`/line bounds → pipe into `get_symbol`), a path to file pages (→ `get_context`), and prose to semantic wiki search (each hit reports `search_method`: `embedding` vs `bm25`). Force a branch with `mode=symbol\|path\|concept\|hybrid`; narrow symbols with `symbol_kind`. || A triage card for specific files/symbols | `get_context(targets=[…])` — title, summary, signatures, hotspot bit, top callers, decision titles, symbol_ids (a row without one is `path::name`). Batch many targets in one call. || The actual source of one symbol | `get_symbol("path/to/file.py::Name")` — exact bytes with line bounds. Cheaper than Read + offset math. Use a `symbol_id` from `get_context`. |@@ -44,7 +44,7 @@ context doesn't cover the specific detail the user asked about.## Trust signals — verify when- `_meta.stale_warning` is present (the index has diverged from HEAD), or-- `retrieval_quality` is `partial`/`weak`, or+- `confidence` is `medium`/`low`, or- a result's `search_method` is `bm25`.Otherwise the response is current — act on it.diff --git a/plugins/codex/skills/codebase-exploration/SKILL.md b/plugins/codex/skills/codebase-exploration/SKILL.mdindex b32139603..e909a4a37 100644--- a/plugins/codex/skills/codebase-exploration/SKILL.md+++ b/plugins/codex/skills/codebase-exploration/SKILL.md@@ -15,7 +15,7 @@ plain file reads don't, usually in one round-trip instead of many.| You want… | Call ||---|---|| First orientation in an unfamiliar repo | `get_overview()` — architecture summary, key modules, entry points, git health, knowledge map. Skip it once you have the map. |-| A direct answer to "how/where/why does X work" | `get_answer(question="…")` — synthesised answer with citations + a `retrieval_quality` signal. Collapses the search → read → reason loop. |+| A direct answer to "how/where/why does X work" | `get_answer(question="…")`: synthesised answer with citations + one `confidence` trust signal. Collapses the search → read → reason loop. || Find a symbol, file, or fuzzy concept | `search_codebase(query="…")` — hybrid search. `mode="auto"` routes an identifier to indexed symbol hits (`symbol_id`/line bounds → pipe into `get_symbol`), a path to file pages (→ `get_context`), and prose to semantic wiki search (each hit reports `search_method`: `embedding` vs `bm25`). Force a branch with `mode=symbol\|path\|concept\|hybrid`; narrow symbols with `symbol_kind`. || A triage card for specific files/symbols | `get_context(targets=[…])` — title, summary, signatures, hotspot bit, top callers, decision titles, symbol_ids (a row without one is `path::name`). Batch many targets in one call. || The actual source of one symbol | `get_symbol("path/to/file.py::Name")` — exact bytes with line bounds. Cheaper than Read + offset math. Use a `symbol_id` from `get_context`. |@@ -39,7 +39,7 @@ context doesn't cover the specific detail the user asked about.## Trust signals — verify when- `_meta.stale_warning` is present (the index has diverged from HEAD), or-- `retrieval_quality` is `partial`/`weak`, or+- `confidence` is `medium`/`low`, or- a result's `search_method` is `bm25`.Otherwise the response is current — act on it.diff --git a/plugins/shared/skills/codebase-exploration.md b/plugins/shared/skills/codebase-exploration.mdindex ac24979f0..ab319d770 100644--- a/plugins/shared/skills/codebase-exploration.md+++ b/plugins/shared/skills/codebase-exploration.md@@ -28,7 +28,7 @@ plain file reads don't, usually in one round-trip instead of many.| You want… | Call ||---|---|| First orientation in an unfamiliar repo | `get_overview()` — architecture summary, key modules, entry points, git health, knowledge map. Skip it once you have the map. |-| A direct answer to "how/where/why does X work" | `get_answer(question="…")` — synthesised answer with citations + a `retrieval_quality` signal. Collapses the search → read → reason loop. |+| A direct answer to "how/where/why does X work" | `get_answer(question="…")`: synthesised answer with citations + one `confidence` trust signal. Collapses the search → read → reason loop. || Find a symbol, file, or fuzzy concept | `search_codebase(query="…")` — hybrid search. `mode="auto"` routes an identifier to indexed symbol hits (`symbol_id`/line bounds → pipe into `get_symbol`), a path to file pages (→ `get_context`), and prose to semantic wiki search (each hit reports `search_method`: `embedding` vs `bm25`). Force a branch with `mode=symbol\|path\|concept\|hybrid`; narrow symbols with `symbol_kind`. || A triage card for specific files/symbols | `get_context(targets=[…])` — title, summary, signatures, hotspot bit, top callers, decision titles, symbol_ids (a row without one is `path::name`). Batch many targets in one call. || The actual source of one symbol | `get_symbol("path/to/file.py::Name")` — exact bytes with line bounds. Cheaper than Read + offset math. Use a `symbol_id` from `get_context`. |@@ -52,7 +52,7 @@ context doesn't cover the specific detail the user asked about.## Trust signals — verify when- `_meta.stale_warning` is present (the index has diverged from HEAD), or-- `retrieval_quality` is `partial`/`weak`, or+- `confidence` is `medium`/`low`, or- a result's `search_method` is `bm25`.Otherwise the response is current — act on it.diff --git a/tests/fixtures/mcp/tool_response_shapes.json b/tests/fixtures/mcp/tool_response_shapes.jsonindex 25a21e86e..987aa0310 100644--- a/tests/fixtures/mcp/tool_response_shapes.json+++ b/tests/fixtures/mcp/tool_response_shapes.json@@ -46,8 +46,7 @@],"quotes_emitted": null,"quotes_reduced_reason": null,- "quotes_total": null,- "retrieval_quality": null+ "quotes_total": null},"get_architecture": {"_meta": {diff --git a/tests/unit/generation/test_editor_file_base.py b/tests/unit/generation/test_editor_file_base.pyindex 0c6760ac2..f287d2610 100644--- a/tests/unit/generation/test_editor_file_base.py+++ b/tests/unit/generation/test_editor_file_base.py@@ -133,7 +133,7 @@ def test_keyless_index_gets_keyless_steering(any_gen):assert "repowise update --full" not in resultassert "[fts]` only has no semantic agreement" not in resultassert 'Cite `confidence: "high"`' not in result- assert "`degraded` means judge by `retrieval_quality`" in result+ assert "`degraded` means no prose; `confidence` rates the evidence" in resultdef test_keyed_index_keeps_model_guidance(any_gen):diff --git a/tests/unit/server/mcp/test_answer_agreement.py b/tests/unit/server/mcp/test_answer_agreement.pyindex 0d0534958..04fe35325 100644--- a/tests/unit/server/mcp/test_answer_agreement.py+++ b/tests/unit/server/mcp/test_answer_agreement.py@@ -127,7 +127,7 @@ def _patch_agreement_pipeline(monkeypatch, answer_mod, *, top_both: bool):for i, h in enumerate(hits):h["target_path"] = h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Upload module summary."+ h["summary"] = "Upload module: streams large uploads in chunks."h["snippet"] = ""h["page_type"] = "file_page"if i == 0:@@ -154,7 +154,7 @@ async def test_agreement_lifts_compressed_retrieval_to_high(setup_mcp, monkeypat_patch_agreement_pipeline(monkeypatch, answer_mod, top_both=True)_patch_provider(monkeypatch, answer_mod, _GOOD_ANSWER)- result = await get_answer("how does upload chunking work")+ result = await get_answer("how does upload chunking work", include=["diagnostics"])assert result["confidence"] == "high"assert result["retrieval_quality"] == "high"@@ -217,7 +217,7 @@ async def test_grounding_earns_high_only_when_opted_in(setup_mcp, monkeypatch):_patch_agreement_pipeline(monkeypatch, answer_mod, top_both=False)_patch_provider(monkeypatch, answer_mod, _GOOD_ANSWER)- result = await get_answer("how are large uploads handled")+ result = await get_answer("how are large uploads handled", include=["diagnostics"])assert result["retrieval_quality"] == "weak"assert result["confidence"] == "medium", "grounded prose does not out-vote weak retrieval"@@ -371,7 +371,7 @@ def _patch_keyless_agreement_pipeline(monkeypatch, answer_mod, pipeline_mod, *,for i, h in enumerate(hits):h["target_path"] = h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Upload module summary."+ h["summary"] = "Upload module: streams large uploads in chunks."h["snippet"] = ""h["page_type"] = "file_page"if i == 0:@@ -440,4 +440,4 @@ async def test_symbol_agreement_flag_is_independently_reversible(setup_mcp, monk# ...and the vector pair still lifts with the symbol flag off._patch_agreement_pipeline(monkeypatch, answer_mod, top_both=True)_patch_provider(monkeypatch, answer_mod, _GOOD_ANSWER)- assert (await get_answer("how does upload chunking work again"))["confidence"] == "high"+ assert (await get_answer("how does upload chunking work in alpha"))["confidence"] == "high"diff --git a/tests/unit/server/mcp/test_answer_always_synthesize.py b/tests/unit/server/mcp/test_answer_always_synthesize.pyindex af332af41..6399cb18f 100644--- a/tests/unit/server/mcp/test_answer_always_synthesize.py+++ b/tests/unit/server/mcp/test_answer_always_synthesize.py@@ -33,7 +33,7 @@ def _patch_pipeline(monkeypatch, answer_mod, scores=(2.0, 1.9), *, with_symbols=for i, h in enumerate(hits):h["target_path"] = h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Module summary for the page. " * 4+ h["summary"] = "The alpha module go function. " * 4h["snippet"] = ""h["page_type"] = "file_page"if with_symbols and i == 0:diff --git a/tests/unit/server/mcp/test_answer_calibration.py b/tests/unit/server/mcp/test_answer_calibration.pyindex 2d69e7bf4..4daf2060e 100644--- a/tests/unit/server/mcp/test_answer_calibration.py+++ b/tests/unit/server/mcp/test_answer_calibration.py@@ -309,7 +309,7 @@ def _patch_pipeline(monkeypatch, answer_mod, *, with_symbols: bool, symbol: dictfor i, h in enumerate(hits):h["target_path"] = h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Auth service summary."+ h["summary"] = "Auth service summary. Callers get a bounded, limited list."h["snippet"] = ""h["page_type"] = "file_page"if with_symbols and i == 0:diff --git a/tests/unit/server/mcp/test_answer_confidence_reasons.py b/tests/unit/server/mcp/test_answer_confidence_reasons.pyindex 6e4f4ac09..d11637376 100644--- a/tests/unit/server/mcp/test_answer_confidence_reasons.py+++ b/tests/unit/server/mcp/test_answer_confidence_reasons.py@@ -35,7 +35,10 @@ import pytest_SYMBOL = "run_sweep"-def _patch_pipeline(monkeypatch, answer_mod, scores, *, with_symbols=True, matched=False):+def _patch_pipeline(+ monkeypatch, answer_mod, scores, *, with_symbols=True, matched=False,+ summary="How the alpha module clears out stale rows. " * 4,+):"""Two file-page hits at *scores*, the top one carrying a hydrated symbol."""async def _fake_retrieve(question, ctx):@@ -48,7 +51,7 @@ def _patch_pipeline(monkeypatch, answer_mod, scores, *, with_symbols=True, matchfor i, h in enumerate(hits):h["target_path"] = h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Module summary for the page. " * 4+ h["summary"] = summaryh["snippet"] = ""h["page_type"] = "file_page"if with_symbols and i == 0:@@ -106,7 +109,7 @@ async def test_ambiguous_retrieval_never_reads_as_dominant(setup_mcp, monkeypatc_patch_pipeline(monkeypatch, answer_mod, scores=(2.91, 2.91))_patch_provider(monkeypatch, answer_mod, _GROUNDED_ANSWER)- result = await get_answer(_QUESTION)+ result = await get_answer(_QUESTION, include=["diagnostics"])note = result.get("note") or ""assert result["confidence"] == "high", "the opt-in must actually reach the high branch"assert result["retrieval_quality"] == "weak", "a tied top pair is ambiguous retrieval"@@ -131,7 +134,7 @@ async def test_grounded_prose_does_not_earn_high_on_weak_retrieval(setup_mcp, mo_patch_pipeline(monkeypatch, answer_mod, scores=(2.91, 2.91))_patch_provider(monkeypatch, answer_mod, _GROUNDED_ANSWER)- result = await get_answer(_QUESTION)+ result = await get_answer(_QUESTION, include=["diagnostics"])assert result["retrieval_quality"] == "weak"assert result["confidence"] == "medium", ("answer-grounding alone must not out-vote an ambiguous retrieval"@@ -164,6 +167,25 @@ async def test_dominance_grade_honours_the_absolute_gap_tier(setup_mcp, monkeypaassert "clearly dominates" in result["note"]+async def test_a_dominant_top_hit_that_misses_the_question_is_not_high(setup_mcp, monkeypatch):+ """Dominance compares hits with each other: an off-topic page beating a worse+ one still dominates. The coverage gate holds it at medium and says why."""+ import repowise.server.mcp_server.tool_answer.answer as answer_mod+ from repowise.server.mcp_server import get_answer++ _patch_pipeline(+ monkeypatch, answer_mod, scores=(8.0, 2.0), summary="Parses yaml config files."+ )+ _patch_provider(monkeypatch, answer_mod, _PLAIN_ANSWER)++ result = await get_answer(_QUESTION, include=["diagnostics"])+ assert result["confidence"] == "medium"+ assert result["retrieval_quality"] == "weak"+ assert "Coverage gate" in result["note"]+ assert "stale" in result["note"]++@pytest.mark.asyncioasync def test_the_gap_tier_does_not_quote_the_ratio_it_did_not_use(setup_mcp, monkeypatch):"""The same self-refuting number, one tier down.@@ -270,7 +292,7 @@ async def test_a_served_named_body_still_earns_high_on_weak_retrieval()_patch_provider(monkeypatch, answer_mod, _GROUNDED_ANSWER)- result = await get_answer(f"How does {_SYMBOL} work?")+ result = await get_answer(f"How does {_SYMBOL} work?", include=["diagnostics"])assert result["retrieval_quality"] == "weak"assert result["symbol_bodies"], "the named body is what the grade rests on"assert result["confidence"] == "high"diff --git a/tests/unit/server/mcp/test_answer_degraded_evidence.py b/tests/unit/server/mcp/test_answer_degraded_evidence.pyindex f15725b56..40827d7ed 100644--- a/tests/unit/server/mcp/test_answer_degraded_evidence.py+++ b/tests/unit/server/mcp/test_answer_degraded_evidence.py@@ -222,9 +222,9 @@ async def test_degraded_with_no_anchor_match_still_describes_a_ranking(tmp_path)# derived from it is pinned in the confidence block further down.-def _scored(*scores: float) -> list[dict]:+def _scored(*scores: float, summary: str = "how the module works") -> list[dict]:return [- {"target_path": f"pkg/m{i}.py", "title": f"m{i}", "summary": "s", "score": s}+ {"target_path": f"pkg/m{i}.py", "title": f"m{i}", "summary": summary, "score": s}for i, s in enumerate(scores)]@@ -253,6 +253,13 @@ async def test_degraded_rates_a_dominant_but_weak_retrieval_partial():assert await _quality(_scored(1.0, 0.1)) == "partial"+async def test_degraded_rates_a_dominant_hit_that_misses_the_question_weak():+ """Dominance compares hits with each other; a winner that never mentions what+ was asked is an off-topic page beating worse ones, so it rates weak."""+ assert await _quality(_scored(6.0, 1.0, summary="parses yaml files")) == "weak"+ assert await _confidence(_scored(6.0, 1.0, summary="parses yaml files")) == "low"++async def test_degraded_does_not_lift_a_genuinely_weak_retrieval():"""The control that must not move.@@ -572,13 +579,13 @@ async def test_degraded_cites_the_retrieval_top_file_before_a_rationale_hub(]-def _rationale_rows(monkeypatch, *paths: str) -> None:+def _rationale_rows(monkeypatch, *paths: str, matched: tuple[str, ...] = ("flask", "app")) -> None:"""Stub the rationale miner to return one row per path, in the given order."""import repowise.server.mcp_server.tool_answer.degraded as degraded_modasync def _rows(ctx, hits, fallback_targets, question):return [- {"path": p, "lines": [100 + i, 101 + i], "comment": "c", "matched_terms": []}+ {"path": p, "lines": [100 + i, 101 + i], "comment": "c", "matched_terms": list(matched)}for i, p in enumerate(paths)]@@ -819,7 +826,7 @@ def test_the_hint_is_the_same_whatever_the_payload_graded():def _ranked(*paths: str, ranks: tuple[int | None, ...] | None = None) -> list[dict]:"""Strong, dominant hits in this order, each stamped with its hybrid rank."""hits = [- {"target_path": p, "title": p, "summary": "s", "score": 6.0 if i == 0 else 2.0}+ {"target_path": p, "title": p, "summary": "the Flask app", "score": 6.0 if i == 0 else 2.0}for i, p in enumerate(paths)]for h, rank in zip(hits, ranks if ranks is not None else range(len(hits)), strict=True):@@ -851,6 +858,16 @@ async def test_the_top_file_rationale_leads_over_a_higher_scoring_hub(tmp_path,assert payload["confidence"] == "medium"+async def test_a_rationale_matching_one_question_word_does_not_lead(tmp_path, monkeypatch):+ """A marker plus one shared word clears the miner's gate but answers nothing."""+ _rationale_rows(monkeypatch, "src/agents/poll-backoff.ts", matched=("flask",))+ payload = await _keyless(tmp_path, _ranked("src/agents/poll-backoff.ts", "src/config/types.ts"))++ answer = _answer(payload)+ assert "Source rationale" not in answer+ assert "Local retrieval points first to src/agents/poll-backoff.ts" in answer++async def test_without_a_top_file_rationale_the_answer_names_the_ranking(tmp_path, monkeypatch):_rationale_rows(monkeypatch, "cmd/completions.go")payload = await _keyless(tmp_path, _ranked("cmd/command.go", "cmd/completions.go"))@@ -975,11 +992,11 @@ async def test_an_unranked_duplicate_does_not_hide_its_files_rank(tmp_path, monk"""An injected symbol page of a ranked file can precede that file's own hit."""_rationale_rows(monkeypatch, "src/hub.ts", "src/other.ts", "src/poll.ts")hits = [- {"target_path": "src/poll.ts::backoff", "title": "t", "summary": "s", "score": 6.0},- {"target_path": "src/hub.ts", "title": "t", "summary": "s", "score": 2.0},- {"target_path": "src/other.ts::Run", "title": "t", "summary": "s", "score": 1.5},- {"target_path": "src/other.ts", "title": "t", "summary": "s", "score": 1.0},- {"target_path": "src/poll.ts", "title": "t", "summary": "s", "score": 1.0},+ {"target_path": "src/poll.ts::backoff", "title": "t", "summary": "the Flask app", "score": 6.0},+ {"target_path": "src/hub.ts", "title": "t", "summary": "the Flask app", "score": 2.0},+ {"target_path": "src/other.ts::Run", "title": "t", "summary": "the Flask app", "score": 1.5},+ {"target_path": "src/other.ts", "title": "t", "summary": "the Flask app", "score": 1.0},+ {"target_path": "src/poll.ts", "title": "t", "summary": "the Flask app", "score": 1.0},]for h, rank in zip(hits, (None, 5, None, 1, 0), strict=True):if rank is not None:diff --git a/tests/unit/server/mcp/test_answer_lookup_truncation.py b/tests/unit/server/mcp/test_answer_lookup_truncation.pyindex 66b30cd15..25670536c 100644--- a/tests/unit/server/mcp/test_answer_lookup_truncation.py+++ b/tests/unit/server/mcp/test_answer_lookup_truncation.py@@ -62,7 +62,7 @@ def _patch_pipeline(monkeypatch, answer_mod, *, symbol: dict, path: str | None =for i, h in enumerate(hits):h["target_path"] = path or h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Policy module summary."+ h["summary"] = "Policy module: the retry loop decides its floor at startup."h["snippet"] = ""h["page_type"] = "file_page"if i == 0:diff --git a/tests/unit/server/mcp/test_answer_payload.py b/tests/unit/server/mcp/test_answer_payload.pyindex 58a6cbf6e..e4c7bd757 100644--- a/tests/unit/server/mcp/test_answer_payload.py+++ b/tests/unit/server/mcp/test_answer_payload.py@@ -209,7 +209,7 @@ def _patch_pipeline(monkeypatch, answer_mod, scores=(5.0, 4.0)):for i, h in enumerate(hits):h["target_path"] = h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Summary text for the page. " * 12+ h["summary"] = "Summary of the beta module function. " * 12h["snippet"] = ""h["page_type"] = "file_page"if i == 0:diff --git a/tests/unit/server/mcp/test_answer_projection.py b/tests/unit/server/mcp/test_answer_projection.pyindex 8f590118f..0ddc355d7 100644--- a/tests/unit/server/mcp/test_answer_projection.py+++ b/tests/unit/server/mcp/test_answer_projection.py@@ -270,6 +270,43 @@ def test_expanded_degraded_answer_names_only_retained_blocks():assert "candidates" not in expanded["answer"][email protected]("confidence", ["high", "medium", "low"])+def test_retrieval_quality_is_served_only_under_diagnostics(confidence):+ """One trust signal by default; the retrieval rating is an opt-in diagnostic."""+ default = project_answer_payload(_raw(confidence), question="how does auth work")+ diagnostics = project_answer_payload(+ _raw(confidence), question="how does auth work", include=["diagnostics"]+ )++ assert "retrieval_quality" not in default+ assert diagnostics["retrieval_quality"] == _raw(confidence)["retrieval_quality"]+ assert "ignored_arguments" not in diagnostics.get("_meta", {})+++def test_a_rationale_matching_one_question_word_does_not_lead():+ raw = {+ "answer": "generic fallback",+ "citations": ["cobra.go"],+ "confidence": "medium",+ "degraded": "no-llm-provider",+ "code_rationale": [+ {+ "path": "cobra.go",+ "lines": [4, 5],+ "comment": "FIXME Gt is unused by cobra and should be removed.",+ "matched_terms": ["cobra"],+ }+ ],+ "best_guesses": [{"file": "cobra.go", "why_relevant": "ranked first"}],+ "_meta": {"contract_version": 1},+ }++ result = project_answer_payload(raw, question="how does cobra choose a subcommand")++ assert "FIXME" not in result["answer"]+ assert "points first to cobra.go" in result["answer"]++def test_rationale_only_degraded_answer_leads_with_the_local_conclusion():raw = {"answer": "generic fallback",@@ -281,6 +318,7 @@ def test_rationale_only_degraded_answer_leads_with_the_local_conclusion():"path": "src/cache.py","lines": [4, 5],"comment": "The TTL matches the upstream feed's five-minute refresh.",+ "matched_terms": ["ttl", "five", "minutes"],}],"_meta": {"contract_version": 1},@@ -318,7 +356,8 @@ async def test_no_llm_and_failed_legs_remain_compact_and_actionable()result = await tool_middleware(get_answer)(- f"how does auth work in {expected_degraded} {legs['vector']} mode"+ "how does auth middleware work",+ include=["diagnostics"],)assert result["degraded"] == expected_degradeddiff --git a/tests/unit/server/mcp/test_answer_shortlist.py b/tests/unit/server/mcp/test_answer_shortlist.pyindex bd32dde07..48c14b939 100644--- a/tests/unit/server/mcp/test_answer_shortlist.py+++ b/tests/unit/server/mcp/test_answer_shortlist.py@@ -276,7 +276,7 @@ def test_keyless_weak_reply_serves_the_ranked_list_and_one_guidance_line():)assert set(out) == {- "answer", "confidence", "retrieval_quality", "degraded", "best_guesses",+ "answer", "confidence", "degraded", "best_guesses","candidate_files", "code_rationale", "_meta",}assert "src/pkg/f0.py first" in out["answer"] and "search_codebase" in out["answer"]diff --git a/tests/unit/server/mcp/test_answer_withheld_symbols.py b/tests/unit/server/mcp/test_answer_withheld_symbols.pyindex 2fca7b8f2..f07f9dcb3 100644--- a/tests/unit/server/mcp/test_answer_withheld_symbols.py+++ b/tests/unit/server/mcp/test_answer_withheld_symbols.py@@ -252,7 +252,7 @@ def _patch_pipeline(monkeypatch, answer_mod, *, symbol: dict):for i, h in enumerate(hits):h["target_path"] = h["page_id"].removeprefix("file_page:")h["title"] = h["target_path"]- h["summary"] = "Store summary."+ h["summary"] = "Store keeps todos."h["snippet"] = ""h["page_type"] = "file_page"if i == 0: