Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Python aicode-complexitycode-healthcode-intelligencecode-qualitydead-codedeveloper-toolsdocumentationgit-analyticsmcpopen-sourcepythonrefactoringstatic-analysistechnical-debt
fix(search): repair a drifted full-text index instead of refusing to open it (#1311)
Raghav Chamadiya5e7bcf98b57a
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.mdindex 8dfd9f65c..827ff42ab 100644--- a/docs/CHANGELOG.md+++ b/docs/CHANGELOG.md@@ -9,6 +9,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0---+## [0.38.1] — 2026-08-05++### Fixed+- **A store upgraded from 0.37.0 no longer refuses to open.** 0.38.0 widened `page_fts` from three columns to five, and FTS5 cannot be altered, so the index is dropped and refilled from `wiki_pages` the first time an upgraded install opens the store. That rebuild refused whenever the index held more rows than `wiki_pages` could account for, and the error it raised told the reader to run `repowise doctor --repair` — which opens the store the same way, hit the same refusal, and died before repairing anything. `serve` and the MCP server died there too, so a search index took the entire wiki down with it. The excess rows are orphans: pages swept from SQL whose index delete never ran, because it runs after the commit, outside the transaction, on a best-effort path, and nothing ever reconciled them. An orphan answers a query in full and 404s when the reader opens it, so the rebuild discards it and reports the count rather than refusing. (#1309)+- **An interrupted sweep heals itself.** `ensure_index` prunes orphaned index rows on every open, which is the only thing that ever reconciled the two halves of the store; the residue could otherwise only grow. The scan is read-only and takes the write lock only when there is something to delete.+- **`doctor --repair` finishes what it was called for.** A failing full-text schema upgrade is reported and stepped over instead of aborting the repair, and orphans are deleted in one transaction rather than one per id.+- **`serve` and the MCP server start on a store whose index cannot be prepared,** falling back to whatever shape the index is in plus the vector arm, with a warning. Every other surface — the wiki, the graph, code health — was unreachable over a keyword index.++---+## [0.38.0] — 2026-08-04Four new languages, an orientation set rebuilt around what a reader actually needs, and a wiki that draws on the repository's own vocabulary instead of writing generic prose about it. Svelte and Vue reach the Full tier through a byte-preserving projection into TypeScript; HTML lands at the import tier; reStructuredText documents are read as reStructuredText rather than silently yielding nothing. Onboarding is six pages now, ending in a glossary built entirely from mined terms, and a directory that heads a subsystem gets a chapter even when it also holds files of its own. Full-text search was rebuilt on both backends after the query shape turned out to match 65% of the corpus on a median question. The agent hooks got measurement first and then acted on it: the Grep flood is replaced by its digest instead of ranked next to it, triage ranks the files the search actually matched, and the three hot index lookups dropped an ORM import that cost a second per hook fire.diff --git a/packages/cli/src/repowise/cli/commands/doctor_cmd/repo_checks.py b/packages/cli/src/repowise/cli/commands/doctor_cmd/repo_checks.pyindex 86a838baf..6deb712be 100644--- a/packages/cli/src/repowise/cli/commands/doctor_cmd/repo_checks.py+++ b/packages/cli/src/repowise/cli/commands/doctor_cmd/repo_checks.py@@ -496,7 +496,31 @@ def _run_repo_checks(# Repair FTS: re-index missing pages, delete orphanedif missing_from_fts or orphaned_fts:fts = FullTextSearch(engine)- await fts.ensure_index()+ try:+ await fts.ensure_index()+ except Exception as exc:+ # The schema upgrade is not what was asked for, and it must+ # not take the repair down with it. It used to: the upgrade+ # refused on a drifted store and the error it raised told+ # the user to run this very command (issue #1309), so the+ # only command that could fix the drift was the one the+ # drift killed. Both repairs below work on either column+ # set, so say what failed and carry on.+ console.print(+ f" [yellow]Full-text index upgrade skipped: {exc}[/yellow]"+ )+ # Orphans first, deliberately. Deleting one needs nothing but+ # its page_id, so it works on any column set this class has+ # ever written — including the one an upgrade just failed to+ # leave behind. Re-indexing a missing page writes every column+ # and would raise there, taking the repair that *can* run down+ # with it.+ if orphaned_fts:+ # One transaction for the lot. Per-id deletes cost a write+ # lock each, and the store that needs this most is the one+ # with thousands of them.+ await fts.delete_many(list(orphaned_fts))+ repaired += len(orphaned_fts)if missing_from_fts:# Fetch full page data for missing pagesasync with get_session(sf) as session:@@ -519,9 +543,6 @@ def _run_repo_checks(target_path=page.target_path,)repaired += 1- for pid in orphaned_fts:- await fts.delete(pid)- repaired += 1# Repair vector store: re-embed missing pages, delete orphanedlance_dir = repowise_dir / "lancedb"diff --git a/packages/core/src/repowise/core/persistence/search.py b/packages/core/src/repowise/core/persistence/search.pyindex a446a68be..79f264c04 100644--- a/packages/core/src/repowise/core/persistence/search.py+++ b/packages/core/src/repowise/core/persistence/search.py@@ -62,6 +62,17 @@ PAGE_FTS_DDL = ("USING fts5(page_id UNINDEXED, title, content, summary, target_path)")+# An indexed row whose page is gone from ``wiki_pages``. Counting and deleting+# share the predicate so the number reported is exactly the number removed.+# ``NOT EXISTS`` rather than ``NOT IN``: the subquery's column is a primary key+# and can never be NULL today, but ``NOT IN`` returns no rows at all the day one+# is, which would turn this into a silent no-op instead of a visible failure.+_ORPHAN_PREDICATE = (+ "NOT EXISTS (SELECT 1 FROM wiki_pages p WHERE p.id = page_fts.page_id)"+)+_ORPHAN_COUNT_SQL = f"SELECT count(*) FROM page_fts WHERE {_ORPHAN_PREDICATE}"+_ORPHAN_DELETE_SQL = f"DELETE FROM page_fts WHERE {_ORPHAN_PREDICATE}"+# The text PostgreSQL indexes and searches. The GIN index built by the Alembic# migration uses this exact expression: any drift between the two silently# drops the index from the query plan, leaving a sequential scan that still@@ -330,6 +341,7 @@ class FullTextSearch:async with self._engine.begin() as conn:await conn.execute(text(PAGE_FTS_DDL))await self._upgrade_sqlite_schema()+ await self.prune_orphans()elif self._dialect == "postgresql":async with self._engine.begin() as conn:await conn.execute(@@ -366,22 +378,53 @@ class FullTextSearch:f"expected {list(PAGE_FTS_COLUMNS)!r}")+ if not await self._page_table_exists(conn):+ # The refill has nothing to read. Leaving the old index in+ # place is the only move that keeps search answering, and it+ # must not raise: every command opens the store through here,+ # so an exception would take the whole CLI down over an index+ # that is still perfectly usable on its old shape.+ _log.warning(+ "page_fts is on an older column set but wiki_pages is missing; "+ "leaving the index alone"+ )+ return+indexed_rows = await conn.execute(text("SELECT count(*) FROM page_fts"))indexed_count = int(indexed_rows.scalar() or 0)page_rows = await conn.execute(text("SELECT count(*) FROM wiki_pages"))page_count = int(page_rows.scalar() or 0)-- # The rebuild is a delete followed by a refill from a different table.- # If that table cannot account for what is already indexed, the two- # halves of the store have drifted and refilling would delete- # searchable pages outright. An index on the old shape still answers- # queries, so leaving it alone is strictly the safer failure.- if indexed_count > page_count:- raise RuntimeError(- f"Refusing to rebuild page_fts: {indexed_count} indexed rows but only "- f"{page_count} rows in wiki_pages to refill them from. The full-text "- f"index and the page table have drifted apart; repair the store "- f"(repowise doctor --repair) before upgrading it."+ orphan_count = await self._count_orphans(conn)++ # The rebuild is a delete followed by a refill from a different table,+ # so an index holding more rows than ``wiki_pages`` used to be refused+ # here on the theory that refilling would delete searchable pages.+ # It does not. Every excess row is a row whose ``page_id`` has no page+ # behind it: the page was swept from SQL and the FTS delete that should+ # have followed never ran, because it runs after the commit, outside+ # the transaction, on a best-effort path (issue #1309). A hit on one of+ # those answers in full and then 404s when the reader opens it, so+ # dropping it is the repair, not the damage.+ #+ # What the refusal actually cost was every command: ``serve``,+ # ``doctor --repair`` and the MCP server all open the store through+ # ``ensure_index``, so the error told the user to run the one command+ # it had already killed. Rebuild, and say what was discarded.+ if orphan_count:+ _log.warning(+ "Dropping %d orphaned page_fts row(s) with no wiki_pages row behind "+ "them while rebuilding the index",+ orphan_count,+ )+ elif indexed_count > page_count:+ # Excess the orphan scan cannot explain: duplicate rows for pages+ # that do exist. The refill writes one row per page and fixes it,+ # but it is worth a line — nothing in normal operation writes two.+ _log.warning(+ "page_fts holds %d rows for %d pages with no orphans among them; "+ "the rebuild will collapse the duplicates",+ indexed_count,+ page_count,)_log.info(@@ -408,6 +451,66 @@ class FullTextSearch:# a mismatch means the page table moved underneath the rebuild._log.warning("page_fts rebuild wrote %d rows for %d pages", refilled, page_count)+ @staticmethod+ async def _page_table_exists(conn) -> bool:+ """Whether ``wiki_pages`` is present on this connection."""+ row = await conn.execute(+ text("SELECT 1 FROM sqlite_master WHERE type='table' AND name='wiki_pages'")+ )+ return row.scalar() is not None++ @staticmethod+ async def _count_orphans(conn) -> int:+ """How many indexed rows name a page ``wiki_pages`` no longer has."""+ row = await conn.execute(text(_ORPHAN_COUNT_SQL))+ return int(row.scalar() or 0)++ async def prune_orphans(self) -> int:+ """Delete indexed rows whose page is gone. Returns how many went.++ Six places delete pages from SQL and then delete their FTS rows+ afterwards: the two writes cannot share a transaction, because on+ SQLite the index lives in the same file as the session and writing to+ it while the session holds the write lock raises "database is locked".+ So the SQL delete is durable and the FTS delete is best-effort, and+ anything that ends the process in between — a Ctrl-C, a crash, a lock+ that did not clear — leaves a row indexed forever. Nothing reconciled+ them, so the residue only ever grew; issue #1309 is a store that+ reached 113 indexed rows against 24 pages that way.++ An orphan is not inert. Search hydrates a hit's title and snippet from+ the FTS row itself, so an orphan answers a query in full and 404s when+ the reader clicks it, while holding a slot a live page wanted.++ Called from :meth:`ensure_index`, which every command runs on the way+ in, so an interrupted sweep heals on the next command instead of+ becoming permanent. The scan is read-only and the write transaction is+ opened only when there is something to delete: the common case is no+ orphans, and taking the write lock to discover that would put every+ command in the way of every other one.+ """+ if self._dialect != "sqlite":+ # The GIN index is an expression over ``wiki_pages`` itself. It+ # cannot hold a row the table does not, so there is nothing to+ # prune and no scan worth paying for.+ return 0++ async with self._engine.connect() as conn:+ if not await self._page_table_exists(conn):+ return 0+ orphans = await self._count_orphans(conn)+ if not orphans:+ return 0++ async with self._engine.begin() as conn:+ await conn.execute(text(_ORPHAN_DELETE_SQL))+ _log.warning(+ "Pruned %d orphaned page_fts row(s): indexed pages that are no longer in "+ "wiki_pages, left behind by a sweep whose index delete did not complete",+ orphans,+ )+ return orphans+async def index(self,page_id: str,diff --git a/packages/core/src/repowise/core/upgrade/_data/CHANGELOG.md b/packages/core/src/repowise/core/upgrade/_data/CHANGELOG.mdindex 8dfd9f65c..827ff42ab 100644--- a/packages/core/src/repowise/core/upgrade/_data/CHANGELOG.md+++ b/packages/core/src/repowise/core/upgrade/_data/CHANGELOG.md@@ -9,6 +9,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0---+## [0.38.1] — 2026-08-05++### Fixed+- **A store upgraded from 0.37.0 no longer refuses to open.** 0.38.0 widened `page_fts` from three columns to five, and FTS5 cannot be altered, so the index is dropped and refilled from `wiki_pages` the first time an upgraded install opens the store. That rebuild refused whenever the index held more rows than `wiki_pages` could account for, and the error it raised told the reader to run `repowise doctor --repair` — which opens the store the same way, hit the same refusal, and died before repairing anything. `serve` and the MCP server died there too, so a search index took the entire wiki down with it. The excess rows are orphans: pages swept from SQL whose index delete never ran, because it runs after the commit, outside the transaction, on a best-effort path, and nothing ever reconciled them. An orphan answers a query in full and 404s when the reader opens it, so the rebuild discards it and reports the count rather than refusing. (#1309)+- **An interrupted sweep heals itself.** `ensure_index` prunes orphaned index rows on every open, which is the only thing that ever reconciled the two halves of the store; the residue could otherwise only grow. The scan is read-only and takes the write lock only when there is something to delete.+- **`doctor --repair` finishes what it was called for.** A failing full-text schema upgrade is reported and stepped over instead of aborting the repair, and orphans are deleted in one transaction rather than one per id.+- **`serve` and the MCP server start on a store whose index cannot be prepared,** falling back to whatever shape the index is in plus the vector arm, with a warning. Every other surface — the wiki, the graph, code health — was unreachable over a keyword index.++---+## [0.38.0] — 2026-08-04Four new languages, an orientation set rebuilt around what a reader actually needs, and a wiki that draws on the repository's own vocabulary instead of writing generic prose about it. Svelte and Vue reach the Full tier through a byte-preserving projection into TypeScript; HTML lands at the import tier; reStructuredText documents are read as reStructuredText rather than silently yielding nothing. Onboarding is six pages now, ending in a glossary built entirely from mined terms, and a directory that heads a subsystem gets a chapter even when it also holds files of its own. Full-text search was rebuilt on both backends after the query shape turned out to match 65% of the corpus on a median question. The agent hooks got measurement first and then acted on it: the Grep flood is replaced by its digest instead of ranked next to it, triage ranks the files the search actually matched, and the three hot index lookups dropped an ORM import that cost a second per hook fire.diff --git a/packages/server/src/repowise/server/app.py b/packages/server/src/repowise/server/app.pyindex d01209d6e..8ace2eb7b 100644--- a/packages/server/src/repowise/server/app.py+++ b/packages/server/src/repowise/server/app.py@@ -185,9 +185,16 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:except Exception as exc:logger.warning("stale_job_reset_failed", extra={"error": str(exc)})- # Full-text search+ # Full-text search. A failure here used to abort startup, so a store whose+ # index could not be upgraded served no documentation at all (issue #1309):+ # the wiki, the graph and the health pages were all unreachable over a+ # search index. Keyword search degrades to whatever shape the index is+ # already in, or to the vector arm alone; everything else keeps working.fts = FullTextSearch(engine)- await fts.ensure_index()+ try:+ await fts.ensure_index()+ except Exception as exc:+ logger.warning("fts_ensure_index_failed", extra={"error": str(exc)})# Reuse the repo-local LanceDB index written by CLI init/update. A fresh# in-memory store is used only when this database cannot be associated withdiff --git a/packages/server/src/repowise/server/mcp_server/_server.py b/packages/server/src/repowise/server/mcp_server/_server.pyindex 716175119..cb381a170 100644--- a/packages/server/src/repowise/server/mcp_server/_server.py+++ b/packages/server/src/repowise/server/mcp_server/_server.py@@ -479,7 +479,14 @@ async def _lifespan(server: FastMCP):)_state._fts = FullTextSearch(engine)- await _state._fts.ensure_index()+ try:+ await _state._fts.ensure_index()+ except Exception:+ # Every tool this server exposes was unreachable when this raised+ # (issue #1309), including the ones that never touch the index. The+ # instance is kept: its search still runs against whatever shape the+ # index is in, and the vector arm answers alongside it.+ _log.warning("repowise MCP: full-text index unavailable", exc_info=True)# Seed InMemory placeholder so tools that don't need vector search# can start immediately, before the background load completes.diff --git a/tests/unit/cli/test_doctor_fts_drift.py b/tests/unit/cli/test_doctor_fts_drift.pynew file mode 100644index 000000000..3ac92e683--- /dev/null+++ b/tests/unit/cli/test_doctor_fts_drift.py@@ -0,0 +1,158 @@+"""``doctor --repair`` repairs a drifted full-text index (issue #1309).++0.38.0 widened ``page_fts`` from three columns to five. FTS5 cannot be+altered, so the index is dropped and refilled from ``wiki_pages`` the first+time an upgraded install opens the store. That rebuild used to refuse when the+index held more rows than ``wiki_pages`` could account for, and the error it+raised told the user to run ``repowise doctor --repair`` — which opens the+store the same way, hit the same refusal, and died before repairing anything.+``serve`` and the MCP server did too, so the whole wiki became unreachable.++The excess rows are orphans: pages swept from SQL whose index delete never+ran, because it runs after the commit, outside the transaction, best-effort.+They point at nothing, so discarding them is the repair.+"""++from __future__ import annotations++import asyncio+import sqlite3+from pathlib import Path++from repowise.cli.commands.doctor_cmd import repo_checks++LIVE_PAGES = 24+ORPHANS = 89++_OLD_SCHEMA_DDL = "CREATE VIRTUAL TABLE page_fts USING fts5(page_id UNINDEXED, title, content)"+++async def _build_drifted_repo(tmp_path: Path) -> Path:+ """A store shaped like the one in the report: 113 indexed rows, 24 pages."""+ import git as gitpython++ from repowise.core.persistence import create_engine, create_session_factory, get_session+ from repowise.core.persistence.crud import upsert_page, upsert_repository+ from repowise.core.persistence.database import init_db++ repo_path = (tmp_path / "repo").resolve()+ repo_path.mkdir()+ gitpython.Repo.init(repo_path)+ repowise_dir = repo_path / ".repowise"+ repowise_dir.mkdir()+ db_path = repowise_dir / "wiki.db"++ engine = create_engine(f"sqlite+aiosqlite:///{db_path}")+ await init_db(engine)+ sf = create_session_factory(engine)+ async with get_session(sf) as session:+ repo = await upsert_repository(+ session, name="repo", local_path=str(repo_path), url="https://example.test/repo"+ )+ for i in range(LIVE_PAGES):+ await upsert_page(+ session,+ page_id=f"file_page:src/mod_{i}.py",+ repository_id=repo.id,+ page_type="file_page",+ title=f"File: src/mod_{i}.py",+ content=f"# Overview\n\nModule {i} does a substantial amount of work. " * 6,+ summary=f"Module {i}.",+ target_path=f"src/mod_{i}.py",+ source_hash="",+ model_name="mock",+ provider_name="mock",+ )+ await session.commit()+ await engine.dispose()++ # Rewind the index to the pre-0.38 shape and leave the orphans of sweeps+ # whose FTS delete never completed. Written through sqlite3 rather than the+ # ORM because no supported code path can produce this state any more.+ conn = sqlite3.connect(db_path)+ conn.execute("DROP TABLE page_fts")+ conn.execute(_OLD_SCHEMA_DDL)+ conn.executemany(+ "INSERT INTO page_fts(page_id, title, content) VALUES (?,?,?)",+ [(f"file_page:src/mod_{i}.py", f"mod_{i}", f"body {i}") for i in range(LIVE_PAGES)]+ + [(f"module:legacy/pkg_{i}", f"pkg_{i}", f"swept body {i}") for i in range(ORPHANS)],+ )+ conn.commit()+ conn.close()++ return repo_path+++def _indexed_ids(repo_path: Path) -> set[str]:+ conn = sqlite3.connect(repo_path / ".repowise" / "wiki.db")+ try:+ return {r[0] for r in conn.execute("SELECT page_id FROM page_fts")}+ finally:+ conn.close()+++def _fts_columns(repo_path: Path) -> list[str]:+ conn = sqlite3.connect(repo_path / ".repowise" / "wiki.db")+ try:+ return [r[1] for r in conn.execute("PRAGMA table_info(page_fts)")]+ finally:+ conn.close()+++def test_the_drift_is_reported(tmp_path: Path) -> None:+ """The check runs on the old column set — it reads ``page_id`` only."""+ repo_path = asyncio.run(_build_drifted_repo(tmp_path))++ _all_ok, checks = repo_checks._run_repo_checks(repo_path, repair=False)+ rows = {c.name: (c.ok, c.detail) for c in checks}++ assert "SQL ↔ FTS Index" in rows+ ok, detail = rows["SQL ↔ FTS Index"]+ assert ok is False+ assert detail == f"0 missing, {ORPHANS} orphaned"+++def test_repair_clears_the_orphans_instead_of_raising(tmp_path: Path) -> None:+ """The reported failure: the repair died on the drift it was called for."""+ repo_path = asyncio.run(_build_drifted_repo(tmp_path))++ repo_checks._run_repo_checks(repo_path, repair=True)++ assert _indexed_ids(repo_path) == {f"file_page:src/mod_{i}.py" for i in range(LIVE_PAGES)}+++def test_repair_leaves_the_index_on_the_current_column_set(tmp_path: Path) -> None:+ """A repair that leaves the store one command away from the same wedge is+ not a repair. The schema upgrade is what the drift was blocking."""+ repo_path = asyncio.run(_build_drifted_repo(tmp_path))++ repo_checks._run_repo_checks(repo_path, repair=True)++ assert _fts_columns(repo_path) == [+ "page_id",+ "title",+ "content",+ "summary",+ "target_path",+ ]+++def test_a_plain_run_heals_the_drift_on_its_own(tmp_path: Path) -> None:+ """Opening the store is enough. ``doctor`` is the recovery path, not the+ only one: every command runs ``ensure_index`` on the way in, which is where+ the orphans of an interrupted sweep now get collected."""+ from repowise.core.persistence import FullTextSearch, create_engine++ repo_path = asyncio.run(_build_drifted_repo(tmp_path))++ async def _open() -> None:+ engine = create_engine(+ f"sqlite+aiosqlite:///{repo_path / '.repowise' / 'wiki.db'}"+ )+ await FullTextSearch(engine).ensure_index()+ await engine.dispose()++ asyncio.run(_open())++ assert _indexed_ids(repo_path) == {f"file_page:src/mod_{i}.py" for i in range(LIVE_PAGES)}+ assert _fts_columns(repo_path)[-1] == "target_path"diff --git a/tests/unit/persistence/test_search_fts_columns.py b/tests/unit/persistence/test_search_fts_columns.pyindex e9057e8d0..ebb949e02 100644--- a/tests/unit/persistence/test_search_fts_columns.py+++ b/tests/unit/persistence/test_search_fts_columns.py@@ -18,7 +18,7 @@ import pytestfrom sqlalchemy.sql import textfrom repowise.core.persistence.crud import upsert_page-from repowise.core.persistence.search import FullTextSearch+from repowise.core.persistence.search import PAGE_FTS_COLUMNS, FullTextSearchfrom tests.unit.persistence.helpers import insert_repo_OLD_SCHEMA_DDL = "CREATE VIRTUAL TABLE page_fts USING fts5(page_id UNINDEXED, title, content)"@@ -186,26 +186,129 @@ async def test_index_with_an_empty_summary_does_not_warn(async_engine):assert fts._warned_missing_fields is False-async def test_rebuild_refuses_to_shrink_the_index(async_engine, async_session, repo):- """The upgrade drops the table, so it must not run against a thinner source.-- The rebuild refills from ``wiki_pages``. If that table cannot account for- the rows already indexed — a store whose two halves have drifted apart, or- an engine pointed at the wrong file — dropping and refilling silently- deletes searchable pages. Raise instead, and leave the old index in place.- """- await _downgrade_to_old_schema(async_engine)- async with async_engine.begin() as conn:- for i in range(3):+async def _insert_old_rows(engine, page_ids: list[str]) -> None:+ async with engine.begin() as conn:+ for pid in page_ids:await conn.execute(text("INSERT INTO page_fts(page_id, title, content) VALUES (:p, :t, :c)"),- {"p": f"orphan-{i}", "t": "Orphan", "c": "no wiki_pages row backs this"},+ {"p": pid, "t": pid, "c": f"body of {pid}"},)- with pytest.raises(RuntimeError, match="page_fts"):++async def _indexed_ids(engine) -> set[str]:+ async with engine.connect() as conn:+ rows = await conn.execute(text("SELECT page_id FROM page_fts"))+ return {r[0] for r in rows.fetchall()}+++async def test_rebuild_discards_orphans_instead_of_refusing(+ async_engine, async_session, repo+):+ """An index holding more rows than ``wiki_pages`` still upgrades (#1309).++ The excess is orphans: rows whose page was swept from SQL while the FTS+ delete that should have followed never ran. Refusing to rebuild protected+ rows that already point at nothing, and did it from inside the call every+ command makes to open the store — including ``doctor --repair``, which the+ error then told the user to run.+ """+ await _seed_page(async_session, repo.id)+ await _downgrade_to_old_schema(async_engine)+ await _insert_old_rows(+ async_engine,+ ["file_page:src/main.py", "orphan-0", "orphan-1", "orphan-2"],+ )++ await FullTextSearch(async_engine).ensure_index()++ assert await _fts_columns(async_engine) == list(PAGE_FTS_COLUMNS)+ assert await _indexed_ids(async_engine) == {"file_page:src/main.py"}+++async def test_rebuild_reports_the_orphans_it_dropped(async_engine, async_session, repo, caplog):+ """Silently discarding indexed rows is how the next drift goes unnoticed."""+ await _seed_page(async_session, repo.id)+ await _downgrade_to_old_schema(async_engine)+ await _insert_old_rows(async_engine, ["file_page:src/main.py", "orphan-0", "orphan-1"])++ with caplog.at_level("WARNING", logger="repowise.core.persistence.search"):await FullTextSearch(async_engine).ensure_index()- async with async_engine.connect() as conn:- rows = await conn.execute(text("SELECT count(*) FROM page_fts"))- assert rows.scalar() == 3+ assert any("2 orphaned page_fts row" in r.getMessage() for r in caplog.records)+++async def test_rebuild_survives_an_index_with_no_pages_behind_it(async_engine):+ """Every row orphaned is the extreme of the same case, not a special one."""+ await _downgrade_to_old_schema(async_engine)+ await _insert_old_rows(async_engine, ["orphan-0", "orphan-1"])++ await FullTextSearch(async_engine).ensure_index()++ assert await _fts_columns(async_engine) == list(PAGE_FTS_COLUMNS)+ assert await _indexed_ids(async_engine) == set()+++async def test_rebuild_leaves_the_old_index_alone_when_wiki_pages_is_missing(async_engine):+ """No source to refill from is the one case where rebuilding loses data.++ It must not raise either: ``ensure_index`` is on the way in to every+ command, so an exception here takes down a CLI whose index still answers+ queries perfectly well on its old shape.+ """+ await _downgrade_to_old_schema(async_engine)+ await _insert_old_rows(async_engine, ["p1"])+ async with async_engine.begin() as conn:+ await conn.execute(text("DROP TABLE wiki_pages"))++ await FullTextSearch(async_engine).ensure_index()+assert await _fts_columns(async_engine) == ["page_id", "title", "content"]+ assert await _indexed_ids(async_engine) == {"p1"}+++async def test_prune_orphans_removes_rows_whose_page_is_gone(+ async_engine, async_session, repo+):+ """The residue of a sweep whose FTS delete never ran.++ Six call sites delete pages from SQL and their index rows afterwards,+ outside the transaction, best-effort. Anything that ends the process in+ between leaves a row that answers queries in full and 404s when opened.+ """+ await _seed_page(async_session, repo.id)+ fts = FullTextSearch(async_engine)+ await fts.ensure_index()+ await fts.index("file_page:src/main.py", "Main", "body", summary="s", target_path="p")+ await fts.index("swept:gone", "Gone", "body of a swept page", summary="s", target_path="p")++ assert await fts.prune_orphans() == 1+ assert await _indexed_ids(async_engine) == {"file_page:src/main.py"}+++async def test_prune_orphans_is_a_no_op_on_a_clean_store(async_engine, async_session, repo):+ """It runs on every command, so the common case must not touch the index."""+ await _seed_page(async_session, repo.id)+ fts = FullTextSearch(async_engine)+ await fts.ensure_index()+ await fts.index("file_page:src/main.py", "Main", "body", summary="s", target_path="p")++ assert await fts.prune_orphans() == 0+ assert await _indexed_ids(async_engine) == {"file_page:src/main.py"}+++async def test_ensure_index_prunes_orphans_on_a_current_schema(+ async_engine, async_session, repo+):+ """The self-heal cannot depend on there being a column upgrade to do.++ A store already on the current shape is where the orphans of an+ interrupted sweep land from now on, and nothing else ever reconciles them.+ """+ await _seed_page(async_session, repo.id)+ fts = FullTextSearch(async_engine)+ await fts.ensure_index()+ await fts.index("swept:gone", "Gone", "body", summary="s", target_path="p")++ await FullTextSearch(async_engine).ensure_index()++ assert await _indexed_ids(async_engine) == set()