Graph-Native Infrastructure for Context and Accountable AI Systems
Python agent-memoryaiai-governanceai-infrastructureartificial-intelligencecontext-engineeringcontext-graphsdata-engineeringdecision-intelligencedeveloper-toolsexplainable-aigenerative-aigraph-ragknowledge-graphllmontologyprovenancepythonreasoningsemantic-search
fix(search-index): restore secondary-scan node ordering and add regression test
KaifAhmad1fe6ca7fccb9a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 00984f03..c30e6f9a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -14,7 +14,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0- **Scoring**: exact ID match → 140, exact term → 120, primary-text substring → 78 + length bonus, token hit → 18, prefix hit → 10, multi-token bonus → 4 per hit. Ties broken deterministically on `(score, exactness, token_hits, node_id)`.- **Mutation sync**: `GraphSession.add_node()`, `add_nodes()`, `add_edges()` and `add_node()` update the index incrementally. `handle_graph_mutation()` integrates with the WebSocket mutation bridge for live updates during reasoning, enrichment, and remote graph changes. `rebuild_search_index()` performs a full O(n) rebuild when needed (session init, merge, reload). `enrich.py` routes node/edge additions through `session.add_node`/`session.add_edge` so reasoning-inferred nodes are indexed immediately.- **Review fixes applied**: replaced `list.sort()` per upsert with `bisect.insort()` (O(log n) vs O(n log n)); replaced `list.remove()` with `bisect.bisect_left` + `pop()` (O(log n) vs O(n)); added `with self._lock` in `handle_graph_mutation()` to prevent index races from the WebSocket thread; removed unnecessary source/target upserts on `add_edge()` (edges don't affect node text); sorted tag values in `_cache_key()` so `["a","b"]` and `["b","a"]` share a cache entry.+ - **Follow-up fix by @KaifAhmad1 and @ZohaibHassan16**: restored `_ordered_node_ids` maintenance during upserts so `secondary_scan` fallback works for terms that are only present in non-curated properties; added regression test coverage to lock this behavior.- 3 new tests: `test_search_exact_and_prefix` (exact match + prefix match), `test_search_filters_and_cache_stability` (type + confidence filter, identical repeated requests), `test_search_sees_new_nodes_after_mutation` (node added via `session.add_node()` immediately visible in search).+ - 1 additional regression test: `test_search_secondary_scan_fallback_matches_non_curated_properties` (verifies fallback matching when a query term appears only in non-curated properties).- **Fix: Provenance traversal now includes multi-hop upstream ancestors + edge direction classification** (closes #470, PR #480 by @Sameer6305, review fixes by @KaifAhmad1):- **Bug — upstream ancestors silently excluded**: `_build_provenance()` built a directed `nx.DiGraph` and seeded first-hop neighbors correctly, but the final subgraph extraction called `nx.ego_graph(..., undirected=False)`. With directed traversal, ego-graph expansion only follows outgoing edges from the focus node, so any node that *points into* the focus node (i.e. an upstream ancestor at depth ≥ 2) was invisible. For the chain `Source → Intermediate → node_id`, `Intermediate` appeared at hop 1 but `Source` was silently dropped. Fixed by changing to `undirected=True` — the radius expansion now traverses both incoming and outgoing edges while the underlying `DiGraph` is preserved, so edge source/target semantics remain correct.- **Enhancement — edge direction classification**: `ProvenanceEdge` gains a `direction: str` field. Each edge in the provenance subgraph is classified relative to the focus node: `"upstream"` when `target == node_id` (edge flows into the focus node), `"downstream"` when `source == node_id` (edge flows out), and `"lateral"` for all other edges between non-focus neighbors. This lets consumers distinguish ancestor provenance from descendant impact without re-traversing the graph.diff --git a/semantica/explorer/search_index.py b/semantica/explorer/search_index.pyindex 1e4df821..d13372ce 100644--- a/semantica/explorer/search_index.py+++ b/semantica/explorer/search_index.py@@ -180,8 +180,7 @@ class GraphSearchIndex:for length in range(self.prefix_min_length, min(len(token), self.prefix_max_length) + 1):self._prefix_index[token[:length]].add(node_id)- if node_id not in self._documents:- bisect.insort(self._ordered_node_ids, node_id)+ bisect.insort(self._ordered_node_ids, node_id)if clear_cache:self.clear_cache()diff --git a/tests/explorer/test_explorer_api.py b/tests/explorer/test_explorer_api.pyindex 34402509..d420fcb7 100644--- a/tests/explorer/test_explorer_api.py+++ b/tests/explorer/test_explorer_api.py@@ -305,6 +305,23 @@ class TestSearchAndStats:assert "metformin" in result_idsassert "metformin_hcl" in result_ids+ def test_search_secondary_scan_fallback_matches_non_curated_properties(self, client):+ session = client.app.state.session+ assert session.add_node(+ "fallback_node",+ "entity",+ content="Alpha",+ description="rareterm",+ )++ response = client.post(+ "/api/graph/search",+ json={"query": "rareterm", "limit": 10},+ )+ assert response.status_code == 200+ result_ids = [item["node"]["id"] for item in response.json()["results"]]+ assert "fallback_node" in result_ids+def test_stats(self, client):response = client.get("/api/graph/stats")assert response.status_code == 200