Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

refactor(workspace): one dialect base for every contract type, and topics that follow RabbitMQ bindings (#2501)

Raghav Chamadiyafd1bb75c77b0
diff --git a/docs/architecture/language-support.md b/docs/architecture/language-support.md
index e4faf6de0..c297715bb 100644
--- a/docs/architecture/language-support.md
+++ b/docs/architecture/language-support.md
@@ -1019,41 +1019,47 @@ language produces no findings rather than wrong ones.
## Workspace contract extraction
In workspace mode (multiple repos indexed together), repowise links
-service-to-service API contracts (HTTP routes, gRPC services, and DB tables) so a
-provider endpoint in one repo connects to its consumers in another. The
-extractors live in `core/workspace/extractors/` and follow the same
-dialect-plugin shape: the orchestrator owns only the file walk, and each
-framework / client library is an independent module registered in a tuple.
+service-to-service contracts (HTTP routes, gRPC services, message topics,
+sockets, and DB tables) so a provider in one repo connects to its consumers in
+another. The extractors live in `core/workspace/extractors/` and share one
+dialect shape: each framework or client library is an independent module
+registered in a tuple, and one walk-and-dispatch loop runs them all.
```
workspace/extractors/
base.py # iter_source_files walk + ScanContext (shared by all)
+ dialect.py # ContractDialect protocol, build_contract, DialectExtractor
+ strings.py # string expressions: argument scanning, constants, Arg selection
langs.py # registry-derived extension sets (JS_TS, PYTHON, RUST, …)
http/
- dialect.py # HttpDialect protocol + build_provider/consumer_contract
+ dialect.py # build_provider/consumer_contract
paths.py # normalize_http_path + URL helpers
- express.py fastapi.py spring.py laravel.py go.py aspnet.py # providers
- js_clients.py python_clients.py csharp_http.py rust_clients.py # consumers
- rust_axum.py mounts.py # providers
- __init__.py # HttpExtractor + PROVIDER_DIALECTS / CONSUMER_DIALECTS
- grpc/
- dialect.py # GrpcDialect protocol + make_grpc_contract
- proto.py go.py java.py python.py typescript.py csharp.py
- __init__.py # GrpcExtractor + DIALECTS
+ client_calls.py # ClientCallMatch + consumer_contracts
+ express.py fastapi.py spring.py laravel.py go.py aspnet.py … # providers
+ js_clients.py python_clients.py php_clients.py … # consumers
+ __init__.py # HttpExtractor (adds router mounts + index passes)
+ grpc/ # proto.py + languages.py (one table of stub shapes)
+ topic/ # dialect.py (TopicCall table) + kafka.py rabbitmq.py nats.py
+ socket/ # dialect.py + dotnet.py python.py
data/ # table providers (DDL / ORM entities) <-> SQL consumers
+workspace/matching/ # shared exact pass + per-type passes (http.py, topic.py)
```
A dialect declares the file extensions it understands (via `langs.py`) and turns
-regex matches into `Contract`s through shared builders, so every dialect emits
-identically-shaped providers/consumers and path-normalization lives in one place.
-**Adding a framework or client** means dropping one module into `http/`, `grpc/`,
-or `data/` and appending its dialect to the relevant registry tuple, with no
-orchestrator edits.
+matches into `Contract`s through shared builders, so every dialect emits
+identically-shaped providers/consumers and normalization lives in one place.
+**Adding a framework or client** means dropping one module into `http/`,
+`grpc/`, `topic/`, `socket/` or `data/` and appending its dialect to the
+relevant registry tuple, with no orchestrator edits. A contract type whose two
+ends can name one thing differently (HTTP mount prefixes, a queue bound to an
+exchange) registers its extra passes in `matching.MATCHERS`.
| Contract | Providers | Consumers |
|----------|-----------|-----------|
| **HTTP** | Express, FastAPI, Spring, Laravel, Go (gin/echo/chi/net-http), ASP.NET (attribute + minimal), Rust (Axum routes, Actix/Rocket attribute macros) | `fetch` / `axios` / URL-literal wrappers (JS/TS), `requests` / `httpx` (Python), `HttpClient` / `UnityWebRequest` / Best.HTTP (C#), `reqwest` (Rust) |
| **gRPC** | `.proto` IDL, Go, Java, Python, NestJS (`@GrpcMethod`), C# (gRPC-dotnet) | Go, Java, Python, C# |
+| **Topic** | Kafka (Spring Kafka, kafkajs, kafka-python/confluent, sarama), RabbitMQ (Spring AMQP, amqplib, pika), NATS | The same libraries' consumers, plus RabbitMQ queue bindings |
+| **Socket** | SignalR `MapHub`, FastAPI `@app.websocket` | ClientWebSocket, SignalR client, NativeWebSocket, WebSocketSharp |
| **Data** | DDL `CREATE`/`ALTER`, Alembic `op.create_table`, ORM entities (SQLAlchemy, SQLModel, Django, JPA, EF Core, ActiveRecord, Eloquent) | SQL string literals in app code (sqlglot-parsed, verb-anchored-regex fallback) |
See [docs/scale/WORKSPACES.md](../scale/WORKSPACES.md) for the user-facing
diff --git a/docs/scale/WORKSPACES.md b/docs/scale/WORKSPACES.md
index 7056f1c98..ef8c3c710 100644
--- a/docs/scale/WORKSPACES.md
+++ b/docs/scale/WORKSPACES.md
@@ -234,11 +234,24 @@ Scans source files for HTTP route handlers, gRPC service definitions, and messag
| HTTP | Express, FastAPI, Spring, Laravel, Go (gin/echo/chi/net-http), ASP.NET (attribute + minimal API), Rust (Axum routes, Actix/Rocket attribute macros) | fetch/axios/URL-literal wrappers (JS/TS), requests/httpx (Python), HttpClient/UnityWebRequest/Best.HTTP (C#), reqwest (Rust) |
| gRPC | `.proto` service definitions, plus per-language dialects (Go, Java, Python, C#, TypeScript, NestJS `@GrpcMethod`) | gRPC client stubs |
| Data / DB | DDL (`CREATE TABLE`/`VIEW`/`MATERIALIZED VIEW`), ORM dialects (SQLAlchemy, Django, JPA, EF Core, ActiveRecord, Eloquent) | Raw SQL string literals in app code (verb-anchored: `SELECT`/`INSERT`/`UPDATE`/`DELETE`/`MERGE`) |
-| Topics | Kafka, RabbitMQ, NATS producers | Corresponding consumers |
+| Topics | Kafka (Spring Kafka, kafkajs, kafka-python/confluent, sarama), RabbitMQ (Spring AMQP, amqplib, pika), NATS producers | The corresponding consumers, plus RabbitMQ queue bindings (`bindQueue`, `queue_bind`) |
| Socket / WebSocket | SignalR `MapHub<T>("/path")`, FastAPI `@app.websocket("/path")` | ClientWebSocket `ConnectAsync`, SignalR `HubConnectionBuilder.WithUrl`, NativeWebSocket and WebSocketSharp `new WebSocket(...)` |
Socket detection is C#/Python only and is toggled by `detect_socket` in the `contracts:` block below.
+A topic, queue or exchange name is read the way a URL is: a literal, or a name
+the same file assigns exactly once to a literal (Python, Go and Java today), is
+resolved; a name built at runtime is skipped rather than guessed. RabbitMQ
+publishers name an exchange and a routing key while consumers name a queue, so
+a queue binding found anywhere in the workspace connects the two: each consumer
+of the bound queue links to the exchange's publishers whose routing key the
+binding pattern accepts (`*` and `#` follow topic-exchange rules; an empty key
+or pattern matches everything, while a key the source does not settle matches
+nothing). Such a link carries the exchange as its `contract_id` and the queue as
+`consumer_contract_id`. When no consumer of the bound queue is found, the
+binding site itself is linked. A publish to the default exchange
+(`publish('', 'jobs')`) is a publish to the queue `jobs`.
+
Data/DB contracts use the id scheme `data::<table>` and render as a `db` edge in the [system graph](#system-graph). The consumer side (SQL string matching) is heuristic and lower-confidence than the ORM-based providers; unlike HTTP and gRPC, there is no field-level breaking-change diffing for data contracts, only table/route-level removal.
HTTP routes are matched on their **full** path: a router mount prefix
diff --git a/packages/api-client/src/types/workspace.ts b/packages/api-client/src/types/workspace.ts
index 519e0879e..cc960e2a2 100644
--- a/packages/api-client/src/types/workspace.ts
+++ b/packages/api-client/src/types/workspace.ts
@@ -117,6 +117,11 @@ export interface WorkspaceContractLinkEntry {
*/
provider_symbol_id: string | null;
consumer_symbol_id: string | null;
+ /**
+ * The consumer's own contract id when it reached the provider under another
+ * name (a queue bound to the exchange `contract_id` names); null otherwise.
+ */
+ consumer_contract_id?: string | null;
}
export interface WorkspaceContractsResponse {
diff --git a/packages/core/src/repowise/core/workspace/contracts.py b/packages/core/src/repowise/core/workspace/contracts.py
index 467f53cf9..5ce7d0ec0 100644
--- a/packages/core/src/repowise/core/workspace/contracts.py
+++ b/packages/core/src/repowise/core/workspace/contracts.py
@@ -4,7 +4,8 @@ Write path: runs during ``repowise update --workspace``.
Results read by ``CrossRepoEnricher`` in the MCP server (read path).
Contracts are persisted as ``.repowise-workspace/contracts.json`` — separate
-from ``cross_repo_edges.json`` so Phase 3 and Phase 4 fail independently.
+from ``cross_repo_edges.json`` so extraction and cross-repo edges fail independently.
+Provider-to-consumer matching lives in :mod:`.matching`.
"""
from __future__ import annotations
@@ -13,8 +14,6 @@ import asyncio
import hashlib
import json
import logging
-import re
-from collections import defaultdict
from dataclasses import asdict, dataclass, field
from datetime import UTC, datetime
from pathlib import Path
@@ -51,7 +50,22 @@ CONTRACTS_FILENAME = "contracts.json"
#: A store written under an older version is readable but not reusable: its
#: rows carry no identity, and nothing short of re-extraction can give them one.
# Version 8 adds bounded OpenAPI 3.x schemas and their extraction diagnostics.
-CONTRACTS_VERSION = 8
+# Version 9 gives topic contracts a kind and routing key, and adds RabbitMQ
+# queue bindings.
+CONTRACTS_VERSION = 9
+
+#: ``meta["kind"]`` of a topic contract: the destination a broker call names.
+#: A queue is read by one consumer group; a topic or subject fans out; an
+#: exchange routes to the queues bound to it; a binding is a queue subscribing
+#: to an exchange, recorded as a consumer of that exchange.
+TOPIC_KIND_TOPIC = "topic"
+TOPIC_KIND_QUEUE = "queue"
+TOPIC_KIND_EXCHANGE = "exchange"
+TOPIC_KIND_SUBJECT = "subject"
+TOPIC_KIND_BINDING = "binding"
+#: Set on a publish or binding whose routing key the file does not settle; the
+#: topic matcher refuses to route through it rather than read it as match-all.
+TOPIC_ROUTING_KEY_UNRESOLVED = "routing_key_unresolved"
# ---------------------------------------------------------------------------
@@ -141,6 +155,10 @@ class ContractLink:
#: a display label. None when the contract never bound to one.
provider_symbol_id: str | None = None
consumer_symbol_id: str | None = None
+ #: The consumer's own contract id, when it reached the provider under
+ #: another name (a queue bound to the exchange ``contract_id`` names).
+ #: None when both ends share ``contract_id``, which is every other link.
+ consumer_contract_id: str | None = None
def to_dict(self) -> dict[str, Any]:
d = asdict(self)
@@ -148,7 +166,7 @@ class ContractLink:
del d["provider_service"]
if d["consumer_service"] is None:
del d["consumer_service"]
- for optional in ("provider_symbol_id", "consumer_symbol_id"):
+ for optional in ("provider_symbol_id", "consumer_symbol_id", "consumer_contract_id"):
if d[optional] is None:
del d[optional]
return d
@@ -170,6 +188,7 @@ class ContractLink:
consumer_service=data.get("consumer_service"),
provider_symbol_id=data.get("provider_symbol_id"),
consumer_symbol_id=data.get("consumer_symbol_id"),
+ consumer_contract_id=data.get("consumer_contract_id"),
)
@@ -284,6 +303,14 @@ def bind_symbol_ids(contracts: list[Contract], index: RepoIndex | None) -> dict[
return counts
+def same_service(repo_a: str, service_a: str | None, repo_b: str, service_b: str | None) -> bool:
+ """True when two ends sit in one repo and one service boundary (or neither has one).
+
+ Such a pair is one program calling itself, not a cross-repo contract.
+ """
+ return repo_a == repo_b and service_a == service_b
+
+
# ---------------------------------------------------------------------------
# Contract ID normalization
# ---------------------------------------------------------------------------
@@ -340,396 +367,6 @@ def normalize_contract_id(contract_id: str) -> str:
return contract_id.lower()
-# ---------------------------------------------------------------------------
-# Matching engine
-# ---------------------------------------------------------------------------
-
-# Common API mount / version path prefixes. When exact matching fails, the
-# candidate pass strips these leading segments (plus unresolved base ``{param}``
-# segments) from both provider and consumer paths so a consumer that hits
-# ``/api/v1/users`` can still link to a provider mounted at ``/users`` (or vice
-# versa). Such links are emitted as lower-confidence ``candidate`` matches.
-#
-# Kept deliberately small: only segments that are almost never real resource
-# names. Words like ``internal``/``public``/``gateway`` are excluded because
-# they double as legitimate route segments and would conflate unrelated routes.
-_MOUNT_PREFIX_SEGMENTS = frozenset({"api", "rest"})
-_VERSION_SEGMENT_RE = re.compile(r"^v\d+$")
-
-# Confidence multiplier applied to candidate (non-exact) links.
-_CANDIDATE_CONFIDENCE_FACTOR = 0.6
-
-# Request paths ending in these suffixes are static assets, never API contracts.
-# They are excluded from the candidate pass so a ``fetch('/static/app.js')``
-# can't spuriously link to a provider route that shares a suffix. ``.json`` and
-# ``.xml`` are intentionally absent — real APIs serve those.
-_STATIC_ASSET_SUFFIXES = (
- ".js",
- ".mjs",
- ".cjs",
- ".css",
- ".map",
- ".html",
- ".htm",
- ".png",
- ".jpg",
- ".jpeg",
- ".gif",
- ".svg",
- ".ico",
- ".webp",
- ".avif",
- ".woff",
- ".woff2",
- ".ttf",
- ".eot",
- ".otf",
- ".pdf",
- ".txt",
- ".wasm",
-)
-
-
-def _find_matching_keys(
- consumer_id: str,
- provider_index: dict[str, list[Contract]],
-) -> list[str]:
- """Find provider index keys that match *consumer_id*."""
- normalized = normalize_contract_id(consumer_id)
-
- if normalized in provider_index:
- return [normalized]
-
- # HTTP wildcard: consumer http::*::/path matches any method on that path
- if normalized.startswith("http::*::"):
- path_suffix = normalized[len("http::*::") :]
- return [
- k for k in provider_index if k.startswith("http::") and k.endswith(f"::{path_suffix}")
- ]
-
- # HTTP: check for wildcard providers (http::*::/path from Go HandleFunc)
- if normalized.startswith("http::") and not normalized.startswith("http::*::"):
- parts = normalized.split("::", 2)
- if len(parts) == 3:
- wildcard_key = f"http::*::{parts[2]}"
- if wildcard_key in provider_index:
- return [wildcard_key]
-
- # gRPC wildcard: grpc::service/* matches grpc::service/Method
- if normalized.endswith("/*"):
- prefix = normalized[:-1] # "grpc::service/"
- return [k for k in provider_index if k.startswith(prefix)]
-
- return []
-
-
-def _split_http_id(normalized_id: str) -> tuple[str, str] | None:
- """Return ``(method, path)`` for a normalized ``http::`` id, else ``None``."""
- parts = normalized_id.split("::", 2)
- if len(parts) != 3 or parts[0] != "http":
- return None
- return parts[1], parts[2]
-
-
-def _candidate_http_path(path: str) -> str:
- """Reduce an HTTP path to its mount-agnostic core for candidate matching.
-
- Strips leading unresolved base ``{param}`` segments and known mount/version
- prefixes so routes that differ only by an API mount or version prefix
- collapse to the same key:
-
- - ``/api/v1/users`` → ``/users``
- - ``/{param}/resource`` → ``/resource``
- - ``/v1/resource`` → ``/resource``
- """
- segments = [s for s in path.split("/") if s]
- while segments and (
- segments[0] == "{param}"
- or segments[0] in _MOUNT_PREFIX_SEGMENTS
- or _VERSION_SEGMENT_RE.match(segments[0])
- ):
- segments.pop(0)
- return "/" + "/".join(segments)
-
-
-def _is_static_asset_path(path: str) -> bool:
- """True when *path*'s final segment looks like a static asset file."""
- last = path.rsplit("/", 1)[-1].split("?")[0].lower()
- return last.endswith(_STATIC_ASSET_SUFFIXES)
-
-
-def _methods_compatible(consumer_method: str, provider_method: str) -> bool:
- """HTTP methods match if equal or either side is the ``*`` wildcard."""
- return consumer_method == provider_method or consumer_method == "*" or provider_method == "*"
-
-
-def _same_repo_same_service(provider: Contract, consumer: Contract) -> bool:
- """Skip internal calls: same repo and same service boundary (or both None)."""
- return provider.repo == consumer.repo and provider.service == consumer.service
-
-
-def _make_link(
- consumer: Contract,
- provider: Contract,
- match_type: str,
- confidence: float,
- seen: set[tuple[str, str, str, str, str]],
-) -> ContractLink | None:
- """Build a deduplicated ContractLink, or ``None`` if already emitted."""
- dedup_key = (
- normalize_contract_id(consumer.contract_id),
- consumer.repo,
- consumer.file_path,
- provider.repo,
- provider.file_path,
- )
- if dedup_key in seen:
- return None
- seen.add(dedup_key)
-
- return ContractLink(
- contract_id=consumer.contract_id,
- contract_type=consumer.contract_type,
- match_type=match_type,
- confidence=confidence,
- provider_repo=provider.repo,
- provider_file=provider.file_path,
- provider_symbol=provider.symbol_name,
- provider_service=provider.service,
- consumer_repo=consumer.repo,
- consumer_file=consumer.file_path,
- consumer_symbol=consumer.symbol_name,
- provider_symbol_id=provider.symbol_id,
- consumer_symbol_id=consumer.symbol_id,
- consumer_service=consumer.service,
- )
-
-
-def _build_candidate_index(
- provider_index: dict[str, list[Contract]],
-) -> dict[str, list[Contract]]:
- """Index HTTP providers by their mount-agnostic candidate path."""
- candidate_index: dict[str, list[Contract]] = defaultdict(list)
- for key, providers in provider_index.items():
- split = _split_http_id(key)
- if split is None:
- continue
- core = _candidate_http_path(split[1])
- if core in ("", "/"):
- continue # nothing concrete left to match on
- candidate_index[core].extend(providers)
- return candidate_index
-
-
-# Hosts that name the local machine, not a specific service. A consumer URL
-# pointing here carries no service identity, so it is resolved by path uniqueness
-# rather than excluded as third-party.
-_LOCALHOST_HOSTS = frozenset({"localhost", "127.0.0.1", "0.0.0.0", "::1"})
-
-# Internal-DNS suffixes whose leading label names a service (k8s / mesh / LAN).
-# Only for these do we map ``<label>.<suffix>`` to a repo alias; a public host
-# like ``backend.stripe.com`` must never be mistaken for the ``backend`` repo.
-_INTERNAL_HOST_SUFFIXES = (".local", ".internal", ".svc.cluster.local")
-
-
-def _resolve_consumer_target(
- consumer: Contract,
- repo_aliases: set[str],
- service_bases: dict[str, str],
-) -> tuple[str | None, bool]:
- """Resolve a consumer's target repo and whether it is a third-party call.
-
- Returns ``(target_repo, is_external)``:
-
- - ``service_bases`` (host or ``${BASE}`` token) wins first;
- - a host equal to a workspace repo alias, or an internal-DNS host whose
- leading label is a repo alias (``backend.svc.cluster.local``), is internal;
- - localhost and bare unknown hostnames (e.g. a docker-compose service we
- can't map) return ``(None, False)`` so path matching still applies;
- - any other *public* dotted host is third-party (``is_external=True``).
- """
- meta = consumer.meta
- host = meta.get("host")
- if host:
- if host in service_bases:
- return service_bases[host], False
- if host in repo_aliases:
- return host, False
- if host in _LOCALHOST_HOSTS:
- return None, False
- if host.endswith(_INTERNAL_HOST_SUFFIXES):
- label = host.split(".")[0]
- return (label, False) if label in repo_aliases else (None, False)
- if "." not in host:
- return None, False # bare unknown hostname, not necessarily third-party
- return None, True # public dotted host, unmapped → third-party
- token = meta.get("base_token")
- if token and token.lower() in service_bases:
- return service_bases[token.lower()], False
- return None, False
-
-
-def annotate_consumer_targets(
- contracts: list[Contract],
- service_bases: dict[str, str] | None = None,
-) -> None:
- """Stamp each HTTP consumer's ``meta`` with its resolved target / external bit.
-
- Mutates the contracts in place so both :func:`match_contracts` and the
- diagnostics builder read one resolution. ``service_bases`` maps a base token
- or host (case-insensitive) to a repo alias.
-
- The two keys this owns are cleared before being recomputed, so the result is
- a function of *contracts* and *service_bases* alone and not of whatever a
- previous run stamped. That matters now that incremental extraction carries
- contracts forward: a consumer resolved to ``gamma`` keeps that ``meta`` when
- it is reused, and if ``gamma`` has since left the workspace the resolution
- no longer fires — so without the clear, the stale target survives and the
- contract claims to call a repo that is not in the workspace any more.
- """
- repo_aliases = {c.repo for c in contracts}
- sb = {k.lower(): v for k, v in (service_bases or {}).items()}
- for c in contracts:
- if c.role != "consumer" or c.contract_type != "http":
- continue
- c.meta.pop("target_repo", None)
- c.meta.pop("external", None)
- target, external = _resolve_consumer_target(c, repo_aliases, sb)
- if external:
- c.meta["external"] = True
- elif target:
- c.meta["target_repo"] = target
-
-
-def _prefer_target_repo(providers: list[Contract], consumer: Contract) -> list[Contract]:
- """Narrow *providers* to the consumer's resolved ``target_repo`` when set.
-
- Falls back to the full list if the target declares no matching provider, so
- a stale/typo'd ``service_bases`` entry never silently drops a real link.
- """
- target = consumer.meta.get("target_repo")
- if not target:
- return providers
- preferred = [p for p in providers if p.repo == target]
- return preferred or providers
-
-
-def match_contracts(contracts: list[Contract]) -> list[ContractLink]:
- """Match providers to consumers across repos.
-
- Passes:
-
- 1. **Exact** — normalized contract IDs must be equal, with HTTP/gRPC
- wildcard handling (``http::*::/path``, ``grpc::Service/*``).
- 1b. **Base-resolved** — a consumer whose URL had an unresolved base prefix
- stripped is matched on its full (host-relative) path; the link is
- ``exact`` when the target service is unambiguous (one matching service, or
- a configured ``service_bases`` target) and ``candidate`` otherwise.
- 2. **Candidate** — remaining unmatched HTTP consumers retry after collapsing
- known mount/version/base prefixes on both sides, at reduced confidence.
-
- Same-repo same-service calls, and consumers resolved to a third-party host
- (``meta['external']``), are filtered from every pass. Target resolution is
- read from ``meta`` (see :func:`annotate_consumer_targets`).
- """
- provider_index: dict[str, list[Contract]] = defaultdict(list)
- consumers: list[Contract] = []
-
- for c in contracts:
- if c.role == "provider":
- key = normalize_contract_id(c.contract_id)
- provider_index[key].append(c)
- elif not c.meta.get("external"):
- consumers.append(c)
-
- links: list[ContractLink] = []
- seen: set[tuple[str, str, str, str, str]] = set()
- matched_consumers: set[int] = set()
-
- # --- Pass 1: exact / wildcard (excludes base-stripped consumers) ---
- for consumer in consumers:
- if consumer.meta.get("base_stripped"):
- continue
- matching_keys = _find_matching_keys(consumer.contract_id, provider_index)
- providers = [p for k in matching_keys for p in provider_index[k]]
- for provider in _prefer_target_repo(providers, consumer):
- if _same_repo_same_service(provider, consumer):
- continue
- link = _make_link(
- consumer,
- provider,
- "exact",
- min(provider.confidence, consumer.confidence),
- seen,
- )
- if link is not None:
- links.append(link)
- matched_consumers.add(id(consumer))
-
- # --- Pass 1b: base-resolved exact path for base-stripped consumers ---
- for consumer in consumers:
- if id(consumer) in matched_consumers or not consumer.meta.get("base_stripped"):
- continue
- matching_keys = _find_matching_keys(consumer.contract_id, provider_index)
- providers = [
- p
- for k in matching_keys
- for p in provider_index[k]
- if not _same_repo_same_service(p, consumer)
- ]
- if not providers:
- continue
- # A config target only resolves the link when it actually narrows to a
- # provider; a stale/typo'd target falls back to all providers and must
- # not be treated as resolved (else an ambiguous link emits as exact).
- target = consumer.meta.get("target_repo")
- narrowed = [p for p in providers if p.repo == target] if target else []
- if narrowed:
- providers = narrowed
- resolved = bool(narrowed) or len({(p.repo, p.service) for p in providers}) == 1
- match_type = "exact" if resolved else "candidate"
- for provider in providers:
- confidence = min(provider.confidence, consumer.confidence)
- if not resolved:
- confidence = round(confidence * _CANDIDATE_CONFIDENCE_FACTOR, 3)
- link = _make_link(consumer, provider, match_type, confidence, seen)
- if link is not None:
- links.append(link)
- matched_consumers.add(id(consumer))
-
- # --- Pass 2: candidate (mount/version/base prefix) for unmatched HTTP ---
- candidate_index = _build_candidate_index(provider_index)
- for consumer in consumers:
- if id(consumer) in matched_consumers:
- continue
-
- split = _split_http_id(normalize_contract_id(consumer.contract_id))
- if split is None:
- continue # candidate matching is HTTP-only
- method, path = split
- if _is_static_asset_path(path):
- continue
- core = _candidate_http_path(path)
- if core in ("", "/"):
- continue
-
- for provider in _prefer_target_repo(candidate_index.get(core, []), consumer):
- if _same_repo_same_service(provider, consumer):
- continue
- psplit = _split_http_id(normalize_contract_id(provider.contract_id))
- if psplit is None or not _methods_compatible(method, psplit[0]):
- continue
- confidence = round(
- min(provider.confidence, consumer.confidence) * _CANDIDATE_CONFIDENCE_FACTOR,
- 3,
- )
- link = _make_link(consumer, provider, "candidate", confidence, seen)
- if link is not None:
- links.append(link)
-
- return links
-
-
# ---------------------------------------------------------------------------
# Manual links
# ---------------------------------------------------------------------------
@@ -901,6 +538,7 @@ async def run_contract_extraction(
)
from .extractors.base import iter_source_files, make_exclude_predicate
from .extractors.from_index import EXTRACTION_LAYER_KEY, LAYER_REGEX
+ from .matching import annotate_consumer_targets, match_contracts
contract_config = ws_config.contracts
exclude = make_exclude_predicate(tuple(contract_config.exclude_globs))
diff --git a/packages/core/src/repowise/core/workspace/diagnostics.py b/packages/core/src/repowise/core/workspace/diagnostics.py
index a9420fe42..9211e752c 100644
--- a/packages/core/src/repowise/core/workspace/diagnostics.py
+++ b/packages/core/src/repowise/core/workspace/diagnostics.py
@@ -23,9 +23,11 @@ from typing import Any
from repowise.core.workspace.code_api import CODE_CONTRACT_TYPE
from repowise.core.workspace.contracts import (
+ TOPIC_KIND_BINDING,
Contract,
ContractLink,
normalize_contract_id,
+ same_service,
)
from repowise.core.workspace.extractors.from_index import EXTRACTION_LAYER_KEY, LAYER_REGEX
from repowise.core.workspace.signature_schema import SCHEMA_SOURCE
@@ -462,7 +464,7 @@ def _classify_unmatched(
# A matching provider exists. If every one shares this consumer's repo AND
# service boundary, the call is intra-service and was filtered on purpose.
all_internal = all(
- p.repo == consumer.repo and p.service == consumer.service for p in candidates
+ same_service(p.repo, p.service, consumer.repo, consumer.service) for p in candidates
)
return UnmatchedReason.INTERNAL_ONLY if all_internal else UnmatchedReason.UNLINKED
@@ -528,7 +530,8 @@ def build_diagnostics(
matched_providers: set[tuple[str, str, str]] = set()
weak_links = 0
for lk in links:
- matched_consumers.add(_contract_key(lk.consumer_repo, lk.consumer_file, lk.contract_id))
+ consumer_id = lk.consumer_contract_id or lk.contract_id
+ matched_consumers.add(_contract_key(lk.consumer_repo, lk.consumer_file, consumer_id))
matched_providers.add(_contract_key(lk.provider_repo, lk.provider_file, lk.contract_id))
if lk.confidence <= WEAK_LINK_CONFIDENCE_THRESHOLD:
weak_links += 1
@@ -542,7 +545,8 @@ def build_diagnostics(
by_reason: dict[str, int] = defaultdict(int)
for c in consumers:
key = _contract_key(c.repo, c.file_path, c.contract_id)
- if key in matched_consumers:
+ # A binding is wiring its queue's consumers are matched through, not a call.
+ if key in matched_consumers or c.meta.get("kind") == TOPIC_KIND_BINDING:
continue
reason = _classify_unmatched(c, providers_by_norm_id)
by_reason[reason] += 1
diff --git a/packages/core/src/repowise/core/workspace/extractors/__init__.py b/packages/core/src/repowise/core/workspace/extractors/__init__.py
index e5bea2f60..8fe913ebd 100644
--- a/packages/core/src/repowise/core/workspace/extractors/__init__.py
+++ b/packages/core/src/repowise/core/workspace/extractors/__init__.py
@@ -4,16 +4,16 @@ database tables, service boundaries."""
from __future__ import annotations
from .data import DataExtractor, normalize_table_name
-from .grpc_extractor import GrpcExtractor
-from .http_extractor import HttpExtractor, normalize_http_path
+from .grpc import GrpcExtractor
+from .http import HttpExtractor, normalize_http_path
from .openapi import OpenApiExtractor, merge_openapi_providers
from .service_boundary import (
ServiceBoundary,
assign_service,
detect_service_boundaries,
)
-from .socket_extractor import SocketExtractor
-from .topic_extractor import TopicExtractor
+from .socket import SocketExtractor
+from .topic import TopicExtractor
__all__ = [
"DataExtractor",
diff --git a/packages/core/src/repowise/core/workspace/extractors/data/__init__.py b/packages/core/src/repowise/core/workspace/extractors/data/__init__.py
index 6f058c500..7f6bf4138 100644
--- a/packages/core/src/repowise/core/workspace/extractors/data/__init__.py
+++ b/packages/core/src/repowise/core/workspace/extractors/data/__init__.py
@@ -1,21 +1,18 @@
"""App-to-database contract extraction.
Scans source files for table *providers* (DDL, migrations, ORM entities) and
-table *consumers* (SQL string literals in app code). Same architecture as the
-HTTP extractor: each recogniser is an independent dialect module registered in
-:data:`PROVIDER_DIALECTS` / :data:`CONSUMER_DIALECTS`; the :class:`DataExtractor`
-orchestrator owns only the file walk and dispatch. Matching happens downstream
-in ``contracts.match_contracts`` on the normalized ``data::<table>`` id, and a
-matched link renders as a ``db`` edge on the Live System Map.
+table *consumers* (SQL string literals in app code). Each recogniser is an
+independent dialect module registered in :data:`PROVIDER_DIALECTS` /
+:data:`CONSUMER_DIALECTS`, run by the shared :class:`..dialect.DialectExtractor`.
+Matching happens downstream in :mod:`repowise.core.workspace.matching` on the
+normalized ``data::<table>`` id, and a matched link renders as a ``db`` edge on
+the Live System Map.
"""
from __future__ import annotations
-from typing import TYPE_CHECKING
-
-from ..base import ScanContext, select_files
+from ..dialect import ContractDialect, DialectExtractor
from .ddl import DdlDialect
-from .dialect import DataDialect
from .names import normalize_table_name
from .orm_models import (
ActiveRecordDialect,
@@ -26,16 +23,8 @@ from .orm_models import (
)
from .sql_strings import SqlStringsDialect
-if TYPE_CHECKING:
- from collections.abc import Callable, Sequence
- from pathlib import Path
-
- from repowise.core.workspace.contracts import Contract
-
- from ..base import SourceFile
-
# Table-ownership recognisers (DDL + one per ORM family).
-PROVIDER_DIALECTS: tuple[DataDialect, ...] = (
+PROVIDER_DIALECTS: tuple[ContractDialect, ...] = (
DdlDialect(),
SqlAlchemyDjangoDialect(),
JpaDialect(),
@@ -45,49 +34,13 @@ PROVIDER_DIALECTS: tuple[DataDialect, ...] = (
)
# Table-access recognisers.
-CONSUMER_DIALECTS: tuple[DataDialect, ...] = (SqlStringsDialect(),)
+CONSUMER_DIALECTS: tuple[ContractDialect, ...] = (SqlStringsDialect(),)
-def _union_extensions(dialects: tuple[DataDialect, ...]) -> frozenset[str]:
- out: set[str] = set()
- for d in dialects:
- out |= d.extensions
- return frozenset(out)
-
-
-class DataExtractor:
+class DataExtractor(DialectExtractor):
"""Extract table contracts from source files via registered dialects."""
- provider_dialects: tuple[DataDialect, ...] = PROVIDER_DIALECTS
- consumer_dialects: tuple[DataDialect, ...] = CONSUMER_DIALECTS
-
- @classmethod
- def source_extensions(cls) -> frozenset[str]:
- """Every extension this extractor's dialects claim."""
- return _union_extensions(cls.provider_dialects) | _union_extensions(
- cls.consumer_dialects
- )
-
- def extract(
- self,
- repo_path: Path,
- repo_alias: str = "",
- exclude: Callable[[str], bool] | None = None,
- files: Sequence[SourceFile] | None = None,
- ) -> list[Contract]:
- """Scan all source files in *repo_path* and return Contract instances."""
- contracts: list[Contract] = []
- for rel_path, suffix, content in select_files(
- repo_path, self.source_extensions(), exclude, files
- ):
- ctx = ScanContext(repo_alias, rel_path, suffix, content)
- for dialect in self.provider_dialects:
- if suffix in dialect.extensions:
- contracts.extend(dialect.extract(ctx))
- for dialect in self.consumer_dialects:
- if suffix in dialect.extensions:
- contracts.extend(dialect.extract(ctx))
- return contracts
+ dialects = PROVIDER_DIALECTS + CONSUMER_DIALECTS
__all__ = [
diff --git a/packages/core/src/repowise/core/workspace/extractors/data/dialect.py b/packages/core/src/repowise/core/workspace/extractors/data/dialect.py
index 7c5f4b3a8..bdb362629 100644
--- a/packages/core/src/repowise/core/workspace/extractors/data/dialect.py
+++ b/packages/core/src/repowise/core/workspace/extractors/data/dialect.py
@@ -9,25 +9,15 @@ scheme (``data::<normalized table>``) and the normalization rules stay in
from __future__ import annotations
-from typing import TYPE_CHECKING, Protocol, runtime_checkable
+from typing import TYPE_CHECKING
-from ..base import ScanContext
+from ..dialect import build_contract
from .names import normalize_table_name
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
-
-@runtime_checkable
-class DataDialect(Protocol):
- """A table-ownership/access recogniser for a set of file extensions."""
-
- name: str
- extensions: frozenset[str]
-
- def extract(self, ctx: ScanContext) -> list[Contract]:
- """Return the contracts found in *ctx* (may be empty)."""
- ...
+ from ..base import ScanContext
def build_table_provider(
@@ -44,20 +34,16 @@ def build_table_provider(
a migration. ``None`` when the raw token does not normalize to a concrete
table name.
"""
- from repowise.core.workspace.contracts import Contract
-
table = normalize_table_name(table_raw)
if table is None:
return None
- return Contract(
- repo=ctx.repo_alias,
+ return build_contract(
+ ctx,
contract_id=f"data::{table}",
contract_type="data",
role="provider",
- file_path=ctx.rel_path,
symbol_name=f"{framework}:{table_raw}",
confidence=confidence,
- service=None,
line=line,
meta={"table": table, "framework": framework},
)
@@ -78,20 +64,16 @@ def build_table_consumer(
``insert`` / ``update`` / ``delete`` / ``join``), carried in ``meta`` so
downstream views can distinguish readers from writers.
"""
- from repowise.core.workspace.contracts import Contract
-
table = normalize_table_name(table_raw)
if table is None:
return None
- return Contract(
- repo=ctx.repo_alias,
+ return build_contract(
+ ctx,
contract_id=f"data::{table}",
contract_type="data",
role="consumer",
- file_path=ctx.rel_path,
symbol_name=f"{client}:{verb} {table}",
confidence=confidence,
- service=None,
line=line,
meta={"table": table, "verb": verb, "client": client},
)
diff --git a/packages/core/src/repowise/core/workspace/extractors/dialect.py b/packages/core/src/repowise/core/workspace/extractors/dialect.py
new file mode 100644
index 000000000..d5655c188
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/dialect.py
@@ -0,0 +1,133 @@
+"""The dialect protocol every contract type shares, and the extractor that runs it.
+
+A *dialect* is one framework's, library's or file format's view of a source
+file: it names the extensions it reads and turns a file into contracts. Each
+contract type is a package of dialects plus a registry tuple, and
+:class:`DialectExtractor` owns the only walk-and-dispatch loop, so adding a
+library is one module and one registry entry.
+"""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, Protocol
+
+from .base import ScanContext, select_files
+
+if TYPE_CHECKING:
+ from collections.abc import Callable, Hashable, Iterable, Sequence
+ from pathlib import Path
+
+ from repowise.core.workspace.contracts import Contract
+
+ from .base import SourceFile
+
+
+class ContractDialect(Protocol):
+ """A recogniser for the files whose extension is in ``extensions``."""
+
+ name: str
+ extensions: frozenset[str]
+
+ def extract(self, ctx: ScanContext) -> list[Contract]:
+ """Return the contracts found in *ctx* (may be empty)."""
+ ...
+
+
+def build_contract(
+ ctx: ScanContext,
+ *,
+ contract_type: str,
+ contract_id: str,
+ role: str,
+ symbol_name: str,
+ confidence: float,
+ line: int | None,
+ meta: dict,
+) -> Contract:
+ """A :class:`Contract` read from *ctx*'s file. The service is assigned later."""
+ from repowise.core.workspace.contracts import Contract
+
+ return Contract(
+ repo=ctx.repo_alias,
+ contract_id=contract_id,
+ contract_type=contract_type,
+ role=role,
+ file_path=ctx.rel_path,
+ symbol_name=symbol_name,
+ confidence=confidence,
+ service=None,
+ line=line,
+ meta=meta,
+ )
+
+
+def union_extensions(dialects: Iterable[ContractDialect]) -> frozenset[str]:
+ """Every extension any of *dialects* reads."""
+ out: set[str] = set()
+ for d in dialects:
+ out |= d.extensions
+ return frozenset(out)
+
+
+def file_identity(c: Contract) -> Hashable:
+ """One fact per file: a contract id read once or ten times in a file is one contract."""
+ return (c.file_path, c.contract_id, c.role)
+
+
+def dedupe(contracts: list[Contract], identity: Callable[[Contract], Hashable]) -> list[Contract]:
+ """The first contract per *identity*, in order."""
+ seen: set[Hashable] = set()
+ out: list[Contract] = []
+ for c in contracts:
+ key = identity(c)
+ if key not in seen:
+ seen.add(key)
+ out.append(c)
+ return out
+
+
+class DialectExtractor:
+ """Walk a repo's files once and hand each to the dialects that read it.
+
+ Subclasses set ``dialects``. ``identity``, when set, keeps the first
+ contract per key within a file, for contract types whose many call sites
+ of one name are one fact.
+ """
+
+ dialects: tuple[ContractDialect, ...] = ()
+ identity: Callable[[Contract], Hashable] | None = None
+
+ @classmethod
+ def source_extensions(cls) -> frozenset[str]:
+ """Every extension this extractor's dialects claim."""
+ return union_extensions(cls.dialects)
+
+ def extract(
+ self,
+ repo_path: Path,
+ repo_alias: str = "",
+ exclude: Callable[[str], bool] | None = None,
+ files: Sequence[SourceFile] | None = None,
+ ) -> list[Contract]:
+ """Scan *repo_path* (or the already-walked *files*) for contracts."""
+ contracts: list[Contract] = []
+ for rel_path, suffix, content in select_files(
+ repo_path, self.source_extensions(), exclude, files
+ ):
+ ctx = ScanContext(repo_alias, rel_path, suffix, content)
+ found: list[Contract] = []
+ for dialect in self.dialects:
+ if suffix in dialect.extensions:
+ found.extend(dialect.extract(ctx))
+ contracts.extend(found if self.identity is None else dedupe(found, self.identity))
+ return contracts
+
+
+__all__ = [
+ "ContractDialect",
+ "DialectExtractor",
+ "build_contract",
+ "dedupe",
+ "file_identity",
+ "union_extensions",
+]
diff --git a/packages/core/src/repowise/core/workspace/extractors/grpc/__init__.py b/packages/core/src/repowise/core/workspace/extractors/grpc/__init__.py
index 5cf04c6a7..c7afe0fab 100644
--- a/packages/core/src/repowise/core/workspace/extractors/grpc/__init__.py
+++ b/packages/core/src/repowise/core/workspace/extractors/grpc/__init__.py
@@ -4,68 +4,29 @@ Scans ``.proto`` files for service/rpc declarations (providers) and
language-specific source files for gRPC server registrations (providers) and
client stubs (consumers). The generated-stub shapes are one table in
:mod:`.languages`; ``.proto`` keeps its own dialect because it is an IDL with a
-grammar rather than a stub shape. :class:`GrpcExtractor` owns only the file walk
-and dispatch.
+grammar rather than a stub shape. :class:`GrpcExtractor` is the registry on the
+shared :class:`..dialect.DialectExtractor`.
"""
from __future__ import annotations
-from typing import TYPE_CHECKING
-
-from ..base import ScanContext, select_files
-from .dialect import GrpcDialect
+from ..dialect import ContractDialect, DialectExtractor
from .languages import LANGUAGES, LanguageGrpcDialect
from .proto import ProtoDialect, _extract_service_blocks
-if TYPE_CHECKING:
- from collections.abc import Callable, Sequence
- from pathlib import Path
-
- from repowise.core.workspace.contracts import Contract
-
- from ..base import SourceFile
-
# One dialect per language/IDL; extension sets are disjoint, so exactly one runs
-# per file.
-DIALECTS: tuple[GrpcDialect, ...] = (
+# per file. Service identity is carried in the contract id
+# (``grpc::<service>/<method-or-*>``).
+DIALECTS: tuple[ContractDialect, ...] = (
ProtoDialect(),
*(LanguageGrpcDialect(lang) for lang in LANGUAGES),
)
-def _union_extensions(dialects: tuple[GrpcDialect, ...]) -> frozenset[str]:
- out: set[str] = set()
- for d in dialects:
- out |= d.extensions
- return frozenset(out)
-
-
-class GrpcExtractor:
+class GrpcExtractor(DialectExtractor):
"""Extract gRPC contracts from proto files and language-specific source."""
- dialects: tuple[GrpcDialect, ...] = DIALECTS
-
- @classmethod
- def source_extensions(cls) -> frozenset[str]:
- """Every extension this extractor's dialects claim."""
- return _union_extensions(cls.dialects)
-
- def extract(
- self,
- repo_path: Path,
- repo_alias: str = "",
- exclude: Callable[[str], bool] | None = None,
- files: Sequence[SourceFile] | None = None,
- ) -> list[Contract]:
- contracts: list[Contract] = []
- for rel_path, suffix, content in select_files(
- repo_path, self.source_extensions(), exclude, files
- ):
- ctx = ScanContext(repo_alias, rel_path, suffix, content)
- for dialect in self.dialects:
- if suffix in dialect.extensions:
- contracts.extend(dialect.extract(ctx))
- return contracts
+ dialects = DIALECTS
__all__ = ["DIALECTS", "GrpcExtractor", "_extract_service_blocks"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/grpc/dialect.py b/packages/core/src/repowise/core/workspace/extractors/grpc/dialect.py
deleted file mode 100644
index f727c1a10..000000000
--- a/packages/core/src/repowise/core/workspace/extractors/grpc/dialect.py
+++ /dev/null
@@ -1,54 +0,0 @@
-"""gRPC dialect protocol and the shared contract builder.
-
-Each language (and the ``.proto`` IDL) is one dialect keyed on its own file
-extensions, so exactly one dialect runs per file. Service identity is carried in
-the ``contract_id`` (``grpc::<service>/<method-or-*>``).
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING, Protocol, runtime_checkable
-
-from ..base import ScanContext
-
-if TYPE_CHECKING:
- from repowise.core.workspace.contracts import Contract
-
-
-@runtime_checkable
-class GrpcDialect(Protocol):
- """A gRPC recogniser for a set of file extensions."""
-
- name: str
- extensions: frozenset[str]
-
- def extract(self, ctx: ScanContext) -> list[Contract]:
- """Return the gRPC contracts found in *ctx* (may be empty)."""
- ...
-
-
-def make_grpc_contract(
- ctx: ScanContext,
- *,
- contract_id: str,
- role: str,
- symbol_name: str,
- confidence: float,
- meta: dict,
- line: int | None = None,
-) -> Contract:
- """Build a gRPC :class:`Contract` with the common fields filled in."""
- from repowise.core.workspace.contracts import Contract
-
- return Contract(
- repo=ctx.repo_alias,
- contract_id=contract_id,
- contract_type="grpc",
- role=role,
- file_path=ctx.rel_path,
- symbol_name=symbol_name,
- confidence=confidence,
- service=None,
- line=line,
- meta=meta,
- )
diff --git a/packages/core/src/repowise/core/workspace/extractors/grpc/languages.py b/packages/core/src/repowise/core/workspace/extractors/grpc/languages.py
index c6ee952b7..340525b63 100644
--- a/packages/core/src/repowise/core/workspace/extractors/grpc/languages.py
+++ b/packages/core/src/repowise/core/workspace/extractors/grpc/languages.py
@@ -17,8 +17,8 @@ from dataclasses import dataclass, field
from typing import TYPE_CHECKING
from ..base import line_at
+from ..dialect import build_contract
from ..langs import CSHARP, GO, JAVA, JS_TS, PYTHON
-from .dialect import make_grpc_contract
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
@@ -142,8 +142,9 @@ class LanguageGrpcDialect:
continue
method = m.group(rule.method_group) if rule.method_group else None
out.append(
- make_grpc_contract(
+ build_contract(
ctx,
+ contract_type="grpc",
contract_id=f"grpc::{service}/{method or '*'}",
role=rule.role,
symbol_name=rule.symbol.format(service=service, method=method),
diff --git a/packages/core/src/repowise/core/workspace/extractors/grpc/proto.py b/packages/core/src/repowise/core/workspace/extractors/grpc/proto.py
index 1918dfc60..543da0816 100644
--- a/packages/core/src/repowise/core/workspace/extractors/grpc/proto.py
+++ b/packages/core/src/repowise/core/workspace/extractors/grpc/proto.py
@@ -19,8 +19,8 @@ from typing import TYPE_CHECKING, NamedTuple
from repowise.core.workspace.contract_schema import ContractSchema, SchemaField
+from ..dialect import build_contract
from ..langs import PROTO
-from .dialect import make_grpc_contract
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
@@ -219,8 +219,9 @@ class ProtoDialect:
out: list[Contract] = []
for svc in services:
for method in svc.methods:
- contract = make_grpc_contract(
+ contract = build_contract(
ctx,
+ contract_type="grpc",
contract_id=f"grpc::{svc.full_path}/{method.name}",
role="provider",
symbol_name=f"{svc.full_path}/{method.name}",
@@ -231,6 +232,7 @@ class ProtoDialect:
"method": method.name,
"source": "proto",
},
+ line=None,
)
schema = ContractSchema(
source="proto",
diff --git a/packages/core/src/repowise/core/workspace/extractors/grpc_extractor.py b/packages/core/src/repowise/core/workspace/extractors/grpc_extractor.py
deleted file mode 100644
index c16dc8436..000000000
--- a/packages/core/src/repowise/core/workspace/extractors/grpc_extractor.py
+++ /dev/null
@@ -1,12 +0,0 @@
-"""Backwards-compatible re-exports for the gRPC contract extractor.
-
-The implementation moved into the :mod:`.grpc` dialect package. This shim keeps
-``from ...extractors.grpc_extractor import GrpcExtractor, _extract_service_blocks``
-working for existing imports.
-"""
-
-from __future__ import annotations
-
-from .grpc import GrpcExtractor, _extract_service_blocks
-
-__all__ = ["GrpcExtractor", "_extract_service_blocks"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/__init__.py b/packages/core/src/repowise/core/workspace/extractors/http/__init__.py
index a126cc147..cd837250b 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/__init__.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/__init__.py
@@ -5,7 +5,10 @@ calls (consumers). Each framework / client library is an independent *dialect*
module registered in :data:`PROVIDER_DIALECTS` / :data:`CONSUMER_DIALECTS`; the
:class:`HttpExtractor` orchestrator owns only the file walk and dispatch. Adding
a framework means dropping one dialect module and appending it to a registry —
-no orchestrator edits.
+no orchestrator edits. HTTP keeps its own orchestrator rather than the shared
+:class:`..dialect.DialectExtractor` because it runs two passes the others do
+not: router mounts collected across files, and the index-backed routes and
+client calls.
"""
from __future__ import annotations
@@ -13,10 +16,10 @@ from __future__ import annotations
from typing import TYPE_CHECKING
from ..base import ScanContext, select_files
+from ..dialect import ContractDialect, union_extensions
from ..langs import PYTHON
from .aspnet import AspNetDialect
from .csharp_http import CSharpHttpDialect
-from .dialect import HttpDialect
from .django import DjangoDialect
from .express import ExpressDialect
from .fastapi import FastApiDialect
@@ -50,7 +53,7 @@ if TYPE_CHECKING:
from ..base import SourceFile
# Route-declaration recognisers (one framework each).
-PROVIDER_DIALECTS: tuple[HttpDialect, ...] = (
+PROVIDER_DIALECTS: tuple[ContractDialect, ...] = (
ExpressDialect(),
FastApiDialect(),
FlaskDialect(),
@@ -67,7 +70,7 @@ PROVIDER_DIALECTS: tuple[HttpDialect, ...] = (
)
# HTTP-client call recognisers (one client/language each).
-CONSUMER_DIALECTS: tuple[HttpDialect, ...] = (
+CONSUMER_DIALECTS: tuple[ContractDialect, ...] = (
JsClientsDialect(),
PythonClientsDialect(),
CSharpHttpDialect(),
@@ -93,25 +96,16 @@ _INDEX_BACKED_DIALECTS = frozenset({"fastapi"})
_INDEX_BACKED_CONSUMER_DIALECTS = frozenset({"js-clients"})
-def _union_extensions(dialects: tuple[HttpDialect, ...]) -> frozenset[str]:
- out: set[str] = set()
- for d in dialects:
- out |= d.extensions
- return frozenset(out)
-
-
class HttpExtractor:
"""Extract HTTP route contracts from source files via registered dialects."""
- provider_dialects: tuple[HttpDialect, ...] = PROVIDER_DIALECTS
- consumer_dialects: tuple[HttpDialect, ...] = CONSUMER_DIALECTS
+ provider_dialects: tuple[ContractDialect, ...] = PROVIDER_DIALECTS
+ consumer_dialects: tuple[ContractDialect, ...] = CONSUMER_DIALECTS
@classmethod
def source_extensions(cls) -> frozenset[str]:
"""Every extension this extractor's dialects claim."""
- return _union_extensions(cls.provider_dialects) | _union_extensions(
- cls.consumer_dialects
- )
+ return union_extensions(cls.provider_dialects + cls.consumer_dialects)
def extract(
self,
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/client_calls.py b/packages/core/src/repowise/core/workspace/extractors/http/client_calls.py
index a90e8f8e7..66574d450 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/client_calls.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/client_calls.py
@@ -5,26 +5,8 @@ route registration once and each consumer of that match builds its own row.
This module is the consumer-side counterpart. A client library's call shape is
recognised in one function yielding :class:`ClientCallMatch` rows, and
:func:`consumer_contracts` turns those rows into contracts through one URL
-resolver, one method inference and one path guard, so a dialect module is a
-table of recognisers and nothing else.
-
-**The URL is an expression, not a literal.** Real call sites spell it as a
-literal, an interpolated string, a format call (``format!``, ``fmt.Sprintf``,
-``String.format``), a concatenation, a name bound to one of those earlier in
-the file, or a wrapper such as ``URI.create(...)``. :func:`resolve_url` reads
-each of these into the ``${expr}`` template text :mod:`.paths` already
-understands, per language through a :class:`UrlSyntax` table, and refuses when
-the expression cannot be settled inside the file. A refused site is a missing
-edge; a guessed one is a wrong edge.
-
-**What folds and what is refused.** A name folds only when the file assigns it
-exactly once, to a single string literal. Assigned twice, or assigned anything
-else anywhere in the file, it is left unresolved. File scope, not lexical
-scope: a single static assignment means the same thing wherever it is read.
-An interpolated expression that is itself such a name is inlined; any other
-expression becomes ``${expr}`` and is stripped as a base or collapsed to a
-parameter downstream. An escaped brace (``{{``) is refused rather than
-corrupted, because no later step can tell it from a parameter.
+resolver (:func:`..strings.resolve_string`), one method inference and one path
+guard, so a dialect module is a table of recognisers and nothing else.
Placed under ``workspace`` because no graph-edge builder reads client calls;
it moves under ``ingestion`` the day one does.
@@ -35,10 +17,10 @@ from __future__ import annotations
import re
from collections.abc import Iterable, Iterator
from dataclasses import dataclass
-from functools import cache
from typing import TYPE_CHECKING
from ..base import line_at
+from ..strings import StringSyntax, literal_span, resolve_string
from .dialect import build_consumer_contract, method_from_callee
if TYPE_CHECKING:
@@ -55,8 +37,8 @@ VERBS = frozenset({"get", "post", "put", "delete", "patch", "head", "options"})
class ClientCallMatch:
"""One HTTP client call a recogniser found in a file's text.
- ``url`` is the URL argument's source text, unparsed; :func:`resolve_url`
- reads it. ``method`` is the verb when the call shape settles it (a verb
+ ``url`` is the URL argument's source text, unparsed; the strings layer's
+ ``resolve_string`` reads it. ``method`` is the verb when the call shape settles it (a verb
argument, a builder terminal, an explicit option) and ``None`` when only
the callee's name carries it, in which case :func:`method_from_callee`
reads ``callee``. ``offset`` is where the call starts, for its line.
@@ -70,202 +52,6 @@ class ClientCallMatch:
confidence: float = 0.75
-@dataclass(frozen=True)
-class UrlSyntax:
- """How one language spells a URL expression.
-
- ``template_quotes`` are quotes whose literal always interpolates (a JS
- backtick); ``template_prefixes`` are prefixes that switch interpolation on
- (Python ``f``, C# ``$``). ``interpolation`` matches one interpolated
- expression inside such a body; its first non-empty group is the expression.
- ``None`` means the body is already in ``${expr}`` form.
-
- ``format_heads`` name calls whose first argument is a template with
- ``placeholder`` holes. ``unwrap_heads`` name calls whose single argument
- *is* the URL. ``concat`` is the string-concatenation operator, empty when
- the language's dialects do not fold concatenation. ``assignment`` locates
- ``name = rhs`` statements for constant folding; ``assignment_strip`` drops a
- trailing decoration from the right-hand side (Ruby's ``.freeze``).
- """
-
- quotes: str = "\"'"
- prefixes: str = ""
- template_quotes: str = ""
- template_prefixes: str = ""
- interpolation: re.Pattern[str] | None = None
- refuse: re.Pattern[str] | None = None
- refuse_prefixes: str = ""
- format_heads: tuple[str, ...] = ()
- placeholder: re.Pattern[str] | None = None
- unwrap_heads: tuple[str, ...] = ()
- concat: str = ""
- assignment: re.Pattern[str] | None = None
- assignment_strip: re.Pattern[str] | None = None
-
-
-# ---------------------------------------------------------------------------
-# Argument scanning
-# ---------------------------------------------------------------------------
-
-_QUOTES = "'\"`"
-
-# How far a scan may run past its opening bracket. A stray quote in a comment
-# desynchronises the scan, and without a bound every candidate after it would
-# read to the end of the file.
-SCAN_LIMIT = 50_000
-
-
-def match_paren(content: str, open_idx: int, limit: int = SCAN_LIMIT, closer: str = ")") -> int:
- """Index of the bracket closing the one at *open_idx*, or -1.
-
- Quote- and template-aware so a parenthesis inside a string literal does not
- unbalance the scan. Nested ``${...}`` inside a template literal is tracked
- as ordinary depth, which is what makes ``` `/a/${f(x)}/b` ``` parse. A
- call that does not close within *limit* characters is -1 as well.
- *closer* is ``}`` for a trailing lambda block.
- """
- depth = 0
- i = open_idx
- n = min(len(content), open_idx + limit)
- # Open template literals, innermost last. A backtick inside a ``${...}``
- # opens a *nested* literal rather than closing the outer one, so the state
- # has to be a stack: ``fetch(`/a/${c ? `x` : `y`}/b`)`` otherwise reads the
- # inner backtick as the end of the string and desynchronises everything
- # after it.
- tmpl: list[int] = []
- quote: str | None = None
- while i < n:
- ch = content[i]
- if quote is not None:
- if ch == "\\":
- i += 2
- continue
- if quote == "`" and ch == "$" and i + 1 < n and content[i + 1] == "{":
- tmpl.append(depth)
- depth += 1
- quote = None
- i += 2
- continue
- if ch == quote:
- quote = None
- i += 1
- continue
- if ch in _QUOTES:
- quote = ch
- elif ch in "([{":
- depth += 1
- elif ch in ")]}":
- depth -= 1
- if tmpl and ch == "}" and depth == tmpl[-1]:
- tmpl.pop()
- quote = "`" # back inside the template literal that opened it
- elif depth == 0 and ch == closer:
- return i
- i += 1
- return -1
-
-
-def _split_top_level(text: str, sep: str) -> list[str]:
- """Split *text* at each top-level *sep* character, outside quotes and brackets."""
- out: list[str] = []
- depth = 0
- tmpl: list[int] = [] # see :func:`match_paren`: nested templates need a stack
- quote: str | None = None
- start = 0
- i = 0
- n = len(text)
- while i < n:
- ch = text[i]
- if quote is not None:
- if ch == "\\":
- i += 2
- continue
- if quote == "`" and ch == "$" and i + 1 < n and text[i + 1] == "{":
- tmpl.append(depth)
- depth += 1
- quote = None
- i += 2
- continue
- if ch == quote:
- quote = None
- i += 1
- continue
- if ch in _QUOTES:
- quote = ch
- elif ch in "([{":
- depth += 1
- elif ch in ")]}":
- depth -= 1
- if tmpl and ch == "}" and depth == tmpl[-1]:
- tmpl.pop()
- quote = "`"
- elif ch == sep and depth == 0:
- out.append(text[start:i])
- start = i + 1
- i += 1
- out.append(text[start:])
- return out
-
-
-def split_first_arg(args: str) -> tuple[str, str]:
- """Split an argument list into ``(first_arg, rest)`` at the top-level comma.
-
- Stops at that comma: the rest is returned unscanned, so a call whose second
- argument is a large options literal costs only its first argument.
- """
- depth = 0
- tmpl: list[int] = [] # see :func:`match_paren`: nested templates need a stack
- quote: str | None = None
- i = 0
- n = len(args)
- while i < n:
- ch = args[i]
- if quote is not None:
- if ch == "\\":
- i += 2
- continue
- if quote == "`" and ch == "$" and i + 1 < n and args[i + 1] == "{":
- tmpl.append(depth)
- depth += 1
- quote = None
- i += 2
- continue
- if ch == quote:
- quote = None
- i += 1
- continue
- if ch in _QUOTES:
- quote = ch
- elif ch in "([{":
- depth += 1
- elif ch in ")]}":
- depth -= 1
- if tmpl and ch == "}" and depth == tmpl[-1]:
- tmpl.pop()
- quote = "`"
- elif ch == "," and depth == 0:
- return args[:i].strip(), args[i + 1 :]
- i += 1
- return args.strip(), ""
-
-
-def call_arguments(content: str, paren_offset: int, close: int | None = None) -> list[str] | None:
- """The top-level arguments of the call whose ``(`` is at *paren_offset*.
-
- ``None`` when the argument list does not close, so a recogniser can tell a
- scanner failure from a call with no arguments. *close* is the matching
- ``)`` when the caller has already found it.
- """
- if close is None:
- close = match_paren(content, paren_offset)
- if close < 0:
- return None
- inner = content[paren_offset + 1 : close]
- if not inner.strip():
- return []
- return [a.strip() for a in _split_top_level(inner, ",")]
-
-
# ---------------------------------------------------------------------------
# Method inference from an argument
# ---------------------------------------------------------------------------
@@ -296,250 +82,6 @@ def method_from_argument(text: str) -> str | None:
return word.upper() if word.lower() in VERBS else None
-# ---------------------------------------------------------------------------
-# URL expression resolution
-# ---------------------------------------------------------------------------
-
-_NAME_RE = re.compile(r"[A-Za-z_$][\w$]*(?:(?:\.|::|->)[A-Za-z_]\w*)*")
-
-# Already-template text: one ``${expr}`` interpolation, for folding a constant
-# inside a body a language wrote in the target form to begin with.
-_TEMPLATE_INTERP_RE = re.compile(r"\$\{([^{}]+)\}")
-
-# Dropped from an interpolated expression (``settings.base.rstrip('/')``) so
-# :func:`.paths.base_token_identifier` reads the attribute, not ``rstrip``.
-_TRAILING_CALL_RE = re.compile(r"\.\s*\w+\s*\([^()]*\)\s*$")
-
-
-@cache
-def _literal_re(quotes: str, prefixes: str) -> re.Pattern[str]:
- alts = []
- if '"' in quotes:
- alts.append('"""')
- if "'" in quotes:
- alts.append("'''")
- alts.extend(re.escape(q) for q in quotes)
- prefix = f"[{re.escape(prefixes)}]*" if prefixes else ""
- return re.compile(
- rf"^(?P<prefix>{prefix})(?P<q>{'|'.join(alts)})(?P<body>.*)(?P=q)$", re.DOTALL
- )
-
-
-@cache
-def _head_re(heads: tuple[str, ...]) -> re.Pattern[str]:
- alts = "|".join(re.escape(h).replace(r"\ ", r"\s+") for h in heads)
- return re.compile(rf"^(?:{alts})\s*\(")
-
-
-def _parse_literal(text: str, syntax: UrlSyntax) -> tuple[str, str, str] | None:
- """``(lowercased prefix, quote, body)`` when *text* is exactly one string literal."""
- m = _literal_re(syntax.quotes, syntax.prefixes).match(text.strip())
- if m is None:
- return None
- body = m.group("body")
- if m.group("q") in body:
- return None # the literal ended and something else followed it
- return m.group("prefix").lower(), m.group("q"), body
-
-
-def _interpolates(prefix: str, quote: str, syntax: UrlSyntax) -> bool:
- # A triple quote interpolates when its single form does (Kotlin raw strings).
- return quote[0] in syntax.template_quotes or any(
- p in syntax.template_prefixes.lower() for p in prefix
- )
-
-
-def _lookup(name: str, constants: dict[str, str]) -> str | None:
- const = constants.get(name)
- if const is None and name.startswith("$"):
- const = constants.get(name[1:]) # a PHP variable, keyed without its sigil
- return const
-
-
-def _fold(expr: str, syntax: UrlSyntax, constants: dict[str, str]) -> str:
- """One interpolated expression as template text, inlining a plain constant.
-
- Only a constant whose own text does not interpolate is inlined: nothing in
- the corpus needs a second level, and a half-folded template would be wrong.
- """
- expr = expr.strip()
- const = _lookup(expr, constants)
- if const is not None:
- lit = _parse_literal(const, syntax)
- text = _literal_text(lit, syntax, {}) if lit is not None else None
- if text is not None and "${" not in text:
- return text
- return "${" + _TRAILING_CALL_RE.sub("", expr) + "}"
-
-
-def _literal_text(
- parsed: tuple[str, str, str], syntax: UrlSyntax, constants: dict[str, str]
-) -> str | None:
- prefix, quote, body = parsed
- if any(p in syntax.refuse_prefixes for p in prefix):
- return None # bytes, not a URL this layer records
- if not _interpolates(prefix, quote, syntax):
- return body
- if syntax.refuse is not None and syntax.refuse.search(body):
- return None
- if syntax.interpolation is None:
- if not constants:
- return body
- return _TEMPLATE_INTERP_RE.sub(lambda m: _fold(m.group(1), syntax, constants), body)
-
- def sub(m: re.Match[str]) -> str:
- expr = next(g for g in m.groups() if g is not None)
- return _fold(expr, syntax, constants)
-
- return syntax.interpolation.sub(sub, body)
-
-
-def _format_template(text: str, syntax: UrlSyntax, constants: dict[str, str]) -> str | None:
- """A ``format!("...")``-style call as template text, or ``None``."""
- m = _head_re(syntax.format_heads).match(text)
- if m is None or syntax.placeholder is None:
- return None
- args = _whole_call_arguments(text, m.end() - 1)
- if not args:
- return None
- parsed = _parse_literal(args[0], syntax)
- if parsed is None:
- return None
- body = _literal_text(parsed, syntax, constants)
- if body is None:
- return None
- # `%%` is a literal percent sign, not a hole.
- return syntax.placeholder.sub(lambda h: "%" if h.group() == "%%" else "${x}", body)
-
-
-def _whole_call_arguments(text: str, paren: int) -> list[str] | None:
- """The arguments of the call at *paren* when the call is all of *text*.
-
- ``URI.create(BASE).resolve("/x")`` is not ``URI.create(BASE)``: reading the
- head alone would drop the path the request reaches.
- """
- close = match_paren(text, paren)
- if close < 0 or text[close + 1 :].strip():
- return None
- return call_arguments(text, paren, close)
-
-
-def _unwrap(text: str, syntax: UrlSyntax) -> str | None:
- """The single argument of a ``URI.create(...)``-style wrapper, or ``None``."""
- m = _head_re(syntax.unwrap_heads).match(text)
- if m is None:
- return None
- args = _whole_call_arguments(text, m.end() - 1)
- return args[0] if args and len(args) == 1 else None
-
-
-def _concat(text: str, syntax: UrlSyntax, constants: dict[str, str]) -> str | None:
- """``a + "/x"`` as template text, or ``None`` when a piece is not readable."""
- if syntax.concat not in text:
- return None
- pieces = _split_top_level(text, syntax.concat)
- if len(pieces) < 2:
- return None
- out: list[str] = []
- for piece in pieces:
- piece = piece.strip()
- parsed = _parse_literal(piece, syntax)
- if parsed is not None:
- body = _literal_text(parsed, syntax, constants)
- if body is None:
- return None
- out.append(body)
- elif _NAME_RE.fullmatch(piece):
- out.append(_fold(piece, syntax, constants))
- else:
- return None # a call or an operator: not settled in this file
- return "".join(out)
-
-
-def resolve_url(
- expr: str, syntax: UrlSyntax, constants: dict[str, str] | None = None
-) -> str | None:
- """The URL text *expr* denotes, or ``None`` when it cannot be resolved here.
-
- Interpolations come out as ``${expr}``, so :func:`.paths.strip_leading_base_expr`
- strips a base placeholder and :func:`.paths.normalize_http_path` collapses a
- parameter with no per-language branch in either.
- """
- constants = constants or {}
- text = expr.strip()
- parsed = _parse_literal(text, syntax)
- if parsed is not None:
- return _literal_text(parsed, syntax, constants)
- if syntax.format_heads:
- folded = _format_template(text, syntax, constants)
- if folded is not None:
- return folded
- if syntax.unwrap_heads:
- inner = _unwrap(text, syntax)
- if inner is not None:
- return resolve_url(inner, syntax, constants)
- if syntax.concat:
- folded = _concat(text, syntax, constants)
- if folded is not None:
- return folded
- if _NAME_RE.fullmatch(text):
- # A bare name. Its value resolved without constants when it was
- # recorded, so this step cannot reach another bare name.
- const = _lookup(text, constants)
- if const is not None:
- return resolve_url(const, syntax, constants)
- return None
-
-
-def string_constants(content: str, syntax: UrlSyntax, code: str | None = None) -> dict[str, str]:
- """Names *content* assigns exactly once, to a URL expression that resolves.
-
- The value is the right-hand side's raw text, so an interpolating literal,
- a format call or a concatenation keeps its parts for :func:`resolve_url`
- to fold at the call site.
-
- *code*, when given, is *content* with comments and string bodies blanked at
- the same offsets. Assignments are located in it and their values read from
- *content*, so the example code in a docstring (`` url = "/docs/example"``)
- is not read as a real assignment.
- """
- if syntax.assignment is None:
- return {}
- text = content if code is None else code
- seen: dict[str, str | None] = {}
- for m in syntax.assignment.finditer(text):
- if code is None and _on_comment_line(content, m.start()):
- continue # an example in a doc comment is not a binding
- name = m.group("name")
- rhs = content[m.start("rhs") : m.end("rhs")].strip()
- if syntax.assignment_strip is not None:
- rhs = syntax.assignment_strip.sub("", rhs).strip()
- # A second assignment retires the name whatever it assigns: the reader
- # cannot tell which one reaches the call site.
- seen[name] = None if name in seen or resolve_url(rhs, syntax) is None else rhs
- # `x += "/v1"` and `x, err = f()` rebind a name without the plain form.
- for m in _COMPOUND_ASSIGN_RE.finditer(text):
- seen[m.group(1)] = None
- for m in _MULTI_ASSIGN_RE.finditer(text):
- for name in m.group(1).replace(" ", "").split(","):
- seen[name.lstrip("$")] = None
- return {name: text for name, text in seen.items() if text is not None}
-
-
-# ``x += ...`` / ``$x .= ...``; the name is keyed without a PHP sigil.
-_COMPOUND_ASSIGN_RE = re.compile(r"(?<![\w.$-])\$?([A-Za-z_]\w*)[ \t]*[+.]=(?!=)")
-# ``a, b := f()`` / ``a, b = f()``: every name on the left is rebound.
-_MULTI_ASSIGN_RE = re.compile(
- r"^[ \t]*(\$?[A-Za-z_]\w*(?:[ \t]*,[ \t]*\$?[A-Za-z_]\w*)+)[ \t]*:?=(?!=)", re.MULTILINE
-)
-
-
-def _on_comment_line(content: str, offset: int) -> bool:
- """True when the line holding *offset* starts as a line or block comment."""
- start = max(content.rfind("\n", 0, offset) + 1, offset - 200)
- return content[start:offset].lstrip().startswith(("//", "#", "*", "/*"))
-
-
# ---------------------------------------------------------------------------
# Contracts
# ---------------------------------------------------------------------------
@@ -557,7 +99,7 @@ def is_rooted_url(url: str) -> bool:
def consumer_contracts(
ctx: ScanContext,
matches: Iterable[ClientCallMatch],
- syntax: UrlSyntax,
+ syntax: StringSyntax,
*,
constants: dict[str, str] | None = None,
path_only: bool = False,
@@ -574,7 +116,7 @@ def consumer_contracts(
"""
out: list[Contract] = []
for m in matches:
- url = resolve_url(m.url, syntax, constants)
+ url = resolve_string(m.url, syntax, constants)
if url is None or (path_only and "/" not in url):
continue
if rooted_only and not is_rooted_url(url):
@@ -593,20 +135,6 @@ def consumer_contracts(
return out
-def literal_span(content: str, m: re.Match[str], group: int) -> str:
- """The source text of a quoted literal whose *body* is *group* of *m*.
-
- For a recogniser whose regex captures the text between the quotes: the
- slice one character wider on each side is the literal as written, which is
- what :func:`resolve_url` reads. A capture that stopped at a different quote
- (one inside a ``${...}`` expression) is closed with its own opening quote,
- so the text read is the prefix the regex saw and the row is kept.
- """
- quote = content[m.start(group) - 1]
- body = content[m.start(group) : m.end(group)]
- return quote + body + quote
-
-
def matches_in(
content: str,
pattern: re.Pattern[str],
@@ -639,121 +167,11 @@ def matches_in(
)
-# ---------------------------------------------------------------------------
-# Per-language syntax tables
-# ---------------------------------------------------------------------------
-
-_C_FORMAT_PLACEHOLDER_RE = re.compile(r"%%|%[-+ #0]*\d*(?:\.\d+)?[a-zA-Z]")
-_BRACE_PLACEHOLDER_RE = re.compile(r"\{[^}]*\}")
-_BRACE_INTERP_RE = re.compile(r"(?<!\{)\{([^{}]+)\}(?!\})")
-_ESCAPED_BRACE_RE = re.compile(r"\{\{|\}\}")
-
-JS_SYNTAX = UrlSyntax(quotes="\"'`", template_quotes="`")
-
-# ``NAME = <expr>`` at any indentation, RHS to end of line. The whitespace
-# around ``=`` is required, which excludes the usual unspaced keyword argument
-# on its own line (``url=str(resp.url),``) that otherwise reads as a second
-# assignment and retires the very name being folded.
-PYTHON_SYNTAX = UrlSyntax(
- prefixes="fFrRbBuU",
- template_prefixes="f",
- interpolation=_BRACE_INTERP_RE,
- refuse=_ESCAPED_BRACE_RE,
- refuse_prefixes="b",
- assignment=re.compile(
- r"^[ \t]*(?P<name>[A-Za-z_]\w*)(?:[ \t]*:[^=\n]+)?[ \t]+=[ \t]+(?P<rhs>.*)$",
- re.MULTILINE,
- ),
-)
-
-RUST_SYNTAX = UrlSyntax(
- quotes='"',
- format_heads=("format!",),
- placeholder=_BRACE_PLACEHOLDER_RE,
-)
-
-CSHARP_SYNTAX = UrlSyntax(
- quotes='"',
- prefixes="$@",
- template_prefixes="$",
- interpolation=_BRACE_INTERP_RE,
- refuse=_ESCAPED_BRACE_RE,
-)
-
-GO_SYNTAX = UrlSyntax(
- quotes='"`',
- format_heads=("fmt.Sprintf",),
- placeholder=_C_FORMAT_PLACEHOLDER_RE,
- concat="+",
- assignment=re.compile(
- r"^[ \t]*(?:(?:const|var)[ \t]+)?(?P<name>[A-Za-z_]\w*)[ \t]*:?=[ \t]*(?P<rhs>[^\n]+)$",
- re.MULTILINE,
- ),
-)
-
-RUBY_SYNTAX = UrlSyntax(
- template_quotes='"',
- interpolation=re.compile(r"#\{([^{}]+)\}"),
- unwrap_heads=("URI.parse", "URI"),
- concat="+",
- assignment=re.compile(
- r"^[ \t]*(?P<name>[A-Za-z_]\w*)[ \t]*=[ \t]*(?P<rhs>[^\n]+)$", re.MULTILINE
- ),
- assignment_strip=re.compile(r"\.freeze$"),
-)
-
-JAVA_SYNTAX = UrlSyntax(
- quotes='"',
- format_heads=("String.format",),
- placeholder=_C_FORMAT_PLACEHOLDER_RE,
- unwrap_heads=("URI.create", "new URI", "new URL", "URI", "HttpUrl.parse"),
- concat="+",
- assignment=re.compile(
- r"(?<![=!<>.\w])(?P<name>[A-Za-z_]\w*)[ \t]*=(?!=)[ \t]*(?P<rhs>[^;\n]+);"
- ),
-)
-
-KOTLIN_SYNTAX = UrlSyntax(
- quotes='"',
- template_quotes='"',
- interpolation=re.compile(r"\$\{([^{}]+)\}|\$([A-Za-z_]\w*)"),
- unwrap_heads=("Url", "URI.create", "URI"),
- concat="+",
- assignment=re.compile(
- r"\b(?:val|var)[ \t]+(?P<name>[A-Za-z_]\w*)(?:[ \t]*:[ \t]*[\w<>?]+)?[ \t]*=[ \t]*(?P<rhs>[^\n]+)$",
- re.MULTILINE,
- ),
-)
-
-PHP_SYNTAX = UrlSyntax(
- template_quotes='"',
- interpolation=re.compile(r"\{\$([^{}]+)\}|\$\{([^{}]+)\}|\$([A-Za-z_]\w*(?:->[A-Za-z_]\w*)?)"),
- concat=".",
- assignment=re.compile(
- r"(?:\bconst[ \t]+|\$)(?P<name>[A-Za-z_]\w*)[ \t]*=(?!=)[ \t]*(?P<rhs>[^;\n]+);"
- ),
-)
-
__all__ = [
- "CSHARP_SYNTAX",
- "GO_SYNTAX",
- "JAVA_SYNTAX",
- "JS_SYNTAX",
- "KOTLIN_SYNTAX",
- "PHP_SYNTAX",
- "PYTHON_SYNTAX",
- "RUBY_SYNTAX",
- "RUST_SYNTAX",
"VERBS",
"ClientCallMatch",
- "call_arguments",
"consumer_contracts",
"is_rooted_url",
- "literal_span",
- "match_paren",
"matches_in",
"method_from_argument",
- "resolve_url",
- "split_first_arg",
- "string_constants",
]
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/csharp_http.py b/packages/core/src/repowise/core/workspace/extractors/http/csharp_http.py
index a3721b3c4..c304048cd 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/csharp_http.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/csharp_http.py
@@ -19,7 +19,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import CSHARP
-from .client_calls import CSHARP_SYNTAX, ClientCallMatch, consumer_contracts, matches_in
+from ..strings import CSHARP_SYNTAX
+from .client_calls import ClientCallMatch, consumer_contracts, matches_in
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/dialect.py b/packages/core/src/repowise/core/workspace/extractors/http/dialect.py
index 400df8ecd..cecdf71d3 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/dialect.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/dialect.py
@@ -1,18 +1,17 @@
-"""HTTP dialect protocol and the shared contract builders.
+"""The shared HTTP contract builders.
-A *dialect* is one framework's or client library's view of a source file. It
-declares the file extensions it understands and turns raw regex matches into
-:class:`Contract` instances via the two builders here, so every dialect emits
-identically-shaped providers/consumers and the normalization rules live in one
-place.
+Every HTTP dialect (see :mod:`..dialect` for the protocol) turns raw matches
+into :class:`Contract` instances via the two builders here, so every dialect
+emits identically-shaped providers/consumers and the normalization rules live
+in one place.
"""
from __future__ import annotations
import re
-from typing import TYPE_CHECKING, Protocol, runtime_checkable
+from typing import TYPE_CHECKING
-from ..base import ScanContext
+from ..dialect import build_contract
from .paths import (
absolute_host,
consumer_meta,
@@ -26,6 +25,8 @@ from .paths import (
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
+ from ..base import ScanContext
+
# Regex fragments for the HTTP method verbs, shared by every dialect's patterns.
METHODS = r"get|post|put|delete|patch"
METHODS_UPPER = r"GET|POST|PUT|DELETE|PATCH"
@@ -51,18 +52,6 @@ def method_from_callee(callee: str, default: str = "GET") -> str:
return verbs[0].upper() if len(verbs) == 1 else default
-@runtime_checkable
-class HttpDialect(Protocol):
- """A framework/client recogniser for a set of file extensions."""
-
- name: str
- extensions: frozenset[str]
-
- def extract(self, ctx: ScanContext) -> list[Contract]:
- """Return the contracts found in *ctx* (may be empty)."""
- ...
-
-
def nearest_prefix(mappings: list[tuple[int, str]], pos: int) -> str:
"""Return the prefix of the nearest class mapping declared before *pos*.
@@ -101,21 +90,17 @@ def build_provider_contract(
lets :func:`.contracts.bind_symbol_ids` reach the handler rather than the
registration site; without it the line lookup binds to ``Program.cs``.
"""
- from repowise.core.workspace.contracts import Contract
-
norm_path = normalize_http_path(path_raw)
if (not norm_path or norm_path == "/") and not path_raw.strip("/"):
return None
- return Contract(
- repo=ctx.repo_alias,
+ return build_contract(
+ ctx,
contract_id=f"http::{method}::{norm_path}",
contract_type="http",
role="provider",
- file_path=ctx.rel_path,
symbol_name=f"{framework}:{method} {path_raw}",
confidence=confidence,
- service=None,
line=line,
meta={
"method": method,
@@ -141,8 +126,6 @@ def build_consumer_contract(
truncated template literal or a path with no concrete segment (see
:func:`is_unusable_consumer_path`).
"""
- from repowise.core.workspace.contracts import Contract
-
host = absolute_host(url)
path = extract_path_from_url(url)
path, base_token = strip_leading_base_expr(path)
@@ -150,15 +133,13 @@ def build_consumer_contract(
norm_path = normalize_http_path(path)
if is_unusable_consumer_path(norm_path):
return None
- return Contract(
- repo=ctx.repo_alias,
+ return build_contract(
+ ctx,
contract_id=f"http::{method}::{norm_path}",
contract_type="http",
role="consumer",
- file_path=ctx.rel_path,
symbol_name=f"{client}:{method} {url}",
confidence=confidence,
- service=None,
line=line,
meta=consumer_meta(method, norm_path, client, base_token, host),
)
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/go_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/go_clients.py
index 764cfca7a..13c794ce4 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/go_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/go_clients.py
@@ -27,14 +27,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import GO
-from .client_calls import (
- GO_SYNTAX,
- ClientCallMatch,
- call_arguments,
- consumer_contracts,
- method_from_argument,
- string_constants,
-)
+from ..strings import GO_SYNTAX, call_arguments, string_constants
+from .client_calls import ClientCallMatch, consumer_contracts, method_from_argument
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/index_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/index_clients.py
index d4644c2c7..fc448f50e 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/index_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/index_clients.py
@@ -32,16 +32,15 @@ from typing import TYPE_CHECKING
from ..base import line_at
from ..langs import PYTHON
-from .client_calls import (
+from ..strings import (
JS_SYNTAX,
PYTHON_SYNTAX,
- VERBS,
- is_rooted_url,
match_paren,
- resolve_url,
+ resolve_string,
split_first_arg,
string_constants,
)
+from .client_calls import VERBS, is_rooted_url
from .dialect import build_consumer_contract
from .wrappers import (
DEFAULT_HOP_BUDGET,
@@ -98,7 +97,7 @@ def _first_arg_url(arg: str, is_python: bool, constants: dict[str, str]) -> str
to a string literal, then faces the same concreteness test as every other
language.
"""
- url = resolve_url(arg, PYTHON_SYNTAX if is_python else JS_SYNTAX, constants)
+ url = resolve_string(arg, PYTHON_SYNTAX if is_python else JS_SYNTAX, constants)
return url if url is not None and is_rooted_url(url) else None
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/java_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/java_clients.py
index 784e88175..b41574758 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/java_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/java_clients.py
@@ -26,16 +26,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import JAVA
-from .client_calls import (
- JAVA_SYNTAX,
- VERBS,
- ClientCallMatch,
- call_arguments,
- consumer_contracts,
- match_paren,
- method_from_argument,
- string_constants,
-)
+from ..strings import JAVA_SYNTAX, call_arguments, match_paren, string_constants
+from .client_calls import VERBS, ClientCallMatch, consumer_contracts, method_from_argument
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/js_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/js_clients.py
index ffbcb0083..dbfe00acc 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/js_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/js_clients.py
@@ -14,7 +14,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import JS_TS
-from .client_calls import JS_SYNTAX, ClientCallMatch, consumer_contracts, literal_span, matches_in
+from ..strings import JS_SYNTAX, literal_span
+from .client_calls import ClientCallMatch, consumer_contracts, matches_in
from .dialect import METHODS
if TYPE_CHECKING:
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/kotlin_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/kotlin_clients.py
index 2abdfc7ab..ebc063718 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/kotlin_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/kotlin_clients.py
@@ -30,15 +30,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import KOTLIN
-from .client_calls import (
- KOTLIN_SYNTAX,
- ClientCallMatch,
- call_arguments,
- consumer_contracts,
- match_paren,
- method_from_argument,
- string_constants,
-)
+from ..strings import KOTLIN_SYNTAX, call_arguments, match_paren, string_constants
+from .client_calls import ClientCallMatch, consumer_contracts, method_from_argument
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/php_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/php_clients.py
index 0e521154b..dcfd34ea8 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/php_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/php_clients.py
@@ -24,16 +24,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import PHP
-from .client_calls import (
- PHP_SYNTAX,
- VERBS,
- ClientCallMatch,
- call_arguments,
- consumer_contracts,
- match_paren,
- method_from_argument,
- string_constants,
-)
+from ..strings import PHP_SYNTAX, call_arguments, match_paren, string_constants
+from .client_calls import VERBS, ClientCallMatch, consumer_contracts, method_from_argument
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/python_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/python_clients.py
index 0387bda23..e8f76ab85 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/python_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/python_clients.py
@@ -7,7 +7,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import PYTHON
-from .client_calls import PYTHON_SYNTAX, ClientCallMatch, consumer_contracts, matches_in
+from ..strings import PYTHON_SYNTAX
+from .client_calls import ClientCallMatch, consumer_contracts, matches_in
from .dialect import METHODS
if TYPE_CHECKING:
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/ruby_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/ruby_clients.py
index e2a72a0b6..bc4aa1678 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/ruby_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/ruby_clients.py
@@ -28,16 +28,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import RUBY
-from .client_calls import (
- RUBY_SYNTAX,
- ClientCallMatch,
- call_arguments,
- consumer_contracts,
- match_paren,
- method_from_argument,
- split_first_arg,
- string_constants,
-)
+from ..strings import RUBY_SYNTAX, call_arguments, match_paren, split_first_arg, string_constants
+from .client_calls import ClientCallMatch, consumer_contracts, method_from_argument
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
diff --git a/packages/core/src/repowise/core/workspace/extractors/http/rust_clients.py b/packages/core/src/repowise/core/workspace/extractors/http/rust_clients.py
index bb59c40f6..6c0e92dda 100644
--- a/packages/core/src/repowise/core/workspace/extractors/http/rust_clients.py
+++ b/packages/core/src/repowise/core/workspace/extractors/http/rust_clients.py
@@ -18,7 +18,8 @@ from collections.abc import Iterator
from typing import TYPE_CHECKING
from ..langs import RUST
-from .client_calls import RUST_SYNTAX, ClientCallMatch, consumer_contracts, matches_in
+from ..strings import RUST_SYNTAX
+from .client_calls import ClientCallMatch, consumer_contracts, matches_in
if TYPE_CHECKING:
from repowise.core.workspace.contracts import Contract
diff --git a/packages/core/src/repowise/core/workspace/extractors/http_extractor.py b/packages/core/src/repowise/core/workspace/extractors/http_extractor.py
deleted file mode 100644
index 4e2ae81ed..000000000
--- a/packages/core/src/repowise/core/workspace/extractors/http_extractor.py
+++ /dev/null
@@ -1,12 +0,0 @@
-"""Backwards-compatible re-exports for the HTTP contract extractor.
-
-The implementation moved into the :mod:`.http` dialect package. This shim keeps
-``from ...extractors.http_extractor import HttpExtractor, normalize_http_path``
-working for existing imports.
-"""
-
-from __future__ import annotations
-
-from .http import HttpExtractor, normalize_http_path
-
-__all__ = ["HttpExtractor", "normalize_http_path"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/socket/__init__.py b/packages/core/src/repowise/core/workspace/extractors/socket/__init__.py
new file mode 100644
index 000000000..38ee447e3
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/socket/__init__.py
@@ -0,0 +1,23 @@
+"""Socket / websocket contract extraction.
+
+Each language's endpoint and connect shapes are a :class:`.dialect.SocketDialect`
+table registered in :data:`DIALECTS`.
+"""
+
+from __future__ import annotations
+
+from ..dialect import ContractDialect, DialectExtractor, file_identity
+from .dotnet import DOTNET
+from .python import PYTHON_SOCKETS
+
+DIALECTS: tuple[ContractDialect, ...] = (DOTNET, PYTHON_SOCKETS)
+
+
+class SocketExtractor(DialectExtractor):
+ """Extract socket/websocket contracts from source files."""
+
+ dialects = DIALECTS
+ identity = staticmethod(file_identity)
+
+
+__all__ = ["DIALECTS", "SocketExtractor"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/socket/dialect.py b/packages/core/src/repowise/core/workspace/extractors/socket/dialect.py
new file mode 100644
index 000000000..bf4893f7e
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/socket/dialect.py
@@ -0,0 +1,106 @@
+"""Socket dialects: a transport's endpoint and connect calls as a pattern table.
+
+A socket contract is identified by its path, normalized exactly as an HTTP path
+is, so ``socket::/hubs/game`` links a hub mapped at ``/hubs/game`` to a client
+connecting to ``wss://host/hubs/game``.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+from ..base import line_at
+from ..dialect import build_contract
+from ..http.paths import (
+ extract_path_from_url,
+ is_unusable_consumer_path,
+ normalize_http_path,
+ strip_leading_base_expr,
+)
+
+if TYPE_CHECKING:
+ from repowise.core.workspace.contracts import Contract
+
+ from ..base import ScanContext
+
+#: A C# string literal, capturing its ``$``/``@`` prefix and its body.
+CSHARP_STRING = r"""(\$?@?)"([^"]*)\""""
+
+
+@dataclass(frozen=True)
+class SocketPattern:
+ """One endpoint or connect shape.
+
+ ``context`` must appear somewhere in the file for the pattern to run, which
+ is how two libraries spelling ``new WebSocket("...")`` are told apart.
+ """
+
+ regex: re.Pattern[str]
+ role: str
+ transport: str
+ confidence: float
+ label: str
+ identity_group: int
+ prefix_group: int | None = None
+ context: re.Pattern[str] | None = None
+
+
+def _to_template(prefix: str, text: str) -> str:
+ """Rewrite a C# interpolated string body into ``${expr}`` template form."""
+ if "$" in prefix:
+ return text.replace("{", "${")
+ return text
+
+
+def socket_identity(raw: str, prefix: str = "") -> str | None:
+ """The stable path identity of a socket URL-like string, or ``None``."""
+ value = _to_template(prefix, raw).strip()
+ if not value or "/" not in value:
+ return None
+ path = extract_path_from_url(value)
+ path, _base_token = strip_leading_base_expr(path)
+ norm = normalize_http_path(path)
+ if norm in ("", "/") or is_unusable_consumer_path(norm):
+ return None
+ return norm
+
+
+class SocketDialect:
+ """One socket library's patterns, for the given file extensions."""
+
+ def __init__(
+ self, name: str, extensions: frozenset[str], patterns: tuple[SocketPattern, ...]
+ ) -> None:
+ self.name = name
+ self.extensions = extensions
+ self.patterns = patterns
+
+ def extract(self, ctx: ScanContext) -> list[Contract]:
+ content = ctx.content
+ out: list[Contract] = []
+ for p in self.patterns:
+ if p.context is not None and not p.context.search(content):
+ continue
+ for m in p.regex.finditer(content):
+ prefix = m.group(p.prefix_group) if p.prefix_group else ""
+ identity = socket_identity(m.group(p.identity_group), prefix)
+ if identity is None:
+ continue
+ out.append(
+ build_contract(
+ ctx,
+ contract_type="socket",
+ contract_id=f"socket::{identity}",
+ role=p.role,
+ symbol_name=f"{p.label}('{identity}')",
+ confidence=p.confidence,
+ line=line_at(content, m.start()),
+ meta={"path": identity, "transport": p.transport},
+ )
+ )
+ return out
+
+
+__all__ = ["CSHARP_STRING", "SocketDialect", "SocketPattern", "socket_identity"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/socket/dotnet.py b/packages/core/src/repowise/core/workspace/extractors/socket/dotnet.py
new file mode 100644
index 000000000..7fea76a5e
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/socket/dotnet.py
@@ -0,0 +1,74 @@
+"""C# socket endpoints and clients: SignalR hubs, ClientWebSocket, Unity websocket libraries."""
+
+from __future__ import annotations
+
+import re
+
+from ..langs import CSHARP
+from .dialect import CSHARP_STRING, SocketDialect, SocketPattern
+
+DOTNET = SocketDialect(
+ name="dotnet-sockets",
+ extensions=CSHARP,
+ patterns=(
+ SocketPattern(
+ regex=re.compile(
+ rf"""\bConnectAsync\s*\(\s*new\s+Uri\s*\(\s*{CSHARP_STRING}\s*\)""",
+ re.IGNORECASE,
+ ),
+ role="consumer",
+ transport="clientwebsocket",
+ confidence=0.75,
+ label="ClientWebSocket.ConnectAsync",
+ prefix_group=1,
+ identity_group=2,
+ ),
+ SocketPattern(
+ regex=re.compile(rf"""\bWithUrl\s*\(\s*{CSHARP_STRING}""", re.IGNORECASE),
+ role="consumer",
+ transport="signalr",
+ confidence=0.8,
+ label="HubConnectionBuilder.WithUrl",
+ prefix_group=1,
+ identity_group=2,
+ context=re.compile(
+ r"\bHubConnectionBuilder\b|SignalR\.Client|Microsoft\.AspNetCore\.SignalR\.Client"
+ ),
+ ),
+ SocketPattern(
+ regex=re.compile(rf"""\bnew\s+WebSocket\s*\(\s*{CSHARP_STRING}""", re.IGNORECASE),
+ role="consumer",
+ transport="nativewebsocket",
+ confidence=0.75,
+ label="NativeWebSocket.WebSocket",
+ prefix_group=1,
+ identity_group=2,
+ context=re.compile(r"\bNativeWebSocket\b"),
+ ),
+ SocketPattern(
+ regex=re.compile(
+ rf"""\bnew\s+(?:\w+\.)?WebSocket\s*\(\s*{CSHARP_STRING}""", re.IGNORECASE
+ ),
+ role="consumer",
+ transport="websocketsharp",
+ confidence=0.75,
+ label="WebSocketSharp.WebSocket",
+ prefix_group=1,
+ identity_group=2,
+ context=re.compile(r"\bWebSocketSharp\b"),
+ ),
+ SocketPattern(
+ regex=re.compile(
+ rf"""\.\s*MapHub\s*<\s*\w+\s*>\s*\(\s*{CSHARP_STRING}""", re.IGNORECASE
+ ),
+ role="provider",
+ transport="signalr",
+ confidence=0.85,
+ label="MapHub",
+ prefix_group=1,
+ identity_group=2,
+ ),
+ ),
+)
+
+__all__ = ["DOTNET"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/socket/python.py b/packages/core/src/repowise/core/workspace/extractors/socket/python.py
new file mode 100644
index 000000000..da85c8183
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/socket/python.py
@@ -0,0 +1,27 @@
+"""Python socket endpoints: FastAPI websocket routes."""
+
+from __future__ import annotations
+
+import re
+
+from ..langs import PYTHON
+from .dialect import SocketDialect, SocketPattern
+
+PYTHON_SOCKETS = SocketDialect(
+ name="python-sockets",
+ extensions=PYTHON,
+ patterns=(
+ SocketPattern(
+ regex=re.compile(
+ r"""@(?:app|router)\.websocket\s*\(\s*['"]([^'"]+)['"]""", re.IGNORECASE
+ ),
+ role="provider",
+ transport="fastapi-websocket",
+ confidence=0.8,
+ label="@app.websocket",
+ identity_group=1,
+ ),
+ ),
+)
+
+__all__ = ["PYTHON_SOCKETS"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/socket_extractor.py b/packages/core/src/repowise/core/workspace/extractors/socket_extractor.py
deleted file mode 100644
index 8bab4515e..000000000
--- a/packages/core/src/repowise/core/workspace/extractors/socket_extractor.py
+++ /dev/null
@@ -1,199 +0,0 @@
-"""Socket / websocket contract extraction.
-
-Scans source files for common websocket-style consumer and provider patterns,
-with an initial focus on Unity / C# clients and path-based server endpoints.
-"""
-
-from __future__ import annotations
-
-import logging
-import re
-from dataclasses import dataclass
-from pathlib import Path
-from typing import TYPE_CHECKING
-
-from .base import line_at, select_files
-from .http.paths import (
- extract_path_from_url,
- is_unusable_consumer_path,
- normalize_http_path,
- strip_leading_base_expr,
-)
-from .langs import CSHARP, PYTHON
-
-if TYPE_CHECKING:
- from collections.abc import Callable, Sequence
-
- from repowise.core.workspace.contracts import Contract
-
- from .base import SourceFile
-
-_log = logging.getLogger("repowise.workspace.extractors.socket")
-
-_EXTENSIONS = CSHARP | PYTHON
-
-_STR = r"""(\$?@?)"([^"]*)\""""
-
-
-@dataclass(frozen=True)
-class _PatternDef:
- regex: re.Pattern[str]
- role: str
- transport: str
- confidence: float
- label: str
- identity_group: int
- prefix_group: int | None = None
- context_regex: re.Pattern[str] | None = None
-
-
-_SIGNALR_CLIENT_CONTEXT_RE = re.compile(
- r"\bHubConnectionBuilder\b|SignalR\.Client|Microsoft\.AspNetCore\.SignalR\.Client"
-)
-
-_NATIVE_WS_CONTEXT_RE = re.compile(r"\bNativeWebSocket\b")
-_WEBSOCKETSHARP_CONTEXT_RE = re.compile(r"\bWebSocketSharp\b")
-
-_CLIENT_PATTERNS: list[_PatternDef] = [
- _PatternDef(
- regex=re.compile(
- rf"""\bConnectAsync\s*\(\s*new\s+Uri\s*\(\s*{_STR}\s*\)""",
- re.IGNORECASE,
- ),
- role="consumer",
- transport="clientwebsocket",
- confidence=0.75,
- label="ClientWebSocket.ConnectAsync",
- prefix_group=1,
- identity_group=2,
- ),
- _PatternDef(
- regex=re.compile(rf"""\bWithUrl\s*\(\s*{_STR}""", re.IGNORECASE),
- role="consumer",
- transport="signalr",
- confidence=0.8,
- label="HubConnectionBuilder.WithUrl",
- prefix_group=1,
- identity_group=2,
- context_regex=_SIGNALR_CLIENT_CONTEXT_RE,
- ),
- _PatternDef(
- regex=re.compile(rf"""\bnew\s+WebSocket\s*\(\s*{_STR}""", re.IGNORECASE),
- role="consumer",
- transport="nativewebsocket",
- confidence=0.75,
- label="NativeWebSocket.WebSocket",
- prefix_group=1,
- identity_group=2,
- context_regex=_NATIVE_WS_CONTEXT_RE,
- ),
- _PatternDef(
- regex=re.compile(rf"""\bnew\s+(?:\w+\.)?WebSocket\s*\(\s*{_STR}""", re.IGNORECASE),
- role="consumer",
- transport="websocketsharp",
- confidence=0.75,
- label="WebSocketSharp.WebSocket",
- prefix_group=1,
- identity_group=2,
- context_regex=_WEBSOCKETSHARP_CONTEXT_RE,
- ),
-]
-
-_PROVIDER_PATTERNS: list[_PatternDef] = [
- _PatternDef(
- regex=re.compile(rf"""\.\s*MapHub\s*<\s*\w+\s*>\s*\(\s*{_STR}""", re.IGNORECASE),
- role="provider",
- transport="signalr",
- confidence=0.85,
- label="MapHub",
- prefix_group=1,
- identity_group=2,
- ),
- _PatternDef(
- regex=re.compile(r"""@(?:app|router)\.websocket\s*\(\s*['"]([^'"]+)['"]""", re.IGNORECASE),
- role="provider",
- transport="fastapi-websocket",
- confidence=0.8,
- label="@app.websocket",
- identity_group=1,
- ),
-]
-
-
-def _to_template(prefix: str, text: str) -> str:
- """Rewrite a C# interpolated string body into ``${expr}`` template form."""
- if "$" in prefix:
- return text.replace("{", "${")
- return text
-
-
-def _normalize_socket_identity(raw: str, prefix: str = "") -> str | None:
- """Extract the stable path/channel identity from a socket URL-like string."""
- value = _to_template(prefix, raw).strip()
- if not value or "/" not in value:
- return None
- path = extract_path_from_url(value)
- path, _base_token = strip_leading_base_expr(path)
- norm = normalize_http_path(path)
- if norm in ("", "/") or is_unusable_consumer_path(norm):
- return None
- return norm
-
-
-class SocketExtractor:
- """Extract socket/websocket contracts from source files."""
-
- @classmethod
- def source_extensions(cls) -> frozenset[str]:
- """Every extension this extractor scans."""
- return _EXTENSIONS
-
- def extract(
- self,
- repo_path: Path,
- repo_alias: str = "",
- exclude: Callable[[str], bool] | None = None,
- files: Sequence[SourceFile] | None = None,
- ) -> list[Contract]:
- from repowise.core.workspace.contracts import Contract
-
- contracts: list[Contract] = []
- seen: set[tuple[str, str, str]] = set()
-
- for rel_path, _suffix, content in select_files(
- repo_path, _EXTENSIONS, exclude, files
- ):
- for pdef in _CLIENT_PATTERNS + _PROVIDER_PATTERNS:
- if pdef.context_regex is not None and not pdef.context_regex.search(content):
- continue
- for match in pdef.regex.finditer(content):
- prefix = match.group(pdef.prefix_group) if pdef.prefix_group else ""
- identity = _normalize_socket_identity(
- match.group(pdef.identity_group),
- prefix,
- )
- if identity is None:
- continue
- contract_id = f"socket::{identity}"
- dedup_key = (rel_path, contract_id, pdef.role)
- if dedup_key in seen:
- continue
- seen.add(dedup_key)
- contracts.append(
- Contract(
- repo=repo_alias,
- contract_id=contract_id,
- contract_type="socket",
- role=pdef.role,
- file_path=rel_path,
- symbol_name=f"{pdef.label}('{identity}')",
- confidence=pdef.confidence,
- service=None,
- line=line_at(content, match.start()),
- meta={
- "path": identity,
- "transport": pdef.transport,
- },
- )
- )
- return contracts
diff --git a/packages/core/src/repowise/core/workspace/extractors/strings.py b/packages/core/src/repowise/core/workspace/extractors/strings.py
new file mode 100644
index 000000000..f9215c27c
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/strings.py
@@ -0,0 +1,673 @@
+"""String expressions as a recogniser reads them: arguments, literals, constants.
+
+A URL, a queue name or a topic is written in source as an expression, not
+always a literal: an interpolated string, a format call (``format!``,
+``fmt.Sprintf``, ``String.format``), a concatenation, a name bound to one of
+those earlier in the file, or a wrapper such as ``URI.create(...)``.
+:func:`resolve_string` reads each of these into ``${expr}`` template text, per
+language through a :class:`StringSyntax` table, and refuses when the expression
+cannot be settled inside the file. A refused site is a missing edge; a guessed
+one is a wrong edge.
+
+**What folds and what is refused.** A name folds only when the file assigns it
+exactly once, to a single string expression that itself resolves. Assigned
+twice, or assigned anything else anywhere in the file, it is left unresolved.
+File scope, not lexical scope: a single static assignment means the same thing
+wherever it is read. An interpolated expression that is itself such a name is
+inlined; any other expression becomes ``${expr}``. An escaped brace (``{{``) is
+refused rather than corrupted, because no later step can tell it from a
+parameter.
+
+The bracket scanner here is quote- and template-aware and knows nothing about
+comments; ``ingestion.framework_routes`` keeps its own comment-aware scanner.
+"""
+
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from functools import cache
+from itertools import pairwise
+
+from .langs import CSHARP, GO, JAVA, JS_TS, KOTLIN, PHP, PYTHON, RUBY, RUST
+
+
+@dataclass(frozen=True)
+class StringSyntax:
+ """How one language spells a string expression.
+
+ ``template_quotes`` are quotes whose literal always interpolates (a JS
+ backtick); ``template_prefixes`` are prefixes that switch interpolation on
+ (Python ``f``, C# ``$``). ``interpolation`` matches one interpolated
+ expression inside such a body; its first non-empty group is the expression.
+ ``None`` means the body is already in ``${expr}`` form.
+
+ ``format_heads`` name calls whose first argument is a template with
+ ``placeholder`` holes. ``unwrap_heads`` name calls whose single argument
+ *is* the value. ``concat`` is the string-concatenation operator, empty when
+ the language's dialects do not fold concatenation. ``assignment`` locates
+ ``name = rhs`` statements for constant folding; ``assignment_strip`` drops a
+ trailing decoration from the right-hand side (Ruby's ``.freeze``).
+ """
+
+ quotes: str = "\"'"
+ prefixes: str = ""
+ template_quotes: str = ""
+ template_prefixes: str = ""
+ interpolation: re.Pattern[str] | None = None
+ refuse: re.Pattern[str] | None = None
+ refuse_prefixes: str = ""
+ format_heads: tuple[str, ...] = ()
+ placeholder: re.Pattern[str] | None = None
+ unwrap_heads: tuple[str, ...] = ()
+ concat: str = ""
+ assignment: re.Pattern[str] | None = None
+ assignment_strip: re.Pattern[str] | None = None
+
+
+# ---------------------------------------------------------------------------
+# Argument scanning
+# ---------------------------------------------------------------------------
+
+_QUOTES = "'\"`"
+
+# How far a scan may run past its opening bracket. A stray quote in a comment
+# desynchronises the scan, and without a bound every candidate after it would
+# read to the end of the file.
+SCAN_LIMIT = 50_000
+
+
+def _scan(
+ text: str,
+ start: int,
+ end: int,
+ *,
+ closer: str | None = None,
+ sep: str | None = None,
+ first_sep_only: bool = False,
+) -> tuple[int, list[int]]:
+ """Walk *text* from *start*, outside quotes and brackets.
+
+ Returns ``(close, seps)``: the index of the *closer* that brings depth back
+ to zero (-1 when none does before *end*), and every top-level *sep* index
+ seen on the way. Nested ``${...}`` inside a template literal is tracked as
+ ordinary depth, which is what makes ``` `/a/${f(x)}/b` ``` parse.
+ """
+ depth = 0
+ # Open template literals, innermost last. A backtick inside a ``${...}``
+ # opens a *nested* literal rather than closing the outer one, so the state
+ # has to be a stack: ``fetch(`/a/${c ? `x` : `y`}/b`)`` otherwise reads the
+ # inner backtick as the end of the string and desynchronises everything
+ # after it.
+ tmpl: list[int] = []
+ quote: str | None = None
+ seps: list[int] = []
+ i = start
+ while i < end:
+ ch = text[i]
+ if quote is not None:
+ if ch == "\\":
+ i += 2
+ continue
+ if quote == "`" and ch == "$" and i + 1 < end and text[i + 1] == "{":
+ tmpl.append(depth)
+ depth += 1
+ quote = None
+ i += 2
+ continue
+ if ch == quote:
+ quote = None
+ i += 1
+ continue
+ if ch in _QUOTES:
+ quote = ch
+ elif ch in "([{":
+ depth += 1
+ elif ch in ")]}":
+ depth -= 1
+ if tmpl and ch == "}" and depth == tmpl[-1]:
+ tmpl.pop()
+ quote = "`" # back inside the template literal that opened it
+ elif closer is not None and depth == 0 and ch == closer:
+ return i, seps
+ elif ch == sep and depth == 0:
+ seps.append(i)
+ if first_sep_only:
+ break
+ i += 1
+ return -1, seps
+
+
+def match_paren(content: str, open_idx: int, limit: int = SCAN_LIMIT, closer: str = ")") -> int:
+ """Index of the bracket closing the one at *open_idx*, or -1.
+
+ Quote- and template-aware so a parenthesis inside a string literal does not
+ unbalance the scan. A call that does not close within *limit* characters
+ is -1 as well. *closer* is ``}`` for a trailing lambda block.
+ """
+ return _scan(content, open_idx, min(len(content), open_idx + limit), closer=closer)[0]
+
+
+def split_top_level(text: str, sep: str) -> list[str]:
+ """Split *text* at each top-level *sep* character, outside quotes and brackets."""
+ _, seps = _scan(text, 0, len(text), sep=sep)
+ bounds = [-1, *seps, len(text)]
+ return [text[a + 1 : b] for a, b in pairwise(bounds)]
+
+
+def split_first_arg(args: str) -> tuple[str, str]:
+ """Split an argument list into ``(first_arg, rest)`` at the top-level comma.
+
+ Stops at that comma: the rest is returned unscanned, so a call whose second
+ argument is a large options literal costs only its first argument.
+ """
+ _, seps = _scan(args, 0, len(args), sep=",", first_sep_only=True)
+ if not seps:
+ return args.strip(), ""
+ return args[: seps[0]].strip(), args[seps[0] + 1 :]
+
+
+def call_arguments(content: str, paren_offset: int, close: int | None = None) -> list[str] | None:
+ """The top-level arguments of the call whose ``(`` is at *paren_offset*.
+
+ ``None`` when the argument list does not close, so a recogniser can tell a
+ scanner failure from a call with no arguments. *close* is the matching
+ ``)`` when the caller has already found it.
+ """
+ if close is None:
+ close = match_paren(content, paren_offset)
+ if close < 0:
+ return None
+ inner = content[paren_offset + 1 : close]
+ if not inner.strip():
+ return []
+ return [a.strip() for a in split_top_level(inner, ",")]
+
+
+# ---------------------------------------------------------------------------
+# Expression resolution
+# ---------------------------------------------------------------------------
+
+NAME_RE = re.compile(r"[A-Za-z_$][\w$]*(?:(?:\.|::|->)[A-Za-z_]\w*)*")
+
+# Already-template text: one ``${expr}`` interpolation, for folding a constant
+# inside a body a language wrote in the target form to begin with.
+_TEMPLATE_INTERP_RE = re.compile(r"\$\{([^{}]+)\}")
+
+# Dropped from an interpolated expression (``settings.base.rstrip('/')``) so a
+# base-token reader sees the attribute, not ``rstrip``.
+_TRAILING_CALL_RE = re.compile(r"\.\s*\w+\s*\([^()]*\)\s*$")
+
+
+@cache
+def _literal_re(quotes: str, prefixes: str) -> re.Pattern[str]:
+ alts = []
+ if '"' in quotes:
+ alts.append('"""')
+ if "'" in quotes:
+ alts.append("'''")
+ alts.extend(re.escape(q) for q in quotes)
+ prefix = f"[{re.escape(prefixes)}]*" if prefixes else ""
+ return re.compile(
+ rf"^(?P<prefix>{prefix})(?P<q>{'|'.join(alts)})(?P<body>.*)(?P=q)$", re.DOTALL
+ )
+
+
+@cache
+def _head_re(heads: tuple[str, ...]) -> re.Pattern[str]:
+ alts = "|".join(re.escape(h).replace(r"\ ", r"\s+") for h in heads)
+ return re.compile(rf"^(?:{alts})\s*\(")
+
+
+def _parse_literal(text: str, syntax: StringSyntax) -> tuple[str, str, str] | None:
+ """``(lowercased prefix, quote, body)`` when *text* is exactly one string literal."""
+ m = _literal_re(syntax.quotes, syntax.prefixes).match(text.strip())
+ if m is None:
+ return None
+ body = m.group("body")
+ if m.group("q") in body:
+ return None # the literal ended and something else followed it
+ return m.group("prefix").lower(), m.group("q"), body
+
+
+def _interpolates(prefix: str, quote: str, syntax: StringSyntax) -> bool:
+ # A triple quote interpolates when its single form does (Kotlin raw strings).
+ return quote[0] in syntax.template_quotes or any(
+ p in syntax.template_prefixes.lower() for p in prefix
+ )
+
+
+def _lookup(name: str, constants: dict[str, str]) -> str | None:
+ const = constants.get(name)
+ if const is None and name.startswith("$"):
+ const = constants.get(name[1:]) # a PHP variable, keyed without its sigil
+ return const
+
+
+def _fold(expr: str, syntax: StringSyntax, constants: dict[str, str]) -> str:
+ """One interpolated expression as template text, inlining a plain constant.
+
+ Only a constant whose own text does not interpolate is inlined: nothing in
+ the corpus needs a second level, and a half-folded template would be wrong.
+ """
+ expr = expr.strip()
+ const = _lookup(expr, constants)
+ if const is not None:
+ lit = _parse_literal(const, syntax)
+ text = _literal_text(lit, syntax, {}) if lit is not None else None
+ if text is not None and "${" not in text:
+ return text
+ return "${" + _TRAILING_CALL_RE.sub("", expr) + "}"
+
+
+def _literal_text(
+ parsed: tuple[str, str, str], syntax: StringSyntax, constants: dict[str, str]
+) -> str | None:
+ prefix, quote, body = parsed
+ if any(p in syntax.refuse_prefixes for p in prefix):
+ return None # bytes, not text this layer records
+ if not _interpolates(prefix, quote, syntax):
+ return body
+ if syntax.refuse is not None and syntax.refuse.search(body):
+ return None
+ if syntax.interpolation is None:
+ if not constants:
+ return body
+ return _TEMPLATE_INTERP_RE.sub(lambda m: _fold(m.group(1), syntax, constants), body)
+
+ def sub(m: re.Match[str]) -> str:
+ expr = next(g for g in m.groups() if g is not None)
+ return _fold(expr, syntax, constants)
+
+ return syntax.interpolation.sub(sub, body)
+
+
+def _format_template(text: str, syntax: StringSyntax, constants: dict[str, str]) -> str | None:
+ """A ``format!("...")``-style call as template text, or ``None``."""
+ m = _head_re(syntax.format_heads).match(text)
+ if m is None or syntax.placeholder is None:
+ return None
+ args = _whole_call_arguments(text, m.end() - 1)
+ if not args:
+ return None
+ parsed = _parse_literal(args[0], syntax)
+ if parsed is None:
+ return None
+ body = _literal_text(parsed, syntax, constants)
+ if body is None:
+ return None
+ # `%%` is a literal percent sign, not a hole.
+ return syntax.placeholder.sub(lambda h: "%" if h.group() == "%%" else "${x}", body)
+
+
+def _whole_call_arguments(text: str, paren: int) -> list[str] | None:
+ """The arguments of the call at *paren* when the call is all of *text*.
+
+ ``URI.create(BASE).resolve("/x")`` is not ``URI.create(BASE)``: reading the
+ head alone would drop the path the request reaches.
+ """
+ close = match_paren(text, paren)
+ if close < 0 or text[close + 1 :].strip():
+ return None
+ return call_arguments(text, paren, close)
+
+
+def _unwrap(text: str, syntax: StringSyntax) -> str | None:
+ """The single argument of a ``URI.create(...)``-style wrapper, or ``None``."""
+ m = _head_re(syntax.unwrap_heads).match(text)
+ if m is None:
+ return None
+ args = _whole_call_arguments(text, m.end() - 1)
+ return args[0] if args and len(args) == 1 else None
+
+
+def _concat(text: str, syntax: StringSyntax, constants: dict[str, str]) -> str | None:
+ """``a + "/x"`` as template text, or ``None`` when a piece is not readable."""
+ if syntax.concat not in text:
+ return None
+ pieces = split_top_level(text, syntax.concat)
+ if len(pieces) < 2:
+ return None
+ out: list[str] = []
+ for piece in pieces:
+ piece = piece.strip()
+ parsed = _parse_literal(piece, syntax)
+ if parsed is not None:
+ body = _literal_text(parsed, syntax, constants)
+ if body is None:
+ return None
+ out.append(body)
+ elif NAME_RE.fullmatch(piece):
+ out.append(_fold(piece, syntax, constants))
+ else:
+ return None # a call or an operator: not settled in this file
+ return "".join(out)
+
+
+def resolve_string(
+ expr: str, syntax: StringSyntax, constants: dict[str, str] | None = None
+) -> str | None:
+ """The text *expr* denotes, or ``None`` when it cannot be resolved here.
+
+ Interpolations come out as ``${expr}``, so a URL's base placeholder can be
+ stripped and a path parameter collapsed with no per-language branch.
+ """
+ constants = constants or {}
+ text = expr.strip()
+ parsed = _parse_literal(text, syntax)
+ if parsed is not None:
+ return _literal_text(parsed, syntax, constants)
+ if syntax.format_heads:
+ folded = _format_template(text, syntax, constants)
+ if folded is not None:
+ return folded
+ if syntax.unwrap_heads:
+ inner = _unwrap(text, syntax)
+ if inner is not None:
+ return resolve_string(inner, syntax, constants)
+ if syntax.concat:
+ folded = _concat(text, syntax, constants)
+ if folded is not None:
+ return folded
+ if NAME_RE.fullmatch(text):
+ # A bare name. Its value resolved without constants when it was
+ # recorded, so this step cannot reach another bare name.
+ const = _lookup(text, constants)
+ if const is not None:
+ return resolve_string(const, syntax, constants)
+ return None
+
+
+def string_constants(
+ content: str, syntax: StringSyntax, code: str | None = None
+) -> dict[str, str]:
+ """Names *content* assigns exactly once, to a string expression that resolves.
+
+ The value is the right-hand side's raw text, so an interpolating literal,
+ a format call or a concatenation keeps its parts for :func:`resolve_string`
+ to fold at the use site.
+
+ *code*, when given, is *content* with comments and string bodies blanked at
+ the same offsets. Assignments are located in it and their values read from
+ *content*, so the example code in a docstring (`` url = "/docs/example"``)
+ is not read as a real assignment.
+ """
+ if syntax.assignment is None:
+ return {}
+ text = content if code is None else code
+ seen: dict[str, str | None] = {}
+ for m in syntax.assignment.finditer(text):
+ if code is None and _on_comment_line(content, m.start()):
+ continue # an example in a doc comment is not a binding
+ name = m.group("name")
+ rhs = content[m.start("rhs") : m.end("rhs")].strip()
+ if syntax.assignment_strip is not None:
+ rhs = syntax.assignment_strip.sub("", rhs).strip()
+ # A second assignment retires the name whatever it assigns: the reader
+ # cannot tell which one reaches the use site.
+ seen[name] = None if name in seen or resolve_string(rhs, syntax) is None else rhs
+ # `x += "/v1"` and `x, err = f()` rebind a name without the plain form.
+ for m in _COMPOUND_ASSIGN_RE.finditer(text):
+ seen[m.group(1)] = None
+ for m in _MULTI_ASSIGN_RE.finditer(text):
+ for name in m.group(1).replace(" ", "").split(","):
+ seen[name.lstrip("$")] = None
+ return {name: text for name, text in seen.items() if text is not None}
+
+
+# ``x += ...`` / ``$x .= ...``; the name is keyed without a PHP sigil.
+_COMPOUND_ASSIGN_RE = re.compile(r"(?<![\w.$-])\$?([A-Za-z_]\w*)[ \t]*[+.]=(?!=)")
+# ``a, b := f()`` / ``a, b = f()``: every name on the left is rebound.
+_MULTI_ASSIGN_RE = re.compile(
+ r"^[ \t]*(\$?[A-Za-z_]\w*(?:[ \t]*,[ \t]*\$?[A-Za-z_]\w*)+)[ \t]*:?=(?!=)", re.MULTILINE
+)
+
+
+def _on_comment_line(content: str, offset: int) -> bool:
+ """True when the line holding *offset* starts as a line or block comment."""
+ start = max(content.rfind("\n", 0, offset) + 1, offset - 200)
+ return content[start:offset].lstrip().startswith(("//", "#", "*", "/*"))
+
+
+def literal_span(content: str, m: re.Match[str], group: int) -> str:
+ """The source text of a quoted literal whose *body* is *group* of *m*.
+
+ For a recogniser whose regex captures the text between the quotes: the
+ slice one character wider on each side is the literal as written, which is
+ what :func:`resolve_string` reads. A capture that stopped at a different
+ quote (one inside a ``${...}`` expression) is closed with its own opening
+ quote, so the text read is the prefix the regex saw and the row is kept.
+ """
+ quote = content[m.start(group) - 1]
+ body = content[m.start(group) : m.end(group)]
+ return quote + body + quote
+
+
+# ---------------------------------------------------------------------------
+# Selecting an argument
+# ---------------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class Arg:
+ """Where a call carries a value: a keyword or map key first, else a position.
+
+ ``keys`` match a keyword argument (``queue='jobs'``, ``topics = "a"``) or a
+ key of a map-literal argument (``{ topic: 'a' }``, ``['queue' => 'a']``).
+ A value that is an array literal yields each element.
+ """
+
+ keys: tuple[str, ...] = ()
+ pos: int | None = None
+
+
+# One map entry's key: a quoted string or an identifier, then `:` or `=>`.
+_MAP_KEY_RE = re.compile(r"""^(?:'[^']*'|"[^"]*"|[A-Za-z_$][\w$]*)\s*(?:=>|:(?!:))""")
+
+
+@cache
+def _keyword_re(keys: tuple[str, ...]) -> re.Pattern[str]:
+ alts = "|".join(re.escape(k) for k in keys)
+ return re.compile(
+ rf"""^['"]?(?:{alts})['"]?\s*(?:=>|:(?!:)|=(?!=))\s*(?P<value>.+)$""", re.DOTALL
+ )
+
+
+def _bracketed(text: str) -> bool:
+ return len(text) >= 2 and text[0] in "[{" and text[-1] in "]}"
+
+
+def _map_entries(text: str) -> list[str] | None:
+ """The entries of a map literal (``{a: 1}``, ``['a' => 1]``), or ``None`` if *text* is not one."""
+ if not _bracketed(text):
+ return None
+ entries = [e.strip() for e in split_top_level(text[1:-1], ",") if e.strip()]
+ # `any`: a JS object may mix `key: value` with shorthand `{ topic, messages }`.
+ if any(_MAP_KEY_RE.match(e) for e in entries):
+ return entries
+ return None
+
+
+def select_argument(args: list[str], arg: Arg) -> list[str]:
+ """The source text of the value(s) *arg* selects from *args*, arrays expanded."""
+ value: str | None = None
+ if arg.keys:
+ pattern = _keyword_re(arg.keys)
+ for a in args:
+ for entry in _map_entries(a) or [a]:
+ m = pattern.match(entry)
+ if m is not None:
+ value = m.group("value").strip()
+ break
+ if value is not None:
+ break
+ if value is None and arg.pos is not None and arg.pos < len(args):
+ value = args[arg.pos]
+ if value is None:
+ return []
+ if _bracketed(value) and _map_entries(value) is None:
+ return [e.strip() for e in split_top_level(value[1:-1], ",") if e.strip()]
+ return [value]
+
+
+def resolve_argument(
+ args: list[str], arg: Arg, syntax: StringSyntax, constants: dict[str, str]
+) -> tuple[list[str], bool]:
+ """``(values, refused)`` for *arg*: each selected value resolved to plain text.
+
+ A value that does not resolve, or resolves to a template with a hole in
+ it, is dropped and ``refused`` is set, so a caller can tell "the call did
+ not say" from "the call said something this file cannot settle".
+ """
+ values: list[str] = []
+ refused = False
+ for raw in select_argument(args, arg):
+ text = resolve_string(raw, syntax, constants)
+ if text is None or "${" in text:
+ refused = True
+ else:
+ values.append(text.strip())
+ return values, refused
+
+
+# ---------------------------------------------------------------------------
+# Per-language syntax tables
+# ---------------------------------------------------------------------------
+
+_C_FORMAT_PLACEHOLDER_RE = re.compile(r"%%|%[-+ #0]*\d*(?:\.\d+)?[a-zA-Z]")
+_BRACE_PLACEHOLDER_RE = re.compile(r"\{[^}]*\}")
+_BRACE_INTERP_RE = re.compile(r"(?<!\{)\{([^{}]+)\}(?!\})")
+_ESCAPED_BRACE_RE = re.compile(r"\{\{|\}\}")
+
+JS_SYNTAX = StringSyntax(quotes="\"'`", template_quotes="`")
+
+# ``NAME = <expr>`` at any indentation, RHS to end of line. The whitespace
+# around ``=`` is required, which excludes the usual unspaced keyword argument
+# on its own line (``url=str(resp.url),``) that otherwise reads as a second
+# assignment and retires the very name being folded.
+PYTHON_SYNTAX = StringSyntax(
+ prefixes="fFrRbBuU",
+ template_prefixes="f",
+ interpolation=_BRACE_INTERP_RE,
+ refuse=_ESCAPED_BRACE_RE,
+ refuse_prefixes="b",
+ assignment=re.compile(
+ r"^[ \t]*(?P<name>[A-Za-z_]\w*)(?:[ \t]*:[^=\n]+)?[ \t]+=[ \t]+(?P<rhs>.*)$",
+ re.MULTILINE,
+ ),
+)
+
+RUST_SYNTAX = StringSyntax(
+ quotes='"',
+ format_heads=("format!",),
+ placeholder=_BRACE_PLACEHOLDER_RE,
+)
+
+CSHARP_SYNTAX = StringSyntax(
+ quotes='"',
+ prefixes="$@",
+ template_prefixes="$",
+ interpolation=_BRACE_INTERP_RE,
+ refuse=_ESCAPED_BRACE_RE,
+)
+
+GO_SYNTAX = StringSyntax(
+ quotes='"`',
+ format_heads=("fmt.Sprintf",),
+ placeholder=_C_FORMAT_PLACEHOLDER_RE,
+ concat="+",
+ assignment=re.compile(
+ r"^[ \t]*(?:(?:const|var)[ \t]+)?(?P<name>[A-Za-z_]\w*)[ \t]*:?=[ \t]*(?P<rhs>[^\n]+)$",
+ re.MULTILINE,
+ ),
+)
+
+RUBY_SYNTAX = StringSyntax(
+ template_quotes='"',
+ interpolation=re.compile(r"#\{([^{}]+)\}"),
+ unwrap_heads=("URI.parse", "URI"),
+ concat="+",
+ assignment=re.compile(
+ r"^[ \t]*(?P<name>[A-Za-z_]\w*)[ \t]*=[ \t]*(?P<rhs>[^\n]+)$", re.MULTILINE
+ ),
+ assignment_strip=re.compile(r"\.freeze$"),
+)
+
+JAVA_SYNTAX = StringSyntax(
+ quotes='"',
+ format_heads=("String.format",),
+ placeholder=_C_FORMAT_PLACEHOLDER_RE,
+ unwrap_heads=("URI.create", "new URI", "new URL", "URI", "HttpUrl.parse"),
+ concat="+",
+ assignment=re.compile(
+ r"(?<![=!<>.\w])(?P<name>[A-Za-z_]\w*)[ \t]*=(?!=)[ \t]*(?P<rhs>[^;\n]+);"
+ ),
+)
+
+KOTLIN_SYNTAX = StringSyntax(
+ quotes='"',
+ template_quotes='"',
+ interpolation=re.compile(r"\$\{([^{}]+)\}|\$([A-Za-z_]\w*)"),
+ unwrap_heads=("Url", "URI.create", "URI"),
+ concat="+",
+ assignment=re.compile(
+ r"\b(?:val|var)[ \t]+(?P<name>[A-Za-z_]\w*)(?:[ \t]*:[ \t]*[\w<>?]+)?[ \t]*=[ \t]*(?P<rhs>[^\n]+)$",
+ re.MULTILINE,
+ ),
+)
+
+PHP_SYNTAX = StringSyntax(
+ template_quotes='"',
+ interpolation=re.compile(r"\{\$([^{}]+)\}|\$\{([^{}]+)\}|\$([A-Za-z_]\w*(?:->[A-Za-z_]\w*)?)"),
+ concat=".",
+ assignment=re.compile(
+ r"(?:\bconst[ \t]+|\$)(?P<name>[A-Za-z_]\w*)[ \t]*=(?!=)[ \t]*(?P<rhs>[^;\n]+);"
+ ),
+)
+
+_SYNTAX_BY_LANGUAGE: tuple[tuple[frozenset[str], StringSyntax], ...] = (
+ (JS_TS, JS_SYNTAX),
+ (PYTHON, PYTHON_SYNTAX),
+ (JAVA, JAVA_SYNTAX),
+ (KOTLIN, KOTLIN_SYNTAX),
+ (GO, GO_SYNTAX),
+ (RUBY, RUBY_SYNTAX),
+ (PHP, PHP_SYNTAX),
+ (CSHARP, CSHARP_SYNTAX),
+ (RUST, RUST_SYNTAX),
+)
+
+
+def syntax_for_suffix(suffix: str) -> StringSyntax | None:
+ """The syntax table for a file extension, or ``None`` for a language without one."""
+ for extensions, syntax in _SYNTAX_BY_LANGUAGE:
+ if suffix in extensions:
+ return syntax
+ return None
+
+
+__all__ = [
+ "CSHARP_SYNTAX",
+ "GO_SYNTAX",
+ "JAVA_SYNTAX",
+ "JS_SYNTAX",
+ "KOTLIN_SYNTAX",
+ "NAME_RE",
+ "PHP_SYNTAX",
+ "PYTHON_SYNTAX",
+ "RUBY_SYNTAX",
+ "RUST_SYNTAX",
+ "SCAN_LIMIT",
+ "Arg",
+ "StringSyntax",
+ "call_arguments",
+ "literal_span",
+ "match_paren",
+ "resolve_argument",
+ "resolve_string",
+ "select_argument",
+ "split_first_arg",
+ "split_top_level",
+ "string_constants",
+ "syntax_for_suffix",
+]
diff --git a/packages/core/src/repowise/core/workspace/extractors/topic/__init__.py b/packages/core/src/repowise/core/workspace/extractors/topic/__init__.py
new file mode 100644
index 000000000..130ec6a4a
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/topic/__init__.py
@@ -0,0 +1,26 @@
+"""Message topic and queue contract extraction.
+
+Each broker is a :class:`.dialect.TopicDialect` table in its own module,
+registered in :data:`DIALECTS`. Matching, including queues reached through an
+exchange binding, happens downstream in :mod:`repowise.core.workspace.matching`.
+"""
+
+from __future__ import annotations
+
+from ..dialect import ContractDialect, DialectExtractor
+from .dialect import topic_identity
+from .kafka import KAFKA
+from .nats import NATS
+from .rabbitmq import RABBITMQ
+
+DIALECTS: tuple[ContractDialect, ...] = (KAFKA, RABBITMQ, NATS)
+
+
+class TopicExtractor(DialectExtractor):
+ """Extract message topic/queue contracts from source files."""
+
+ dialects = DIALECTS
+ identity = staticmethod(topic_identity)
+
+
+__all__ = ["DIALECTS", "TopicExtractor"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/topic/dialect.py b/packages/core/src/repowise/core/workspace/extractors/topic/dialect.py
new file mode 100644
index 000000000..f3c37c3d3
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/topic/dialect.py
@@ -0,0 +1,148 @@
+"""Topic dialects: a broker library's calls as a table, read through one resolver.
+
+A :class:`TopicCall` names a call head (a regex ending at the opening
+parenthesis), the role it plays, and where in the argument list the
+destination name sits (:class:`..strings.Arg`). :class:`TopicDialect` reads
+the argument list with the shared bracket scanner and resolves each value
+through :func:`..strings.resolve_argument`, so a queue named by a constant
+folds exactly as a URL does. A value that cannot be settled inside the file is
+refused.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+from repowise.core.workspace.contracts import (
+ TOPIC_KIND_QUEUE,
+ TOPIC_KIND_TOPIC,
+ TOPIC_ROUTING_KEY_UNRESOLVED,
+)
+
+from ..base import line_at
+from ..dialect import build_contract
+from ..strings import call_arguments, resolve_argument, string_constants, syntax_for_suffix
+
+if TYPE_CHECKING:
+ import re
+
+ from repowise.core.workspace.contracts import Contract
+
+ from ..base import ScanContext
+ from ..strings import Arg, StringSyntax
+
+
+@dataclass(frozen=True)
+class TopicCall:
+ """One broker call shape.
+
+ ``default_exchange`` marks a publish whose empty exchange name is the
+ broker's default exchange, which delivers to the queue the routing key
+ names. ``queue`` is the bound queue of a binding call.
+ """
+
+ head: re.Pattern[str]
+ role: str
+ label: str
+ extensions: frozenset[str]
+ name: Arg
+ kind: str = TOPIC_KIND_TOPIC
+ confidence: float = 0.8
+ routing_key: Arg | None = None
+ queue: Arg | None = None
+ default_exchange: bool = False
+
+
+class TopicDialect:
+ """One broker library's calls, for the files whose extension its calls read."""
+
+ def __init__(self, name: str, calls: tuple[TopicCall, ...]) -> None:
+ self.name = name
+ self.calls = calls
+ self.extensions = frozenset().union(*(c.extensions for c in calls))
+
+ def extract(self, ctx: ScanContext) -> list[Contract]:
+ syntax = syntax_for_suffix(ctx.suffix)
+ if syntax is None:
+ return []
+ content = ctx.content
+ constants: dict[str, str] | None = None
+ out: list[Contract] = []
+ for call in self.calls:
+ if ctx.suffix not in call.extensions:
+ continue
+ for m in call.head.finditer(content):
+ args = call_arguments(content, m.end() - 1)
+ if not args:
+ continue
+ if constants is None:
+ constants = string_constants(content, syntax)
+ out.extend(self._contracts(ctx, call, args, m.start(), syntax, constants))
+ return out
+
+ def _contracts(
+ self,
+ ctx: ScanContext,
+ call: TopicCall,
+ args: list[str],
+ offset: int,
+ syntax: StringSyntax,
+ constants: dict[str, str],
+ ) -> list[Contract]:
+ keys, key_refused = (
+ resolve_argument(args, call.routing_key, syntax, constants)
+ if call.routing_key
+ else ([], False)
+ )
+ routing_key = keys[0] if keys else ""
+ queues: list[str] = []
+ if call.queue is not None:
+ queues, _ = resolve_argument(args, call.queue, syntax, constants)
+ if not queues:
+ return [] # a binding with no readable queue binds nothing we can name
+ names, _ = resolve_argument(args, call.name, syntax, constants)
+ out: list[Contract] = []
+ for name in names:
+ kind, key, unresolved = call.kind, routing_key, key_refused
+ if not name and call.default_exchange:
+ if not routing_key:
+ continue # the queue the default exchange delivers to is unknown
+ name, kind, key, unresolved = routing_key, TOPIC_KIND_QUEUE, "", False
+ if not name:
+ continue
+ meta: dict[str, str | bool] = {"topic": name, "broker": self.name, "kind": kind}
+ if key:
+ meta["routing_key"] = key
+ if unresolved:
+ meta[TOPIC_ROUTING_KEY_UNRESOLVED] = True
+ if queues:
+ meta["queue"] = queues[0]
+ out.append(
+ build_contract(
+ ctx,
+ contract_type="topic",
+ contract_id=f"topic::{name.lower()}",
+ role=call.role,
+ symbol_name=f"{call.label}('{name}')",
+ confidence=call.confidence,
+ line=line_at(ctx.content, offset),
+ meta=meta,
+ )
+ )
+ return out
+
+
+def topic_identity(c: Contract) -> tuple[str, ...]:
+ """One contract per file, id, role and route: a second key or bound queue is new."""
+ meta = c.meta
+ return (
+ c.file_path,
+ c.contract_id,
+ c.role,
+ meta.get("routing_key", ""),
+ meta.get("queue", ""),
+ )
+
+
+__all__ = ["TopicCall", "TopicDialect", "topic_identity"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/topic/kafka.py b/packages/core/src/repowise/core/workspace/extractors/topic/kafka.py
new file mode 100644
index 000000000..5bf42a1f1
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/topic/kafka.py
@@ -0,0 +1,72 @@
+"""Kafka producers and consumers: Spring Kafka, kafkajs, kafka-python / confluent, sarama."""
+
+from __future__ import annotations
+
+import re
+
+from ..langs import GO, JAVA, JS_TS, PYTHON
+from ..strings import Arg
+from .dialect import TopicCall, TopicDialect
+
+_TOPIC = Arg(keys=("topics", "topic"))
+_FIRST = Arg(pos=0)
+
+KAFKA = TopicDialect(
+ name="kafka",
+ calls=(
+ TopicCall(
+ head=re.compile(r"@KafkaListener\s*\("),
+ role="consumer",
+ label="@KafkaListener",
+ extensions=JAVA,
+ name=_TOPIC,
+ ),
+ TopicCall(
+ head=re.compile(r"kafkaTemplate\.send\s*\("),
+ role="provider",
+ label="kafkaTemplate.send",
+ extensions=JAVA,
+ name=_FIRST,
+ ),
+ TopicCall(
+ head=re.compile(r"producer\.send\s*\("),
+ role="provider",
+ label="producer.send({topic})",
+ extensions=JS_TS,
+ name=_TOPIC,
+ ),
+ TopicCall(
+ head=re.compile(r"consumer\.subscribe\s*\("),
+ role="consumer",
+ label="consumer.subscribe({topic})",
+ extensions=JS_TS,
+ name=_TOPIC,
+ ),
+ TopicCall(
+ head=re.compile(r"KafkaConsumer\s*\("),
+ role="consumer",
+ label="KafkaConsumer",
+ extensions=PYTHON,
+ name=_FIRST,
+ confidence=0.7,
+ ),
+ TopicCall(
+ head=re.compile(r"producer\.produce\s*\("),
+ role="provider",
+ label="producer.produce",
+ extensions=PYTHON | JS_TS,
+ name=Arg(keys=("topic",), pos=0),
+ confidence=0.7,
+ ),
+ TopicCall(
+ head=re.compile(r"ConsumePartition\s*\("),
+ role="consumer",
+ label="ConsumePartition",
+ extensions=GO,
+ name=_FIRST,
+ confidence=0.7,
+ ),
+ ),
+)
+
+__all__ = ["KAFKA"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/topic/nats.py b/packages/core/src/repowise/core/workspace/extractors/topic/nats.py
new file mode 100644
index 000000000..697a3cdc0
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/topic/nats.py
@@ -0,0 +1,42 @@
+"""NATS publishers and subscribers.
+
+The receiver must carry a NATS-idiomatic name, so an RxJS or EventEmitter
+``subscribe``/``publish`` is not read as a subject.
+"""
+
+from __future__ import annotations
+
+import re
+
+from repowise.core.workspace.contracts import TOPIC_KIND_SUBJECT
+
+from ..langs import GO, JAVA, JS_TS, PYTHON
+from ..strings import Arg
+from .dialect import TopicCall, TopicDialect
+
+_RECEIVER = r"(?:nc|nats|conn|js|sub|client)\s*\.\s*"
+_LANGUAGES = GO | JAVA | JS_TS | PYTHON
+
+NATS = TopicDialect(
+ name="nats",
+ calls=(
+ TopicCall(
+ head=re.compile(_RECEIVER + r"(?:Subscribe|subscribe)\s*\("),
+ role="consumer",
+ label="nc.Subscribe",
+ extensions=_LANGUAGES,
+ name=Arg(pos=0),
+ kind=TOPIC_KIND_SUBJECT,
+ ),
+ TopicCall(
+ head=re.compile(_RECEIVER + r"(?:Publish|publish)\s*\("),
+ role="provider",
+ label="nc.Publish",
+ extensions=_LANGUAGES,
+ name=Arg(pos=0),
+ kind=TOPIC_KIND_SUBJECT,
+ ),
+ ),
+)
+
+__all__ = ["NATS"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/topic/rabbitmq.py b/packages/core/src/repowise/core/workspace/extractors/topic/rabbitmq.py
new file mode 100644
index 000000000..dad606a9f
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/extractors/topic/rabbitmq.py
@@ -0,0 +1,113 @@
+"""RabbitMQ publishers, consumers and queue bindings: Spring AMQP, amqplib, pika.
+
+A publish names an exchange and a routing key, a consume names a queue, and a
+binding connects the two; see :mod:`repowise.core.workspace.matching.topic`.
+"""
+
+from __future__ import annotations
+
+import re
+
+from repowise.core.workspace.contracts import (
+ TOPIC_KIND_BINDING,
+ TOPIC_KIND_EXCHANGE,
+ TOPIC_KIND_QUEUE,
+)
+
+from ..langs import JAVA, JS_TS, PYTHON
+from ..strings import Arg
+from .dialect import TopicCall, TopicDialect
+
+RABBITMQ = TopicDialect(
+ name="rabbitmq",
+ calls=(
+ TopicCall(
+ head=re.compile(r"@RabbitListener\s*\("),
+ role="consumer",
+ label="@RabbitListener",
+ extensions=JAVA,
+ name=Arg(keys=("queues", "queue")),
+ kind=TOPIC_KIND_QUEUE,
+ ),
+ TopicCall(
+ head=re.compile(r"rabbitTemplate\.convertAndSend\s*\("),
+ role="provider",
+ label="rabbitTemplate.convertAndSend",
+ extensions=JAVA,
+ name=Arg(pos=0),
+ kind=TOPIC_KIND_EXCHANGE,
+ # `(exchange, routingKey, message)`; in the two-argument form the
+ # second is the message, which does not resolve, so it routes nowhere.
+ routing_key=Arg(pos=1),
+ ),
+ TopicCall(
+ head=re.compile(r"channel\.consume\s*\("),
+ role="consumer",
+ label="channel.consume",
+ extensions=JS_TS,
+ name=Arg(pos=0),
+ kind=TOPIC_KIND_QUEUE,
+ ),
+ TopicCall(
+ head=re.compile(r"channel\.publish\s*\("),
+ role="provider",
+ label="channel.publish",
+ extensions=JS_TS,
+ name=Arg(pos=0),
+ kind=TOPIC_KIND_EXCHANGE,
+ routing_key=Arg(pos=1),
+ default_exchange=True,
+ ),
+ TopicCall(
+ head=re.compile(r"channel\.sendToQueue\s*\("),
+ role="provider",
+ label="channel.sendToQueue",
+ extensions=JS_TS,
+ name=Arg(pos=0),
+ kind=TOPIC_KIND_QUEUE,
+ ),
+ TopicCall(
+ head=re.compile(r"channel\.bindQueue\s*\("),
+ role="consumer",
+ label="channel.bindQueue",
+ extensions=JS_TS,
+ name=Arg(pos=1),
+ kind=TOPIC_KIND_BINDING,
+ routing_key=Arg(pos=2),
+ queue=Arg(pos=0),
+ ),
+ TopicCall(
+ head=re.compile(r"channel\.basic_consume\s*\("),
+ role="consumer",
+ label="basic_consume",
+ extensions=PYTHON,
+ name=Arg(keys=("queue",), pos=0),
+ kind=TOPIC_KIND_QUEUE,
+ confidence=0.7,
+ ),
+ TopicCall(
+ head=re.compile(r"channel\.basic_publish\s*\("),
+ role="provider",
+ label="basic_publish",
+ extensions=PYTHON,
+ name=Arg(keys=("exchange",)),
+ kind=TOPIC_KIND_EXCHANGE,
+ confidence=0.7,
+ routing_key=Arg(keys=("routing_key",)),
+ default_exchange=True,
+ ),
+ TopicCall(
+ head=re.compile(r"channel\.queue_bind\s*\("),
+ role="consumer",
+ label="queue_bind",
+ extensions=PYTHON,
+ name=Arg(keys=("exchange",), pos=1),
+ kind=TOPIC_KIND_BINDING,
+ confidence=0.7,
+ routing_key=Arg(keys=("routing_key",), pos=2),
+ queue=Arg(keys=("queue",), pos=0),
+ ),
+ ),
+)
+
+__all__ = ["RABBITMQ"]
diff --git a/packages/core/src/repowise/core/workspace/extractors/topic_extractor.py b/packages/core/src/repowise/core/workspace/extractors/topic_extractor.py
deleted file mode 100644
index 6c272112f..000000000
--- a/packages/core/src/repowise/core/workspace/extractors/topic_extractor.py
+++ /dev/null
@@ -1,282 +0,0 @@
-"""Message topic/queue contract extraction.
-
-Scans source files for Kafka, RabbitMQ, and NATS producer (provider) and
-consumer patterns.
-"""
-
-from __future__ import annotations
-
-import logging
-import re
-from dataclasses import dataclass
-from pathlib import Path
-from typing import TYPE_CHECKING
-
-from repowise.core.ingestion.languages.registry import REGISTRY as _LANG_REGISTRY
-
-from .base import line_at, select_files
-
-if TYPE_CHECKING:
- from collections.abc import Callable, Sequence
-
- from repowise.core.workspace.contracts import Contract
-
- from .base import SourceFile
-
-_log = logging.getLogger("repowise.workspace.extractors.topic")
-
-# ---------------------------------------------------------------------------
-# Constants
-# ---------------------------------------------------------------------------
-
-_EXTENSIONS = _LANG_REGISTRY.extensions_for(["python", "typescript", "javascript", "java", "go"])
-
-
-# ---------------------------------------------------------------------------
-# Pattern definitions
-# ---------------------------------------------------------------------------
-
-
-@dataclass(frozen=True)
-class _PatternDef:
- regex: re.Pattern[str]
- role: str # "provider" | "consumer"
- broker: str # "kafka" | "rabbitmq" | "nats"
- confidence: float
- topic_group: int # capture group index for topic name
- label: str # human-readable name
-
-
-# --- Kafka ---
-
-_KAFKA_PATTERNS: list[_PatternDef] = [
- # Java: @KafkaListener(topics = "orders")
- _PatternDef(
- regex=re.compile(r"""@KafkaListener\s*\([^)]*topics?\s*=\s*['"]([^'"]+)['"]"""),
- role="consumer",
- broker="kafka",
- confidence=0.8,
- topic_group=1,
- label="@KafkaListener",
- ),
- # Java: kafkaTemplate.send("orders", ...)
- _PatternDef(
- regex=re.compile(r"""kafkaTemplate\.send\s*\(\s*['"]([^'"]+)['"]"""),
- role="provider",
- broker="kafka",
- confidence=0.8,
- topic_group=1,
- label="kafkaTemplate.send",
- ),
- # Node: producer.send({ topic: 'orders' })
- _PatternDef(
- regex=re.compile(r"""producer\.send\s*\(\s*\{\s*topic\s*:\s*['"]([^'"]+)['"]"""),
- role="provider",
- broker="kafka",
- confidence=0.8,
- topic_group=1,
- label="producer.send({topic})",
- ),
- # Node: consumer.subscribe({ topic: 'orders' })
- _PatternDef(
- regex=re.compile(r"""consumer\.subscribe\s*\(\s*\{\s*topic\s*:\s*['"]([^'"]+)['"]"""),
- role="consumer",
- broker="kafka",
- confidence=0.8,
- topic_group=1,
- label="consumer.subscribe({topic})",
- ),
- # Python: KafkaConsumer('orders')
- _PatternDef(
- regex=re.compile(r"""KafkaConsumer\s*\(\s*['"]([^'"]+)['"]"""),
- role="consumer",
- broker="kafka",
- confidence=0.7,
- topic_group=1,
- label="KafkaConsumer",
- ),
- # Python: producer.produce('orders')
- _PatternDef(
- regex=re.compile(r"""producer\.produce\s*\(\s*['"]([^'"]+)['"]"""),
- role="provider",
- broker="kafka",
- confidence=0.7,
- topic_group=1,
- label="producer.produce",
- ),
- # Go: ConsumePartition("orders", ...)
- _PatternDef(
- regex=re.compile(r"""ConsumePartition\s*\(\s*['"]([^'"]+)['"]"""),
- role="consumer",
- broker="kafka",
- confidence=0.7,
- topic_group=1,
- label="ConsumePartition",
- ),
-]
-
-# --- RabbitMQ ---
-
-_RABBITMQ_PATTERNS: list[_PatternDef] = [
- # Java: @RabbitListener(queues = "jobs")
- _PatternDef(
- regex=re.compile(r"""@RabbitListener\s*\([^)]*queues?\s*=\s*['"]([^'"]+)['"]"""),
- role="consumer",
- broker="rabbitmq",
- confidence=0.8,
- topic_group=1,
- label="@RabbitListener",
- ),
- # Java: rabbitTemplate.convertAndSend("exchange", ...)
- _PatternDef(
- regex=re.compile(r"""rabbitTemplate\.convertAndSend\s*\(\s*['"]([^'"]+)['"]"""),
- role="provider",
- broker="rabbitmq",
- confidence=0.8,
- topic_group=1,
- label="rabbitTemplate.convertAndSend",
- ),
- # Node: channel.consume("queue", ...)
- _PatternDef(
- regex=re.compile(r"""channel\.consume\s*\(\s*['"]([^'"]+)['"]"""),
- role="consumer",
- broker="rabbitmq",
- confidence=0.8,
- topic_group=1,
- label="channel.consume",
- ),
- # Node: channel.publish("exchange", ...)
- _PatternDef(
- regex=re.compile(r"""channel\.publish\s*\(\s*['"]([^'"]+)['"]"""),
- role="provider",
- broker="rabbitmq",
- confidence=0.8,
- topic_group=1,
- label="channel.publish",
- ),
- # Node: channel.sendToQueue("queue", ...)
- _PatternDef(
- regex=re.compile(r"""channel\.sendToQueue\s*\(\s*['"]([^'"]+)['"]"""),
- role="provider",
- broker="rabbitmq",
- confidence=0.8,
- topic_group=1,
- label="channel.sendToQueue",
- ),
- # Python: channel.basic_consume(queue='jobs')
- _PatternDef(
- regex=re.compile(r"""channel\.basic_consume\s*\([^)]*queue\s*=\s*['"]([^'"]+)['"]"""),
- role="consumer",
- broker="rabbitmq",
- confidence=0.7,
- topic_group=1,
- label="basic_consume",
- ),
- # Python: channel.basic_publish(exchange='events')
- _PatternDef(
- regex=re.compile(r"""channel\.basic_publish\s*\([^)]*exchange\s*=\s*['"]([^'"]+)['"]"""),
- role="provider",
- broker="rabbitmq",
- confidence=0.7,
- topic_group=1,
- label="basic_publish",
- ),
-]
-
-# --- NATS ---
-
-_NATS_PATTERNS: list[_PatternDef] = [
- # Go/Python/Node: nc.Subscribe("events") or nc.subscribe("events")
- # Requires a NATS-idiomatic variable name (nc, nats, conn, js, sub, client)
- # to avoid false-positives from RxJS, EventEmitter, custom publishers, etc.
- _PatternDef(
- regex=re.compile(
- r"""(?:nc|nats|conn|js|sub|client)\s*\.\s*(?:Subscribe|subscribe)\s*\(\s*['"]([^'"]+)['"]"""
- ),
- role="consumer",
- broker="nats",
- confidence=0.8,
- topic_group=1,
- label="nc.Subscribe",
- ),
- # Go/Python/Node: nc.Publish("events") or nc.publish("events")
- # Requires a NATS-idiomatic variable name to avoid false-positives.
- _PatternDef(
- regex=re.compile(
- r"""(?:nc|nats|conn|js|sub|client)\s*\.\s*(?:Publish|publish)\s*\(\s*['"]([^'"]+)['"]"""
- ),
- role="provider",
- broker="nats",
- confidence=0.8,
- topic_group=1,
- label="nc.Publish",
- ),
-]
-
-_ALL_PATTERNS = _KAFKA_PATTERNS + _RABBITMQ_PATTERNS + _NATS_PATTERNS
-
-
-# ---------------------------------------------------------------------------
-# Extractor
-# ---------------------------------------------------------------------------
-
-
-class TopicExtractor:
- """Extract message topic/queue contracts from source files."""
-
- @classmethod
- def source_extensions(cls) -> frozenset[str]:
- """Every extension this extractor scans."""
- return _EXTENSIONS
-
- def extract(
- self,
- repo_path: Path,
- repo_alias: str = "",
- exclude: Callable[[str], bool] | None = None,
- files: Sequence[SourceFile] | None = None,
- ) -> list[Contract]:
- from repowise.core.workspace.contracts import Contract
-
- contracts: list[Contract] = []
- seen: set[tuple[str, str, str]] = set() # (file, contract_id, role) dedup
-
- # File discovery is gitignore- and nested-repo-aware (see
- # ``iter_source_files``) so a workspace repo whose root contains nested
- # repos or large ignored trees is not scanned end-to-end.
- for rel_path, _suffix, content in select_files(
- repo_path, _EXTENSIONS, exclude, files
- ):
- for pdef in _ALL_PATTERNS:
- for match in pdef.regex.finditer(content):
- topic_name = match.group(pdef.topic_group).strip()
- if not topic_name:
- continue
-
- contract_id = f"topic::{topic_name.lower()}"
-
- # Deduplicate within the same file
- dedup_key = (rel_path, contract_id, pdef.role)
- if dedup_key in seen:
- continue
- seen.add(dedup_key)
-
- contracts.append(
- Contract(
- repo=repo_alias,
- contract_id=contract_id,
- contract_type="topic",
- role=pdef.role,
- file_path=rel_path,
- symbol_name=f"{pdef.label}('{topic_name}')",
- confidence=pdef.confidence,
- service=None,
- line=line_at(content, match.start()),
- meta={
- "topic": topic_name,
- "broker": pdef.broker,
- },
- )
- )
-
- return contracts
diff --git a/packages/core/src/repowise/core/workspace/matching/__init__.py b/packages/core/src/repowise/core/workspace/matching/__init__.py
new file mode 100644
index 000000000..f682b5ca6
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/matching/__init__.py
@@ -0,0 +1,84 @@
+"""Provider-to-consumer matching across repos.
+
+Every contract type shares one exact pass over normalized ids (with the HTTP and
+gRPC wildcards). A type whose two ends can name one thing differently registers
+a :class:`TypeMatcher` in :data:`MATCHERS`: consumers the exact pass must leave
+alone, a routing check the exact pass must honour, and the extra passes that
+run after it. Adding a transport's rules is one module and one registry entry.
+
+Same-repo same-service pairs, and consumers resolved to a third-party host
+(``meta['external']``), are filtered from every pass.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass
+from typing import TYPE_CHECKING
+
+from . import http, topic
+from .common import MatchState, find_matching_keys, internal, prefer_target_repo
+from .http import annotate_consumer_targets
+
+if TYPE_CHECKING:
+ from repowise.core.workspace.contracts import Contract, ContractLink
+
+
+@dataclass(frozen=True)
+class TypeMatcher:
+ """What one contract type adds to the shared exact pass."""
+
+ deferred: Callable[[Contract], bool] | None = None
+ accepts: Callable[[Contract, Contract], bool] | None = None
+ passes: tuple[Callable[[MatchState], None], ...] = ()
+
+
+MATCHERS: dict[str, TypeMatcher] = {
+ "http": TypeMatcher(
+ deferred=http.is_deferred,
+ passes=(http.base_resolved_pass, http.candidate_pass),
+ ),
+ "topic": TypeMatcher(
+ deferred=topic.is_binding, accepts=topic.accepts, passes=(topic.binding_pass,)
+ ),
+}
+
+_NO_RULES = TypeMatcher()
+
+
+def _exact_pass(state: MatchState) -> None:
+ for consumer in state.consumers:
+ rules = MATCHERS.get(consumer.contract_type, _NO_RULES)
+ if rules.deferred is not None and rules.deferred(consumer):
+ continue
+ keys = find_matching_keys(consumer.contract_id, state.provider_index)
+ providers = [p for k in keys for p in state.provider_index[k]]
+ for provider in prefer_target_repo(providers, consumer):
+ if internal(provider, consumer):
+ continue
+ if rules.accepts is not None and not rules.accepts(provider, consumer):
+ continue
+ state.add(consumer, provider, "exact", min(provider.confidence, consumer.confidence))
+
+
+def match_contracts(contracts: list[Contract]) -> list[ContractLink]:
+ """Match providers to consumers across repos.
+
+ The exact pass runs first for every type, then each registered type's own
+ passes in registry order. Target resolution for HTTP consumers is read from
+ ``meta`` (see :func:`annotate_consumer_targets`).
+ """
+ state = MatchState.build(contracts)
+ _exact_pass(state)
+ for rules in MATCHERS.values():
+ for run in rules.passes:
+ run(state)
+ return state.links
+
+
+__all__ = [
+ "MATCHERS",
+ "TypeMatcher",
+ "annotate_consumer_targets",
+ "match_contracts",
+]
diff --git a/packages/core/src/repowise/core/workspace/matching/common.py b/packages/core/src/repowise/core/workspace/matching/common.py
new file mode 100644
index 000000000..258f8863b
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/matching/common.py
@@ -0,0 +1,154 @@
+"""The state every matching pass shares, and the helpers they all use."""
+
+from __future__ import annotations
+
+from collections import defaultdict
+from dataclasses import dataclass, field
+
+from repowise.core.workspace.contracts import (
+ Contract,
+ ContractLink,
+ normalize_contract_id,
+ same_service,
+)
+
+
+def internal(provider: Contract, consumer: Contract) -> bool:
+ """One program calling itself, which no pass links."""
+ return same_service(provider.repo, provider.service, consumer.repo, consumer.service)
+
+
+@dataclass
+class MatchState:
+ """The provider index, the consumers to place, and the links placed so far.
+
+ Built once per :func:`..match_contracts` call and threaded through every
+ pass, so a later pass sees which consumers an earlier one already linked.
+ """
+
+ provider_index: dict[str, list[Contract]]
+ consumers: list[Contract]
+ links: list[ContractLink] = field(default_factory=list)
+ seen: set[tuple[str, ...]] = field(default_factory=set)
+ matched: set[int] = field(default_factory=set)
+
+ @classmethod
+ def build(cls, contracts: list[Contract]) -> MatchState:
+ """Index providers by normalized id; consumers resolved to a third party are dropped."""
+ provider_index: dict[str, list[Contract]] = defaultdict(list)
+ consumers: list[Contract] = []
+ for c in contracts:
+ if c.role == "provider":
+ provider_index[normalize_contract_id(c.contract_id)].append(c)
+ elif not c.meta.get("external"):
+ consumers.append(c)
+ return cls(provider_index, consumers)
+
+ def unmatched(self, contract_type: str) -> list[Contract]:
+ """Consumers of *contract_type* no pass has linked yet."""
+ return [
+ c
+ for c in self.consumers
+ if c.contract_type == contract_type and id(c) not in self.matched
+ ]
+
+ def add(
+ self,
+ consumer: Contract,
+ provider: Contract,
+ match_type: str,
+ confidence: float,
+ *,
+ via_alias: bool = False,
+ ) -> bool:
+ """Record one link unless an identical one exists; True when it was added.
+
+ A link's ``contract_id`` is the id both ends share. *via_alias* marks a
+ consumer that reached the provider under another name (a queue bound to
+ an exchange): the link then carries the provider's id, and the
+ consumer's own id as ``consumer_contract_id``. One queue bound to two
+ exchanges is two links, so an aliased key names the provider's id too.
+ """
+ dedup_key = (
+ normalize_contract_id(consumer.contract_id),
+ consumer.repo,
+ consumer.file_path,
+ provider.repo,
+ provider.file_path,
+ normalize_contract_id(provider.contract_id) if via_alias else "",
+ )
+ if dedup_key in self.seen:
+ return False
+ self.seen.add(dedup_key)
+ self.links.append(
+ ContractLink(
+ contract_id=provider.contract_id if via_alias else consumer.contract_id,
+ contract_type=consumer.contract_type,
+ match_type=match_type,
+ confidence=confidence,
+ provider_repo=provider.repo,
+ provider_file=provider.file_path,
+ provider_symbol=provider.symbol_name,
+ provider_service=provider.service,
+ consumer_repo=consumer.repo,
+ consumer_file=consumer.file_path,
+ consumer_symbol=consumer.symbol_name,
+ provider_symbol_id=provider.symbol_id,
+ consumer_symbol_id=consumer.symbol_id,
+ consumer_service=consumer.service,
+ consumer_contract_id=consumer.contract_id if via_alias else None,
+ )
+ )
+ self.matched.add(id(consumer))
+ return True
+
+
+def find_matching_keys(consumer_id: str, provider_index: dict[str, list[Contract]]) -> list[str]:
+ """Provider index keys that match *consumer_id*, wildcards included."""
+ normalized = normalize_contract_id(consumer_id)
+
+ if normalized in provider_index:
+ return [normalized]
+
+ # HTTP wildcard: consumer http::*::/path matches any method on that path
+ if normalized.startswith("http::*::"):
+ path_suffix = normalized[len("http::*::") :]
+ return [
+ k for k in provider_index if k.startswith("http::") and k.endswith(f"::{path_suffix}")
+ ]
+
+ # HTTP: check for wildcard providers (http::*::/path from Go HandleFunc)
+ if normalized.startswith("http::"):
+ parts = normalized.split("::", 2)
+ if len(parts) == 3:
+ wildcard_key = f"http::*::{parts[2]}"
+ if wildcard_key in provider_index:
+ return [wildcard_key]
+
+ # gRPC wildcard: grpc::service/* matches grpc::service/Method
+ if normalized.endswith("/*"):
+ prefix = normalized[:-1] # "grpc::service/"
+ return [k for k in provider_index if k.startswith(prefix)]
+
+ return []
+
+
+def prefer_target_repo(providers: list[Contract], consumer: Contract) -> list[Contract]:
+ """Narrow *providers* to the consumer's resolved ``target_repo`` when set.
+
+ Falls back to the full list if the target declares no matching provider, so
+ a stale/typo'd ``service_bases`` entry never silently drops a real link.
+ """
+ target = consumer.meta.get("target_repo")
+ if not target:
+ return providers
+ preferred = [p for p in providers if p.repo == target]
+ return preferred or providers
+
+
+__all__ = [
+ "MatchState",
+ "find_matching_keys",
+ "internal",
+ "prefer_target_repo",
+]
diff --git a/packages/core/src/repowise/core/workspace/matching/http.py b/packages/core/src/repowise/core/workspace/matching/http.py
new file mode 100644
index 000000000..b7c075322
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/matching/http.py
@@ -0,0 +1,281 @@
+"""HTTP matching: consumer target resolution, then the base-resolved and candidate passes.
+
+The exact pass is shared by every contract type. HTTP adds two because its
+consumers routinely name a path without the prefix the provider serves it
+under: a base placeholder that was stripped (``${API_BASE}/users``), or a mount
+or version segment one side carries and the other does not (``/api/v1``).
+"""
+
+from __future__ import annotations
+
+import re
+from collections import defaultdict
+from typing import TYPE_CHECKING
+
+from repowise.core.workspace.contracts import normalize_contract_id
+
+from .common import find_matching_keys, internal, prefer_target_repo
+
+if TYPE_CHECKING:
+ from repowise.core.workspace.contracts import Contract
+
+ from .common import MatchState
+
+# Common API mount / version path prefixes. When exact matching fails, the
+# candidate pass strips these leading segments (plus unresolved base ``{param}``
+# segments) from both provider and consumer paths so a consumer that hits
+# ``/api/v1/users`` can still link to a provider mounted at ``/users`` (or vice
+# versa). Such links are emitted as lower-confidence ``candidate`` matches.
+#
+# Kept deliberately small: only segments that are almost never real resource
+# names. Words like ``internal``/``public``/``gateway`` are excluded because
+# they double as legitimate route segments and would conflate unrelated routes.
+_MOUNT_PREFIX_SEGMENTS = frozenset({"api", "rest"})
+_VERSION_SEGMENT_RE = re.compile(r"^v\d+$")
+
+# Confidence multiplier applied to candidate (non-exact) links.
+CANDIDATE_CONFIDENCE_FACTOR = 0.6
+
+# Request paths ending in these suffixes are static assets, never API contracts.
+# They are excluded from the candidate pass so a ``fetch('/static/app.js')``
+# can't spuriously link to a provider route that shares a suffix. ``.json`` and
+# ``.xml`` are intentionally absent — real APIs serve those.
+_STATIC_ASSET_SUFFIXES = (
+ ".js",
+ ".mjs",
+ ".cjs",
+ ".css",
+ ".map",
+ ".html",
+ ".htm",
+ ".png",
+ ".jpg",
+ ".jpeg",
+ ".gif",
+ ".svg",
+ ".ico",
+ ".webp",
+ ".avif",
+ ".woff",
+ ".woff2",
+ ".ttf",
+ ".eot",
+ ".otf",
+ ".pdf",
+ ".txt",
+ ".wasm",
+)
+
+# Hosts that name the local machine, not a specific service. A consumer URL
+# pointing here carries no service identity, so it is resolved by path uniqueness
+# rather than excluded as third-party.
+_LOCALHOST_HOSTS = frozenset({"localhost", "127.0.0.1", "0.0.0.0", "::1"})
+
+# Internal-DNS suffixes whose leading label names a service (k8s / mesh / LAN).
+# Only for these do we map ``<label>.<suffix>`` to a repo alias; a public host
+# like ``backend.stripe.com`` must never be mistaken for the ``backend`` repo.
+_INTERNAL_HOST_SUFFIXES = (".local", ".internal", ".svc.cluster.local")
+
+
+# ---------------------------------------------------------------------------
+# Consumer target resolution
+# ---------------------------------------------------------------------------
+
+
+def _resolve_consumer_target(
+ consumer: Contract,
+ repo_aliases: set[str],
+ service_bases: dict[str, str],
+) -> tuple[str | None, bool]:
+ """Resolve a consumer's target repo and whether it is a third-party call.
+
+ Returns ``(target_repo, is_external)``:
+
+ - ``service_bases`` (host or ``${BASE}`` token) wins first;
+ - a host equal to a workspace repo alias, or an internal-DNS host whose
+ leading label is a repo alias (``backend.svc.cluster.local``), is internal;
+ - localhost and bare unknown hostnames (e.g. a docker-compose service we
+ can't map) return ``(None, False)`` so path matching still applies;
+ - any other *public* dotted host is third-party (``is_external=True``).
+ """
+ meta = consumer.meta
+ host = meta.get("host")
+ if host:
+ if host in service_bases:
+ return service_bases[host], False
+ if host in repo_aliases:
+ return host, False
+ if host in _LOCALHOST_HOSTS:
+ return None, False
+ if host.endswith(_INTERNAL_HOST_SUFFIXES):
+ label = host.split(".")[0]
+ return (label, False) if label in repo_aliases else (None, False)
+ if "." not in host:
+ return None, False # bare unknown hostname, not necessarily third-party
+ return None, True # public dotted host, unmapped → third-party
+ token = meta.get("base_token")
+ if token and token.lower() in service_bases:
+ return service_bases[token.lower()], False
+ return None, False
+
+
+def annotate_consumer_targets(
+ contracts: list[Contract],
+ service_bases: dict[str, str] | None = None,
+) -> None:
+ """Stamp each HTTP consumer's ``meta`` with its resolved target / external bit.
+
+ Mutates the contracts in place so both :func:`..match_contracts` and the
+ diagnostics builder read one resolution. ``service_bases`` maps a base token
+ or host (case-insensitive) to a repo alias.
+
+ The two keys this owns are cleared before being recomputed, so the result is
+ a function of *contracts* and *service_bases* alone and not of whatever a
+ previous run stamped. That matters now that incremental extraction carries
+ contracts forward: a consumer resolved to ``gamma`` keeps that ``meta`` when
+ it is reused, and if ``gamma`` has since left the workspace the resolution
+ no longer fires — so without the clear, the stale target survives and the
+ contract claims to call a repo that is not in the workspace any more.
+ """
+ repo_aliases = {c.repo for c in contracts}
+ sb = {k.lower(): v for k, v in (service_bases or {}).items()}
+ for c in contracts:
+ if c.role != "consumer" or c.contract_type != "http":
+ continue
+ c.meta.pop("target_repo", None)
+ c.meta.pop("external", None)
+ target, external = _resolve_consumer_target(c, repo_aliases, sb)
+ if external:
+ c.meta["external"] = True
+ elif target:
+ c.meta["target_repo"] = target
+
+
+# ---------------------------------------------------------------------------
+# Passes
+# ---------------------------------------------------------------------------
+
+
+def _split_http_id(normalized_id: str) -> tuple[str, str] | None:
+ """Return ``(method, path)`` for a normalized ``http::`` id, else ``None``."""
+ parts = normalized_id.split("::", 2)
+ if len(parts) != 3 or parts[0] != "http":
+ return None
+ return parts[1], parts[2]
+
+
+def _candidate_http_path(path: str) -> str:
+ """Reduce an HTTP path to its mount-agnostic core for candidate matching.
+
+ Strips leading unresolved base ``{param}`` segments and known mount/version
+ prefixes so routes that differ only by an API mount or version prefix
+ collapse to the same key:
+
+ - ``/api/v1/users`` → ``/users``
+ - ``/{param}/resource`` → ``/resource``
+ - ``/v1/resource`` → ``/resource``
+ """
+ segments = [s for s in path.split("/") if s]
+ while segments and (
+ segments[0] == "{param}"
+ or segments[0] in _MOUNT_PREFIX_SEGMENTS
+ or _VERSION_SEGMENT_RE.match(segments[0])
+ ):
+ segments.pop(0)
+ return "/" + "/".join(segments)
+
+
+def _is_static_asset_path(path: str) -> bool:
+ """True when *path*'s final segment looks like a static asset file."""
+ last = path.rsplit("/", 1)[-1].split("?")[0].lower()
+ return last.endswith(_STATIC_ASSET_SUFFIXES)
+
+
+def _methods_compatible(consumer_method: str, provider_method: str) -> bool:
+ """HTTP methods match if equal or either side is the ``*`` wildcard."""
+ return consumer_method == provider_method or consumer_method == "*" or provider_method == "*"
+
+
+def is_deferred(consumer: Contract) -> bool:
+ """A base-stripped consumer skips the exact pass; :func:`base_resolved_pass` owns it."""
+ return bool(consumer.meta.get("base_stripped"))
+
+
+def base_resolved_pass(state: MatchState) -> None:
+ """Match consumers whose unresolved base was stripped, on their host-relative path.
+
+ The link is ``exact`` when the target service is unambiguous (one matching
+ service, or a configured ``service_bases`` target that narrows to a
+ provider) and ``candidate`` otherwise.
+ """
+ for consumer in state.unmatched("http"):
+ if not is_deferred(consumer):
+ continue
+ matching_keys = find_matching_keys(consumer.contract_id, state.provider_index)
+ providers = [
+ p
+ for k in matching_keys
+ for p in state.provider_index[k]
+ if not internal(p, consumer)
+ ]
+ if not providers:
+ continue
+ # A config target only resolves the link when it actually narrows to a
+ # provider; a stale/typo'd target falls back to all providers and must
+ # not be treated as resolved (else an ambiguous link emits as exact).
+ target = consumer.meta.get("target_repo")
+ narrowed = [p for p in providers if p.repo == target] if target else []
+ if narrowed:
+ providers = narrowed
+ resolved = bool(narrowed) or len({(p.repo, p.service) for p in providers}) == 1
+ match_type = "exact" if resolved else "candidate"
+ for provider in providers:
+ confidence = min(provider.confidence, consumer.confidence)
+ if not resolved:
+ confidence = round(confidence * CANDIDATE_CONFIDENCE_FACTOR, 3)
+ state.add(consumer, provider, match_type, confidence)
+
+
+def candidate_pass(state: MatchState) -> None:
+ """Retry unmatched consumers after collapsing mount/version/base prefixes on both sides."""
+ candidate_index: dict[str, list[Contract]] = defaultdict(list)
+ for key, providers in state.provider_index.items():
+ split = _split_http_id(key)
+ if split is None:
+ continue
+ core = _candidate_http_path(split[1])
+ if core in ("", "/"):
+ continue # nothing concrete left to match on
+ candidate_index[core].extend(providers)
+
+ for consumer in state.unmatched("http"):
+ split = _split_http_id(normalize_contract_id(consumer.contract_id))
+ if split is None:
+ continue
+ method, path = split
+ if _is_static_asset_path(path):
+ continue
+ core = _candidate_http_path(path)
+ if core in ("", "/"):
+ continue
+
+ for provider in prefer_target_repo(candidate_index.get(core, []), consumer):
+ if internal(provider, consumer):
+ continue
+ psplit = _split_http_id(normalize_contract_id(provider.contract_id))
+ if psplit is None or not _methods_compatible(method, psplit[0]):
+ continue
+ confidence = round(
+ min(provider.confidence, consumer.confidence) * CANDIDATE_CONFIDENCE_FACTOR,
+ 3,
+ )
+ state.add(consumer, provider, "candidate", confidence)
+
+
+__all__ = [
+ "CANDIDATE_CONFIDENCE_FACTOR",
+ "annotate_consumer_targets",
+ "base_resolved_pass",
+ "candidate_pass",
+ "is_deferred",
+]
diff --git a/packages/core/src/repowise/core/workspace/matching/topic.py b/packages/core/src/repowise/core/workspace/matching/topic.py
new file mode 100644
index 000000000..f4e2bbeb6
--- /dev/null
+++ b/packages/core/src/repowise/core/workspace/matching/topic.py
@@ -0,0 +1,119 @@
+"""Topic matching: routing keys, and queues reached through an exchange binding.
+
+A message broker's producer and consumer often name different things. A
+RabbitMQ producer publishes to an *exchange* with a routing key; a consumer
+reads a *queue*; a binding (``bindQueue(queue, exchange, pattern)``) is what
+connects the two, and it can live in either service or a third. Extraction
+records a binding as a consumer of the exchange carrying ``queue`` and
+``routing_key``. A binding is wiring, not a call: the exact pass leaves it
+alone, and the pass here links each consumer of the bound queue to the
+exchange's publishers, or the binding site itself when no consumer of the
+queue was found. The routing check keeps a binding from linking to a publisher
+whose key it would never receive.
+"""
+
+from __future__ import annotations
+
+from collections import defaultdict
+from functools import cache
+from typing import TYPE_CHECKING
+
+from repowise.core.workspace.contracts import (
+ TOPIC_KIND_BINDING,
+ TOPIC_KIND_QUEUE,
+ TOPIC_ROUTING_KEY_UNRESOLVED,
+ normalize_contract_id,
+)
+
+from .common import internal
+
+if TYPE_CHECKING:
+ from repowise.core.workspace.contracts import Contract
+
+ from .common import MatchState
+
+
+@cache
+def _words_match(pattern: tuple[str, ...], key: tuple[str, ...]) -> bool:
+ """AMQP topic matching over dot-separated words: ``*`` is one, ``#`` is zero or more."""
+ if not pattern:
+ return not key
+ head, rest = pattern[0], pattern[1:]
+ if head == "#":
+ return _words_match(rest, key) or (bool(key) and _words_match(pattern, key[1:]))
+ if not key:
+ return False
+ return head in ("*", key[0]) and _words_match(rest, key[1:])
+
+
+def routing_matches(pattern: str, key: str) -> bool:
+ """True when a message published with *key* reaches a binding on *pattern*.
+
+ Topic-exchange semantics, which reduce to equality for a pattern with no
+ wildcard (a direct exchange). An empty side is a fanout binding or a
+ publish whose key nothing inspects, so it matches everything.
+ """
+ if not pattern or not key:
+ return True
+ return _words_match(tuple(pattern.split(".")), tuple(key.split(".")))
+
+
+def is_binding(consumer: Contract) -> bool:
+ """A binding row is linked by :func:`binding_pass`, never by the exact pass."""
+ return consumer.meta.get("kind") == TOPIC_KIND_BINDING
+
+
+def accepts(provider: Contract, consumer: Contract) -> bool:
+ """Whether *consumer* receives what *provider* publishes, on routing grounds.
+
+ A key the source did not settle routes nowhere: reading it as match-all
+ would link every binding on the exchange.
+ """
+ if provider.meta.get(TOPIC_ROUTING_KEY_UNRESOLVED) or consumer.meta.get(
+ TOPIC_ROUTING_KEY_UNRESOLVED
+ ):
+ return False
+ return routing_matches(
+ consumer.meta.get("routing_key", ""), provider.meta.get("routing_key", "")
+ )
+
+
+def binding_pass(state: MatchState) -> None:
+ """Link every consumer of a bound queue, else the binding, to the exchange's publishers."""
+ queue_consumers: dict[str, list[Contract]] = defaultdict(list)
+ bindings: list[Contract] = []
+ for c in state.consumers:
+ if c.contract_type != "topic":
+ continue
+ if is_binding(c):
+ bindings.append(c)
+ elif c.meta.get("kind") == TOPIC_KIND_QUEUE:
+ queue_consumers[str(c.meta.get("topic", "")).lower()].append(c)
+
+ for binding in bindings:
+ publishers = [
+ p
+ for p in state.provider_index.get(normalize_contract_id(binding.contract_id), [])
+ if accepts(p, binding)
+ ]
+ readers = queue_consumers.get(str(binding.meta.get("queue", "")).lower())
+ for provider in publishers:
+ if not readers:
+ if not internal(provider, binding):
+ state.add(
+ binding, provider, "exact", min(provider.confidence, binding.confidence)
+ )
+ continue
+ for consumer in readers:
+ if internal(provider, consumer):
+ continue
+ state.add(
+ consumer,
+ provider,
+ "exact",
+ min(provider.confidence, consumer.confidence, binding.confidence),
+ via_alias=True,
+ )
+
+
+__all__ = ["accepts", "binding_pass", "is_binding", "routing_matches"]
diff --git a/packages/core/src/repowise/core/workspace/test_impact.py b/packages/core/src/repowise/core/workspace/test_impact.py
index 3a5da034a..eff5f9963 100644
--- a/packages/core/src/repowise/core/workspace/test_impact.py
+++ b/packages/core/src/repowise/core/workspace/test_impact.py
@@ -27,7 +27,7 @@ from repowise.core.analysis.test_reachability import (
)
from repowise.core.persistence.crud.analysis.coverage_map import tests_covering_files
from repowise.core.workspace.config import WorkspaceConfig
-from repowise.core.workspace.contracts import ContractLink, load_contract_store
+from repowise.core.workspace.contracts import ContractLink, load_contract_store, same_service
from repowise.core.workspace.repo_index import WorkspaceIndex, open_workspace_index
if TYPE_CHECKING:
@@ -118,13 +118,9 @@ def _norm(path: str) -> str:
def _internal_call(link: ContractLink) -> bool:
- """Mirror of ``contracts._same_repo_same_service``, which takes Contracts.
-
- A provider and consumer in the same repo and the same service boundary are
- one program calling itself, not a cross-repo contract.
- """
- return link.provider_repo == link.consumer_repo and (
- link.provider_service == link.consumer_service
+ """A link whose two ends are one program calling itself, as the matcher defines it."""
+ return same_service(
+ link.provider_repo, link.provider_service, link.consumer_repo, link.consumer_service
)
diff --git a/packages/server/src/repowise/server/routers/workspace.py b/packages/server/src/repowise/server/routers/workspace.py
index 62118842a..9420f702c 100644
--- a/packages/server/src/repowise/server/routers/workspace.py
+++ b/packages/server/src/repowise/server/routers/workspace.py
@@ -287,6 +287,7 @@ def _contract_link(lk: dict) -> WorkspaceContractLinkEntry:
consumer_service=lk.get("consumer_service"),
provider_symbol_id=lk.get("provider_symbol_id"),
consumer_symbol_id=lk.get("consumer_symbol_id"),
+ consumer_contract_id=lk.get("consumer_contract_id"),
)
diff --git a/packages/server/src/repowise/server/schemas/workspace.py b/packages/server/src/repowise/server/schemas/workspace.py
index 584e593cd..cb1ff07bf 100644
--- a/packages/server/src/repowise/server/schemas/workspace.py
+++ b/packages/server/src/repowise/server/schemas/workspace.py
@@ -129,6 +129,9 @@ class WorkspaceContractLinkEntry(BaseModel):
#: than a display label. None when that side never bound to one.
provider_symbol_id: str | None = None
consumer_symbol_id: str | None = None
+ #: The consumer's own contract id when it reached the provider under another
+ #: name (a queue bound to the exchange ``contract_id`` names); None otherwise.
+ consumer_contract_id: str | None = None
class WorkspaceContractsResponse(BaseModel):
diff --git a/packages/types/src/generated/http.ts b/packages/types/src/generated/http.ts
index 29920cd5e..ebb397611 100644
--- a/packages/types/src/generated/http.ts
+++ b/packages/types/src/generated/http.ts
@@ -2987,6 +2987,7 @@ export interface WorkspaceContractLinkEntry {
consumer_service?: string | null;
provider_symbol_id?: string | null;
consumer_symbol_id?: string | null;
+ consumer_contract_id?: string | null;
}
export interface WorkspaceContractSummary {
diff --git a/packages/types/src/workspace.ts b/packages/types/src/workspace.ts
index aa52057ea..36b966dc1 100644
--- a/packages/types/src/workspace.ts
+++ b/packages/types/src/workspace.ts
@@ -57,6 +57,11 @@ export interface WorkspaceContractLinkEntry {
*/
provider_symbol_id: string | null;
consumer_symbol_id: string | null;
+ /**
+ * The consumer's own contract id when it reached the provider under another
+ * name (a queue bound to the exchange `contract_id` names); null otherwise.
+ */
+ consumer_contract_id?: string | null;
}
export interface WorkspaceCoChangeEntry {
diff --git a/packages/ui/__tests__/workspace/contract-drawer.test.tsx b/packages/ui/__tests__/workspace/contract-drawer.test.tsx
index c0bffc366..c4f0000eb 100644
--- a/packages/ui/__tests__/workspace/contract-drawer.test.tsx
+++ b/packages/ui/__tests__/workspace/contract-drawer.test.tsx
@@ -185,4 +185,17 @@ describe("linksForContract", () => {
const mine = link();
expect(linksForContract(c, [mine])).toEqual([mine]);
});
+
+ it("finds a consumer that reached its provider under another id", () => {
+ const c = contract({
+ role: "consumer",
+ contract_id: "topic::audit",
+ repo: "web-app",
+ file_path: "src/api/users-client.ts",
+ });
+ const bridged = link({ contract_id: "topic::orders", consumer_contract_id: "topic::audit" });
+ const provider = contract({ contract_id: "topic::orders" });
+ expect(linksForContract(c, [bridged])).toEqual([bridged]);
+ expect(linksForContract(provider, [bridged])).toEqual([bridged]);
+ });
});
diff --git a/packages/ui/src/workspace/contract-facts.ts b/packages/ui/src/workspace/contract-facts.ts
index e3f5e2703..a9c4e16ac 100644
--- a/packages/ui/src/workspace/contract-facts.ts
+++ b/packages/ui/src/workspace/contract-facts.ts
@@ -218,6 +218,7 @@ export function schemaFieldConstraints(field: SchemaField): string {
export function linksForContract<
T extends {
contract_id: string;
+ consumer_contract_id?: string | null;
provider_repo: string;
provider_file: string;
consumer_repo: string;
@@ -225,11 +226,15 @@ export function linksForContract<
},
>(contract: ContractEntry, links: T[]): T[] {
const isProvider = contract.role === "provider";
- return links.filter(
- (l) =>
- l.contract_id === contract.contract_id &&
- (isProvider
- ? l.provider_repo === contract.repo && l.provider_file === contract.file_path
- : l.consumer_repo === contract.repo && l.consumer_file === contract.file_path),
+ // A consumer that reached its provider under another name (a queue bound to
+ // an exchange) carries its own id on the link, not the provider's.
+ return links.filter((l) =>
+ isProvider
+ ? l.contract_id === contract.contract_id &&
+ l.provider_repo === contract.repo &&
+ l.provider_file === contract.file_path
+ : (l.consumer_contract_id ?? l.contract_id) === contract.contract_id &&
+ l.consumer_repo === contract.repo &&
+ l.consumer_file === contract.file_path,
);
}
diff --git a/packages/ui/src/workspace/contract-links-table.tsx b/packages/ui/src/workspace/contract-links-table.tsx
index e0efb7f93..3df9fe1a7 100644
--- a/packages/ui/src/workspace/contract-links-table.tsx
+++ b/packages/ui/src/workspace/contract-links-table.tsx
@@ -102,7 +102,7 @@ export function ContractLinksTable({ links }: ContractLinksTableProps) {
<VirtualizedTable<WorkspaceContractLinkEntry>
rows={links}
rowKey={(link) =>
- `${link.contract_id}|${link.provider_repo}|${link.provider_file}|${link.consumer_repo}|${link.consumer_file}`
+ `${link.contract_id}|${link.consumer_contract_id ?? ""}|${link.provider_repo}|${link.provider_file}|${link.consumer_repo}|${link.consumer_file}`
}
header={header}
renderRow={renderRow}
diff --git a/tests/unit/workspace/test_client_calls.py b/tests/unit/workspace/test_client_calls.py
index 797e02ffb..36a06b41f 100644
--- a/tests/unit/workspace/test_client_calls.py
+++ b/tests/unit/workspace/test_client_calls.py
@@ -11,6 +11,11 @@ import pytest
from repowise.core.workspace.extractors.base import ScanContext
from repowise.core.workspace.extractors.http.client_calls import (
+ ClientCallMatch,
+ consumer_contracts,
+ method_from_argument,
+)
+from repowise.core.workspace.extractors.strings import (
CSHARP_SYNTAX,
GO_SYNTAX,
JAVA_SYNTAX,
@@ -20,11 +25,8 @@ from repowise.core.workspace.extractors.http.client_calls import (
PYTHON_SYNTAX,
RUBY_SYNTAX,
RUST_SYNTAX,
- ClientCallMatch,
call_arguments,
- consumer_contracts,
- method_from_argument,
- resolve_url,
+ resolve_string,
split_first_arg,
string_constants,
)
@@ -58,7 +60,7 @@ class TestLiterals:
],
)
def test_literal_bodies(self, syntax, expr, expected):
- assert resolve_url(expr, syntax) == expected
+ assert resolve_string(expr, syntax) == expected
@pytest.mark.parametrize(
("syntax", "expr"),
@@ -73,50 +75,50 @@ class TestLiterals:
],
)
def test_unsettled_expressions_are_refused(self, syntax, expr):
- assert resolve_url(expr, syntax) is None
+ assert resolve_string(expr, syntax) is None
class TestFormatCalls:
def test_rust_format(self):
expr = 'format!("{}/systems/{}", base, id)'
- assert resolve_url(expr, RUST_SYNTAX) == "${x}/systems/${x}"
+ assert resolve_string(expr, RUST_SYNTAX) == "${x}/systems/${x}"
def test_go_sprintf(self):
expr = 'fmt.Sprintf("%s/%s/@v/%s.zip", proxy, mod, ver)'
- assert resolve_url(expr, GO_SYNTAX) == "${x}/${x}/@v/${x}.zip"
+ assert resolve_string(expr, GO_SYNTAX) == "${x}/${x}/@v/${x}.zip"
def test_java_string_format(self):
expr = 'String.format("https://api.example.com/v3/price?ids=%s", ids)'
- assert resolve_url(expr, JAVA_SYNTAX) == "https://api.example.com/v3/price?ids=${x}"
+ assert resolve_string(expr, JAVA_SYNTAX) == "https://api.example.com/v3/price?ids=${x}"
def test_a_format_call_whose_template_is_not_a_literal_is_refused(self):
- assert resolve_url("fmt.Sprintf(tmpl, a)", GO_SYNTAX) is None
+ assert resolve_string("fmt.Sprintf(tmpl, a)", GO_SYNTAX) is None
class TestUnwrapAndConcat:
def test_java_uri_create_is_unwrapped(self):
- assert resolve_url('URI.create(base + "/echo")', JAVA_SYNTAX) == "${base}/echo"
+ assert resolve_string('URI.create(base + "/echo")', JAVA_SYNTAX) == "${base}/echo"
def test_go_concat_with_a_trailing_name(self):
- assert resolve_url('"/users/" + id', GO_SYNTAX) == "/users/${id}"
+ assert resolve_string('"/users/" + id', GO_SYNTAX) == "/users/${id}"
def test_php_concat(self):
- assert resolve_url("$this->base . '/contacts/' . $id", PHP_SYNTAX) == (
+ assert resolve_string("$this->base . '/contacts/' . $id", PHP_SYNTAX) == (
"${$this->base}/contacts/${$id}"
)
def test_concat_with_a_call_is_refused(self):
- assert resolve_url('base + "/" + fmt(x)', GO_SYNTAX) is None
+ assert resolve_string('base + "/" + fmt(x)', GO_SYNTAX) is None
def test_a_concat_of_two_names_carries_no_path(self):
- assert resolve_url("c.InfoAPIURL + id.Name", GO_SYNTAX) == "${c.InfoAPIURL}${id.Name}"
+ assert resolve_string("c.InfoAPIURL + id.Name", GO_SYNTAX) == "${c.InfoAPIURL}${id.Name}"
class TestConstantFolding:
def test_python_single_assignment_folds(self):
content = 'BASE = "http://svc"\nurl = f"{BASE}/users"\nrequests.get(url)\n'
constants = string_constants(content, PYTHON_SYNTAX)
- assert resolve_url("url", PYTHON_SYNTAX, constants) == "http://svc/users"
+ assert resolve_string("url", PYTHON_SYNTAX, constants) == "http://svc/users"
def test_a_reassigned_name_is_retired(self):
content = 'url = "/a"\nurl = "/b"\n'
@@ -126,18 +128,18 @@ class TestConstantFolding:
content = 'u := fmt.Sprintf("%s/x", base)\nresp, err := http.Get(u)\n'
constants = string_constants(content, GO_SYNTAX)
# A format call bound once folds at the call site like a literal.
- assert resolve_url("u", GO_SYNTAX, constants) == "${x}/x"
- assert resolve_url("other", GO_SYNTAX, constants) is None
+ assert resolve_string("u", GO_SYNTAX, constants) == "${x}/x"
+ assert resolve_string("other", GO_SYNTAX, constants) is None
content = 'const liveReloadSourceURL = "https://example.com/livereload.js"\n'
constants = string_constants(content, GO_SYNTAX)
- assert resolve_url("liveReloadSourceURL", GO_SYNTAX, constants) == (
+ assert resolve_string("liveReloadSourceURL", GO_SYNTAX, constants) == (
"https://example.com/livereload.js"
)
def test_ruby_frozen_constant(self):
content = "CLEARBIT_ENDPOINT = 'https://person.clearbit.com/v2/find'.freeze\n"
constants = string_constants(content, RUBY_SYNTAX)
- assert resolve_url("CLEARBIT_ENDPOINT", RUBY_SYNTAX, constants) == (
+ assert resolve_string("CLEARBIT_ENDPOINT", RUBY_SYNTAX, constants) == (
"https://person.clearbit.com/v2/find"
)
@@ -145,26 +147,26 @@ class TestConstantFolding:
content = "BASE_URL = 'https://api.dyte.io/v2'\n"
constants = string_constants(content, RUBY_SYNTAX)
expr = '"#{BASE_URL}/accounts/#{@account_id}/kit"'
- assert resolve_url(expr, RUBY_SYNTAX, constants) == (
+ assert resolve_string(expr, RUBY_SYNTAX, constants) == (
"https://api.dyte.io/v2/accounts/${@account_id}/kit"
)
def test_java_field_initialiser(self):
content = 'private static final String BASE = "https://svc/api";\n'
constants = string_constants(content, JAVA_SYNTAX)
- assert resolve_url('BASE + "/users"', JAVA_SYNTAX, constants) == "https://svc/api/users"
+ assert resolve_string('BASE + "/users"', JAVA_SYNTAX, constants) == "https://svc/api/users"
def test_kotlin_const_val(self):
content = 'const val TEST_SERVER = "http://127.0.0.1:8080"\n'
constants = string_constants(content, KOTLIN_SYNTAX)
- assert resolve_url('"$TEST_SERVER/echo"', KOTLIN_SYNTAX, constants) == (
+ assert resolve_string('"$TEST_SERVER/echo"', KOTLIN_SYNTAX, constants) == (
"http://127.0.0.1:8080/echo"
)
def test_php_variable_and_const(self):
content = "$base = 'https://svc';\nconst PATH = '/x';\n"
constants = string_constants(content, PHP_SYNTAX)
- assert resolve_url("$base . PATH", PHP_SYNTAX, constants) == "https://svc/x"
+ assert resolve_string("$base . PATH", PHP_SYNTAX, constants) == "https://svc/x"
def test_a_comparison_is_not_an_assignment(self):
assert string_constants('if (a == "x") {}\n', JAVA_SYNTAX) == {}
@@ -198,15 +200,15 @@ class TestConstantFolding:
def test_an_unwrap_or_format_call_must_be_the_whole_expression(self):
constants = {"BASE": '"https://h/root"'}
- assert resolve_url('URI.create(BASE).resolve("/v2")', JAVA_SYNTAX, constants) is None
- assert resolve_url('fmt.Sprintf("%s/api", b) + "/v2"', GO_SYNTAX) is None
- assert resolve_url("URI.parse(BASE) + '/x'", RUBY_SYNTAX, constants) is None
+ assert resolve_string('URI.create(BASE).resolve("/v2")', JAVA_SYNTAX, constants) is None
+ assert resolve_string('fmt.Sprintf("%s/api", b) + "/v2"', GO_SYNTAX) is None
+ assert resolve_string("URI.parse(BASE) + '/x'", RUBY_SYNTAX, constants) is None
def test_a_doubled_percent_is_a_literal(self):
- assert resolve_url('fmt.Sprintf("/a/100%%/%s", x)', GO_SYNTAX) == "/a/100%/${x}"
+ assert resolve_string('fmt.Sprintf("/a/100%%/%s", x)', GO_SYNTAX) == "/a/100%/${x}"
def test_csharp_escaped_braces_are_refused(self):
- assert resolve_url('$"/a/{{lit}}/{id}"', CSHARP_SYNTAX) is None
+ assert resolve_string('$"/a/{{lit}}/{id}"', CSHARP_SYNTAX) is None
class TestMethodFromArgument:
@@ -257,7 +259,7 @@ class TestLiteralSpan:
content = "fetch(`https://h/v1/links?${new URLSearchParams({ a: 'x' })}`)"
(row,) = fetch_calls(content)
assert row.url == "`https://h/v1/links?${new URLSearchParams({ a: `"
- assert resolve_url(row.url, JS_SYNTAX) == "https://h/v1/links?${new URLSearchParams({ a: "
+ assert resolve_string(row.url, JS_SYNTAX) == "https://h/v1/links?${new URLSearchParams({ a: "
class TestConsumerContracts:
diff --git a/tests/unit/workspace/test_contracts.py b/tests/unit/workspace/test_contracts.py
index 836485615..8709641f6 100644
--- a/tests/unit/workspace/test_contracts.py
+++ b/tests/unit/workspace/test_contracts.py
@@ -1,4 +1,4 @@
-"""Tests for contract extraction — HTTP, gRPC, topic extractors, matching, persistence."""
+"""Tests for contract extraction — HTTP, gRPC, socket extractors, matching, persistence."""
from __future__ import annotations
@@ -9,14 +9,12 @@ from repowise.core.workspace.contracts import (
Contract,
ContractLink,
ContractStore,
- annotate_consumer_targets,
load_contract_store,
- match_contracts,
normalize_contract_id,
save_contract_store,
)
-from repowise.core.workspace.extractors.grpc_extractor import GrpcExtractor
-from repowise.core.workspace.extractors.http_extractor import (
+from repowise.core.workspace.extractors.grpc import GrpcExtractor
+from repowise.core.workspace.extractors.http import (
HttpExtractor,
normalize_http_path,
)
@@ -25,8 +23,8 @@ from repowise.core.workspace.extractors.service_boundary import (
assign_service,
detect_service_boundaries,
)
-from repowise.core.workspace.extractors.socket_extractor import SocketExtractor
-from repowise.core.workspace.extractors.topic_extractor import TopicExtractor
+from repowise.core.workspace.extractors.socket import SocketExtractor
+from repowise.core.workspace.matching import annotate_consumer_targets, match_contracts
# ---------------------------------------------------------------------------
# normalize_http_path
@@ -664,87 +662,6 @@ class TestGrpcExtractor:
assert consumers == []
-# ---------------------------------------------------------------------------
-# TopicExtractor
-# ---------------------------------------------------------------------------
-
-
-class TestTopicExtractor:
- def _write_file(self, repo: Path, rel: str, content: str) -> None:
- fpath = repo / rel
- fpath.parent.mkdir(parents=True, exist_ok=True)
- fpath.write_text(content, encoding="utf-8")
-
- def test_kafka_listener_consumer(self, tmp_path: Path) -> None:
- self._write_file(tmp_path, "Consumer.java", """
- @KafkaListener(topics = "orders")
- public void listen(String message) {}
- """)
- contracts = TopicExtractor().extract(tmp_path, "worker")
- consumers = [c for c in contracts if c.role == "consumer"]
- assert len(consumers) == 1
- assert consumers[0].contract_id == "topic::orders"
- assert consumers[0].meta["broker"] == "kafka"
-
- def test_kafka_template_provider(self, tmp_path: Path) -> None:
- self._write_file(tmp_path, "Publisher.java", """
- kafkaTemplate.send("orders", payload);
- """)
- contracts = TopicExtractor().extract(tmp_path, "api")
- providers = [c for c in contracts if c.role == "provider"]
- assert len(providers) == 1
- assert providers[0].contract_id == "topic::orders"
-
- def test_kafka_node_provider(self, tmp_path: Path) -> None:
- self._write_file(tmp_path, "producer.js", """
- await producer.send({ topic: 'payments', messages });
- """)
- contracts = TopicExtractor().extract(tmp_path, "svc")
- providers = [c for c in contracts if c.role == "provider"]
- assert len(providers) == 1
- assert providers[0].contract_id == "topic::payments"
-
- def test_rabbitmq_listener_consumer(self, tmp_path: Path) -> None:
- self._write_file(tmp_path, "Worker.java", """
- @RabbitListener(queues = "jobs")
- public void process(Message msg) {}
- """)
- contracts = TopicExtractor().extract(tmp_path, "worker")
- consumers = [c for c in contracts if c.role == "consumer"]
- assert len(consumers) == 1
- assert consumers[0].contract_id == "topic::jobs"
- assert consumers[0].meta["broker"] == "rabbitmq"
-
- def test_nats_subscribe_consumer(self, tmp_path: Path) -> None:
- self._write_file(tmp_path, "listener.go", """
- nc.Subscribe("events.created", handler)
- """)
- contracts = TopicExtractor().extract(tmp_path, "svc")
- consumers = [c for c in contracts if c.role == "consumer"]
- assert len(consumers) == 1
- assert consumers[0].contract_id == "topic::events.created"
- assert consumers[0].meta["broker"] == "nats"
-
- def test_nats_publish_provider(self, tmp_path: Path) -> None:
- self._write_file(tmp_path, "notifier.go", """
- nc.Publish("events.created", data)
- """)
- contracts = TopicExtractor().extract(tmp_path, "notifier")
- providers = [c for c in contracts if c.role == "provider"]
- assert len(providers) == 1
- assert providers[0].contract_id == "topic::events.created"
-
- def test_dedup_same_pattern_in_file(self, tmp_path: Path) -> None:
- self._write_file(tmp_path, "multi.java", """
- kafkaTemplate.send("orders", payload1);
- kafkaTemplate.send("orders", payload2);
- """)
- contracts = TopicExtractor().extract(tmp_path, "svc")
- # Same topic, same role, same file → deduplicated
- providers = [c for c in contracts if c.role == "provider"]
- assert len(providers) == 1
-
-
# ---------------------------------------------------------------------------
# ServiceBoundary
# ---------------------------------------------------------------------------
diff --git a/tests/unit/workspace/test_data_contracts.py b/tests/unit/workspace/test_data_contracts.py
index 2b794f447..7cdf0a82a 100644
--- a/tests/unit/workspace/test_data_contracts.py
+++ b/tests/unit/workspace/test_data_contracts.py
@@ -9,12 +9,13 @@ from __future__ import annotations
from pathlib import Path
-from repowise.core.workspace.contracts import match_contracts, normalize_contract_id
+from repowise.core.workspace.contracts import normalize_contract_id
from repowise.core.workspace.extractors.data import DataExtractor
from repowise.core.workspace.extractors.data.names import (
normalize_table_name,
split_qualified,
)
+from repowise.core.workspace.matching import match_contracts
from repowise.core.workspace.system_graph import edge_kind_for_contract_type
diff --git a/tests/unit/workspace/test_dialect_regressions.py b/tests/unit/workspace/test_dialect_regressions.py
index c3dacb479..f177c1315 100644
--- a/tests/unit/workspace/test_dialect_regressions.py
+++ b/tests/unit/workspace/test_dialect_regressions.py
@@ -100,7 +100,7 @@ class TestExtractorSelfExclusion:
SELF_PATHS = (
"packages/core/src/repowise/core/workspace/extractors/http/fastapi.py",
"packages/core/src/repowise/core/workspace/extractors/http/mounts.py",
- "packages/core/src/repowise/core/workspace/extractors/topic_extractor.py",
+ "packages/core/src/repowise/core/workspace/extractors/topic/rabbitmq.py",
"packages/core/src/repowise/core/workspace/extractors/grpc/python.py",
"packages/core/src/repowise/core/workspace/extractors/data/sql_strings.py",
)
diff --git a/tests/unit/workspace/test_dotnet_workspace.py b/tests/unit/workspace/test_dotnet_workspace.py
index 1bf41f401..74dfb2284 100644
--- a/tests/unit/workspace/test_dotnet_workspace.py
+++ b/tests/unit/workspace/test_dotnet_workspace.py
@@ -22,8 +22,8 @@ from repowise.core.workspace.cross_repo import (
_scan_csproj,
detect_package_dependencies,
)
-from repowise.core.workspace.extractors.grpc_extractor import GrpcExtractor
-from repowise.core.workspace.extractors.http_extractor import HttpExtractor
+from repowise.core.workspace.extractors.grpc import GrpcExtractor
+from repowise.core.workspace.extractors.http import HttpExtractor
# ---------------------------------------------------------------------------
# HTTP — ASP.NET
diff --git a/tests/unit/workspace/test_extractor_traversal.py b/tests/unit/workspace/test_extractor_traversal.py
index cdbfacbe8..9002079fa 100644
--- a/tests/unit/workspace/test_extractor_traversal.py
+++ b/tests/unit/workspace/test_extractor_traversal.py
@@ -14,7 +14,7 @@ from pathlib import Path
from repowise.core.workspace.extractors.base import iter_source_files
from repowise.core.workspace.extractors.service_boundary import detect_service_boundaries
-from repowise.core.workspace.extractors.topic_extractor import TopicExtractor
+from repowise.core.workspace.extractors.topic import TopicExtractor
_PY = frozenset({".py"})
diff --git a/tests/unit/workspace/test_flask_dialect.py b/tests/unit/workspace/test_flask_dialect.py
index 4a1710aef..4134936ea 100644
--- a/tests/unit/workspace/test_flask_dialect.py
+++ b/tests/unit/workspace/test_flask_dialect.py
@@ -23,9 +23,9 @@ from repowise.core.ingestion.framework_routes import flask_blueprints, flask_rou
from repowise.core.ingestion.models import FileInfo
from repowise.core.ingestion.parser import ASTParser
from repowise.core.ingestion.resolvers.context import ResolverContext
+from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.fastapi import FastApiDialect
from repowise.core.workspace.extractors.http.flask import FlaskDialect
-from repowise.core.workspace.extractors.http_extractor import HttpExtractor
APP_PY = """\
from flask import Flask
diff --git a/tests/unit/workspace/test_framework_dialects.py b/tests/unit/workspace/test_framework_dialects.py
index 8d805c223..9239427e8 100644
--- a/tests/unit/workspace/test_framework_dialects.py
+++ b/tests/unit/workspace/test_framework_dialects.py
@@ -31,7 +31,7 @@ from repowise.core.ingestion.models import FileInfo
from repowise.core.ingestion.parser import ASTParser
from repowise.core.ingestion.resolvers.context import ResolverContext
from repowise.core.workspace.contracts import bind_symbol_ids
-from repowise.core.workspace.extractors.http_extractor import HttpExtractor
+from repowise.core.workspace.extractors.http import HttpExtractor
from ._repo_index import make_repo_index
diff --git a/tests/unit/workspace/test_framework_routes.py b/tests/unit/workspace/test_framework_routes.py
index 150e0dc5a..2afabff0d 100644
--- a/tests/unit/workspace/test_framework_routes.py
+++ b/tests/unit/workspace/test_framework_routes.py
@@ -32,8 +32,8 @@ from repowise.core.ingestion.models import FileInfo, Symbol
from repowise.core.ingestion.parser import ASTParser
from repowise.core.ingestion.resolvers.context import ResolverContext
from repowise.core.workspace.contracts import bind_symbol_ids
+from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.mounts import group_prefixes
-from repowise.core.workspace.extractors.http_extractor import HttpExtractor
from ._repo_index import make_repo_index
diff --git a/tests/unit/workspace/test_go_clients.py b/tests/unit/workspace/test_go_clients.py
index e4c47e91e..0921dfe9f 100644
--- a/tests/unit/workspace/test_go_clients.py
+++ b/tests/unit/workspace/test_go_clients.py
@@ -9,11 +9,11 @@ from __future__ import annotations
from pathlib import Path
-from repowise.core.workspace.contracts import match_contracts
from repowise.core.workspace.extractors.base import ScanContext
from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.fastapi import FastApiDialect
from repowise.core.workspace.extractors.http.go_clients import GoClientsDialect, net_http_calls
+from repowise.core.workspace.matching import match_contracts
PACKAGE_VERBS = """package main
diff --git a/tests/unit/workspace/test_index_clients.py b/tests/unit/workspace/test_index_clients.py
index bd65033b6..fb16d11bc 100644
--- a/tests/unit/workspace/test_index_clients.py
+++ b/tests/unit/workspace/test_index_clients.py
@@ -472,8 +472,8 @@ export class HostedApiClient {
decorator paths, so the join runs against whatever normalisation the
product actually applies rather than an id hand-written to match.
"""
- from repowise.core.workspace.contracts import match_contracts
from repowise.core.workspace.extractors.http.dialect import build_provider_contract
+ from repowise.core.workspace.matching import match_contracts
consumers, _u, _c = _run(self.SSE_CLIENT, "src/lib/api/client.ts")
backend_ctx = ScanContext("backend", "app/routers/chat.py", ".py", "", {})
@@ -681,9 +681,9 @@ class TestSupersedeCannotSubtract:
],
)
def test_literal_url_recognition(args: str, expected: str | None):
- from repowise.core.workspace.extractors.http.client_calls import JS_SYNTAX, resolve_url
+ from repowise.core.workspace.extractors.strings import JS_SYNTAX, resolve_string
- assert resolve_url(args, JS_SYNTAX) == expected
+ assert resolve_string(args, JS_SYNTAX) == expected
# ---------------------------------------------------------------------------
@@ -943,8 +943,8 @@ export class HostedApiClient {
def test_it_now_reaches_the_provider_that_exists(self):
"""End to end against a provider built by the product's own normalizer."""
- from repowise.core.workspace.contracts import match_contracts
from repowise.core.workspace.extractors.http.dialect import build_provider_contract
+ from repowise.core.workspace.matching import match_contracts
consumers, _u, _c = _run(self.QUERY_SUFFIX_TS)
backend_ctx = ScanContext("backend", "app/routers/graph.py", ".py", "", {})
diff --git a/tests/unit/workspace/test_java_clients.py b/tests/unit/workspace/test_java_clients.py
index 78e6acecc..83860b904 100644
--- a/tests/unit/workspace/test_java_clients.py
+++ b/tests/unit/workspace/test_java_clients.py
@@ -10,7 +10,6 @@ from __future__ import annotations
from pathlib import Path
-from repowise.core.workspace.contracts import match_contracts
from repowise.core.workspace.extractors.base import ScanContext
from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.fastapi import FastApiDialect
@@ -20,6 +19,7 @@ from repowise.core.workspace.extractors.http.java_clients import (
java_net_http_calls,
resttemplate_calls,
)
+from repowise.core.workspace.matching import match_contracts
FEIGN_CLIENT = """package com.example.client;
diff --git a/tests/unit/workspace/test_kotlin_clients.py b/tests/unit/workspace/test_kotlin_clients.py
index 8abff7550..ed061c403 100644
--- a/tests/unit/workspace/test_kotlin_clients.py
+++ b/tests/unit/workspace/test_kotlin_clients.py
@@ -12,7 +12,6 @@ from __future__ import annotations
from pathlib import Path
-from repowise.core.workspace.contracts import match_contracts
from repowise.core.workspace.extractors.base import ScanContext
from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.fastapi import FastApiDialect
@@ -20,6 +19,7 @@ from repowise.core.workspace.extractors.http.kotlin_clients import (
KotlinClientsDialect,
ktor_calls,
)
+from repowise.core.workspace.matching import match_contracts
LITERALS = """package com.example.sync
diff --git a/tests/unit/workspace/test_micronaut_dialect.py b/tests/unit/workspace/test_micronaut_dialect.py
index 4a2ecbc6d..9a5a072a5 100644
--- a/tests/unit/workspace/test_micronaut_dialect.py
+++ b/tests/unit/workspace/test_micronaut_dialect.py
@@ -28,10 +28,10 @@ from repowise.core.ingestion.framework_routes import (
from repowise.core.ingestion.models import FileInfo
from repowise.core.ingestion.parser import ASTParser
from repowise.core.ingestion.resolvers.context import ResolverContext
+from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.jaxrs import JaxRsDialect
from repowise.core.workspace.extractors.http.micronaut import MicronautDialect
from repowise.core.workspace.extractors.http.spring import SpringDialect
-from repowise.core.workspace.extractors.http_extractor import HttpExtractor
USER_CONTROLLER_JAVA = """\
package com.example;
diff --git a/tests/unit/workspace/test_php_clients.py b/tests/unit/workspace/test_php_clients.py
index 1a6bc8497..2dbee3ec0 100644
--- a/tests/unit/workspace/test_php_clients.py
+++ b/tests/unit/workspace/test_php_clients.py
@@ -10,7 +10,6 @@ from __future__ import annotations
from pathlib import Path
-from repowise.core.workspace.contracts import match_contracts
from repowise.core.workspace.extractors.base import ScanContext
from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.fastapi import FastApiDialect
@@ -20,6 +19,7 @@ from repowise.core.workspace.extractors.http.php_clients import (
guzzle_verb_calls,
laravel_http_calls,
)
+from repowise.core.workspace.matching import match_contracts
GUZZLE_REQUEST = """<?php
diff --git a/tests/unit/workspace/test_remix_dialect.py b/tests/unit/workspace/test_remix_dialect.py
index bfb56e28f..51e387320 100644
--- a/tests/unit/workspace/test_remix_dialect.py
+++ b/tests/unit/workspace/test_remix_dialect.py
@@ -27,10 +27,10 @@ from repowise.core.ingestion.framework_routes import (
from repowise.core.ingestion.models import FileInfo
from repowise.core.ingestion.parser import ASTParser
from repowise.core.ingestion.resolvers.context import ResolverContext
+from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.express import ExpressDialect
from repowise.core.workspace.extractors.http.next_app import NextAppDialect
from repowise.core.workspace.extractors.http.remix import RemixDialect
-from repowise.core.workspace.extractors.http_extractor import HttpExtractor
NOTE_ROUTE_TSX = """\
import type { ActionFunctionArgs, LoaderFunctionArgs } from "@remix-run/node";
diff --git a/tests/unit/workspace/test_ruby_clients.py b/tests/unit/workspace/test_ruby_clients.py
index f4a368a8f..62e5456d6 100644
--- a/tests/unit/workspace/test_ruby_clients.py
+++ b/tests/unit/workspace/test_ruby_clients.py
@@ -13,7 +13,6 @@ from __future__ import annotations
from pathlib import Path
-from repowise.core.workspace.contracts import match_contracts
from repowise.core.workspace.extractors.base import ScanContext
from repowise.core.workspace.extractors.http import HttpExtractor
from repowise.core.workspace.extractors.http.fastapi import FastApiDialect
@@ -23,6 +22,7 @@ from repowise.core.workspace.extractors.http.ruby_clients import (
module_calls,
net_http_calls,
)
+from repowise.core.workspace.matching import match_contracts
HTTPARTY_RB = """\
class Dyte
diff --git a/tests/unit/workspace/test_strings.py b/tests/unit/workspace/test_strings.py
new file mode 100644
index 000000000..73c75f2a1
--- /dev/null
+++ b/tests/unit/workspace/test_strings.py
@@ -0,0 +1,58 @@
+"""The shared string-expression layer: argument selection and syntax lookup."""
+
+from __future__ import annotations
+
+import pytest
+
+from repowise.core.workspace.extractors.strings import (
+ JS_SYNTAX,
+ PHP_SYNTAX,
+ PYTHON_SYNTAX,
+ Arg,
+ resolve_argument,
+ select_argument,
+ syntax_for_suffix,
+)
+
+
+class TestSelectArgument:
+ @pytest.mark.parametrize(
+ ("args", "arg", "expected"),
+ [
+ (["'a'", "'b'"], Arg(pos=1), ["'b'"]),
+ (["queue='jobs'", "cb"], Arg(keys=("queue",), pos=0), ["'jobs'"]),
+ (["{ topic: 'a', messages }"], Arg(keys=("topic",)), ["'a'"]),
+ (['{"topic": "a"}'], Arg(keys=("topic",)), ['"a"']),
+ (["['queue' => 'jobs', 'delay' => 5]"], Arg(keys=("queue",)), ["'jobs'"]),
+ (['topics = {"a", "b"}'], Arg(keys=("topics",)), ['"a"', '"b"']),
+ (["['orders:a', 'orders:b']"], Arg(pos=0), ["'orders:a'", "'orders:b'"]),
+ (["'a'"], Arg(pos=3), []),
+ (["x == 'a'"], Arg(keys=("x",)), []),
+ ],
+ )
+ def test_selection(self, args: list[str], arg: Arg, expected: list[str]) -> None:
+ assert select_argument(args, arg) == expected
+
+
+class TestResolveArgument:
+ def test_a_constant_folds(self) -> None:
+ got = resolve_argument(["QUEUE"], Arg(pos=0), PYTHON_SYNTAX, {"QUEUE": "'jobs'"})
+ assert got == (["jobs"], False)
+
+ def test_a_template_with_a_hole_is_refused(self) -> None:
+ assert resolve_argument(["`jobs.${env}`"], Arg(pos=0), JS_SYNTAX, {}) == ([], True)
+
+ def test_an_absent_argument_is_not_a_refusal(self) -> None:
+ assert resolve_argument(["'a'"], Arg(pos=2), JS_SYNTAX, {}) == ([], False)
+
+ def test_php_map_value(self) -> None:
+ got = resolve_argument(["['queue' => 'jobs']"], Arg(keys=("queue",)), PHP_SYNTAX, {})
+ assert got == (["jobs"], False)
+
+
+ ("suffix", "expected"),
+ [(".ts", JS_SYNTAX), (".py", PYTHON_SYNTAX), (".php", PHP_SYNTAX), (".sql", None)],
+)
+def test_syntax_for_suffix(suffix: str, expected: object) -> None:
+ assert syntax_for_suffix(suffix) is expected
diff --git a/tests/unit/workspace/test_topic_dialects.py b/tests/unit/workspace/test_topic_dialects.py
new file mode 100644
index 000000000..2a8058916
--- /dev/null
+++ b/tests/unit/workspace/test_topic_dialects.py
@@ -0,0 +1,237 @@
+"""Topic dialects: what each broker call reads, and what it refuses."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+
+from repowise.core.workspace.extractors.topic import TopicExtractor
+
+
+def _extract(tmp_path: Path, rel: str, content: str, alias: str = "svc"):
+ path = tmp_path / rel
+ path.parent.mkdir(parents=True, exist_ok=True)
+ path.write_text(content, encoding="utf-8")
+ return TopicExtractor().extract(tmp_path, alias)
+
+
+def _one(contracts, role: str):
+ rows = [c for c in contracts if c.role == role]
+ assert len(rows) == 1, rows
+ return rows[0]
+
+
+class TestKafka:
+ def test_kafka_listener_consumer(self, tmp_path: Path) -> None:
+ c = _one(
+ _extract(tmp_path, "Consumer.java", '@KafkaListener(topics = "orders")\nvoid f() {}'),
+ "consumer",
+ )
+ assert c.contract_id == "topic::orders"
+ assert c.meta == {"topic": "orders", "broker": "kafka", "kind": "topic"}
+ assert c.symbol_name == "@KafkaListener('orders')"
+
+ def test_kafka_listener_reads_every_topic_of_an_array(self, tmp_path: Path) -> None:
+ rows = _extract(
+ tmp_path, "Consumer.java", '@KafkaListener(id = "g", topics = {"orders", "refunds"})'
+ )
+ assert sorted(c.contract_id for c in rows) == ["topic::orders", "topic::refunds"]
+
+ def test_kafka_template_provider(self, tmp_path: Path) -> None:
+ c = _one(_extract(tmp_path, "Publisher.java", 'kafkaTemplate.send("orders", p);'), "provider")
+ assert c.contract_id == "topic::orders"
+
+ def test_kafkajs_producer_reads_the_topic_key(self, tmp_path: Path) -> None:
+ c = _one(
+ _extract(tmp_path, "producer.js", "await producer.send({ topic: 'payments', messages });"),
+ "provider",
+ )
+ assert c.contract_id == "topic::payments"
+
+ def test_kafkajs_subscribe_reads_a_topics_array(self, tmp_path: Path) -> None:
+ rows = _extract(
+ tmp_path, "consumer.ts", "await consumer.subscribe({ topics: ['a.created', 'a.deleted'] });"
+ )
+ assert sorted(c.contract_id for c in rows) == ["topic::a.created", "topic::a.deleted"]
+
+ def test_python_constant_folds(self, tmp_path: Path) -> None:
+ src = 'ORDERS_TOPIC = "orders"\n\nproducer.produce(ORDERS_TOPIC, value=b"x")\n'
+ c = _one(_extract(tmp_path, "pub.py", src), "provider")
+ assert c.contract_id == "topic::orders"
+
+ def test_a_name_assigned_twice_is_refused(self, tmp_path: Path) -> None:
+ src = 'T = "a"\nT = "b"\nproducer.produce(T)\n'
+ assert _extract(tmp_path, "pub.py", src) == []
+
+ def test_kafka_listener_singular_topic_key(self, tmp_path: Path) -> None:
+ c = _one(_extract(tmp_path, "C.java", '@KafkaListener(topic = "orders")'), "consumer")
+ assert c.contract_id == "topic::orders"
+
+ def test_kafkajs_subscribe_singular_topic(self, tmp_path: Path) -> None:
+ src = "await consumer.subscribe({ topic: 'orders' });"
+ c = _one(_extract(tmp_path, "c.ts", src), "consumer")
+ assert c.contract_id == "topic::orders"
+
+ def test_confluent_produce_topic_keyword(self, tmp_path: Path) -> None:
+ src = "producer.produce(topic='orders', value=b'x')\n"
+ c = _one(_extract(tmp_path, "p.py", src), "provider")
+ assert c.contract_id == "topic::orders"
+
+ def test_an_array_keeps_the_elements_it_can_read(self, tmp_path: Path) -> None:
+ rows = _extract(tmp_path, "C.java", '@KafkaListener(topics = {"orders", dynamicTopic})')
+ assert [c.contract_id for c in rows] == ["topic::orders"]
+
+ def test_a_spring_property_placeholder_is_refused(self, tmp_path: Path) -> None:
+ assert _extract(tmp_path, "C.java", '@KafkaListener(topics = "${orders.topic}")') == []
+
+ @pytest.mark.parametrize(
+ ("rel", "src"),
+ [
+ ("pub.go", 'const Orders = "orders"\nc.ConsumePartition(Orders, 0, 0)\n'),
+ ("Pub.java", 'static final String ORDERS = "orders";\nkafkaTemplate.send(ORDERS, p);\n'),
+ ],
+ )
+ def test_constants_fold_in_go_and_java(self, tmp_path: Path, rel: str, src: str) -> None:
+ assert [c.contract_id for c in _extract(tmp_path, rel, src)] == ["topic::orders"]
+
+ def test_java_listener_shape_is_not_read_in_python(self, tmp_path: Path) -> None:
+ assert _extract(tmp_path, "x.py", 'kafkaTemplate.send("orders", p)') == []
+
+
+class TestRabbitMq:
+ def test_send_to_queue_is_a_queue(self, tmp_path: Path) -> None:
+ c = _one(
+ _extract(tmp_path, "pub.ts", "channel.sendToQueue('ticket.sold', Buffer.from(x));"),
+ "provider",
+ )
+ assert c.contract_id == "topic::ticket.sold"
+ assert c.meta["kind"] == "queue"
+
+ def test_publish_carries_exchange_and_routing_key(self, tmp_path: Path) -> None:
+ c = _one(
+ _extract(tmp_path, "pub.ts", "channel.publish('orders', 'order.created', body);"),
+ "provider",
+ )
+ assert c.contract_id == "topic::orders"
+ assert c.meta["kind"] == "exchange"
+ assert c.meta["routing_key"] == "order.created"
+
+ def test_publish_to_the_default_exchange_names_the_queue(self, tmp_path: Path) -> None:
+ c = _one(_extract(tmp_path, "pub.ts", "channel.publish('', 'jobs', body);"), "provider")
+ assert c.contract_id == "topic::jobs"
+ assert c.meta["kind"] == "queue"
+ assert "routing_key" not in c.meta
+
+ def test_two_routing_keys_on_one_exchange_are_two_rows(self, tmp_path: Path) -> None:
+ src = "channel.publish('orders', 'a', b);\nchannel.publish('orders', 'b', b);\n"
+ keys = sorted(c.meta["routing_key"] for c in _extract(tmp_path, "pub.ts", src))
+ assert keys == ["a", "b"]
+
+ def test_repeated_identical_call_is_one_row(self, tmp_path: Path) -> None:
+ src = "channel.consume('jobs', a);\nchannel.consume('jobs', b);\n"
+ assert len(_extract(tmp_path, "w.js", src)) == 1
+
+ def test_bind_queue_is_a_binding_consumer_of_the_exchange(self, tmp_path: Path) -> None:
+ c = _one(
+ _extract(tmp_path, "w.js", "await channel.bindQueue('audit', 'orders', 'order.*');"),
+ "consumer",
+ )
+ assert c.contract_id == "topic::orders"
+ assert c.meta == {
+ "topic": "orders",
+ "broker": "rabbitmq",
+ "kind": "binding",
+ "routing_key": "order.*",
+ "queue": "audit",
+ }
+
+ def test_pika_queue_bind_reads_keywords(self, tmp_path: Path) -> None:
+ src = "channel.queue_bind(exchange='orders', queue='audit', routing_key='order.#')\n"
+ c = _one(_extract(tmp_path, "w.py", src), "consumer")
+ assert (c.contract_id, c.meta["queue"], c.meta["routing_key"]) == (
+ "topic::orders",
+ "audit",
+ "order.#",
+ )
+
+ def test_pika_basic_publish_default_exchange(self, tmp_path: Path) -> None:
+ src = "channel.basic_publish(exchange='', routing_key='jobs', body=b'x')\n"
+ c = _one(_extract(tmp_path, "p.py", src), "provider")
+ assert (c.contract_id, c.meta["kind"]) == ("topic::jobs", "queue")
+
+ def test_bind_queue_with_an_unreadable_queue_binds_nothing(self, tmp_path: Path) -> None:
+ src = "channel.bindQueue(q.queue, 'orders', 'order.*');"
+ assert _extract(tmp_path, "w.js", src) == []
+
+ def test_bind_queue_with_an_unreadable_pattern_is_marked(self, tmp_path: Path) -> None:
+ src = "channel.bindQueue('audit', 'orders', pattern);"
+ c = _one(_extract(tmp_path, "w.js", src), "consumer")
+ assert c.meta["routing_key_unresolved"] is True
+ assert "routing_key" not in c.meta
+
+ def test_spring_convert_and_send_reads_the_routing_key(self, tmp_path: Path) -> None:
+ src = 'rabbitTemplate.convertAndSend("orders", "order.created", event);'
+ c = _one(_extract(tmp_path, "P.java", src), "provider")
+ assert (c.contract_id, c.meta["kind"], c.meta["routing_key"]) == (
+ "topic::orders",
+ "exchange",
+ "order.created",
+ )
+
+ def test_spring_two_argument_send_routes_nowhere(self, tmp_path: Path) -> None:
+ src = 'rabbitTemplate.convertAndSend("jobs", event);'
+ c = _one(_extract(tmp_path, "P.java", src), "provider")
+ assert c.meta["routing_key_unresolved"] is True
+
+ @pytest.mark.parametrize(
+ "call",
+ [
+ "channel.basic_consume(queue='jobs', on_message_callback=cb)",
+ "channel.basic_consume('jobs', cb)",
+ ],
+ )
+ def test_pika_basic_consume(self, tmp_path: Path, call: str) -> None:
+ c = _one(_extract(tmp_path, "w.py", call + "\n"), "consumer")
+ assert (c.contract_id, c.meta["kind"]) == ("topic::jobs", "queue")
+
+ def test_rabbit_listener_consumer(self, tmp_path: Path) -> None:
+ c = _one(_extract(tmp_path, "W.java", '@RabbitListener(queues = "jobs")'), "consumer")
+ assert (c.contract_id, c.meta["kind"]) == ("topic::jobs", "queue")
+
+ @pytest.mark.parametrize(
+ "call",
+ [
+ "channel.consume(`jobs.${env}`, h);",
+ "channel.consume(queueName, h);",
+ "channel.consume('jobs' + suffix, h);",
+ ],
+ )
+ def test_a_name_not_settled_in_the_file_is_refused(self, tmp_path: Path, call: str) -> None:
+ assert _extract(tmp_path, "w.js", call) == []
+
+
+class TestNats:
+ def test_subscribe_and_publish(self, tmp_path: Path) -> None:
+ src = 'nc.Subscribe("events.created", h)\nnc.Publish("events.deleted", d)\n'
+ rows = _extract(tmp_path, "main.go", src)
+ assert {(c.role, c.contract_id) for c in rows} == {
+ ("consumer", "topic::events.created"),
+ ("provider", "topic::events.deleted"),
+ }
+ assert {c.meta["kind"] for c in rows} == {"subject"}
+
+ @pytest.mark.parametrize(
+ ("rel", "src", "role"),
+ [
+ ("sub.py", 'await nc.subscribe("events.created", cb=h)\n', "consumer"),
+ ("Pub.java", 'nc.publish("events.created", data);', "provider"),
+ ("pub.ts", "nc.publish('events.created', sc.encode(x));", "provider"),
+ ],
+ )
+ def test_nats_in_other_languages(self, tmp_path: Path, rel: str, src: str, role: str) -> None:
+ c = _one(_extract(tmp_path, rel, src), role)
+ assert c.contract_id == "topic::events.created"
+
+ def test_an_unrelated_receiver_is_not_nats(self, tmp_path: Path) -> None:
+ assert _extract(tmp_path, "x.ts", "store.subscribe('state', h);") == []
diff --git a/tests/unit/workspace/test_topic_matching.py b/tests/unit/workspace/test_topic_matching.py
new file mode 100644
index 000000000..a7404e827
--- /dev/null
+++ b/tests/unit/workspace/test_topic_matching.py
@@ -0,0 +1,180 @@
+"""Topic matching: routing keys and queues reached through an exchange binding."""
+
+from __future__ import annotations
+
+import pytest
+
+from repowise.core.workspace.contracts import Contract, ContractLink
+from repowise.core.workspace.diagnostics import build_diagnostics
+from repowise.core.workspace.matching import match_contracts
+from repowise.core.workspace.matching.topic import routing_matches
+
+
+def _topic(repo: str, role: str, name: str, file: str, **meta) -> Contract:
+ return Contract(
+ repo=repo,
+ contract_id=f"topic::{name}",
+ contract_type="topic",
+ role=role,
+ file_path=file,
+ symbol_name=f"call('{name}')",
+ confidence=0.8,
+ meta={"topic": name, "broker": "rabbitmq", **meta},
+ )
+
+
+def _publisher(key: str = "order.created") -> Contract:
+ return _topic("orders", "provider", "orders", "pub.ts", kind="exchange", routing_key=key)
+
+
+def _binding(repo: str, queue: str, pattern: str, file: str = "setup.ts") -> Contract:
+ return _topic(repo, "consumer", "orders", file, kind="binding", queue=queue, routing_key=pattern)
+
+
+def _queue_consumer(repo: str, queue: str, file: str = "worker.ts") -> Contract:
+ return _topic(repo, "consumer", queue, file, kind="queue")
+
+
+class TestRoutingMatches:
+ @pytest.mark.parametrize(
+ ("pattern", "key", "expected"),
+ [
+ ("order.created", "order.created", True),
+ ("order.created", "order.deleted", False),
+ ("order.*", "order.created", True),
+ ("order.*", "order.created.v2", False),
+ ("order.#", "order", True),
+ ("order.#", "order.created.v2", True),
+ ("#.v2", "order.created.v2", True),
+ ("*.created", "order.created", True),
+ ("#", "anything.at.all", True),
+ ("", "order.created", True),
+ ("order.*", "", True),
+ ],
+ )
+ def test_amqp_topic_semantics(self, pattern: str, key: str, expected: bool) -> None:
+ assert routing_matches(pattern, key) is expected
+
+
+class TestBindingPass:
+ def test_a_queue_bound_in_the_producer_links_its_consumer(self) -> None:
+ # The producer declares the binding; the consumer only names its queue.
+ contracts = [
+ _publisher(),
+ _binding("orders", "audit", "order.*"),
+ _queue_consumer("audit-worker", "audit"),
+ ]
+ links = match_contracts(contracts)
+ assert len(links) == 1
+ link = links[0]
+ assert (link.provider_repo, link.consumer_repo) == ("orders", "audit-worker")
+ assert link.contract_id == "topic::orders"
+ assert link.consumer_contract_id == "topic::audit"
+ assert link.match_type == "exact"
+
+ def test_a_binding_beside_its_consumer_is_wiring_not_a_second_link(self) -> None:
+ contracts = [
+ _publisher(),
+ _binding("audit-worker", "audit", "order.*", file="worker.ts"),
+ _queue_consumer("audit-worker", "audit", file="worker.ts"),
+ ]
+ links = match_contracts(contracts)
+ assert [(lk.contract_id, lk.consumer_contract_id) for lk in links] == [
+ ("topic::orders", "topic::audit")
+ ]
+
+ def test_a_binding_with_no_consumer_found_links_itself(self) -> None:
+ links = match_contracts([_publisher(), _binding("audit-worker", "audit", "order.*")])
+ assert len(links) == 1
+ assert (links[0].consumer_repo, links[0].consumer_contract_id) == ("audit-worker", None)
+
+ def test_an_unresolved_routing_key_routes_nowhere(self) -> None:
+ publisher = _publisher()
+ publisher.meta["routing_key_unresolved"] = True
+ del publisher.meta["routing_key"]
+ contracts = [publisher, _binding("orders", "audit", "#"), _queue_consumer("w", "audit")]
+ assert match_contracts(contracts) == []
+
+ def test_an_unresolved_binding_pattern_routes_nowhere(self) -> None:
+ binding = _binding("orders", "audit", "")
+ binding.meta["routing_key_unresolved"] = True
+ assert match_contracts([_publisher(), binding, _queue_consumer("w", "audit")]) == []
+
+ def test_a_queue_fed_directly_is_still_bridged_to_its_exchange(self) -> None:
+ contracts = [
+ _topic("jobs-api", "provider", "audit", "direct.ts", kind="queue"),
+ _publisher(),
+ _binding("orders", "audit", "order.*"),
+ _queue_consumer("audit-worker", "audit"),
+ ]
+ links = match_contracts(contracts)
+ assert sorted((lk.provider_repo, lk.contract_id) for lk in links) == [
+ ("jobs-api", "topic::audit"),
+ ("orders", "topic::orders"),
+ ]
+
+ def test_one_queue_bound_to_two_exchanges_published_from_one_file(self) -> None:
+ refunds = _topic("orders", "provider", "refunds", "pub.ts", kind="exchange", routing_key="r")
+ contracts = [
+ _publisher(),
+ refunds,
+ _binding("orders", "audit", "order.*"),
+ _topic("orders", "consumer", "refunds", "setup.ts", kind="binding", queue="audit", routing_key="#"),
+ _queue_consumer("audit-worker", "audit"),
+ ]
+ assert sorted(lk.contract_id for lk in match_contracts(contracts)) == [
+ "topic::orders",
+ "topic::refunds",
+ ]
+
+ def test_a_routing_key_the_binding_never_receives_does_not_link(self) -> None:
+ contracts = [
+ _publisher(key="payment.captured"),
+ _binding("orders", "audit", "order.*"),
+ _queue_consumer("audit-worker", "audit"),
+ ]
+ assert match_contracts(contracts) == []
+
+ def test_a_binding_alone_honours_the_routing_key_too(self) -> None:
+ contracts = [_publisher(key="payment.captured"), _binding("audit-worker", "audit", "order.*")]
+ assert match_contracts(contracts) == []
+
+ def test_a_queue_named_directly_still_links_exactly(self) -> None:
+ contracts = [
+ _topic("pub", "provider", "jobs", "p.ts", kind="queue"),
+ _queue_consumer("worker", "jobs"),
+ ]
+ links = match_contracts(contracts)
+ assert len(links) == 1
+ assert links[0].consumer_contract_id is None
+
+ def test_same_service_is_not_bridged(self) -> None:
+ contracts = [_publisher(), _binding("orders", "audit", "order.*"), _queue_consumer("orders", "audit")]
+ assert match_contracts(contracts) == []
+
+
+class TestDiagnosticsReadTheConsumerId:
+ def test_a_bridged_consumer_is_not_reported_unmatched(self) -> None:
+ contracts = [
+ _publisher(),
+ _binding("orders", "audit", "order.*"),
+ _queue_consumer("audit-worker", "audit"),
+ ]
+ links = match_contracts(contracts)
+ diag = build_diagnostics(contracts, links)
+ unmatched = {(u.repo, u.contract_id) for u in diag.unmatched_consumers}
+ assert ("audit-worker", "topic::audit") not in unmatched
+
+ def test_a_binding_row_is_never_reported_unmatched(self) -> None:
+ # The binding sits in the producer's own service, so nothing links it.
+ contracts = [_publisher(), _binding("orders", "audit", "order.*")]
+ diag = build_diagnostics(contracts, match_contracts(contracts))
+ assert diag.unmatched_consumers == []
+
+ def test_the_link_round_trips_its_consumer_id(self) -> None:
+ link = match_contracts(
+ [_publisher(), _binding("orders", "audit", "#"), _queue_consumer("w", "audit")]
+ )[0]
+ assert ContractLink.from_dict(link.to_dict()).consumer_contract_id == "topic::audit"
+ plain = ContractLink.from_dict({**link.to_dict(), "consumer_contract_id": None})
+ assert "consumer_contract_id" not in plain.to_dict()