Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Python aicode-complexitycode-healthcode-intelligencecode-qualitydead-codedeveloper-toolsdocumentationgit-analyticsmcpopen-sourcepythonrefactoringstatic-analysistechnical-debt
feat(workspace): compose ${VAR:-default} defaults, env_file and .env files feed the service-name map (#4656)
Raghav Chamadiyaf79a72c2d19a
diff --git a/docs/reference/WORKSPACE_CONTRACTS.md b/docs/reference/WORKSPACE_CONTRACTS.mdindex a5423ff82..1df1364c3 100644--- a/docs/reference/WORKSPACE_CONTRACTS.md+++ b/docs/reference/WORKSPACE_CONTRACTS.md@@ -138,12 +138,13 @@ records `target_via` (the source) and `target_evidence` (the file).| Source | Files read | Name | Repo ||---|---|---|---|| `aspire_apphost` | `AppHost.cs`, `Program.cs` holding `DistributedApplication.CreateBuilder` | `AddProject("catalog-api", "../Catalog.API")`, or `AddProject<Projects.X>(...)` through the AppHost's `<ProjectReference>` | The project's directory |-| `compose` | `docker-compose*.yml`, `compose*.yml` | Service name, `container_name`, `hostname`; `environment` keys whose value is an address | `build` context (or its Dockerfile's directory), else an image basename equal to a repo alias |+| `compose` | `docker-compose*.yml`, `compose*.yml` | Service name, `container_name`, `hostname`; `environment` keys whose value is an address. `${VAR-default}` and `${VAR:-default}` read as the default | `build` context (or its Dockerfile's directory), else an image basename equal to a repo alias. A missing or empty context (an uninitialised submodule) counts as no build || `k8s_service` | `*.yaml` under `k8s/`, `deploy/`, `manifests/`, ... | `kind: Service` `metadata.name` | A selector or label value equal to a repo alias, else the manifest's repo when it declares no other Service || `spring_application_name` | `application` / `bootstrap` `.yml` or `.properties` | `spring.application.name` | The module the file sits in || `spring_base_path` | `.yml`, `.yaml`, `.properties` under `resources/`, `config/`, `configs/` or `shared/` | `server.servlet.context-path`, `server.context-path`, `spring.webflux.base-path`: a base path, not a name | A module's own `src/main/resources/application` or `bootstrap` file (not a profile file): that module. A config server's `<app>.yml`: the service named `<app>` || `spring_gateway` | `application` / `bootstrap` in a module's resources, or a config-server file named `*gateway*`, `*zuul*`, `*proxy*`, `*edge*` (`.yml` or `.properties`) holding `routes` | Spring Cloud Gateway `routes[]` (`lb://name`, `http://host`) and Zuul `zuul.routes` | A gateway route (below) || `dockerfile_env` | `Dockerfile`, `Dockerfile.*`, `*.Dockerfile` holding `ENV` | `ENV` keys whose value is an address (`CATALOGUE_HOST=catalogue`) | The host the value names |+| `env_file` | `.env`, `.env.*`, `*.env` (what a compose `env_file:` names) | Keys whose value is an address; no other value is kept | The host the value names || `nginx` | `*.conf`, `*.conf.template`, `*.conf.tmpl` holding `proxy_pass` | Prefix `location` blocks with `proxy_pass` to a host, an `upstream` of one host, or an envsubst `${X_HOST}` in a template | A gateway route (below) || `ocelot` | `ocelot*.json`, `configuration.json` holding `UpstreamPathTemplate` | Ocelot `Routes` (or `ReRoutes`): `ServiceName`, or the one host in `DownstreamHostAndPorts` | A gateway route (below) || `yarp` | `appsettings*.json`, `yarp*.json` holding `ReverseProxy` | YARP `Routes` and the one host their cluster's `Destinations` name | A gateway route (below) |diff --git a/packages/core/src/repowise/core/workspace/service_names/__init__.py b/packages/core/src/repowise/core/workspace/service_names/__init__.pyindex 3b2270f9b..7ed07520b 100644--- a/packages/core/src/repowise/core/workspace/service_names/__init__.py+++ b/packages/core/src/repowise/core/workspace/service_names/__init__.py@@ -40,6 +40,7 @@ from . import (aspire,compose,dockerfile,+ env_file,k8s_service,nginx,ocelot,@@ -57,6 +58,7 @@ SOURCES: tuple[ServiceNameSource, ...] = (aspire.SOURCE,compose.SOURCE,dockerfile.SOURCE,+ env_file.SOURCE,k8s_service.SOURCE,nginx.SOURCE,ocelot.SOURCE,diff --git a/packages/core/src/repowise/core/workspace/service_names/base.py b/packages/core/src/repowise/core/workspace/service_names/base.pyindex 934c0f144..948fc776c 100644--- a/packages/core/src/repowise/core/workspace/service_names/base.py+++ b/packages/core/src/repowise/core/workspace/service_names/base.py@@ -189,6 +189,18 @@ def template_upstream(pattern: str, forwarded: str) -> str | None:return heads[1] if tails[0] == tails[1] else None+_DEFAULT_RE = re.compile(r"\$\{[A-Za-z_][A-Za-z0-9_]*:?-([^${}]*)\}")+++def with_defaults(value: str) -> str:+ """*value* with each ``${VAR-default}`` / ``${VAR:-default}`` read as its default.++ The value compose (and a shell) uses when nothing sets the variable. Any other+ placeholder stays, so :func:`host_of` still refuses it.+ """+ return _DEFAULT_RE.sub(r"\1", value) if "${" in value else value++def host_of(key: str, value: str) -> str | None:"""The host config entry *key* = *value* names, or None when it is not an address.@@ -218,4 +230,5 @@ __all__ = ["join_config_path","literal_prefix","template_upstream",+ "with_defaults",]diff --git a/packages/core/src/repowise/core/workspace/service_names/compose.py b/packages/core/src/repowise/core/workspace/service_names/compose.pyindex 530b2b7e2..19446a510 100644--- a/packages/core/src/repowise/core/workspace/service_names/compose.py+++ b/packages/core/src/repowise/core/workspace/service_names/compose.py@@ -3,10 +3,14 @@``services.catalogue.build.context: ../catalogue`` puts the name ``catalogue``in the repo at that path; a ``dockerfile`` narrows it to the Dockerfile'sdirectory. A service with no build names a repo only through its image's-basename (``org/cart:1.2`` -> ``cart``). ``container_name`` and ``hostname``-are the same container under another name. Every ``environment`` value that-reads as an address (``CATALOGUE_URL: http://catalogue:8080``) makes its key a-reference to that host.+basename (``org/cart:1.2`` -> ``cart``), as does a build context that is+missing or empty on disk (an uninitialised submodule, the service checked out+as its own repo). ``container_name``+and ``hostname`` are the same container under another name. Every+``environment`` value that reads as an address (``CATALOGUE_URL:+http://catalogue:8080``) makes its key a reference to that host. A+``${VAR-default}`` or ``${VAR:-default}`` reads as its default, the value+compose uses when nothing sets the variable."""from __future__ import annotations@@ -16,14 +20,22 @@ from collections.abc import Iterable, Iteratorfrom pathlib import Pathfrom typing import Any-from .base import ENV, Claim, ServiceNameSource, SourceFile, host_of, join_config_path+from .base import (+ ENV,+ Claim,+ ServiceNameSource,+ SourceFile,+ host_of,+ join_config_path,+ with_defaults,+)from .yaml_docs import load_yaml_docs_PLACEHOLDER_RE = re.compile(r"\$\{[^}]*\}")def _image_basename(image: str) -> str | None:- name = _PLACEHOLDER_RE.sub("", image).rsplit("/", 1)[-1]+ name = _PLACEHOLDER_RE.sub("", with_defaults(image)).rsplit("/", 1)[-1]name = name.split("@", 1)[0].split(":", 1)[0].strip()return name or None@@ -37,10 +49,20 @@ def _build_dir(build: Any, base: Path) -> Path | None:return Noneif not isinstance(context, str) or "$" in context or "://" in context or context.startswith("git@"):return None- where = join_config_path(base, context)+ root = join_config_path(base, context)+ if not _has_files(root):+ return Noneif isinstance(dockerfile, str) and "$" not in dockerfile and "/" in dockerfile.replace("\\", "/"):- where = join_config_path(where, dockerfile).parent- return where+ return join_config_path(root, dockerfile).parent+ return root+++def _has_files(where: Path) -> bool:+ # A missing or empty context (an uninitialised git submodule) builds nothing here.+ try:+ return next(where.iterdir(), None) is not None+ except OSError:+ return Falsedef _environment(env: Any) -> Iterator[tuple[str, str]]:@@ -70,7 +92,7 @@ def _service_claims(svc: str, conf: dict[str, Any], base: Path) -> Iterator[Claifor name in sorted(names):yield Claim(name=name, at=at, aliases=aliases)for key, value in _environment(conf.get("environment")):- host = host_of(key, value)+ host = host_of(key, with_defaults(value))if host:yield Claim(name=key.lower(), kind=ENV, ref=host)diff --git a/packages/core/src/repowise/core/workspace/service_names/env_file.py b/packages/core/src/repowise/core/workspace/service_names/env_file.pynew file mode 100644index 000000000..d118aa848--- /dev/null+++ b/packages/core/src/repowise/core/workspace/service_names/env_file.py@@ -0,0 +1,42 @@+"""Env files (``.env``, ``.env.*``, ``*.env``): the addresses a stack's env keys default to.++``CATALOGUE_URL=http://catalogue:8080`` makes ``CATALOGUE_URL`` a reference to+the host ``catalogue``, as a compose ``environment`` entry does; a compose+``env_file:`` points at these files. Only values :func:`.base.host_of` reads as+addresses become claims: every other value (a password, a token) is never kept.+A sample file's ``localhost`` names no workspace repo, so it resolves to nothing.+A key an env file and a compose file (or two env files) point at different+repos is dropped as ambiguous, as for every other source.++Ceiling: an ``env_file:`` with another file name (``api.conf``) is not read;+following each compose reference is the upgrade if estates use such names.+"""++from __future__ import annotations++import re+from collections.abc import Iterable++from .base import ENV, Claim, ServiceNameSource, SourceFile, host_of, with_defaults++_LINE_RE = re.compile(r"^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_.]*)\s*=(.*)$")+_COMMENT_RE = re.compile(r"\s#")+++def read(file: SourceFile) -> Iterable[Claim]:+ for line in file.text.splitlines():+ m = _LINE_RE.match(line)+ if m is None:+ continue+ key, value = m.groups()+ host = host_of(key, with_defaults(_COMMENT_RE.split(value, maxsplit=1)[0]))+ if host:+ yield Claim(name=key.lower(), kind=ENV, ref=host)+++SOURCE = ServiceNameSource(+ name="env_file",+ filenames=(".env", ".env.*", "*.env"),+ marker="=",+ read=read,+)diff --git a/tests/unit/workspace/test_service_names.py b/tests/unit/workspace/test_service_names.pyindex 9fcc19b01..7d8a49039 100644--- a/tests/unit/workspace/test_service_names.py+++ b/tests/unit/workspace/test_service_names.py@@ -148,6 +148,60 @@ def test_a_name_claimed_for_two_repos_is_left_unmapped(tmp_path: Path) -> None:assert m.ambiguous[(HOST, "api")] == ("a", "b")+def test_compose_defaults_and_an_empty_submodule_context_name_the_image(tmp_path: Path) -> None:+ # ReportPortal's stack: each service builds from a submodule that is empty+ # in the deploy checkout, and its image sits behind ${X_IMAGE-default}.+ for r in ("reportportal/service-api", "service-api", "service-jobs", "other"):+ (tmp_path / r).mkdir(parents=True)+ (tmp_path / "other" / "Dockerfile").write_text("FROM scratch\n")+ (tmp_path / "reportportal" / "docker-compose.yml").write_text(+ "services:\n"+ " api:\n"+ " image: ${API_IMAGE-reportportal/service-api:5.15.5}\n"+ " build: ./service-api\n"+ " environment:\n"+ " RP_JOBS_BASEURL: ${JOBS_URL:-http://jobs:8686}\n"+ " RP_AMQP_HOST: ${AMQP_HOST}\n"+ " jobs: {image: '${JOBS_IMAGE:-reportportal/service-jobs}', build: ./service-jobs}\n"+ " analyzer: {image: reportportal/service-auto-analyzer, build: ./service-auto-analyzer}\n"+ " built: {image: org/service-jobs, build: ../other}\n"+ " split: {image: org/split, build: {context: ../other, dockerfile: deploy/Dockerfile}}\n"+ )+ m = build_service_name_map(+ _repos(tmp_path, "reportportal", "service-api", "service-jobs", "other")+ )+ assert _mapped(m) == {+ "api": ("service-api", "compose"),+ "jobs": ("service-jobs", "compose"),+ "built": ("other", "compose"), # a context with files wins over the image+ "split": ("other", "compose"), # the context decides, not the Dockerfile's dir+ }+ assert _mapped(m, ENV) == {"rp_jobs_baseurl": ("service-jobs", "compose")}+++def test_env_files_feed_token_entries_and_keep_only_addresses(tmp_path: Path) -> None:+ for r in ("web", "catalogue", "cart"):+ (tmp_path / r).mkdir()+ (tmp_path / "web" / ".env").write_text(+ "# local stack\n"+ "CATALOGUE_URL=http://catalogue:8080\n"+ "export\tCART_HOST='cart'\t# the cart service\r\n"+ "DB_PASSWORD=catalogue\n"+ "API_TOKEN=cart:8080\n"+ "STRIPE_URL=https://api.stripe.com\n"+ )+ (tmp_path / "web" / ".env.example").write_text("SEARCH_URL=http://localhost:9200\n")+ (tmp_path / "web" / "orders.env").write_text("ORDERS_URL=${ORDERS:-http://cart:80}\n")+ m = build_service_name_map(_repos(tmp_path, "web", "catalogue", "cart"))+ assert _mapped(m, ENV) == {+ "catalogue_url": ("catalogue", "env_file"),+ "cart_host": ("cart", "env_file"),+ "api_token": ("cart", "env_file"), # host:port is an address under any key+ "orders_url": ("cart", "env_file"),+ }+ assert not _mapped(m)++@pytest.mark.parametrize("rel",[