Skip to content

Build AI methods with your coding agent, then run them anywhere: as an MCP for chatbots, as a webapp for people, or via API for your software.

Teach the config-sync gate to look past what the migrator writes

Louis Choquel3293ad75c123
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 53692b312..2a6116371 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,11 @@
# Changelog
+## [Unreleased]
+
+### Fixed
+
+- **The config-sync gate no longer goes red on what the migrator leaves behind:** `make check` compares the repository's own `.pipelex/` against the `pipelex/kit/configs/` templates by contents, and it had never been taught about the two things pipelex itself writes into a configuration directory — the `.gitignore` added in v0.46.2 (which has no kit counterpart by design) and the timestamped `*.bak.<stamp>` copies a real `pipelex migrate` leaves beside every file it rewrites. Dogfooding the `.gitignore` in v0.46.4 broke `make check` outright, and any developer who ran `pipelex migrate` in a checkout broke it again — putting the dirtied-repository problem that `.gitignore` exists to solve straight back, one gate over. Both the check and `make up-kit-configs` now look past those artifacts, matching backups and `.rescue.` copies by a glob built from the same infixes and stamp shape the backup namer writes, so a rename there cannot orphan the rule. This also closes a latent hole in `make up-kit-configs`, which would have mirrored a developer's backup copy into the shipped kit.
+
## [v0.46.4] - 2026-08-18
### Changed
diff --git a/docs/contribute/configuration-defaults-and-overrides.md b/docs/contribute/configuration-defaults-and-overrides.md
index 8e18ef954..05e785fa0 100644
--- a/docs/contribute/configuration-defaults-and-overrides.md
+++ b/docs/contribute/configuration-defaults-and-overrides.md
@@ -56,6 +56,7 @@ Load order:
- If you change **templates**, keep them user-focused and stable; avoid adding internal-only details.
- If you change **merge order or override semantics**, treat it as a potentially breaking change and document it (changelog + migration notes if needed).
- If you change the **shape** of a configuration model — renaming, moving or removing a key, or dropping an enumerated value — `make check` will stop you until the change is recorded in that surface's migration ledger. Adding a key is not a shape change and needs nothing, provided the key has a default: the defaults layer is what an old file falls back on, and a required key with no default breaks every existing file. The gate is `make check-migration-schemas` (alias `cmig`) and its regenerator is `make up-migration-schemas` (alias `umig`). The contract, including what a ledger may contain and what is guaranteed to a user whose files are migrated, is in [The Migration Ledger](../migration-ledger.md).
+- The repository's own `.pipelex/` dogfoods the templates, and `make check-config-sync` (alias `ccs`) holds the two directories to the same contents; `make up-kit-configs` (alias `ukc`) mirrors `.pipelex/` back onto `pipelex/kit/configs/`. Both look past the same exclusions, declared together in `pipelex/cli/dev_cli/config_sync_exclusions.py`, so the check can never go red on something the sync would not fix. Three kinds of file are excluded: the per-developer overrides and the templates that intentionally differ between the two sides (`CONFIG_SYNC_EXCLUDED_FILES`, each with its reason beside it); the `.gitignore` pipelex writes into a configuration directory it owns, which is a runtime artifact of that directory and has no kit counterpart; and, by glob, the timestamped copies a migration leaves behind (`CONFIG_SYNC_EXCLUDED_PATTERNS`, built from the same infixes and stamp shape `pipelex/migration/backup.py` writes, so a rename there cannot leave a pattern matching nothing). That last one matters because migrating in a checkout is an ordinary thing to do: the run's own `.gitignore` already keeps those copies out of `git status`, and a sync check going red on them would put the dirtied-repository problem straight back, one gate over. The declaration sits under the dev CLI rather than beside the kit paths for a reason worth knowing before you move it: deriving the patterns from the namer means importing `pipelex.migration`, which reaches the pipeline layer, and `pipelex/kit/paths.py` is inside the kernel boot closure that [hub layering](hub-layering.md) keeps free of exactly that. When you add a file to `.pipelex/` that genuinely has no kit counterpart, add it to one of those two sets with a comment saying why — do not delete it to get a green check.
---
diff --git a/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py b/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py
index f7d385125..5920b8244 100644
--- a/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py
+++ b/pipelex/cli/dev_cli/commands/check_config_sync_cmd.py
@@ -9,7 +9,8 @@ from pathlib import Path
from rich.markup import escape
from rich.panel import Panel
-from pipelex.kit.paths import CONFIG_SYNC_EXCLUDED_FILES, GIT_IGNORED_CONFIG_DIRS
+from pipelex.cli.dev_cli.config_sync_exclusions import CONFIG_SYNC_EXCLUDED_FILES, CONFIG_SYNC_EXCLUDED_PATTERNS
+from pipelex.kit.paths import GIT_IGNORED_CONFIG_DIRS
from pipelex.runtime_hub import get_console
from pipelex.tools.misc.diff import has_diff_dirs, make_diff_dirs_pretty
@@ -76,7 +77,13 @@ def check_config_sync_cmd(
# Check for differences (excluding files and directories that intentionally differ)
try:
- has_diff = has_diff_dirs(dir1=left_dir, dir2=right_dir, exclude_files=CONFIG_SYNC_EXCLUDED_FILES, exclude_dirs=GIT_IGNORED_CONFIG_DIRS)
+ has_diff = has_diff_dirs(
+ dir1=left_dir,
+ dir2=right_dir,
+ exclude_files=CONFIG_SYNC_EXCLUDED_FILES,
+ exclude_dirs=GIT_IGNORED_CONFIG_DIRS,
+ exclude_patterns=CONFIG_SYNC_EXCLUDED_PATTERNS,
+ )
except OSError as exc:
# Handle race condition where directories are deleted/modified after existence checks
if quiet:
@@ -121,7 +128,11 @@ def check_config_sync_cmd(
if show_diff:
console.print()
pretty_diff = make_diff_dirs_pretty(
- dir1=left_dir, dir2=right_dir, exclude_files=CONFIG_SYNC_EXCLUDED_FILES, exclude_dirs=GIT_IGNORED_CONFIG_DIRS
+ dir1=left_dir,
+ dir2=right_dir,
+ exclude_files=CONFIG_SYNC_EXCLUDED_FILES,
+ exclude_dirs=GIT_IGNORED_CONFIG_DIRS,
+ exclude_patterns=CONFIG_SYNC_EXCLUDED_PATTERNS,
)
console.print(pretty_diff)
console.print()
diff --git a/pipelex/cli/dev_cli/commands/sync_kit_configs_cmd.py b/pipelex/cli/dev_cli/commands/sync_kit_configs_cmd.py
index 8fdd2783b..00a35cef9 100644
--- a/pipelex/cli/dev_cli/commands/sync_kit_configs_cmd.py
+++ b/pipelex/cli/dev_cli/commands/sync_kit_configs_cmd.py
@@ -9,7 +9,8 @@ from rich.markup import escape
from rich.panel import Panel
from rich.table import Table
-from pipelex.kit.paths import CONFIG_SYNC_EXCLUDED_FILES, GIT_IGNORED_CONFIG_DIRS
+from pipelex.cli.dev_cli.config_sync_exclusions import CONFIG_SYNC_EXCLUDED_FILES, CONFIG_SYNC_EXCLUDED_PATTERNS
+from pipelex.kit.paths import GIT_IGNORED_CONFIG_DIRS
from pipelex.runtime_hub import get_console
from pipelex.tools.misc.file_utils import MirrorDirResult, mirror_dir
@@ -112,6 +113,7 @@ def sync_kit_configs_cmd(*, quiet: bool = False, dry_run: bool = False) -> None:
target_dir=Path(KIT_CONFIGS_DIR),
exclude_files=CONFIG_SYNC_EXCLUDED_FILES,
exclude_dirs=GIT_IGNORED_CONFIG_DIRS,
+ exclude_patterns=CONFIG_SYNC_EXCLUDED_PATTERNS,
dry_run=dry_run,
)
except OSError as exc:
diff --git a/pipelex/cli/dev_cli/config_sync_exclusions.py b/pipelex/cli/dev_cli/config_sync_exclusions.py
new file mode 100644
index 000000000..333a914f3
--- /dev/null
+++ b/pipelex/cli/dev_cli/config_sync_exclusions.py
@@ -0,0 +1,57 @@
+"""What the `.pipelex` ↔ `pipelex/kit/configs` comparison is told to look past.
+
+`pipelex/kit/configs/` holds the templates a project's `.pipelex/` is initialised from, and this
+repository's own `.pipelex/` dogfoods them — so `check-config-sync` holds the two directories to the
+same contents and `sync-kit-configs` mirrors one onto the other. A configuration directory in *use*
+also holds things the kit has no counterpart for, and every one of them has to be declared here or
+the check reports a project that has simply been used as out of sync.
+
+**Why the two commands share one declaration.** `check-config-sync` reports and `sync-kit-configs`
+acts, and its report names the sync as the remedy. If the check saw a file the sync would not touch,
+it would send a developer to run a command that cannot make it pass; if the sync saw one the check
+does not, it would quietly copy a local artifact into the shipped kit. Both read these constants.
+
+**Why they live under the dev CLI rather than beside the kit paths.** Deriving the backup patterns
+from the namer means importing `pipelex.migration`, which reaches the pipeline layer — and
+`pipelex.kit.paths` is inside the kernel boot closure, which `tests/unit/pipelex/
+test_kernel_layer_import_closure.py` keeps free of exactly that. These two commands are the only
+consumers, and `pipelex-dev` is above the boundary, so the declaration belongs on this side of it.
+See `docs/contribute/hub-layering.md`.
+"""
+
+from pipelex.kit.paths import GIT_IGNORED_CONFIG_FILES
+from pipelex.migration.backup import BACKUP_INFIX, BACKUP_STAMP_GLOB, RESCUE_INFIX
+from pipelex.migration.gitignore import CONFIG_DIR_GITIGNORE_NAME
+
+# Files excluded from config sync checks but still copied during `pipelex init config`.
+# Extends GIT_IGNORED_CONFIG_FILES with files that intentionally differ between the two sides:
+# - telemetry.toml: the kit's holds the active global template, while the pipelex repo's
+# `.pipelex/telemetry.toml` dogfoods the commented-out project template.
+# - plxt.toml: the repo's `.pipelex/plxt.toml` adds a `[rule.schema]` override pointing at the
+# locally generated `derived/mthds_schema.json` (this repo is the source of truth for the MTHDS
+# language); the kit template must not reference that repo-internal artifact — plxt hard-fails
+# on every .mthds file when the configured schema path does not exist.
+# - .gitignore: written into a configuration directory by `ensure_config_dir_gitignore`, at init and
+# at migrate. It is a runtime artifact of the directory, never a kit template — the kit has no
+# copy to be in sync with, and giving it one would put two mechanisms in charge of one file.
+CONFIG_SYNC_EXCLUDED_FILES: frozenset[str] = GIT_IGNORED_CONFIG_FILES | {"telemetry.toml", "plxt.toml", CONFIG_DIR_GITIGNORE_NAME}
+
+# Name patterns excluded from config sync checks, for the artifacts no fixed name can name: every
+# one of them carries the timestamp of the run that wrote it.
+# - `<file>.bak.<stamp>`: the copy `pipelex migrate` takes before it rewrites a file. Migrating in a
+# checkout is a thing a developer does, and the run's own `.gitignore` already keeps these out of
+# `git status`; a sync check that went red on them would put the dirtied-repository problem that
+# `.gitignore` solves straight back, one gate over.
+# - `<file>.rescue.<stamp>`: the copy kept when a write could not be vouched for. It is deliberately
+# NOT git-ignored, because turning up in `git status` is how the user is told to go and get it —
+# and that reminder is untouched by this. What excluding it here avoids is a sync check answering
+# a transient artifact with "run make up-kit-configs", which would vendor the copy into the kit.
+#
+# Built from the same infixes and stamp shape `backup_path_for` and `rescue_path_for` actually
+# write, so renaming an infix or restamping cannot leave a pattern that silently matches nothing.
+CONFIG_SYNC_EXCLUDED_PATTERNS: frozenset[str] = frozenset(
+ {
+ f"*{BACKUP_INFIX}{BACKUP_STAMP_GLOB}",
+ f"*{RESCUE_INFIX}{BACKUP_STAMP_GLOB}",
+ }
+)
diff --git a/pipelex/kit/paths.py b/pipelex/kit/paths.py
index 6f8eee578..e459054c8 100644
--- a/pipelex/kit/paths.py
+++ b/pipelex/kit/paths.py
@@ -8,6 +8,10 @@ from importlib.resources.abc import Traversable
# - telemetry_override.toml: Personal telemetry settings
# - telemetry.project.toml: Kit-internal commented-out template used by project init
# only; must never be propagated into ~/.pipelex/ or a project's .pipelex/ as-is
+# The config-sync check and the kit mirror extend this set with the files that intentionally
+# differ between the two sides; that declaration lives in
+# `pipelex/cli/dev_cli/config_sync_exclusions.py`, which is above the kernel boot closure this
+# module sits inside and so may derive its patterns from `pipelex.migration`.
GIT_IGNORED_CONFIG_FILES: frozenset[str] = frozenset(
{
".DS_Store",
@@ -24,16 +28,6 @@ GIT_IGNORED_CONFIG_FILES: frozenset[str] = frozenset(
}
)
-# Files excluded from config sync checks but still copied during `pipelex init config`.
-# Extends GIT_IGNORED_CONFIG_FILES with files that intentionally differ between the two sides:
-# - telemetry.toml: the kit's holds the active global template, while the pipelex repo's
-# `.pipelex/telemetry.toml` dogfoods the commented-out project template.
-# - plxt.toml: the repo's `.pipelex/plxt.toml` adds a `[rule.schema]` override pointing at the
-# locally generated `derived/mthds_schema.json` (this repo is the source of truth for the MTHDS
-# language); the kit template must not reference that repo-internal artifact — plxt hard-fails
-# on every .mthds file when the configured schema path does not exist.
-CONFIG_SYNC_EXCLUDED_FILES: frozenset[str] = GIT_IGNORED_CONFIG_FILES | {"telemetry.toml", "plxt.toml"}
-
# Directories that should not be synced between .pipelex and kit/configs.
# These are runtime directories created locally:
# - storage: Local storage directory for runtime data
diff --git a/pipelex/migration/backup.py b/pipelex/migration/backup.py
index df418398f..6c7c0a857 100644
--- a/pipelex/migration/backup.py
+++ b/pipelex/migration/backup.py
@@ -53,6 +53,24 @@ _BACKUP_STAMP_PATTERN = re.compile(r"\d{8}T\d{6}Z")
"""The stamp, as the shape `BACKUP_STAMP_FORMAT` renders — what tells one of our backups from a
file the user named `<file>.bak.notes` themselves."""
+BACKUP_STAMP_GLOB = f"{'[0-9]' * 8}T{'[0-9]' * 6}Z"
+"""The same shape again, in the only vocabulary a `.gitignore` rule or an `fnmatch` filter has.
+
+It lives beside the regex rather than wherever a glob happens to be needed, so the two spellings
+of one stamp cannot drift apart, and so a change to `BACKUP_STAMP_FORMAT` has one obvious list of
+things to change with it.
+
+A glob has no repetition count, so the eight-digit date and the six-digit time are spelled out
+rather than abbreviated to a `[0-9]*` run. The run looks equivalent and is not: its `*` accepts
+anything at all, so it also matches `pipelex.toml.bak.1-notesZ` — a name `existing_backups_of`
+reads as the user's own and refuses to prune. Anything that declines to manage a file should not
+be hiding it either.
+
+Nothing translates the strftime string into this, on purpose: a parser for format directives is
+more machinery than one constant is worth. What holds the two together is the test matching this
+pattern against a name `backup_path_for` actually produced, which goes red if the stamp restamps.
+"""
+
class WrittenBackup(NamedTuple):
"""Where this file's backup is, and whether this run is the one that put it there.
diff --git a/pipelex/migration/gitignore.py b/pipelex/migration/gitignore.py
index 0587434c8..989e2bc3c 100644
--- a/pipelex/migration/gitignore.py
+++ b/pipelex/migration/gitignore.py
@@ -19,9 +19,9 @@ is migrated through to the file it names, and its backup lands beside *that* fil
deliberately does not follow it there: writing into the `.gitignore` of somebody's whole dotfiles
repository is a larger liberty than the one untracked copy it would hide.
-**The pattern is derived from the namer, not retyped.** `BACKUP_INFIX` is what
-`backup_path_for` actually writes; spelling `*.bak.…` again here would let a rename of the infix
-leave a rule that silently matches nothing.
+**The pattern is derived from the namer, not retyped.** `BACKUP_INFIX` and
+`BACKUP_STAMP_GLOB` are what `backup_path_for` actually writes; spelling `*.bak.…` again here would
+let a rename of the infix — or a restamp — leave a rule that silently matches nothing.
Two omissions are deliberate:
@@ -38,26 +38,12 @@ See `docs/migration-ledger.md` → "Backups".
import os
from pathlib import Path
-from pipelex.migration.backup import BACKUP_INFIX
+from pipelex.migration.backup import BACKUP_INFIX, BACKUP_STAMP_GLOB
CONFIG_DIR_GITIGNORE_NAME = ".gitignore"
"""What the file is called inside `~/.pipelex/` or a project's `.pipelex/`."""
-_STAMP_GLOB = f"{'[0-9]' * 8}T{'[0-9]' * 6}Z"
-"""The shape `BACKUP_STAMP_FORMAT` renders, written in the only vocabulary a `.gitignore` has.
-
-A glob has no repetition count, so the eight-digit date and the six-digit time are spelled out
-rather than abbreviated to a `[0-9]*` run. The run looks equivalent and is not: its `*` accepts
-anything at all, so it also matches `pipelex.toml.bak.1-notesZ` — a name `existing_backups_of`
-reads as the user's own and refuses to prune. Hiding a file we have just declined to manage is the
-worse half of both answers.
-
-Nothing translates the strftime string into this, on purpose: a parser for format directives is
-more machinery than one constant is worth. What holds the two together is the test matching this
-pattern against a name `backup_path_for` actually produced, which goes red if the stamp restamps.
-"""
-
-BACKUP_IGNORE_PATTERN = f"*{BACKUP_INFIX}{_STAMP_GLOB}"
+BACKUP_IGNORE_PATTERN = f"*{BACKUP_INFIX}{BACKUP_STAMP_GLOB}"
"""The one rule the file carries: any name ending in the infix followed by exactly the stamp.
Exactly the stamp, rather than something stamp-shaped, is what keeps `pipelex.toml.bak.notes` — and
diff --git a/pipelex/tools/misc/diff.py b/pipelex/tools/misc/diff.py
index 729874818..11a880ab3 100644
--- a/pipelex/tools/misc/diff.py
+++ b/pipelex/tools/misc/diff.py
@@ -10,6 +10,7 @@ from rich.syntax import Syntax
from rich.table import Table
from rich.text import Text
+from pipelex.tools.misc.file_utils import is_excluded_by_name
from pipelex.tools.misc.pretty import PrettyPrinter
if TYPE_CHECKING:
@@ -24,6 +25,7 @@ def has_diff_dirs(
dir2: str | Path,
exclude_files: AbstractSet[str] | None = None,
exclude_dirs: AbstractSet[str] | None = None,
+ exclude_patterns: AbstractSet[str] | None = None,
) -> bool:
"""Check if there are any differences between two directories.
@@ -32,17 +34,21 @@ def has_diff_dirs(
dir2: Second directory path.
exclude_files: Set of file names to exclude from comparison (e.g., {"pipelex_service.toml"}).
exclude_dirs: Set of directory names to exclude from comparison (e.g., {"storage"}).
+ exclude_patterns: Set of glob patterns whose matching file names are excluded from
+ comparison (e.g., {"*.bak.*"}). For artifacts whose names carry a timestamp or another
+ varying part, so no fixed name can name them.
Returns:
True if there are any files only in left, only in right, or different files.
"""
dir1 = Path(dir1)
dir2 = Path(dir2)
- exclude_files = exclude_files or set()
exclude_dirs = exclude_dirs or set()
+ excluded_files = frozenset(exclude_files or set())
+ excluded_patterns = frozenset(exclude_patterns or set())
def _filter_excluded_files(file_list: list[str]) -> list[str]:
- return [file for file in file_list if file not in exclude_files]
+ return [file for file in file_list if not is_excluded_by_name(name=file, exclude_files=excluded_files, exclude_patterns=excluded_patterns)]
def _has_diff(dir_comparison: filecmp.dircmp[str]) -> bool:
# Filter out excluded directories from left_only and right_only
@@ -214,6 +220,7 @@ def make_diff_dirs_pretty(
dir2: str | Path,
exclude_files: AbstractSet[str] | None = None,
exclude_dirs: AbstractSet[str] | None = None,
+ exclude_patterns: AbstractSet[str] | None = None,
) -> PrettyPrintable:
"""Generate a PrettyPrintable representation of directory differences.
@@ -222,6 +229,9 @@ def make_diff_dirs_pretty(
dir2: Second directory path.
exclude_files: Set of file names to exclude from comparison (e.g., {"pipelex_service.toml"}).
exclude_dirs: Set of directory names to exclude from comparison (e.g., {"storage"}).
+ exclude_patterns: Set of glob patterns whose matching file names are excluded from
+ comparison (e.g., {"*.bak.*"}). Keep it equal to what the matching `has_diff_dirs` call
+ passes, or the report contradicts the verdict that produced it.
Returns:
A Rich renderable showing files only in left, only in right,
@@ -230,13 +240,14 @@ def make_diff_dirs_pretty(
"""
dir1 = Path(dir1)
dir2 = Path(dir2)
- exclude_files = exclude_files or set()
exclude_dirs = exclude_dirs or set()
+ excluded_files = frozenset(exclude_files or set())
+ excluded_patterns = frozenset(exclude_patterns or set())
sections: list[PrettyPrintable] = []
def _filter_excluded_files(file_list: list[str]) -> list[str]:
- return [file for file in file_list if file not in exclude_files]
+ return [file for file in file_list if not is_excluded_by_name(name=file, exclude_files=excluded_files, exclude_patterns=excluded_patterns)]
def _collect_diffs(dir_comparison: filecmp.dircmp[str], *, relative_path: str = "") -> None:
# Files only in left directory (excluding excluded files and directories)
diff --git a/pipelex/tools/misc/file_utils.py b/pipelex/tools/misc/file_utils.py
index c0856b696..e96f377fb 100644
--- a/pipelex/tools/misc/file_utils.py
+++ b/pipelex/tools/misc/file_utils.py
@@ -1,4 +1,5 @@
import filecmp
+import fnmatch
import importlib.resources
import os
import shutil
@@ -245,6 +246,30 @@ def remove_folder(folder_path: Path) -> None:
shutil.rmtree(folder_path)
+def is_excluded_by_name(*, name: str, exclude_files: frozenset[str], exclude_patterns: frozenset[str]) -> bool:
+ """Whether a directory entry is one the caller asked a directory walk to look past.
+
+ Two vocabularies, because two kinds of entry need excluding. A fixed name covers a file that is
+ always called the same thing; a glob covers the ones whose name carries a timestamp or another
+ part that varies per run, which no fixed name can ever match.
+
+ Matching is case-sensitive (`fnmatchcase`) rather than `fnmatch`, whose case-folding follows the
+ host filesystem — the same pattern would otherwise exclude on macOS what it keeps on Linux, and
+ a rule that answers differently per developer is worse than one that answers strictly.
+
+ Args:
+ name: The basename of the entry being considered.
+ exclude_files: Basenames excluded outright.
+ exclude_patterns: Glob patterns whose matching basenames are excluded (e.g. "*.bak.*").
+
+ Returns:
+ True when the entry should be skipped.
+ """
+ if name in exclude_files:
+ return True
+ return any(fnmatch.fnmatchcase(name, pattern) for pattern in exclude_patterns)
+
+
class MirrorDirResult(BaseModel):
"""Outcome of a mirror_dir operation.
@@ -281,6 +306,7 @@ def mirror_dir(
target_dir: Path,
exclude_files: frozenset[str] | None = None,
exclude_dirs: frozenset[str] | None = None,
+ exclude_patterns: frozenset[str] | None = None,
dry_run: bool = False,
) -> MirrorDirResult:
"""Mirrors a source directory onto a target directory (recursive copy + delete).
@@ -297,6 +323,8 @@ def mirror_dir(
target_dir (Path): The directory brought in sync. Created if missing.
exclude_files (frozenset[str] | None): File basenames to skip entirely.
exclude_dirs (frozenset[str] | None): Directory basenames to skip entirely.
+ exclude_patterns (frozenset[str] | None): Glob patterns whose matching file basenames are
+ skipped entirely, for the entries whose name varies per run and so cannot be named.
dry_run (bool): If True, report changes without touching the filesystem.
Returns:
@@ -312,6 +340,7 @@ def mirror_dir(
target_root = Path(target_dir)
excluded_files = exclude_files or frozenset()
excluded_dirs = exclude_dirs or frozenset()
+ excluded_patterns = exclude_patterns or frozenset()
# Validate the source before Pass 1: an invalid source makes every
# is_dir()/is_file() probe return false, which would delete the entire
@@ -350,7 +379,7 @@ def mirror_dir(
remove_folder(dir_to_remove)
dir_names[:] = kept_dir_names
for file_name in sorted(file_names):
- if file_name in excluded_files:
+ if is_excluded_by_name(name=file_name, exclude_files=excluded_files, exclude_patterns=excluded_patterns):
continue
if (source_root / relative_root / file_name).is_file():
continue
@@ -373,7 +402,7 @@ def mirror_dir(
if not dry_run:
ensure_directory_exists(target_subdir)
for file_name in sorted(file_names):
- if file_name in excluded_files:
+ if is_excluded_by_name(name=file_name, exclude_files=excluded_files, exclude_patterns=excluded_patterns):
continue
source_file = Path(current_root) / file_name
target_file = target_root / relative_root / file_name
diff --git a/tests/unit/pipelex/cli/dev/test_config_sync_exclusions.py b/tests/unit/pipelex/cli/dev/test_config_sync_exclusions.py
new file mode 100644
index 000000000..449bc5cea
--- /dev/null
+++ b/tests/unit/pipelex/cli/dev/test_config_sync_exclusions.py
@@ -0,0 +1,80 @@
+"""What the `.pipelex` ↔ `pipelex/kit/configs` comparison must look past.
+
+The kit holds the templates; a configuration directory in use also holds what pipelex itself writes
+there — the `.gitignore` `ensure_config_dir_gitignore` creates, and the timestamped copies a
+migration leaves behind. None of those has a kit counterpart, so a check comparing the two
+directories by contents has to be told about each one, or it reports a project that has simply been
+migrated as out of sync.
+
+Every assertion below is made against a name the writing code actually produces, so a rename in
+`pipelex/migration/` goes red here instead of quietly widening what the check reports.
+"""
+
+from datetime import UTC, datetime
+from fnmatch import fnmatchcase
+from pathlib import Path
+
+from pipelex.cli.dev_cli.config_sync_exclusions import CONFIG_SYNC_EXCLUDED_FILES, CONFIG_SYNC_EXCLUDED_PATTERNS
+from pipelex.kit.paths import GIT_IGNORED_CONFIG_DIRS
+from pipelex.migration.backup import backup_path_for, rescue_path_for
+from pipelex.migration.gitignore import CONFIG_DIR_GITIGNORE_NAME, ensure_config_dir_gitignore
+from pipelex.tools.misc.diff import has_diff_dirs
+
+MOMENT = datetime(2026, 8, 18, 12, 14, 41, tzinfo=UTC)
+
+
+class TestConfigSyncExclusions:
+ def test_the_config_directory_gitignore_is_excluded(self) -> None:
+ """It is written into the directory at init and at migrate, and is never a kit template."""
+ assert CONFIG_DIR_GITIGNORE_NAME in CONFIG_SYNC_EXCLUDED_FILES
+
+ def test_a_real_backup_name_is_excluded(self) -> None:
+ """Migrating in a checkout must not make the next `make check` red."""
+ backup = backup_path_for(path=Path("pipelex_override.toml"), moment=MOMENT)
+
+ assert any(fnmatchcase(backup.name, pattern) for pattern in CONFIG_SYNC_EXCLUDED_PATTERNS)
+
+ def test_a_real_rescue_name_is_excluded(self) -> None:
+ """Still visible in `git status`, where the reminder belongs — just not read as a config."""
+ rescue = rescue_path_for(path=Path("pipelex.toml"), moment=MOMENT)
+
+ assert any(fnmatchcase(rescue.name, pattern) for pattern in CONFIG_SYNC_EXCLUDED_PATTERNS)
+
+ def test_every_file_a_real_walk_can_back_up_is_excluded(self) -> None:
+ """The walk claims files by (directory, name), so the patterns must not lean on a stem."""
+ for name in ("pipelex.toml", "pipelex_override.toml", "telemetry.toml", "anthropic.toml"):
+ backup = backup_path_for(path=Path(name), moment=MOMENT)
+ assert any(fnmatchcase(backup.name, pattern) for pattern in CONFIG_SYNC_EXCLUDED_PATTERNS), name
+
+ def test_a_copy_the_user_named_themselves_is_not_excluded(self) -> None:
+ """Pruning refuses to manage one of these; the check has no business hiding it either."""
+ for name in ("pipelex.toml.bak.notes", "pipelex.toml.bak.1-notesZ", "pipelex.toml.rescue.keep"):
+ assert not any(fnmatchcase(name, pattern) for pattern in CONFIG_SYNC_EXCLUDED_PATTERNS), name
+
+ def test_a_migrated_config_directory_reads_as_in_sync(self, tmp_path: Path) -> None:
+ """The whole chain, against a directory carrying everything a real migration leaves.
+
+ Asserted through `has_diff_dirs` with the real constants rather than against the constants
+ alone, so a command that stopped passing them would fail here too.
+ """
+ kit_side = tmp_path / "kit"
+ project_side = tmp_path / "project"
+ kit_side.mkdir()
+ project_side.mkdir()
+ (kit_side / "pipelex.toml").write_text("shared = true", encoding="utf-8")
+ (project_side / "pipelex.toml").write_text("shared = true", encoding="utf-8")
+
+ ensure_config_dir_gitignore(directory=project_side)
+ backup_path_for(path=project_side / "pipelex.toml", moment=MOMENT).write_text("shared = false", encoding="utf-8")
+ rescue_path_for(path=project_side / "pipelex.toml", moment=MOMENT).write_text("shared = false", encoding="utf-8")
+
+ assert (
+ has_diff_dirs(
+ dir1=project_side,
+ dir2=kit_side,
+ exclude_files=CONFIG_SYNC_EXCLUDED_FILES,
+ exclude_dirs=GIT_IGNORED_CONFIG_DIRS,
+ exclude_patterns=CONFIG_SYNC_EXCLUDED_PATTERNS,
+ )
+ is False
+ )
diff --git a/tests/unit/pipelex/tools/misc/test_diff.py b/tests/unit/pipelex/tools/misc/test_diff.py
index 4c1061ad4..b2190428c 100644
--- a/tests/unit/pipelex/tools/misc/test_diff.py
+++ b/tests/unit/pipelex/tools/misc/test_diff.py
@@ -65,6 +65,63 @@ class TestDiff:
assert has_diff_dirs(dir1=dir1, dir2=dir2) is True
+ def test_has_diff_dirs_pattern_excluded_file_only_in_left(self):
+ """A left-only file matching an exclude pattern does not count as a difference."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ dir1 = Path(temp_dir) / "dir1"
+ dir2 = Path(temp_dir) / "dir2"
+ dir1.mkdir()
+ dir2.mkdir()
+
+ (dir1 / "common.txt").write_text("content")
+ (dir2 / "common.txt").write_text("content")
+ (dir1 / "common.txt.bak.20260818T084913Z").write_text("pre-migration copy")
+
+ assert has_diff_dirs(dir1=dir1, dir2=dir2) is True
+ assert has_diff_dirs(dir1=dir1, dir2=dir2, exclude_patterns={"*.bak.*"}) is False
+
+ def test_has_diff_dirs_pattern_exclusion_reaches_subdirectories(self):
+ """An exclude pattern applies at every depth of the comparison."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ dir1 = Path(temp_dir) / "dir1"
+ dir2 = Path(temp_dir) / "dir2"
+ (dir1 / "sub").mkdir(parents=True)
+ (dir2 / "sub").mkdir(parents=True)
+
+ (dir1 / "sub" / "common.txt").write_text("content")
+ (dir2 / "sub" / "common.txt").write_text("content")
+ (dir1 / "sub" / "common.txt.bak.20260818T084913Z").write_text("pre-migration copy")
+
+ assert has_diff_dirs(dir1=dir1, dir2=dir2, exclude_patterns={"*.bak.*"}) is False
+
+ def test_has_diff_dirs_pattern_leaves_unmatched_file_visible(self):
+ """A pattern excludes only what it matches — a neighbouring extra file still differs."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ dir1 = Path(temp_dir) / "dir1"
+ dir2 = Path(temp_dir) / "dir2"
+ dir1.mkdir()
+ dir2.mkdir()
+
+ (dir1 / "common.txt.bak.20260818T084913Z").write_text("pre-migration copy")
+ (dir1 / "common.txt.bak.notes").write_text("a copy the user named")
+
+ assert has_diff_dirs(dir1=dir1, dir2=dir2, exclude_patterns={"*.bak.[0-9]*Z"}) is True
+
+ def test_make_diff_dirs_pretty_omits_pattern_excluded_file(self):
+ """The report leaves out what the verdict looked past, so the two cannot contradict."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ dir1 = Path(temp_dir) / "dir1"
+ dir2 = Path(temp_dir) / "dir2"
+ dir1.mkdir()
+ dir2.mkdir()
+
+ (dir1 / "common.txt.bak.20260818T084913Z").write_text("pre-migration copy")
+
+ result = make_diff_dirs_pretty(dir1=dir1, dir2=dir2, exclude_patterns={"*.bak.*"})
+
+ assert isinstance(result, Text)
+ assert "No differences found" in result.plain
+
def test_has_diff_dirs_recursive(self):
"""Test that subdirectory differences are detected."""
with tempfile.TemporaryDirectory() as temp_dir:
diff --git a/tests/unit/pipelex/tools/misc/test_mirror_dir.py b/tests/unit/pipelex/tools/misc/test_mirror_dir.py
index c13528b6e..a6105d297 100644
--- a/tests/unit/pipelex/tools/misc/test_mirror_dir.py
+++ b/tests/unit/pipelex/tools/misc/test_mirror_dir.py
@@ -149,6 +149,58 @@ class TestMirrorDir:
assert result.copied_files == []
assert not (target / "telemetry.toml").exists()
+ def test_pattern_excluded_file_is_not_copied(self):
+ """A file matching an exclude pattern in the source is not copied into the target."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ source = Path(temp_dir) / "source"
+ target = Path(temp_dir) / "target"
+ _write(source / "pipelex.toml.bak.20260818T084913Z", "pre-migration copy")
+ target.mkdir()
+
+ result = mirror_dir(source_dir=source, target_dir=target, exclude_patterns=frozenset({"*.bak.*"}))
+
+ assert result.copied_files == []
+ assert not (target / "pipelex.toml.bak.20260818T084913Z").exists()
+
+ def test_pattern_excluded_file_target_only_is_preserved(self):
+ """A file matching an exclude pattern present only in the target is not deleted."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ source = Path(temp_dir) / "source"
+ target = Path(temp_dir) / "target"
+ source.mkdir()
+ _write(target / "pipelex.toml.bak.20260818T084913Z", "pre-migration copy")
+
+ result = mirror_dir(source_dir=source, target_dir=target, exclude_patterns=frozenset({"*.bak.*"}))
+
+ assert result.deleted_files == []
+ assert (target / "pipelex.toml.bak.20260818T084913Z").exists()
+
+ def test_pattern_exclusion_reaches_nested_directories(self):
+ """An exclude pattern applies at every depth of the walk, not just the top level."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ source = Path(temp_dir) / "source"
+ target = Path(temp_dir) / "target"
+ _write(source / "inference" / "backends.toml", "backends")
+ _write(source / "inference" / "backends.toml.bak.20260818T084913Z", "pre-migration copy")
+ target.mkdir()
+
+ result = mirror_dir(source_dir=source, target_dir=target, exclude_patterns=frozenset({"*.bak.*"}))
+
+ assert result.copied_files == ["inference/backends.toml"]
+ assert not (target / "inference" / "backends.toml.bak.20260818T084913Z").exists()
+
+ def test_pattern_exclusion_is_case_sensitive(self):
+ """Matching does not fold case, so the same pattern answers alike on every filesystem."""
+ with tempfile.TemporaryDirectory() as temp_dir:
+ source = Path(temp_dir) / "source"
+ target = Path(temp_dir) / "target"
+ _write(source / "PIPELEX.BAK.file", "not one of ours")
+ target.mkdir()
+
+ result = mirror_dir(source_dir=source, target_dir=target, exclude_patterns=frozenset({"*.bak.*"}))
+
+ assert result.copied_files == ["PIPELEX.BAK.file"]
+
def test_excluded_dir_target_only_is_preserved(self):
"""An excluded directory present only in the target is left untouched."""
with tempfile.TemporaryDirectory() as temp_dir: