The OSINT investigator's workbench. One case, one folder, every tool. Local-first: your media and your investigations never leave your machine.
JavaScript
Put the hidden attribute back on the directories Windows rebuilt (#47)
OsintMeThat2e971b6f865f
diff --git a/docs/SPEC.md b/docs/SPEC.mdindex 3a60694..32a0682 100644--- a/docs/SPEC.md+++ b/docs/SPEC.md@@ -158,7 +158,6 @@ Each version delivers one complete daily workflow. Firm ideas move here from| **Case Sheet** | The same case as a table: a row is an entity, columns are its attributes plus free ones the analyst adds, sorted, filtered and edited in place. Imports a CSV as loose rows that stay out of the graph until promoted, and exports back to CSV or GeoJSON. || **Camera Resection (GCP)** | Marks matching points photo↔map, then solves camera position, viewing azimuth and rough FOV (OpenCV `solvePnP`) and saves the match as evidence. Its photo canvas and pixel↔angle camera frame are built for two callers: Sky Clock fills the same frame by hand. || **Command palette** | Ctrl+K reaches a tool, a case or an artifact. |-| **Capture scale and north** | Preference-controlled scale bar, north arrow and graticule on app and extension captures. |Toward v2: split Satellite.svelte into `lib/` modules, before the map enginechanges under it.@@ -169,6 +168,7 @@ changes under it.|------|--------------|| **Map engine (MapLibre)** | Replaces Leaflet with MapLibre GL at 2D parity: same providers, same captures, and the capture tests still verify the pixels. Ships before the 3D map, which builds on it. || **3D map** | Pitch, public DEM terrain and extruded OSM buildings on the MapLibre map. An oblique capture records its pitch beside the bearing, so the view can be reproduced. |+| **Capture scale and north** | Preference-controlled scale bar, north arrow and graticule on app and extension captures. Follows the map engine, which redraws what a capture is made of. || **Satellite Compare** | Same coords across providers (Esri / Sentinel-2 date slider / Bing / keyed), synced pan/zoom. Copernicus easy link. || **Image Compare** | Overlay two images with opacity, swipe and pixel diff. Assist satellite-to-screen alignment without presenting a verdict. || **Metadata follow-up** | Explains which common image/video fields were stripped and proposes events from capture times. |diff --git a/docs/STORAGE_AND_PERFORMANCE.md b/docs/STORAGE_AND_PERFORMANCE.mdindex 416ea77..56efb4b 100644--- a/docs/STORAGE_AND_PERFORMANCE.md+++ b/docs/STORAGE_AND_PERFORMANCE.md@@ -114,8 +114,11 @@ wrapper arrive without rewriting a stored path, and what keeps bundles writtenbefore it importable after it.Visible means openable and useful in another program; a dot-directory means only-Azimut reads it. `ensure_dir` sets `FILE_ATTRIBUTE_HIDDEN` on those, since a-leading dot means nothing to Windows Explorer.+Azimut reads it. `ensure_dir` adds `FILE_ATTRIBUTE_HIDDEN` to those, since a+leading dot means nothing to Windows Explorer. The attribute belongs to the+directory rather than to its name, so a rebuilt tree loses it: opening a case+puts it back over `layout.hidden_dirs()`, which is what covers a workspace+copied from another machine.`layout.py` owns this shape. It answers where each file goes for a given caseroot and carries the path budget — the caps on case, media, document and folderdiff --git a/src/azimut/api/proofs.py b/src/azimut/api/proofs.pyindex 43b261d..e16c524 100644--- a/src/azimut/api/proofs.py+++ b/src/azimut/api/proofs.py@@ -163,7 +163,7 @@ def _write_assets(folder: Path, incoming: dict[str, bytes], keep: set[str]) -> Npasted images keeps no folder either."""if incoming:- folder.mkdir(parents=True, exist_ok=True)+ ensure_dir(folder)for asset_name, data in incoming.items():path = folder / asset_nameif not path.exists():diff --git a/src/azimut/config.py b/src/azimut/config.pyindex 02ce5d3..d80e609 100644--- a/src/azimut/config.py+++ b/src/azimut/config.py@@ -410,7 +410,7 @@ def ensure_workspace() -> None:# Windows ignores POSIX modes, which is fine (its ACLs already scope $HOME)._restrict(workspace_root(), 0o700)_restrict(internal_dir(), 0o700)- _hide_if_dotted(internal_dir())+ hide_if_dotted(internal_dir())_migrate_workspace_settings()_migrate_workspace_directories()for directory in (scratch_dir(), bundles_dir(), runtime_dir(), tile_cache_dir()):@@ -437,15 +437,38 @@ def _restrict(path: Path, mode: int) -> None:pass-def _hide_if_dotted(path: Path) -> None:- """Make a dot-directory hidden in Windows Explorer too."""+#: Windows attribute bits. `INVALID` is what `GetFileAttributesW` answers for a+#: path it cannot read, including one that is not there.+_FILE_ATTRIBUTE_HIDDEN = 0x02+_INVALID_FILE_ATTRIBUTES = -1+++def hide_if_dotted(path: Path) -> None:+ """Make a dot-directory hidden in Windows Explorer too.++ A leading dot hides a directory on Linux and macOS and means nothing to+ Windows Explorer, which needs `FILE_ATTRIBUTE_HIDDEN` set. Without this the+ whole visible/hidden split in `layout.py` would work on two platforms out of+ three, and `.data` — the case database — would sit in plain view on the one+ where people are most likely to go tidying.++ The bit is *added* to whatever the directory already carries rather than+ written over it: this runs again on paths that already exist, and a+ wholesale write would drop attributes Windows or the analyst had set.++ Best effort: a filesystem that refuses the attribute (a network share, a FAT+ stick) is not a reason to fail a directory creation.+ """if sys.platform != "win32" or not path.name.startswith("."):returnimport ctypes- file_attribute_hidden = 0x02try:- ctypes.windll.kernel32.SetFileAttributesW(str(path), file_attribute_hidden)+ kernel32 = ctypes.windll.kernel32+ current = kernel32.GetFileAttributesW(str(path))+ if current == _INVALID_FILE_ATTRIBUTES or current & _FILE_ATTRIBUTE_HIDDEN:+ return+ kernel32.SetFileAttributesW(str(path), current | _FILE_ATTRIBUTE_HIDDEN)except (AttributeError, OSError): # pragma: no cover - Windows onlypass@@ -471,7 +494,7 @@ def _migrate_workspace_settings() -> None:directory = settings_dir()directory.mkdir(parents=True, exist_ok=True)_restrict(directory, 0o700)- _hide_if_dotted(directory)+ hide_if_dotted(directory)for source_dir in (workspace_root(), workspace_root() / ".settings"):for name in ("settings.json", "templates.json", "signature.png", "cookies.txt"):source = source_dir / namediff --git a/src/azimut/engine/doctor.py b/src/azimut/engine/doctor.pyindex 61e05c7..3059ccf 100644--- a/src/azimut/engine/doctor.py+++ b/src/azimut/engine/doctor.py@@ -10,7 +10,7 @@ from typing import Any, Iteratorfrom .. import layoutfrom ..sqlite_backend import SqliteCase-from ..workspace import Case, CaseError+from ..workspace import Case, CaseError, ensure_dirfrom . import media as media_engineDATABASE_MISSING = "database-missing"@@ -163,7 +163,7 @@ def rebuild_database(case: Case) -> dict[str, Any]:if case.db_path.exists():raise CaseError("the case database already exists")manifest = case.read()- case.db_path.parent.mkdir(parents=True, exist_ok=True)+ ensure_dir(case.db_path.parent)staging = case.db_path.with_name(f".{case.db_path.name}.doctor-{uuid.uuid4().hex}.tmp")counts = {"media": 0, "notes": 0, "proofs": 0}try:diff --git a/src/azimut/engine/workspacemove.py b/src/azimut/engine/workspacemove.pyindex c725b95..674301f 100644--- a/src/azimut/engine/workspacemove.py+++ b/src/azimut/engine/workspacemove.py@@ -34,6 +34,7 @@ from pathlib import Pathfrom typing import Any, Callablefrom .. import config, layout+from ..workspace import ensure_dirfrom . import workspacelockfrom .bundles import disk_reserve@@ -586,7 +587,10 @@ def _step_copy(move: Move, staging: Path) -> None:move.files, move.total_bytes = measure(move.source)staging.mkdir(parents=True)for source_dir in _copyable_dirs(move.source):- (staging / source_dir.relative_to(move.source)).mkdir(parents=True, exist_ok=True)+ # `ensure_dir`, not `mkdir`: a rebuilt directory carries none of the+ # attributes the original had, so a plain copy is how a workspace used+ # to arrive at its new home with every hidden directory on show.+ ensure_dir(staging / source_dir.relative_to(move.source))for source_file in _copyable_files(move.source):destination = staging / source_file.relative_to(move.source)shutil.copy2(source_file, destination)diff --git a/src/azimut/layout.py b/src/azimut/layout.pyindex fa24829..c55aadb 100644--- a/src/azimut/layout.py+++ b/src/azimut/layout.py@@ -398,6 +398,23 @@ def content_dirs(root: Path) -> tuple[Path, ...]:return tuple(dirs)+def hidden_dirs(root: Path) -> tuple[Path, ...]:+ """The directories a leading dot is supposed to keep out of the way.++ Windows hides by attribute rather than by name, and an attribute only+ travels with the directory it was set on: a case that arrives by any route+ other than being born here — a workspace copied to another machine, a folder+ restored from a backup — has these in plain view. Whoever opens a case walks+ this list and puts the attribute back.++ The regenerable caches under `media/` are not in it. They are re-created+ through `ensure_dir` on the next thumbnail or download, which hides them+ again by itself.+ """+ dirs = [path for path in content_dirs(root) if path.name.startswith(".")]+ return (data_dir(root), trash(root), *dirs)++def unwrapped_manifest(root: Path) -> Path:"""Where the manifest sat before the case gained its `azimut/` wrapper.diff --git a/src/azimut/workspace.py b/src/azimut/workspace.pyindex 9f95782..63241cf 100644--- a/src/azimut/workspace.py+++ b/src/azimut/workspace.py@@ -14,7 +14,6 @@ import jsonimport loggingimport reimport shutil-import sysimport sqlite3import threadingimport time@@ -159,7 +158,7 @@ def ensure_dir(path: Path) -> Path:for attempt in range(20):try:path.mkdir(parents=True, exist_ok=True)- _hide_if_dotted(path)+ _hide_dotted_chain(path)return pathexcept PermissionError:if path.is_dir():@@ -170,27 +169,37 @@ def ensure_dir(path: Path) -> Path:return path # pragma: no cover - loop always returns or raises above-def _hide_if_dotted(path: Path) -> None:- """Make a dot-directory actually hidden on Windows.+def _hide_dotted_chain(path: Path) -> None:+ """Hide *path* on Windows, and the dot-directories `parents=True` just made.- A leading dot hides a directory on Linux and macOS and means nothing to- Windows Explorer, which needs `FILE_ATTRIBUTE_HIDDEN` set. Without this the- whole visible/hidden split in `layout.py` would work on two platforms out of- three, and `.data` — the case database — would sit in plain view on the one- where people are most likely to go tidying.-- Best effort: a filesystem that refuses the attribute (a network share, a FAT- stick) is not a reason to fail a directory creation.+ `media/.meta/<proof>.assets` creates `.meta` on the way to a leaf that is+ not itself dotted, so hiding the leaf alone would leave the directory that+ matters visible. The walk stops at the first ancestor without a leading dot,+ which is always the visible half of the case (`media/`, `proofs/`, the tool+ root), so it never climbs out of the workspace."""- if sys.platform != "win32" or not path.name.startswith("."):- return- import ctypes+ current = path+ while True:+ config.hide_if_dotted(current)+ parent = current.parent+ if parent == current or not parent.name.startswith("."):+ return+ current = parent- FILE_ATTRIBUTE_HIDDEN = 0x02- try:- ctypes.windll.kernel32.SetFileAttributesW(str(path), FILE_ATTRIBUTE_HIDDEN)- except (AttributeError, OSError): # pragma: no cover - Windows only- pass++def _follow_hidden_dirs(root: Path) -> None:+ """Put the hidden attribute back on a case's internal directories.++ `.azimut` gets this at every startup (`config.ensure_workspace`); a case only+ got it the day it was born, so any copy of a workspace — a move onto another+ drive, a folder carried between machines, a backup unpacked — showed the+ analyst directories the layout means to keep out of sight.++ Costs nothing off Windows, where `hide_if_dotted` is what returns early: one+ guard in one place beats a second copy of the platform test here.+ """+ for directory in layout.hidden_dirs(root):+ config.hide_if_dotted(directory)def write_readme(root: Path) -> None:@@ -402,6 +411,7 @@ class Case:def open(cls, case_id: str) -> "Case":case = cls.locate(case_id)case.migrate()+ _follow_hidden_dirs(case.path)from .engine import trash as trash_enginetrash_engine.recover(case)diff --git a/tests/test_hidden_dirs.py b/tests/test_hidden_dirs.pynew file mode 100644index 0000000..39b06b0--- /dev/null+++ b/tests/test_hidden_dirs.py@@ -0,0 +1,170 @@+"""A leading dot hides a directory on two operating systems out of three.++Windows hides by attribute instead, and an attribute belongs to the directory it+was set on — not to its name. Every way a case tree can be rebuilt (a workspace+copied to another drive, a folder carried between machines, a database the+doctor recreates) therefore drops it, and the case's internals come back into+view. These tests hold the two halves of the answer: the attribute is set+wherever a directory is created, and it is put back wherever a case is opened.++The platform is faked rather than skipped. The behaviour has to be verifiable on+the machine the suite actually runs on, or the one OS it exists for is the one+OS nobody tests it on.+"""++from __future__ import annotations++import ctypes+import sys+from pathlib import Path+from types import SimpleNamespace++import pytest++from azimut import config, layout+from azimut.api import proofs+from azimut.workspace import Case, ensure_dir++#: What Windows answers for a path it cannot read, and the bits in play. Every+#: real directory reports `DIRECTORY`, which is why the fake below answers with+#: it: an attribute set is never empty, and the code has to add to one.+INVALID = -1+HIDDEN = 0x02+READONLY = 0x01+DIRECTORY = 0x10+++class FakeKernel32:+ """The two calls `hide_if_dotted` makes, over an in-memory attribute table."""++ def __init__(self, attributes: dict[str, int] | None = None) -> None:+ self.attributes = attributes or {}+ self.reads: list[str] = []++ def GetFileAttributesW(self, path: str) -> int: # noqa: N802 - Windows spelling+ self.reads.append(path)+ if path in self.attributes:+ return self.attributes[path]+ return DIRECTORY if Path(path).is_dir() else INVALID++ def SetFileAttributesW(self, path: str, value: int) -> int: # noqa: N802+ self.attributes[path] = value+ return 1+++def windows(monkeypatch):+ """Run the code under test as if it were on Windows, with a fake kernel32."""+ kernel32 = FakeKernel32()+ monkeypatch.setattr(sys, "platform", "win32")+ monkeypatch.setattr(ctypes, "windll", SimpleNamespace(kernel32=kernel32), raising=False)+ return kernel32+++def test_the_hidden_bit_is_added_to_the_attributes_already_there(windows, tmp_path):+ """A directory Windows or the analyst had marked keeps that mark. Writing+ the bit on its own would silently clear everything else."""+ directory = tmp_path / ".data"+ windows.attributes[str(directory)] = READONLY++ config.hide_if_dotted(directory)++ assert windows.attributes[str(directory)] == READONLY | HIDDEN+++def test_an_already_hidden_directory_is_not_written_again(windows, tmp_path):+ directory = tmp_path / ".data"+ windows.attributes[str(directory)] = HIDDEN++ config.hide_if_dotted(directory)++ assert windows.attributes[str(directory)] == HIDDEN+ assert windows.reads == [str(directory)]+++def test_a_directory_windows_cannot_read_is_left_alone(windows, tmp_path):+ """`GetFileAttributesW` answers -1 for a path that is not there. Treating+ that as an attribute set would write `-1 | HIDDEN` over the directory."""+ missing = tmp_path / ".gone"++ config.hide_if_dotted(missing)++ assert str(missing) not in windows.attributes+++def test_a_visible_directory_is_never_hidden(windows, tmp_path):+ config.hide_if_dotted(tmp_path / "media")++ assert windows.attributes == {}+++def test_nothing_is_touched_off_windows(monkeypatch, tmp_path):+ """The whole mechanism is one platform's. Elsewhere it must not even look:+ `ctypes.windll` does not exist there."""+ monkeypatch.setattr(sys, "platform", "linux")+ monkeypatch.delattr(ctypes, "windll", raising=False)++ config.hide_if_dotted(tmp_path / ".data") # no AttributeError+++def test_creating_a_directory_hides_the_dotted_parent_it_had_to_make(windows, tmp_path):+ """`proofs/.meta/<name>.assets` is a visible leaf under a hidden parent, and+ `parents=True` creates both. Hiding only the leaf hides nothing."""+ leaf = tmp_path / "proofs" / layout.META_DIR / "harbour.assets"++ ensure_dir(leaf)++ assert windows.attributes == {str(leaf.parent): DIRECTORY | HIDDEN}+++def test_the_walk_up_stops_at_the_first_visible_directory(windows, tmp_path):+ """It climbs to fix what `parents=True` just made, not to redecorate the+ machine the workspace happens to sit on."""+ ensure_dir(tmp_path / ".azimut" / "cases" / "harbour" / "azimut" / "media")++ assert windows.attributes == {}+++def test_a_case_is_born_with_every_internal_directory_hidden(windows, tmp_workspace):+ case = Case.create("Harbour survey")++ for directory in layout.hidden_dirs(case.path):+ if directory.is_dir():+ assert windows.attributes.get(str(directory)) == DIRECTORY | HIDDEN, directory+++def test_opening_a_case_puts_the_attribute_back(tmp_workspace, monkeypatch):+ """The case that arrives from somewhere else: a workspace copied between+ machines carries no attributes at all, and nothing else would ever restore+ them — unlike `.azimut`, which `ensure_workspace` re-hides at every start."""+ case = Case.create("Harbour survey")+ offered: list[Path] = []+ monkeypatch.setattr(config, "hide_if_dotted", offered.append)++ Case.open(case.id)++ assert set(layout.hidden_dirs(case.path)) <= set(offered)+++def test_the_hidden_list_holds_every_dot_directory_a_case_is_born_with(tmp_workspace):+ """The sweep is a written-down list, so what it misses stays visible for+ good. This is what fails the day a directory joins the layout."""+ case = Case.create("Harbour survey")+ born = {+ path+ for path in layout.tool_root(case.path).rglob("*")+ if path.is_dir() and path.name.startswith(".")+ }++ assert born <= set(layout.hidden_dirs(case.path))+++def test_saving_a_proof_asset_hides_the_folder_it_lands_in(windows, tmp_workspace):+ """The one writer that reaches a hidden directory through a visible child."""+ case = Case.create("Harbour survey")+ folder = case.resolve_inside(layout.proof_assets_rel("harbour"))+ windows.attributes.clear()++ proofs._write_assets(folder, {"a1b2c3d4e5f60718.webp": b"pretend"}, {"a1b2c3d4e5f60718.webp"})++ assert windows.attributes.get(str(folder.parent)) == DIRECTORY | HIDDENdiff --git a/tests/test_workspace_move.py b/tests/test_workspace_move.pyindex 0ac4ba1..af941c7 100644--- a/tests/test_workspace_move.py+++ b/tests/test_workspace_move.py@@ -370,6 +370,23 @@ def test_a_move_carries_the_empty_directories_a_case_is_born_with(workspace, tmpassert (target / ".azimut" / "bundles").is_dir()+def test_a_move_rebuilds_the_hidden_directories_hidden(workspace, tmp_path, monkeypatch):+ """A copy is new directories, and a new directory carries none of the+ attributes the original had. On Windows that put every case's internals on+ show in the new folder, which is how the whole thing was noticed."""+ from azimut import layout++ Case.create("Harbour survey")+ hidden = []+ monkeypatch.setattr(config, "hide_if_dotted", hidden.append)++ workspacemove.start(str(tmp_path / "elsewhere"))+ _finish()++ staged = {path for path in hidden if "harbour-survey" in path.parts}+ assert {path.name for path in staged} >= {layout.DATA_DIR, layout.INSPECT_DIR}++def test_a_move_carries_scratch_cases(workspace, tmp_path):Case.create("One shot", scratch=True)