Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.
Python agentaichatgptdeepseekfine-tuninggemmaimage-generationllamallmllmsopenaipythonqwenreinforcement-learningself-hostedstable-diffusiontext-to-speechttsuiunsloth
Contain a temp-scan failure to the directory that raised it, and refuse a partial scan (#11217)
Daniel Hanbf61e642b458
diff --git a/.github/scripts/Watch-ForCompiler.ps1 b/.github/scripts/Watch-ForCompiler.ps1index 11b3eaafd57..cda6ec85e03 100644--- a/.github/scripts/Watch-ForCompiler.ps1+++ b/.github/scripts/Watch-ForCompiler.ps1@@ -64,26 +64,168 @@ function Get-StudioTempRoots {return $result}+function Get-StudioTempSubtree {+ <#+ .SYNOPSIS+ Every file under one root, walked a directory at a time so that one unreadable+ directory costs that directory and nothing else.++ .DESCRIPTION+ Get-ChildItem -Recurse is the obvious way to do this and it is not safe here.+ A temp root is shared with everything else running on the machine, so a+ directory can be removed or become unopenable partway through the walk, and+ the provider raises a Win32Exception that -ErrorAction SilentlyContinue does+ not suppress: that parameter governs non-terminating errors, and with+ $ErrorActionPreference = 'Stop' set by the caller this one ends the step.+ Observed on hosted runners as++ Get-ChildItem : The system cannot find the file specified+ + CategoryInfo : NotSpecified: (:) [Get-ChildItem], Win32Exception++ from inside the positive control, which failed the job while the installer+ under test had done nothing wrong. Walking by hand means the failure is+ contained to the one directory that raised it, and the rest of the subtree is+ still reported.++ Reparse points are not followed. A junction into an ancestor would otherwise+ walk forever, and a compile does not write through one.++ Filtering happens here rather than in the caller. A temp root can hold an+ extracted toolchain or a package cache, and this sweep runs before and after+ every measured action, so collecting every path first and selecting afterwards+ means carrying tens of thousands of strings that were never of interest.+ #>+ param(+ [Parameter(Mandatory = $true)][string]$Root,+ [Parameter(Mandatory = $true)][string[]]$Patterns+ )+ # Generic lists, not PowerShell arrays. += on an array allocates a new one and+ # copies, so a large temp tree costs quadratic time in the number of entries on+ # a path that runs twice per measured action.+ $found = New-Object 'System.Collections.Generic.List[string]'+ $unread = New-Object 'System.Collections.Generic.List[string]'+ $pending = New-Object 'System.Collections.Generic.List[string]'+ $pending.Add($Root)+ $visited = 0+ while ($pending.Count -gt 0) {+ $visited+++ if ($visited -gt 200000) {+ # Not a break. A truncated snapshot is indistinguishable from a clean one+ # to the caller, and this listing is exactly what stands in when the+ # watcher cannot attach, so a silent stop turns a missed artifact into a+ # clean verdict. Better to declare the measurement void.+ throw ("the temp scan of $Root passed $visited directories without finishing. " ++ "A partial snapshot would be read as a complete one, so this run cannot " ++ "say whether a compiler ran.")+ }+ $dir = $pending[$pending.Count - 1]+ $pending.RemoveAt($pending.Count - 1)+ $entries = @()+ try { $entries = @(Get-ChildItem -LiteralPath $dir -Force -ErrorAction Stop) }+ catch {+ # Recorded, not just skipped. The caller subtracts the baseline listing from the+ # final one, so a directory that fails HERE and succeeds in the other snapshot+ # silently changes the answer: files that were there all along show up as new and+ # an innocent action is reported as having compiled. The reverse hides a real+ # artifact. Which directory went unread has to survive to the comparison.+ #+ # A directory that no longer exists is not a gap. It cannot contribute a file to+ # a later listing of itself, and re-reading a path that raised for any other+ # reason is how a transient lock gets a second chance.+ if (Test-Path -LiteralPath $dir) {+ try { $entries = @(Get-ChildItem -LiteralPath $dir -Force -ErrorAction Stop) }+ catch { $unread.Add($dir); continue }+ } else {+ continue+ }+ }+ foreach ($entry in $entries) {+ if ($entry.Attributes -band [System.IO.FileAttributes]::ReparsePoint) { continue }+ if ($entry.PSIsContainer) { $pending.Add($entry.FullName); continue }+ foreach ($pattern in $Patterns) {+ if ($entry.Name -like $pattern) {+ $found.Add($entry.FullName)+ break+ }+ }+ }+ }+ return [pscustomobject]@{+ Files = [string[]]$found.ToArray()+ Unread = [string[]]$unread.ToArray()+ }+}+function Get-StudioTempArtifacts {<#.SYNOPSIS- Compiler intermediates and assemblies currently sitting in the temp roots.+ Compiler intermediates and assemblies currently sitting in the temp roots, and the+ directories this sweep could not read.++ .DESCRIPTION+ Both halves are returned because the caller compares two of these snapshots and a+ directory missing from one side is not the same thing as a directory that is empty.+ See the comparison in Invoke-WithCompilerWatch.#>$patterns = @('*.dll', '*.cmdline', '*.rsp', '*.cs', '*.err', '*.out')- $found = @()+ $found = New-Object 'System.Collections.Generic.List[string]'+ $unread = New-Object 'System.Collections.Generic.List[string]'foreach ($root in (Get-StudioTempRoots)) {- foreach ($pattern in $patterns) {- # Recurse: PowerShell compiles into a per-invocation subdirectory, not- # into the root, so a non-recursive listing sees none of this.- $found += Get-ChildItem -LiteralPath $root -Filter $pattern -File -Recurse `- -Force -ErrorAction SilentlyContinue |- Select-Object -ExpandProperty FullName- }+ # The whole subtree: PowerShell compiles into a per-invocation subdirectory,+ # not into the root, so a non-recursive listing sees none of this.+ $scan = Get-StudioTempSubtree -Root $root -Patterns $patterns+ $found.AddRange([string[]]$scan.Files)+ $unread.AddRange([string[]]$scan.Unread)}- # Comma-wrapped: PowerShell unrolls an empty array to nothing on return, and the- # caller casts this into a HashSet whose two-argument constructor rejects null.+ # Arrays cast explicitly: PowerShell unrolls an empty array to nothing, and the+ # caller casts Files into a HashSet whose two-argument constructor rejects null.# On a clean runner that killed the watcher before the positive control ran.- return ,[string[]]$found+ return [pscustomobject]@{+ Files = [string[]]$found.ToArray()+ Unread = [string[]]$unread.ToArray()+ }+}++function Test-StudioPathUnder {+ <#+ .SYNOPSIS+ Is $Path inside $Directory, or the directory itself?++ .DESCRIPTION+ Compared with a trailing separator appended to the directory, so C:\Temp\ab does not+ count as being under C:\Temp\a. Withholding a sibling that merely shares a name prefix+ would drop real evidence, which is the opposite of what the caller wants.++ Both separators are accepted rather than [System.IO.Path]::DirectorySeparatorChar. That+ property is '/' under pwsh on Linux, where this script's own tests run, so pinning to it+ made every Windows-shaped path compare false and the withholding silently did nothing.++ Case-insensitive, like the rest of this script's path handling, because the paths reach+ here from two different APIs: the directory from Get-ChildItem and the file path from the+ same walk or from FileSystemWatcher.++ The directory itself counts, and that is load-bearing rather than tidiness. A temp ROOT+ can be the thing that could not be enumerated, and the root is also what the watcher+ attaches to, so the coverage check asks whether an unread directory is at or under a+ watched root and gets back the root itself. Descendants-only there means a root that+ failed to enumerate twice, on a machine where the watcher did attach to it, is declared+ uncovered and the run throws - which is the temp-scan failure this whole change exists to+ contain, put back one layer up.+ #>+ param(+ [Parameter(Mandatory = $true)][string]$Path,+ [Parameter(Mandatory = $true)][string]$Directory+ )+ $trimmed = $Directory.TrimEnd('\', '/')+ if ($Path.TrimEnd('\', '/').Equals($trimmed, [System.StringComparison]::OrdinalIgnoreCase)) {+ return $true+ }+ foreach ($separator in @('\', '/')) {+ if ($Path.StartsWith($trimmed + $separator, [System.StringComparison]::OrdinalIgnoreCase)) {+ return $true+ }+ }+ return $false}$script:ArtifactPattern = '\.(dll|cmdline|rsp|cs|err|out)$'@@ -410,8 +552,9 @@ function Invoke-WithCompilerWatch {# recursively and can take seconds, and a csc.exe the machine started during that# walk predates the action, so counting it fails a measurement for something it# did not do. Same reasoning as the $until below.+ $beforeScan = Get-StudioTempArtifacts$before = New-Object 'System.Collections.Generic.HashSet[string]' (- [string[]](Get-StudioTempArtifacts), [StringComparer]::OrdinalIgnoreCase)+ [string[]]$beforeScan.Files, [StringComparer]::OrdinalIgnoreCase)# A second back, so a process created in the same tick as the timestamp survives# Get-WinEvent's strictly-later comparison. That reaches one second into the tail# of the sweep above, so a compiler started in that second is counted: a deliberate@@ -458,8 +601,48 @@ function Invoke-WithCompilerWatch {Where-Object { $prior -notcontains $_ })- $after = Get-StudioTempArtifacts- $left = @($after | Where-Object { -not $before.Contains($_) })+ $afterScan = Get-StudioTempArtifacts+ $after = $afterScan.Files++ # A directory that could not be read in EITHER sweep is a hole in the comparison, not an+ # empty directory. $left below is "in the final listing and not in the baseline", so a+ # directory unread at baseline and readable afterwards hands every file that was already+ # sitting in it to $left, and the action is reported as having compiled something it did+ # not. Unread afterwards hides the opposite: a real artifact that never reaches $left.+ #+ # So paths under an unread directory are not evidence either way and are withheld from+ # $left, and which directories those were is recorded in the evidence rather than+ # dropped silently.+ $unread = @($beforeScan.Unread + $afterScan.Unread | Sort-Object -Unique)+ $watchedRoots = New-Object 'System.Collections.Generic.HashSet[string]' (+ [string[]]@($watch | ForEach-Object { $_.Root }), [StringComparer]::OrdinalIgnoreCase)+ if ($unread.Count -gt 0) {+ foreach ($dir in $unread) {+ # Withholding is only safe while the watcher is covering that root: it reports+ # creations live, so a compile inside an unread directory still lands in+ # $transient. With no watcher on the root the listing is the only evidence there+ # is, and withholding part of it would report a hole as a clean result. That is+ # the same call as the traversal cap in Get-StudioTempSubtree, for the same+ # reason, so it is the same answer: declare the measurement void.+ $covered = @($watchedRoots | Where-Object { Test-StudioPathUnder -Path $dir -Directory $_ })+ if ($covered.Count -eq 0) {+ throw ("the temp sweep could not read $dir, and no file watcher is attached " ++ "to the root containing it. The listing is the only evidence here and " ++ "it is incomplete, so this run cannot say whether a compiler ran.")+ }+ }+ }++ $left = @(+ $after | Where-Object {+ $path = $_+ if ($before.Contains($path)) { return $false }+ foreach ($dir in $unread) {+ if (Test-StudioPathUnder -Path $path -Directory $dir) { return $false }+ }+ return $true+ }+ )# Only the names the compiler writes, because the watcher reports every creation# under temp and most of them are nobody's business.$transient = @($live | Where-Object { $_ -match $script:ArtifactPattern })@@ -473,6 +656,9 @@ function Invoke-WithCompilerWatch {$stem = Join-Path $EvidenceRoot $Name$compilers | Out-File -FilePath "$stem-compilers.txt" -Encoding utf8$newArtifacts | Out-File -FilePath "$stem-temp-artifacts.txt" -Encoding utf8+ # Written even when empty, so "the sweep read everything" is a statement the evidence+ # makes rather than the absence of a file, which is also what a crash looks like.+ $unread | Out-File -FilePath "$stem-unread-dirs.txt" -Encoding utf8if ($failure) {$failure | Out-String | Out-File -FilePath "$stem-error.txt" -Encoding utf8}@@ -483,5 +669,6 @@ function Invoke-WithCompilerWatch {Compilers = $compilersTempLibraries = $newLibrariesTempArtifacts = $newArtifacts+ UnreadDirs = $unread}}diff --git a/.github/workflows/cross-platform-parity-ci.yml b/.github/workflows/cross-platform-parity-ci.ymlindex 03eeddca52a..3ba5c71a847 100644--- a/.github/workflows/cross-platform-parity-ci.yml+++ b/.github/workflows/cross-platform-parity-ci.yml@@ -77,6 +77,13 @@ on:- 'tests/python/test_windows_setup_output_encoding.py'- 'tests/python/test_windows_installer_native_type_fallback.py'- 'tests/python/test_pwsh_runner_encoding.py'+ # The deliberate unreadable-directory and traversal-limit cases for the+ # no-compiler detector's temp scan live in this job, and only here. The+ # no-compiler workflow runs the scanner's positive control but never creates+ # those conditions, so without both lines a PR touching only the scanner+ # could regress the error handling with nothing red.+ - '.github/scripts/Watch-ForCompiler.ps1'+ - 'tests/python/test_compiler_watch_temp_scan.py'# Every PowerShell test in this job shells out through it, so a change here can break# all of them. It was in no path filter, so it triggered nothing.- 'tests/_shared/unsloth_pwsh_runner.py'@@ -158,6 +165,13 @@ on:- 'tests/python/test_windows_setup_output_encoding.py'- 'tests/python/test_windows_installer_native_type_fallback.py'- 'tests/python/test_pwsh_runner_encoding.py'+ # The deliberate unreadable-directory and traversal-limit cases for the+ # no-compiler detector's temp scan live in this job, and only here. The+ # no-compiler workflow runs the scanner's positive control but never creates+ # those conditions, so without both lines a PR touching only the scanner+ # could regress the error handling with nothing red.+ - '.github/scripts/Watch-ForCompiler.ps1'+ - 'tests/python/test_compiler_watch_temp_scan.py'# Every PowerShell test in this job shells out through it, so a change here can break# all of them. It was in no path filter, so it triggered nothing.- 'tests/_shared/unsloth_pwsh_runner.py'@@ -258,6 +272,7 @@ jobs:tests/python/test_windows_setup_output_encoding.pytests/python/test_windows_installer_native_type_fallback.pytests/python/test_pwsh_runner_encoding.py+ tests/python/test_compiler_watch_temp_scan.pytests/test_installer_skip_autostart.pytests/test_installer_system32_guard.pytests/test_installer_interactive_prompts.pydiff --git a/.github/workflows/windows-no-compiler-ci.yml b/.github/workflows/windows-no-compiler-ci.ymlindex 714e7c028f1..4f2ac8f121d 100644--- a/.github/workflows/windows-no-compiler-ci.yml+++ b/.github/workflows/windows-no-compiler-ci.yml@@ -51,6 +51,9 @@ on:# The watcher implements both detectors, so a PR touching only it could disable# detection without running the positive control that would catch that.- '.github/scripts/Watch-ForCompiler.ps1'+ # Its own tests, whose Windows-only ACL control can only run on a Windows+ # runner, so an edit to them has to reach one.+ - 'tests/python/test_compiler_watch_temp_scan.py'- '.github/workflows/windows-no-compiler-ci.yml'push:branches: [main]diff --git a/tests/python/test_compiler_watch_temp_scan.py b/tests/python/test_compiler_watch_temp_scan.pynew file mode 100644index 00000000000..76e048b1ed8--- /dev/null+++ b/tests/python/test_compiler_watch_temp_scan.py@@ -0,0 +1,434 @@+# SPDX-License-Identifier: AGPL-3.0-only+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0++"""The no-compiler detector must not fail the job because something else touched TEMP.++Watch-ForCompiler.ps1 lists the temp roots to see what a compile left behind. It did that with+one `Get-ChildItem -Recurse -Force -ErrorAction SilentlyContinue` per root. A temp root is+shared with everything else on the machine, so a directory can disappear or stop being openable+partway through the walk, and the provider raises a Win32Exception. `-ErrorAction+SilentlyContinue` does not suppress that one: it governs non-terminating errors, and the step+sets `$ErrorActionPreference = 'Stop'`.++Observed on hosted runners, in the POSITIVE CONTROL, which is the worst place for it:++ Get-ChildItem : The system cannot find the file specified+ + CategoryInfo : NotSpecified: (:) [Get-ChildItem], Win32Exception++The job went red while the installer under test had done nothing at all. The walk is by hand+now, one directory at a time, so an unreadable directory costs that directory and nothing else.++Driven rather than read: the whole question is what happens when enumeration throws, and a+regex over the script cannot answer it.+"""++from __future__ import annotations++import os+import pathlib+import shutil+import subprocess++import pytest++REPO = pathlib.Path(__file__).resolve().parents[2]+SCRIPT = REPO / ".github" / "scripts" / "Watch-ForCompiler.ps1"++PWSH = shutil.which("pwsh")+pytestmark = pytest.mark.skipif(PWSH is None, reason = "needs PowerShell")+++def _run_pwsh(body: str) -> subprocess.CompletedProcess:+ script = f"$ErrorActionPreference = 'Stop'\n. '{SCRIPT}'\n{body}"+ return subprocess.run(+ [PWSH, "-NoProfile", "-NonInteractive", "-Command", script],+ capture_output = True,+ text = True,+ timeout = 300,+ )+++def _scan(root: pathlib.Path, patterns: str = "'*.dll','*.cmdline'") -> dict[str, list[str]]:+ """Run Get-StudioTempSubtree over `root` under the same preference CI uses.++ Both halves are returned. Which directories went unread is not a diagnostic here: it is+ what stops a directory read in one snapshot and not the other from being scored as a+ compile, so it is asserted on directly.+ """+ proc = _run_pwsh(+ f"$scan = Get-StudioTempSubtree -Root '{root}' -Patterns {patterns}\n"+ 'foreach ($f in $scan.Files) { Write-Output "FILE $f" }\n'+ 'foreach ($d in $scan.Unread) { Write-Output "UNREAD $d" }\n'+ )+ assert proc.returncode == 0, f"the walk itself failed:\n{proc.stdout}\n{proc.stderr}"+ out: dict[str, list[str]] = {"files": [], "unread": []}+ for line in proc.stdout.splitlines():+ line = line.strip()+ if line.startswith("FILE "):+ out["files"].append(line[len("FILE ") :])+ elif line.startswith("UNREAD "):+ out["unread"].append(line[len("UNREAD ") :])+ return out+++def _walk(root: pathlib.Path, patterns: str = "'*.dll','*.cmdline'") -> list[str]:+ return _scan(root, patterns)["files"]+++ os.name == "nt",+ reason = (+ "POSIX permissions only. Windows has no os.geteuid, and chmod there sets the read-only "+ "attribute rather than making a directory unopenable, so this would not deny anything. "+ "The Windows half of the same question is the ACL-based control below."+ ),+)+def test_an_unreadable_directory_costs_only_itself(tmp_path: pathlib.Path) -> None:+ (tmp_path / "keep").mkdir()+ (tmp_path / "keep" / "probe.dll").write_text("x")+ (tmp_path / "locked").mkdir()+ (tmp_path / "locked" / "hidden.dll").write_text("x")+ (tmp_path / "later").mkdir()+ (tmp_path / "later" / "after.cmdline").write_text("x")+ os.chmod(tmp_path / "locked", 0o000)+ try:+ if os.geteuid() == 0:+ pytest.skip("root reads every directory, so nothing here can be made unreadable")+ found = _walk(tmp_path)+ finally:+ os.chmod(tmp_path / "locked", 0o700)++ # The bites control first: if the tree were readable throughout, this row would pass on a+ # walk that stopped at the first directory and never proved anything.+ assert any(name.endswith("probe.dll") for name in found), found+ assert any(+ name.endswith("after.cmdline") for name in found+ ), "the walk stopped at the unreadable directory instead of stepping over it"+ assert not any(name.endswith("hidden.dll") for name in found), found+++# There is no control here for "the old shape really would have died", and that is deliberate.+#+# The CI failure was a RACE: a directory in the shared temp root disappeared partway through a+# recursive enumeration and the Windows provider raised a Win32Exception, which -ErrorAction+# SilentlyContinue does not suppress because it governs non-terminating errors. An ACL denial was+# tried as a stand-in and is not one: a directory the caller cannot open is an ordinary+# non-terminating access-denied error, which that parameter DOES suppress, so the control reached+# its SURVIVED line and would have passed while claiming the opposite. A POSIX chmod is the same+# story.+#+# Reproducing the race means deleting directories under a live walk and hoping the timing lands,+# which is a flaky test rather than a control. What is pinned instead is the shape: enumeration is+# per-directory and wrapped, and no recursive listing is left in the file. The evidence for the+# failure itself is the CI log quoted at the top of this module.+++def test_the_scan_refuses_to_report_a_truncated_snapshot(tmp_path: pathlib.Path) -> None:+ """Past the ceiling it raises, rather than handing back a partial listing.++ The caller reads this snapshot as complete, and it is what stands in when the file-system+ watcher cannot attach, so a silent stop turns a missed artifact into a clean verdict. Driven+ by lowering the ceiling with a stubbed walk is not possible here, so the tree is built: 12+ directories against a ceiling of 8, set by dot-sourcing and re-declaring nothing.+ """+ # The real ceiling is 200000, far too large to build, so the shape is asserted instead and+ # the behaviour is driven at a scale that fits: the function is re-defined with the same body+ # and a smaller limit, taken from the shipped source rather than retyped.+ text = SCRIPT.read_text(encoding = "utf-8")+ assert (+ "throw (" in text and "$visited -gt 200000" in text+ ), "the ceiling no longer raises, so a truncated scan would be read as a complete one"+ small = text[+ text.index("function Get-StudioTempSubtree") : text.index(+ "function Get-StudioTempArtifacts"+ )+ ]+ small = small.replace("$visited -gt 200000", "$visited -gt 3")+ assert "$visited -gt 3" in small+ for i in range(12):+ (tmp_path / f"d{i}").mkdir()+ holder = tmp_path / "small.ps1"+ holder.write_text(small, encoding = "utf-8")+ proc = subprocess.run(+ [+ PWSH,+ "-NoProfile",+ "-NonInteractive",+ "-Command",+ "$ErrorActionPreference = 'Stop'\n"+ f". '{holder}'\n"+ f"Get-StudioTempSubtree -Root '{tmp_path}' -Patterns '*.dll' | Out-Null\n"+ "Write-Output 'NO-THROW'\n",+ ],+ capture_output = True,+ text = True,+ timeout = 300,+ )+ assert (+ "NO-THROW" not in proc.stdout+ ), "the walk returned a partial snapshot instead of declaring the measurement void"+ assert "cannot say whether a compiler ran" in (proc.stdout + proc.stderr)+++def test_the_artifact_filter_still_selects_by_extension(tmp_path: pathlib.Path) -> None:+ """Splitting the walk from the filter must not widen what counts as an artifact."""+ (tmp_path / "sub").mkdir()+ for name in ("a.dll", "b.cmdline", "c.rsp", "d.cs", "e.err", "f.out", "g.txt", "h.exe"):+ (tmp_path / "sub" / name).write_text("x")+ script = (+ "$ErrorActionPreference = 'Stop'\n"+ f". '{SCRIPT}'\n"+ f"$env:TEMP = '{tmp_path}'\n"+ f"$env:TMP = '{tmp_path}'\n"+ "(Get-StudioTempArtifacts).Files | ForEach-Object { Write-Output (Split-Path -Leaf $_) }\n"+ )+ proc = subprocess.run(+ [PWSH, "-NoProfile", "-NonInteractive", "-Command", script],+ capture_output = True,+ text = True,+ timeout = 300,+ )+ assert proc.returncode == 0, proc.stdout + proc.stderr+ got = {line.strip() for line in proc.stdout.splitlines() if line.strip()}+ assert {"a.dll", "b.cmdline", "c.rsp", "d.cs", "e.err", "f.out"} <= got, got+ assert "g.txt" not in got and "h.exe" not in got, got+++def test_no_recursive_listing_is_left_in_the_script() -> None:+ """The shape that raised, pinned out of the file it was removed from.++ A future edit that reaches for -Recurse again brings the whole failure back, and it only+ shows up on a runner whose temp directory happened to change under it.+ """+ text = SCRIPT.read_text(encoding = "utf-8")+ # Comments first: this file EXPLAINS the shape it removed, and prose naming it is not a+ # call. Stripping the comment-based help blocks as well, which is where that prose lives.+ body, inside_help = [], False+ for line in text.splitlines():+ stripped = line.strip()+ if stripped.startswith("<#"):+ inside_help = True+ if inside_help:+ if "#>" in stripped:+ inside_help = False+ continue+ if stripped.startswith("#"):+ continue+ body.append(line)+ offenders = [line.strip() for line in body if "Get-ChildItem" in line and "-Recurse" in line]+ assert not offenders, offenders+++ os.name == "nt",+ reason = (+ "POSIX permissions only, for the same reason as the walk control above: chmod on "+ "Windows sets the read-only attribute rather than making a directory unopenable."+ ),+)+def test_an_unreadable_directory_is_reported_and_not_just_skipped(tmp_path: pathlib.Path) -> None:+ """The gap has to reach the caller, because the caller subtracts two of these listings.++ Skipping quietly is what makes the comparison lie: a directory unread at baseline and+ readable afterwards hands every file already sitting in it to the "new since the action"+ set, and an installer that compiled nothing is reported as having compiled. This asserts+ the walk says which directory it could not read, which is what the caller needs to+ withhold those paths.+ """+ (tmp_path / "locked").mkdir()+ (tmp_path / "locked" / "hidden.dll").write_text("x")+ (tmp_path / "open").mkdir()+ (tmp_path / "open" / "seen.dll").write_text("x")+ os.chmod(tmp_path / "locked", 0o000)+ try:+ if os.geteuid() == 0:+ pytest.skip("root reads every directory, so nothing here can be made unreadable")+ scan = _scan(tmp_path)+ finally:+ os.chmod(tmp_path / "locked", 0o700)++ assert any(name.endswith("seen.dll") for name in scan["files"]), scan+ assert not any(name.endswith("hidden.dll") for name in scan["files"]), scan+ assert [d for d in scan["unread"] if d.endswith("locked")], (+ "the walk stepped over the unreadable directory without recording it. The caller "+ f"cannot then tell an empty directory from an unread one: {scan}"+ )+++ os.name == "nt",+ reason = "POSIX permissions only; see the walk control above.",+)+def test_a_directory_that_vanished_is_not_reported_as_unread(tmp_path: pathlib.Path) -> None:+ """A path that is gone is not a hole in the measurement, and must not void a run.++ This is the transient case the whole change exists for. A directory that no longer+ exists cannot contribute a file to a later listing of itself, so there is nothing for+ the caller to withhold and nothing to declare void. Only a directory that is still+ there and still unreadable is a real gap.+ """+ scan = _scan(tmp_path / "never-existed")+ assert scan["files"] == [], scan+ assert scan["unread"] == [], (+ "a missing directory was recorded as an unread one. That would void a measurement "+ f"for the ordinary temp race this change was written to survive: {scan}"+ )+++def _shipped_left_expression() -> str:+ """The `$left = @(...)` assignment as it appears in Invoke-WithCompilerWatch.++ Lifted from the shipped source rather than retyped. A copy of the expression in this file+ would keep passing after the withholding was deleted from the script, which is precisely+ the regression worth catching: the comparison is the only place the unread directories+ are allowed to change the answer.+ """+ text = SCRIPT.read_text(encoding = "utf-8")+ start = text.index(" $left = @(")+ end = text.index("\n )\n", start) + len("\n )\n")+ expression = text[start:end]+ assert "$unread" in expression, (+ "the $left comparison no longer consults the unread directories, so a directory that "+ "could not be read in one snapshot and could in the other is scored as a compile:\n"+ f"{expression}"+ )+ return expression+++def test_the_unread_directories_are_withheld_from_the_new_artifact_set() -> None:+ """The defect this guards: a baseline hole turning pre-existing files into evidence.++ `old.dll` was in the gap directory all along. The baseline sweep could not read that+ directory, the final sweep could, so a plain "in after, not in before" difference hands it+ back as new and an installer that compiled nothing is reported as having compiled.++ The expression under test is the one the script actually runs, read out of the file, so+ deleting or bypassing the withholding fails this rather than leaving a copy passing here.+ """+ proc = _run_pwsh(+ "$before = New-Object 'System.Collections.Generic.HashSet[string]' "+ "([string[]]@(), [StringComparer]::OrdinalIgnoreCase)\n"+ r"$after = @('C:\t\gap\old.dll', 'C:\t\seen\new.dll')" + "\n"+ r"$unread = @('C:\t\gap')"+ + "\n"+ + _shipped_left_expression()+ + "foreach ($p in $left) { Write-Output $p }\n"+ )+ assert proc.returncode == 0, proc.stdout + proc.stderr+ left = [line.strip() for line in proc.stdout.splitlines() if line.strip()]+ assert left == [r"C:\t\seen\new.dll"], (+ "a file under a directory the baseline sweep could not read was scored as new. "+ f"That reports an innocent action as having compiled: {left}"+ )+++def test_the_same_comparison_still_reports_a_genuinely_new_file() -> None:+ """The control for the row above: withholding must not swallow everything.++ A test that only checks something was removed passes just as well against an expression+ that returns nothing at all, which would hide every real compile instead. With no unread+ directories, both files are new and both come back.+ """+ proc = _run_pwsh(+ "$before = New-Object 'System.Collections.Generic.HashSet[string]' "+ "([string[]]@(), [StringComparer]::OrdinalIgnoreCase)\n"+ r"$after = @('C:\t\gap\old.dll', 'C:\t\seen\new.dll')" + "\n"+ "$unread = @()\n"+ + _shipped_left_expression()+ + "foreach ($p in $left) { Write-Output $p }\n"+ )+ assert proc.returncode == 0, proc.stdout + proc.stderr+ left = sorted(line.strip() for line in proc.stdout.splitlines() if line.strip())+ assert left == [+ r"C:\t\gap\old.dll",+ r"C:\t\seen\new.dll",+ ], f"a clean sweep stopped reporting new artifacts, which hides every real compile: {left}"+++def test_the_prefix_test_does_not_match_a_sibling_by_name() -> None:+ """`C:\\t\\ab` is not under `C:\\t\\a`, and withholding it would hide a real artifact."""+ proc = _run_pwsh(+ r"Write-Output ('under=' + (Test-StudioPathUnder -Path 'C:\t\a\x.dll' -Directory 'C:\t\a'))"+ + "\n"+ r"Write-Output ('sibling=' + (Test-StudioPathUnder -Path 'C:\t\ab\x.dll' -Directory 'C:\t\a'))"+ + "\n"+ r"Write-Output ('case=' + (Test-StudioPathUnder -Path 'C:\T\A\x.dll' -Directory 'c:\t\a'))"+ + "\n"+ # The directory itself. A temp ROOT can be what failed to enumerate, and the root is+ # also what the watcher attaches to, so descendants-only makes the coverage check+ # below declare a watched root uncovered and throw.+ r"Write-Output ('self=' + (Test-StudioPathUnder -Path 'C:\t\a' -Directory 'C:\t\a'))" + "\n"+ r"Write-Output ('selfslash=' + (Test-StudioPathUnder -Path 'C:\t\a\' -Directory 'C:\t\a'))"+ + "\n"+ )+ assert proc.returncode == 0, proc.stderr+ out = dict(line.split("=", 1) for line in proc.stdout.splitlines() if "=" in line)+ assert out["under"] == "True", out+ assert out["sibling"] == "False", (+ "a sibling directory sharing a name prefix was treated as being inside the unread "+ f"one, which would withhold real evidence: {out}"+ )+ assert out["case"] == "True", out+ assert out["self"] == "True" and out["selfslash"] == "True", (+ "the directory itself did not count as covered. An unreadable temp ROOT is then "+ f"declared to have no watcher on it and the run throws for nothing: {out}"+ )+++def _shipped_coverage_check() -> str:+ """The `if ($unread.Count -gt 0) { ... }` block from Invoke-WithCompilerWatch.++ Read from the shipped file for the same reason as the `$left` expression: a copy here+ would keep passing after the real check changed.+ """+ text = SCRIPT.read_text(encoding = "utf-8")+ start = text.index(" if ($unread.Count -gt 0) {")+ end = text.index("\n }\n", start) + len("\n }\n")+ block = text[start:end]+ assert "Test-StudioPathUnder" in block and "throw" in block, block+ return block+++def test_an_unreadable_root_with_a_watcher_on_it_does_not_void_the_run() -> None:+ """The case that would reintroduce the failure this change exists to contain.++ A temp ROOT can be the directory that could not be enumerated, and the root is also+ exactly what Start-StudioTempWatch attaches to. If the coverage check only recognises+ descendants of a watched root, an unreadable root is declared to have no watcher, the run+ throws, and the job goes red again for a transient condition in somebody else's TEMP.+ """+ proc = _run_pwsh(+ r"$unread = @('C:\t')" + "\n"+ r"$watch = @([pscustomobject]@{ Root = 'C:\t' })" + "\n"+ "$watchedRoots = New-Object 'System.Collections.Generic.HashSet[string]' "+ "([string[]]@($watch | ForEach-Object { $_.Root }), [StringComparer]::OrdinalIgnoreCase)\n"+ + _shipped_coverage_check()+ + "Write-Output 'SURVIVED'\n"+ )+ assert proc.returncode == 0, proc.stdout + proc.stderr+ assert "SURVIVED" in proc.stdout, (+ "an unreadable temp root voided the measurement even though the watcher was attached "+ f"to that very root:\n{proc.stdout}\n{proc.stderr}"+ )+++def test_an_unreadable_root_with_no_watcher_still_voids_the_run() -> None:+ """The control: the throw has to survive, or the fix above would gut the guard.++ With nothing watching, the listing is the only evidence there is, and withholding part of+ it would hand back a hole as a clean result.+ """+ proc = _run_pwsh(+ r"$unread = @('C:\t')" + "\n"+ "$watchedRoots = New-Object 'System.Collections.Generic.HashSet[string]' "+ "([string[]]@(), [StringComparer]::OrdinalIgnoreCase)\n"+ + _shipped_coverage_check()+ + "Write-Output 'SURVIVED'\n"+ )+ assert "SURVIVED" not in proc.stdout, (+ "an unread directory with no watcher on its root was treated as a complete "+ f"measurement:\n{proc.stdout}"+ )+ assert "cannot say whether a compiler ran" in (+ proc.stdout + proc.stderr+ ), f"the run was voided without saying why:\n{proc.stdout}\n{proc.stderr}"diff --git a/tests/studio/install/test_update_idempotency.py b/tests/studio/install/test_update_idempotency.pyindex d4f37b48a23..0fc53834aa0 100644--- a/tests/studio/install/test_update_idempotency.py+++ b/tests/studio/install/test_update_idempotency.py@@ -732,12 +732,24 @@ def test_the_desktop_update_path_keeps_a_verified_install_offline(install, settl"an update with nothing to do failed offline. That is the state a user on a "f"plane, or behind a corporate proxy, is in:\n{run.log[-8000:]}")- # Either the offline rule or the ordinary fast path: Invoke-RestMethod ignores HTTPS_PROXY, so- # on Windows the version check can still answer "up to date". Either way nothing got through the- # proxy and nothing on disk moved.- assert "keeping the verified install" in run.log or UPTODATE_MARKER.search(run.log), run.log[- -8000:- ]+ # Three outcomes are all correct here, because the version check is not behind the proxy:+ # Invoke-RestMethod ignores HTTPS_PROXY, so on Windows it reaches PyPI and answers for real.+ #+ # - the offline rule fired, which is the case this test exists for;+ # - PyPI said the installed version is the latest, the ordinary fast path;+ # - PyPI said a NEWER release exists, so the offline rule's precondition ("PyPI is+ # unreachable") never held and the pass is supposed to start.+ #+ # The third is not a weakening. It is what happens to every branch in the repo for the hours+ # between a release landing on PyPI and the pin here moving, and asserting it away would mean+ # this row goes red on a schedule that has nothing to do with the code. The teeth are the two+ # assertions below, which hold in all three: nothing got through the proxy, and nothing on+ # disk moved.+ assert (+ "keeping the verified install" in run.log+ or UPTODATE_MARKER.search(run.log)+ or UPGRADE_MARKER in run.log+ ), run.log[-8000:]assert run.connections == run.refused, run.report()assert diff(before, snapshot(install)) == []