Skip to content

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4.1, MiniMax-H3, Gemma 4, FLUX and more.

Windows installer: shorten the native-type comments (#10621)

Daniel Han3cb17d7a5418
diff --git a/.github/scripts/Watch-ForCompiler.ps1 b/.github/scripts/Watch-ForCompiler.ps1
index 2a3ea7cfa10..03a8726aa4c 100644
--- a/.github/scripts/Watch-ForCompiler.ps1
+++ b/.github/scripts/Watch-ForCompiler.ps1
@@ -7,23 +7,22 @@ Run a script block and report whether a C# compiler ran, or a DLL landed in a
temporary directory, while it did.
.DESCRIPTION
-Two independent detectors, because either one alone can be defeated by something
-that is not the code under test:
+Two independent detectors, because either alone can be defeated by something that
+is not the code under test:
1. Security log event 4688, process creation. Authoritative about what ran, and
- it sees a compiler spawned through any depth of child process, which is the
- case a text search of our own scripts cannot reach. Needs auditing enabled
- by the caller; the workflow does that and verifies it took.
+ it sees a compiler spawned through any depth of child process, which a text
+ search of our own scripts cannot reach. Needs auditing enabled by the caller;
+ the workflow does that and verifies it took.
2. New *.dll, *.cmdline, *.rsp and *.?.cs files under every temporary directory
- in play. This is the artefact half of the same shape, and it survives the
- case where auditing is silently overridden by a machine policy. csc.exe
- writes the source and the response file next to the assembly, so the
- supporting names are watched too: an assembly written and cleaned up leaves
- the job with nothing to show, and a .cmdline left behind still names it.
+ in play. The artefact half of the same shape, and it survives auditing being
+ silently overridden by machine policy. csc.exe writes the source and the
+ response file next to the assembly, so those names are watched too: an
+ assembly cleaned up leaves nothing to show, and a .cmdline still names it.
-Both are reported. A caller decides what to do with them; the positive control
-requires both to fire, and the real measurement requires neither to.
+Both are reported. The positive control requires both to fire, and the real
+measurement requires neither to.
TEMP is read from the environment rather than assumed, and the machine-wide
C:\Windows\Temp is watched alongside it, because a compile launched from a
@@ -68,9 +67,8 @@ function Get-StudioTempArtifacts {
}
}
# Comma-wrapped: PowerShell unrolls an empty array to nothing on return, and the
- # caller casts this into a HashSet whose two-argument constructor rejects null. On a
- # clean runner with no matching temp files that killed the watcher before it ran even
- # the positive control.
+ # caller casts this into a HashSet whose two-argument constructor rejects null.
+ # On a clean runner that killed the watcher before the positive control ran.
return ,[string[]]$found
}
@@ -79,12 +77,10 @@ function Get-StudioProcessImageName {
.SYNOPSIS
The image a 4688 record says was created, from the record's own field.
.DESCRIPTION
- NewProcessName, read out of the event XML by name rather than by position, since
- the position is a property of the schema and this has to keep meaning the same
- thing if the schema gains a field. Nothing else in the record is consulted: the
- rendered message also contains the command line, so matching the message scores
- `cmd.exe /c echo csc.exe` as a compiler, which is a false report of the one thing
- this job exists to detect.
+ NewProcessName, read out of the event XML by name rather than by position, so a
+ schema that gains a field still means the same thing. Nothing else is consulted:
+ the rendered message also carries the command line, so matching it would score
+ `cmd.exe /c echo csc.exe` as a compiler.
#>
param([Parameter(Mandatory = $true)]$Event)
@@ -102,9 +98,8 @@ function Select-StudioCompilerHits {
.SYNOPSIS
The records among $Events whose created image is a compiler.
.DESCRIPTION
- Separate from the query so it can be exercised without a Security log, which is
- the only way to test the classification anywhere but a Windows runner that has
- just compiled something.
+ Separate from the query so the classification can be exercised without a
+ Security log, which is the only way to test it off a Windows runner.
#>
param([Parameter(Mandatory = $true)][AllowEmptyCollection()][array]$Events)
@@ -112,10 +107,10 @@ function Select-StudioCompilerHits {
foreach ($record in $Events) {
$image = Get-StudioProcessImageName -Event $record
if ([string]::IsNullOrEmpty($image)) { continue }
- # Split explicitly rather than through [System.IO.Path]::GetFileName, which splits on
- # the separators of the HOST it runs on: under a Linux pwsh, which is where the
- # classification is tested, a backslash is an ordinary character and the whole path
- # came back as the leaf. The records are always Windows paths whatever reads them.
+ # Split explicitly, not via [System.IO.Path]::GetFileName, which splits on the
+ # HOST's separators: under the Linux pwsh where this is tested a backslash is an
+ # ordinary character and the whole path came back as the leaf. The records are
+ # always Windows paths whatever reads them.
$leaf = ($image -split '[\\/]')[-1]
foreach ($name in $script:CompilerNames) {
if ($leaf -eq $name) {
@@ -139,14 +134,13 @@ function Get-StudioCompilerEvents {
filtering afterwards by a fixed window, so a slow installer cannot outrun it.
.PARAMETER Until
The instant it ended. Both ends are needed: a runner is a shared machine, and an
- unrelated service starting a compiler after the action would otherwise be scored
- against it.
+ unrelated service starting a compiler after the action would be scored against it.
- This window is the honest bound rather than the process tree the workflow prose
- describes. 4688 carries the creator's pid, but a compile can be several processes
- deep and the intermediate pids have exited by the time this reads the log, so an
- ancestry walk is not reconstructable after the fact. The window is what can be
- measured; the positive control is what proves it measures anything.
+ A window, not the process tree the workflow prose describes. 4688 carries the
+ creator's pid, but a compile can be several processes deep and the intermediate
+ pids have exited by the time this reads the log, so ancestry is not
+ reconstructable after the fact. The positive control proves the window measures
+ anything.
#>
param(
[Parameter(Mandatory = $true)][datetime]$Since,
@@ -155,9 +149,8 @@ function Get-StudioCompilerEvents {
$events = @()
try {
- # Bounded at both ends. Open-ended, a compiler started by something else while the
- # recursive temp scan was still running counted against the action that had
- # already finished.
+ # Bounded at both ends. Open-ended, a compiler started by something else during
+ # the recursive temp scan counted against the action that had already finished.
$events = Get-WinEvent -FilterHashtable @{
LogName = 'Security'
Id = 4688
@@ -165,17 +158,15 @@ function Get-StudioCompilerEvents {
EndTime = $Until
} -ErrorAction Stop
} catch [System.Exception] {
- # No matching events is an exception from Get-WinEvent, not an empty set, and on a
- # clean run that is the expected result. A log this cannot READ throws the same way,
- # and swallowing both made an unreadable Security log indistinguishable from a clean
- # measurement: the job would print "no compiler" having seen nothing at all. The
- # positive control does not cover this, because it runs in an earlier step and says
- # nothing about whether the log was still readable during the two measurements.
+ # No matching events is an exception from Get-WinEvent, not an empty set, and on
+ # a clean run that is expected. A log this cannot READ throws the same way, so
+ # swallowing both would print "no compiler" having seen nothing at all. The
+ # positive control runs in an earlier step and says nothing about whether the log
+ # was readable during the measurements.
#
- # So the two are separated structurally rather than by matching the message text,
- # which is localised: ask the log for any one record at all. If that succeeds the log
- # is readable and the filter genuinely matched nothing; if it fails too, this
- # measurement is void and the caller must not report it as clean.
+ # Separate them structurally rather than by the localised message text: ask the
+ # log for any one record. If that succeeds the filter genuinely matched nothing;
+ # if it fails too, this measurement is void, not clean.
# Bound before the probe below, whose own catch rebinds $_.
$reason = $_.Exception.Message
$readable = $false
@@ -199,9 +190,8 @@ function Invoke-WithCompilerWatch {
Run $Action and return what the two detectors saw while it ran.
.OUTPUTS
A hashtable with Compilers and TempLibraries, each an array of strings, plus
- the exit state of the action. Evidence is written under $EvidenceRoot whether
- the action succeeded or not, since a failed installer is exactly when the
- trace is wanted.
+ the exit state of the action. Evidence is written under $EvidenceRoot even when
+ the action fails, which is exactly when the trace is wanted.
#>
param(
[Parameter(Mandatory = $true)][string]$Name,
@@ -212,39 +202,32 @@ function Invoke-WithCompilerWatch {
New-Item -ItemType Directory -Force -Path $EvidenceRoot | Out-Null
# Baseline first, THEN open the window. The sweep walks every temp root
- # recursively and can take seconds on a runner with a large temp tree, and a
- # csc.exe the machine started during that walk predates the action: counting it
- # fails the positive control or the installer measurement for something neither
- # one did. Same reasoning as the $until below, which is taken before the second
- # sweep for the same reason.
+ # recursively and can take seconds, and a csc.exe the machine started during that
+ # walk predates the action, so counting it fails a measurement for something it
+ # did not do. Same reasoning as the $until below.
$before = New-Object 'System.Collections.Generic.HashSet[string]' (
[string[]](Get-StudioTempArtifacts), [StringComparer]::OrdinalIgnoreCase)
- # A second back, so a process created in the same tick as the timestamp is not
- # filtered out by a strictly-later comparison inside Get-WinEvent. Stated exactly,
- # since the sweep above runs before this: the window still reaches one second into
- # the tail of that sweep, so a compiler the machine started in that last second is
- # counted. The trade is deliberate and it errs towards a false alarm, which fails a
- # measurement loudly, rather than towards dropping a real compile.
+ # A second back, so a process created in the same tick as the timestamp survives
+ # Get-WinEvent's strictly-later comparison. That reaches one second into the tail
+ # of the sweep above, so a compiler started in that second is counted: a deliberate
+ # trade towards a loud false alarm rather than a dropped real compile.
$since = (Get-Date).AddSeconds(-1)
$failure = $null
try {
- # Piped to Out-Host, not left on the success stream. The installer action tees its
- # log, and every one of those lines would otherwise be emitted as function output
- # ahead of the result hashtable, so the caller's $seen became an object array and
- # $seen.Compilers failed under Set-StrictMode against the string elements rather
- # than reporting the measurement.
+ # Out-Host, not the success stream. The installer action tees its log, and those
+ # lines would be emitted as function output ahead of the result hashtable, making
+ # the caller's $seen an object array whose $seen.Compilers fails under
+ # Set-StrictMode instead of reporting the measurement.
& $Action | Out-Host
} catch {
- # Recorded and re-thrown by the caller if it cares. The detectors still
- # report, because "the installer died AND spawned a compiler" is a more
- # useful message than either half alone.
+ # Recorded and re-thrown below. The detectors still report, because "the
+ # installer died AND spawned a compiler" beats either half alone.
$failure = $_
}
- # Closed before the temp sweep, not after it: the sweep walks every temp root
- # recursively and can take seconds, and anything the machine starts during that walk
- # belongs to nobody's measurement.
+ # Closed before the temp sweep, which can take seconds: anything the machine
+ # starts during that walk belongs to nobody's measurement.
$until = Get-Date
$compilers = @(Get-StudioCompilerEvents -Since $since -Until $until)
diff --git a/.github/workflows/windows-no-compiler-ci.yml b/.github/workflows/windows-no-compiler-ci.yml
index 1354a01cc6f..aa77f758705 100644
--- a/.github/workflows/windows-no-compiler-ci.yml
+++ b/.github/workflows/windows-no-compiler-ci.yml
@@ -4,39 +4,33 @@
# The behavioural half of "the installer does not run a compiler".
#
# tests/studio/test_installer_av_shapes.py reads the scripts and fails on an
-# Add-Type that is not -AssemblyName. That is a text check, and text checks are
-# blind to the two ways this regresses in practice: a compile reached through a
-# module, a dot-sourced file or a generated here-string, and a compile that some
-# dependency performs on our behalf while our own process tree is what a scanner
-# scores. Bitdefender flagged the chain, not the bytes:
+# Add-Type that is not -AssemblyName. Text checks are blind to the two ways this
+# regresses in practice: a compile reached through a module, a dot-sourced file or
+# a generated here-string, and a compile a dependency performs on our behalf while
+# our own process tree is what a scanner scores. Bitdefender flagged the chain:
#
# unsloth-studio.exe -> powershell.exe -> csc.exe -> %TEMP%\<random>.dll
#
# so the assertion that matters is about processes, not source. This job turns on
# 4688 process-creation auditing with command lines, runs the real installer, and
-# fails if csc.exe, vbc.exe or cvtres.exe ever appears, or if a DLL lands in a
-# temporary directory while the installer is running.
+# fails if csc.exe, vbc.exe or cvtres.exe appears, or if a DLL lands in a temporary
+# directory while the installer is running.
#
-# The positive control is not optional and runs FIRST. A detector that sees
-# nothing is indistinguishable from a clean run, and every part of this is
-# something that can silently fail to apply: auditpol can be overridden by a
-# machine policy, the Security log can be full, command-line capture can be off.
-# So the job deliberately compiles one throwaway type with Add-Type and REQUIRES
-# both signals to fire. If they do not, the job fails as a broken detector and
-# never reaches the real measurement.
+# The positive control is not optional and runs FIRST, because a detector that sees
+# nothing looks exactly like a clean run and every part of this can silently fail to
+# apply: auditpol overridden by machine policy, a full Security log, command-line
+# capture off. So the job compiles one throwaway type with Add-Type and REQUIRES both
+# signals to fire, failing as a broken detector before the real measurement.
#
-# Scope, stated because a green run here does not mean more than it means: this
-# proves nothing about a machine where App Control or Smart App Control is
-# enforcing, since a hosted runner has neither. It proves that the shipped
-# install path spawns no compiler and drops no temporary DLL, which is the shape
-# that was reported.
+# Scope: this proves nothing about a machine where App Control or Smart App Control
+# is enforcing, since a hosted runner has neither. It proves the shipped install path
+# spawns no compiler and drops no temporary DLL, which is the reported shape.
#
-# Which revision of studio/setup.ps1 each leg measures is also part of that scope.
-# The installer delegates setup to the managed CLI, so a default run executes the
-# setup.ps1 inside the released package, not the one in the checkout. The shell leg
-# is therefore a measurement of the release; the desktop leg passes --local, which
-# overlays the checkout as an editable install, and refuses to report a result
-# unless the overlay is in the log.
+# Which revision of studio/setup.ps1 each leg measures is part of that scope. The
+# installer delegates setup to the managed CLI, so a default run executes the
+# setup.ps1 inside the released package, not the checkout. The shell leg therefore
+# measures the release; the desktop leg passes --local to overlay the checkout as an
+# editable install, and refuses to report a result unless the overlay is in the log.
name: Windows No Compiler CI
@@ -49,8 +43,8 @@ on:
- 'studio/src-tauri/src/install.rs'
- 'studio/src-tauri/src/process.rs'
- 'tests/studio/test_installer_av_shapes.py'
- # The watcher implements both detectors. A PR touching only it could disable the
- # detection without ever running the positive control that would catch that.
+ # The watcher implements both detectors, so a PR touching only it could disable
+ # detection without running the positive control that would catch that.
- '.github/scripts/Watch-ForCompiler.ps1'
- '.github/workflows/windows-no-compiler-ci.yml'
push:
@@ -71,8 +65,8 @@ jobs:
timeout-minutes: 60
env:
- # Outside any user profile, for the same reason the Application Control leg
- # pins it: the paths under test must not depend on whose profile ran.
+ # Outside any user profile, like the Application Control leg: the paths under
+ # test must not depend on whose profile ran.
UNSLOTH_STUDIO_HOME: 'C:\Users\Public\no-compiler-home'
UNSLOTH_EVIDENCE: 'C:\Users\Public\no-compiler-evidence'
@@ -85,26 +79,26 @@ jobs:
$ErrorActionPreference = 'Stop'
New-Item -ItemType Directory -Force -Path $env:UNSLOTH_EVIDENCE | Out-Null
- # 4688 with the command line, which is what distinguishes "csc.exe ran"
- # from "csc.exe ran for us". Both are needed: the subcategory produces
- # the event, the policy value fills in the command line.
+ # 4688 with the command line, which distinguishes "csc.exe ran" from
+ # "csc.exe ran for us". The subcategory produces the event, the policy
+ # value fills in the command line.
auditpol /set /subcategory:"Process Creation" /success:enable | Out-Null
$auditKey = 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\Audit'
New-Item -Path $auditKey -Force | Out-Null
New-ItemProperty -Path $auditKey -Name 'ProcessCreationIncludeCmdLine_Enabled' `
-Value 1 -PropertyType DWord -Force | Out-Null
- # A Security log large enough that the installer's own noise cannot roll
- # the window under test off the end.
+ # Large enough that the installer's own noise cannot roll the window under
+ # test off the end of the Security log.
wevtutil sl Security /ms:268435456
$state = auditpol /get /subcategory:"Process Creation"
$state | Tee-Object -FilePath "$env:UNSLOTH_EVIDENCE\auditpol.txt"
- # Joined to one string, and matched on the subcategory line rather than the bare
- # word. auditpol returns an ARRAY of lines, and -notmatch on an array is a filter,
- # not a test: it returns the elements that do not match, so on a runner where
- # auditing turned on correctly it still returned the three header lines, and a
- # non-empty array is truthy. This threw on every healthy machine.
+ # Joined to one string, and matched on the subcategory line rather than the
+ # bare word: auditpol returns an ARRAY of lines, and -notmatch on an array
+ # filters rather than tests, so a healthy runner still returned the three
+ # header lines and a non-empty array is truthy. This threw on every healthy
+ # machine.
if (($state | Out-String) -notmatch 'Process Creation\s+Success') {
throw "process creation auditing did not turn on; this job cannot measure anything"
}
@@ -116,9 +110,8 @@ jobs:
. "$env:GITHUB_WORKSPACE\.github\scripts\Watch-ForCompiler.ps1"
$probe = {
- # One throwaway type, compiled the way the installer used to. This is
- # the only Add-Type in this repository's CI and it exists to prove the
- # measurement works.
+ # One throwaway type, compiled the way the installer used to: the only
+ # Add-Type in this repository's CI, here to prove the detector works.
Add-Type -TypeDefinition 'public static class UnslothCompilerProbe { public static int N() { return 1; } }'
[UnslothCompilerProbe]::N() | Out-Null
}
@@ -144,11 +137,10 @@ jobs:
$install = {
# Spawned the way the desktop spawns it, not dot-sourced: the shape
- # under test is a child powershell.exe, and a compile started by the
- # installer's own children is exactly what a dot-source would hide
- # from the 4688 ancestry. --no-torch for time; the native
- # declarations are made before any of that. Not --tauri, which
- # refuses UNSLOTH_STUDIO_HOME, and the pinned home is what keeps
+ # under test is a child powershell.exe, and a dot-source would hide a
+ # compile started by the installer's own children from the 4688
+ # ancestry. --no-torch for time; the native declarations happen before
+ # that. Not --tauri, which refuses the UNSLOTH_STUDIO_HOME that keeps
# every step reading one tree.
New-Item -ItemType Directory -Force -Path logs | Out-Null
& powershell.exe -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass `
@@ -182,21 +174,18 @@ jobs:
$ErrorActionPreference = 'Stop'
. "$env:GITHUB_WORKSPACE\.github\scripts\Watch-ForCompiler.ps1"
- # --tauri is what the launcher actually passes: install.rs initialises its
- # argument list with it unconditionally. The step above runs the shell mode
- # because that is the one that honours a pinned UNSLOTH_STUDIO_HOME, and the
- # desktop mode refuses it; running only that left a compile reached through a
- # Tauri-only branch invisible to a job whose whole claim is the desktop shape.
- # So both modes run, and the home is unpinned here rather than the mode changed.
+ # --tauri is what the launcher passes: install.rs initialises its argument
+ # list with it unconditionally. The step above runs shell mode because only
+ # that honours a pinned UNSLOTH_STUDIO_HOME; running only it left a compile
+ # reached through a Tauri-only branch invisible to a job whose claim is the
+ # desktop shape. So both modes run, with the home unpinned here.
#
- # --local, which is the only thing that makes this leg measure the setup.ps1 in
- # THIS checkout. Without it the installer installs the released unsloth and
- # delegates setup to the copy inside that release (install.ps1 hands the work to
- # the managed CLI), so a PR touching only studio/setup.ps1 triggered this workflow
- # and then measured a different revision of the file it changed. --local overlays
- # the checkout as an editable install, so the delegated `studio setup` runs the
- # bytes under review. The gate below is what proves the overlay took; a run that
- # fell back to the release is void here, not clean.
+ # --local is the only thing that makes this leg measure the setup.ps1 in THIS
+ # checkout. Without it install.ps1 hands the work to the managed CLI, which
+ # runs the setup.ps1 inside the released package, so a PR touching only
+ # studio/setup.ps1 measured a different revision of the file it changed. The
+ # gate below proves the overlay took; a run that fell back to the release is
+ # void, not clean.
$desktop = {
Remove-Item Env:\UNSLOTH_STUDIO_HOME -ErrorAction SilentlyContinue
& powershell.exe -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Bypass `
@@ -204,15 +193,14 @@ jobs:
Tee-Object -FilePath "$env:UNSLOTH_EVIDENCE\install-tauri.log"
$exit = $LASTEXITCODE
$global:LASTEXITCODE = 0
- # Not gated on exit 0. A desktop-mode install can decline for reasons that
- # have nothing to do with this measurement, and the question here is only
- # whether a compiler ran on the way.
+ # Not gated on exit 0: a desktop-mode install can decline for unrelated
+ # reasons, and the question is only whether a compiler ran on the way.
Write-Host "install.ps1 --tauri exited $exit"
- # It IS gated on having got as far as the work. A run that stopped at
- # argument validation spawns nothing at all, so both detectors come back
- # empty and this step prints "no compiler" having measured nothing. The
- # STEP line is emitted after the install lock is taken, which is where the
- # native declarations happen, so seeing it means the branch under test ran.
+ # It IS gated on reaching the work. A run that stopped at argument
+ # validation spawns nothing, so both detectors come back empty and the
+ # step prints "no compiler" having measured nothing. This STEP line is
+ # emitted after the install lock is taken, where the native declarations
+ # happen, so seeing it means the branch under test ran.
$log = ""
if (Test-Path "$env:UNSLOTH_EVIDENCE\install-tauri.log") {
$log = Get-Content -Raw "$env:UNSLOTH_EVIDENCE\install-tauri.log"
@@ -221,15 +209,15 @@ jobs:
throw ("the desktop-mode install stopped before it reached any install " +
"work (exit $exit), so this measurement is void rather than clean.")
}
- # And that the checkout is what setup ran from. This line is printed by the
- # installer's --local overlay step; without it the delegated setup came out of
- # the released package and this leg measured someone else's file.
+ # And that setup ran from the checkout. This line is printed by the
+ # --local overlay step; without it the delegated setup came out of the
+ # released package.
if ($log -notmatch 'overlaying local repo') {
throw ("the local overlay never ran (exit $exit), so the setup.ps1 that " +
"executed was not the one in this checkout. Void, not clean.")
}
- # Which revision, recorded rather than asserted, so the artifact says what was
- # measured instead of leaving it to be inferred from the commit.
+ # Which revision, recorded rather than asserted, so the artifact says
+ # what was measured instead of leaving it inferred from the commit.
$setup = Join-Path $env:GITHUB_WORKSPACE 'studio\setup.ps1'
"path=$setup" | Out-File "$env:UNSLOTH_EVIDENCE\setup-measured.txt" -Encoding utf8
"sha256=$((Get-FileHash -LiteralPath $setup -Algorithm SHA256).Hash)" |
diff --git a/install.ps1 b/install.ps1
index 75b008c1fa5..f5630c2b8e4 100644
--- a/install.ps1
+++ b/install.ps1
@@ -346,10 +346,11 @@ function Install-UnslothStudio {
# Windows picks the temp directory from TMP, then TEMP, then the profile, and
# never checks it exists or is writable. The desktop app passes on whatever it
# inherited (studio/src-tauri/src/install.rs sets neither); one report had it at
- # C:\Windows\TEMP, unwritable, which used to break the install outright because
- # the native helper was compiled through it (issue #9140). Nothing compiles now,
- # but the Python, uv and VC++ downloads still stage through it. Probe it once and, if it cannot hold a file, point BOTH
- # variables at a directory we own: every child process and every
+ # C:\Windows\TEMP, unwritable, which broke the install when the native helper was
+ # still compiled through it (issue #9140). Nothing compiles now, but the Python,
+ # uv and VC++ downloads still stage through it. Probe it once and, if it cannot
+ # hold a file, point BOTH variables at a directory we own: every child process
+ # and every
# [System.IO.Path]::GetTempPath() call reads the process environment block.
function Test-StudioDirectoryUsable {
param(
@@ -830,45 +831,35 @@ function Install-UnslothStudio {
try { $defaultProfile = [Environment]::GetFolderPath("UserProfile") } catch {}
$tauriProfile = if ($defaultProfile) { $defaultProfile } else { $env:USERPROFILE }
- # Every native declaration in this script goes through here, and none of them
- # goes through Add-Type.
+ # Every native declaration in this script goes through here, never Add-Type.
#
- # Add-Type on Windows PowerShell 5.1 (the interpreter the desktop app spawns)
- # has no in-process compiler: it writes C# to %TEMP% and runs csc.exe to get a
- # DLL back. That is true of -TypeDefinition and of -MemberDefinition alike.
- # Bitdefender blocks the DLL that produces, and the verdict lands on the
- # generated file rather than on anything shipped: a windowless PowerShell,
- # spawned by a GUI binary, launching a compiler and writing executable content
- # to %TEMP% is a dropper's shape, whatever the code says. It also used to fail
- # outright with CS2001 when %TEMP% was unusable (issue #9140).
+ # Add-Type on Windows PowerShell 5.1 (the interpreter the desktop app spawns) has
+ # no in-process compiler: -TypeDefinition and -MemberDefinition alike write C# to
+ # %TEMP% and run csc.exe. Bitdefender blocks the resulting DLL, because a
+ # windowless PowerShell spawned by a GUI binary, running a compiler and writing
+ # executable content to %TEMP%, is a dropper's shape whatever the code says. It
+ # also failed with CS2001 when %TEMP% was unusable (issue #9140). Reflection emit
+ # builds the same interop stubs in memory: no compiler process, no source, no DLL,
+ # empty assembly Location. Available on .NET Framework 4 and .NET 5+, so 5.1 and 7
+ # take the same path.
#
- # Reflection emit builds the same interop stubs in memory. No compiler
- # process, no source on disk, no DLL, nothing in %TEMP%, and the resulting
- # assembly reports an empty Location. Available on .NET Framework 4 and on
- # .NET 5+, so 5.1 and 7 take the same path.
+ # Throws rather than reporting: each caller wants a different answer to "the native
+ # side is unavailable", and the two cosmetic ones must not print the resolver's
+ # warning.
#
- # Throws on failure rather than reporting it: each caller wants a different
- # answer to "the native side is unavailable", and the two that are cosmetic
- # must not print the resolver's warning.
- #
- # Test-StudioCanDefineNativeTypes is the gate every caller checks first, and it
- # is a gate rather than a try/catch because one of the two conditions is not
- # catchable. App Control's Dynamic Code Security (policy option 19) always
- # blocks loading unsigned assemblies built with System.Reflection.Emit, and
- # Microsoft documents the parent process as usually stopped or crashing rather
- # than raising, so nothing below could recover from it:
+ # Test-StudioCanDefineNativeTypes is the gate every caller checks first, rather than
+ # a try/catch, because App Control's Dynamic Code Security (policy option 19) blocks
+ # loading unsigned System.Reflection.Emit assemblies by usually stopping or crashing
+ # the parent instead of raising:
# learn.microsoft.com/en-us/windows/security/application-security/application-control/app-control-for-business/design/appcontrol-and-dotnet
- # A machine enforcing user-mode code integrity puts PowerShell in Constrained
- # Language, which is the first check and the one that fires in practice; the
- # Device Guard probe is the belt to that pair of braces, for a policy that
- # somehow left the language mode alone. When one IS active, the answer comes from
- # a child process that tries the emit, not from a guess about which options the
- # policy set.
+ # Such a machine puts PowerShell in Constrained Language, the first check and the one
+ # that fires in practice; the Device Guard probe covers a policy that left the
+ # language mode alone. When one IS active, a child process tries the emit rather than
+ # guessing which options the policy set.
$script:StudioCanDefineNativeTypes = $null
# Why the last probe answered as it did, so a caller can tell "the child ran and
- # said no" from "the child never got to answer". Those are the same boolean and
- # they are not the same fact: one is a policy, the other is a process that failed
- # to start, was killed at the deadline, or lost its output.
+ # said no" (a policy) from "the child never answered" (failed to start, killed
+ # at the deadline, or lost its output). Same boolean, different facts.
$script:StudioEmitProbeOutcome = $null
function Test-StudioCanDefineNativeTypes {
if ($null -ne $script:StudioCanDefineNativeTypes) { return $script:StudioCanDefineNativeTypes }
@@ -878,23 +869,20 @@ function Install-UnslothStudio {
$script:StudioCanDefineNativeTypes = $false
return $false
}
- # Three outcomes, not two. A status that was READ and says 0 is the only one that
- # skips the probe: a query that threw, returned nothing, or returned an object
- # without the property is UNKNOWN, and unknown used to be treated as unrestricted.
- # That is the wrong direction here. The case it lets through is option 19 enforced
- # on a host whose CIM query happens to fail, and the cost of being wrong there is a
- # stopped installer, while the cost of probing unnecessarily is one short-lived
- # process.
+ # Three outcomes, not two. Only a status that was READ and says 0 skips the
+ # probe; a query that threw, returned nothing, or lacked the property is
+ # UNKNOWN, and unknown must not mean unrestricted. Treating it as such lets
+ # through option 19 enforced on a host whose CIM query fails, costing a
+ # stopped installer; probing unnecessarily costs one short-lived process.
$known = $false
$active = $false
try {
- # Bounded, with what that bound actually covers stated rather than implied:
- # -OperationTimeoutSec limits the CIM operation on a responsive target. It does
- # not interrupt DCOM connection setup and it does not override a provider that
- # has wedged, where the server's own timeout wins. It is worth having for the
- # slow case and it is not a hang guard. The installer already depends on CIM
- # for adapter and process queries, so this query does not introduce that
- # exposure; the child probe below is the part that carries a real deadline.
+ # -OperationTimeoutSec bounds the CIM operation on a responsive target
+ # only: it does not interrupt DCOM connection setup, and a wedged
+ # provider's own timeout wins. Good for the slow case, not a hang guard.
+ # The installer already depends on CIM for adapter and process queries,
+ # so this adds no exposure; the child probe below carries the real
+ # deadline.
$guard = Get-CimInstance -Namespace "root\Microsoft\Windows\DeviceGuard" `
-ClassName "Win32_DeviceGuard" -OperationTimeoutSec 10 -ErrorAction Stop
# 0 off, 1 audit, 2 enforced. A null property is not a zero.
@@ -909,21 +897,20 @@ function Install-UnslothStudio {
$script:StudioCanDefineNativeTypes = $true
return $true
}
- # A policy is active, or its state could not be read, and WHICH policy decides
- # this. Option 19 Dynamic Code
- # Security always blocks unsigned System.Reflection.Emit assemblies and has no
- # audit mode on Windows 10 or Windows 11 before 24H2, where it is enforced even
- # in an audit policy; an audit policy WITHOUT that option emits perfectly well.
- # Win32_DeviceGuard does not report the option bit, so either guess costs a
- # population: refusing everything sends every audit-mode machine down the lexical
- # path, and allowing everything risks the process. Ask the machine instead.
+ # A policy is active or unreadable, and WHICH policy decides this. Option 19
+ # Dynamic Code Security always blocks unsigned System.Reflection.Emit
+ # assemblies, with no audit mode on Windows 10 or Windows 11 before 24H2
+ # (enforced even in an audit policy); an audit policy WITHOUT that option
+ # emits fine. Win32_DeviceGuard does not report the option bit, so either
+ # guess costs a population: refusing sends every audit-mode machine down the
+ # lexical path, allowing risks the process. Ask the machine instead.
$script:StudioCanDefineNativeTypes = Test-StudioEmitInChildProcess
- # One retry, and only when the first attempt never reached an answer. The compiled
- # version this replaces also tried twice before caching a negative, and without
- # that a single transient process failure is indistinguishable from a policy: it
- # is cached for the whole run and sends the installer down the lexical path, where
- # two unequal roots compare as unknown and a second lock gets taken. A child that
- # RAN and said no is not retried, so a genuinely blocked machine still pays for
+ # One retry, only when the first attempt never reached an answer (the
+ # compiled version this replaces also tried twice before caching a
+ # negative). Otherwise one transient process failure is cached for the whole
+ # run as if it were a policy, sending the installer down the lexical path
+ # where two unequal roots compare as unknown and a second lock gets taken. A
+ # child that RAN and said no is not retried, so a blocked machine pays for
# one probe.
if (-not $script:StudioCanDefineNativeTypes -and
$script:StudioEmitProbeOutcome -eq "indeterminate") {
@@ -932,17 +919,15 @@ function Install-UnslothStudio {
return $script:StudioCanDefineNativeTypes
}
- # The same emit, in a process that is allowed to die. Microsoft documents a blocked
- # dynamic load as usually stopping or crashing the parent, which is precisely why
- # this is not attempted in-process: a child that vanishes is an answer, and the same
- # event here would be the installer vanishing. Silence is refusal, so a probe that
- # cannot be spawned at all lands on the lexical path rather than on optimism.
+ # The same emit, in a process that is allowed to die. A blocked dynamic load
+ # usually stops or crashes the parent, so doing this in-process would be the
+ # installer vanishing; a child that vanishes is just an answer. Silence is
+ # refusal, so an unspawnable probe lands on the lexical path.
function Test-StudioEmitInChildProcess {
- # HostPath is for the tests, which have no policy to trigger the real path and cannot
- # shadow $PSHOME, since it is read-only. Production never passes it.
+ # HostPath is for the tests, which have no policy to trigger the real path and
+ # cannot shadow the read-only $PSHOME. Production never passes it.
param([string]$HostPath)
- # Until something here establishes otherwise. Every exit below either leaves this
- # alone or says what it learned.
+ # Until something below establishes otherwise.
$script:StudioEmitProbeOutcome = "indeterminate"
$probe = @'
try {
@@ -971,10 +956,9 @@ try {
if ('UnslothStudioEmitProbe' -as [type]) { Write-Output ('STUDIO_EMIT_OK ' + [string]$ExecutionContext.SessionState.LanguageMode); exit 0 }
exit 1
'@
- # This host, not a guessed one: a 5.1 answer does not carry to pwsh or the other
- # way, and the emit that matters is the one this interpreter will make. Both
- # spellings of the leaf, so the function is the same one a non-Windows lane can
- # execute end to end rather than a Windows-only path nothing tests.
+ # This host, not a guessed one: a 5.1 answer does not carry to pwsh or back.
+ # Both spellings of the leaf, so a non-Windows lane can execute this function
+ # end to end rather than leaving a Windows-only path untested.
$hostExe = $HostPath
if (-not $hostExe) {
try {
@@ -987,20 +971,18 @@ exit 1
} catch {}
}
if (-not $hostExe) { return $false }
- # Through a Process object rather than the call operator, for a deadline. The call
- # operator waits for the child forever, and "forever" is reachable: a security
- # product inspecting a freshly spawned interpreter, a wedged runtime start, a child
- # that blocks on shutdown. A probe that exists to keep the installer alive must not
- # be the thing that hangs it.
+ # A Process object rather than the call operator, for a deadline: the call
+ # operator waits forever, and forever is reachable (a security product
+ # inspecting a fresh interpreter, a wedged runtime start, a child blocking on
+ # shutdown). A probe meant to keep the installer alive must not hang it.
#
- # BOTH streams are redirected and drained asynchronously. Draining is what stops a
- # chatty child filling a pipe and deadlocking against the wait. Redirecting stderr as
- # well is the difference between a probe that is invisible and one that can write
- # into the installer's own stderr, which the desktop app reads, and which anything
- # the child spawns would inherit and hold open.
+ # BOTH streams are redirected and drained asynchronously. Draining stops a
+ # chatty child filling a pipe and deadlocking against the wait. Redirecting
+ # stderr keeps the probe out of the installer's own stderr, which the desktop
+ # app reads and anything the child spawns would inherit and hold open.
#
- # The body is embedded in double quotes here, which is safe for exactly one reason:
- # it contains none, asserted by a test. See the note above.
+ # The body is embedded in double quotes, safe only because it contains none,
+ # asserted by a test. See the note above.
$info = New-Object System.Diagnostics.ProcessStartInfo
$info.FileName = $hostExe
$info.Arguments = "-NoProfile -NonInteractive -Command `"$probe`""
@@ -1017,11 +999,11 @@ exit 1
try { $child.Kill() } catch {}
return $false
}
- # Exit code AND an exact record. A marker followed by a crash is a crash: the
- # question is whether this machine can emit and live, and a child that printed
- # and then died has answered no. FullLanguage because an approved script can run
- # in FullLanguage while a fresh inline command does not, and a child restricted
- # differently from its parent has measured a different machine.
+ # Exit code AND an exact record. A marker followed by a crash is a crash:
+ # the question is whether this machine can emit and live. FullLanguage
+ # because an approved script can run in FullLanguage while a fresh inline
+ # command does not, and a child restricted differently from its parent
+ # has measured a different machine.
if ($child.ExitCode -ne 0) {
$script:StudioEmitProbeOutcome = "blocked"
return $false
@@ -1032,23 +1014,23 @@ exit 1
$script:StudioEmitProbeOutcome = "ok"
return $true
}
- # Emitted, but in a language mode this parent is not in. The child measured a
- # different machine, which is an answer rather than a missed one.
+ # Emitted, but in a language mode this parent is not in: the child
+ # measured a different machine, which is an answer, not a miss.
if ($line.Trim() -like "STUDIO_EMIT_OK *") {
$script:StudioEmitProbeOutcome = "blocked"
return $false
}
}
- # Exit 0 with no marker at all: the child cannot have got past the emit and then
- # reported nothing, so its output was lost rather than negative.
+ # Exit 0 with no marker: the child cannot have emitted and reported
+ # nothing, so its output was lost rather than negative.
return $false
} catch {
return $false
} finally {
if ($child) {
- # The read end goes first. A killed child can leave a grandchild holding the
- # write end of that pipe, and the pending async read then keeps this process
- # alive past the deadline it just enforced.
+ # The read end goes first: a killed child can leave a grandchild
+ # holding the write end, and the pending async read would then keep
+ # this process alive past the deadline it just enforced.
try { $child.StandardOutput.Close() } catch {}
try { $child.StandardError.Close() } catch {}
try { $child.Dispose() } catch {}
@@ -1061,16 +1043,16 @@ exit 1
Both spellings of "define a dynamic assembly", because the two PowerShell
hosts that run this file are on different runtimes. The static
AssemblyBuilder::DefineDynamicAssembly is documented for .NET Framework
- 4.5 through 4.8.1 as well as .NET Core, so Windows PowerShell 5.1 should
- take the first branch. It is tried rather than assumed because nothing
- here can test a .NET Framework host, and getting it wrong is not a visible
- error: the catch would cache the resolver as unavailable and every desktop
- install would silently use the lexical fallback, which can assign
- different mutex names to a long path and its 8.3 alias.
+ 4.5 through 4.8.1 as well as .NET Core, so 5.1 should take the first
+ branch; it is tried rather than assumed because nothing here can test a
+ .NET Framework host and getting it wrong is invisible: the catch would
+ cache the resolver as unavailable and every desktop install would use the
+ lexical fallback, which can give a long path and its 8.3 alias different
+ mutex names.
AppDomain.CurrentDomain.DefineDynamicAssembly is the .NET Framework
- spelling and is absent on .NET Core, so it is the fallback rather than the
- first try; pwsh would fail on it.
+ spelling and is absent on .NET Core, so it is the fallback; pwsh would
+ fail on it.
#>
param([Parameter(Mandatory = $true)][System.Reflection.AssemblyName]$AssemblyName)
$access = [System.Reflection.Emit.AssemblyBuilderAccess]::Run
@@ -1080,10 +1062,10 @@ exit 1
} catch [System.Management.Automation.MethodException] {
return [AppDomain]::CurrentDomain.DefineDynamicAssembly($AssemblyName, $access)
} catch [System.Management.Automation.RuntimeException] {
- # A missing static surfaces as a RuntimeException on some hosts rather
- # than a MethodException. Both mean "no such method here", and a real
- # emit failure (Dynamic Code Security, Constrained Language) throws
- # from the AppDomain call too, so the caller still sees it.
+ # Some hosts surface a missing static as RuntimeException, not
+ # MethodException. Both mean "no such method here", and a real emit
+ # failure (Dynamic Code Security, Constrained Language) throws from the
+ # AppDomain call too, so the caller still sees it.
return [AppDomain]::CurrentDomain.DefineDynamicAssembly($AssemblyName, $access)
}
}
@@ -1102,13 +1084,12 @@ exit 1
$TypeName, "Public, Class, AutoClass, AnsiClass, BeforeFieldInit")
$winapi = [System.Runtime.InteropServices.CallingConvention]::Winapi
- # Per import, because CharSet is not decoration: it selects name mangling
- # as well as marshalling. Unicode probes <Name>W before <Name>, Ansi probes
- # <Name> before <Name>A. Every import here names an export that exists
- # exactly as written, so both orders arrive; declaring the one the C# these
- # replace declared keeps the metadata honest and puts the export that does
- # exist first. Unicode is the default, since the calls carrying text are
- # the ones already spelled W.
+ # Per import, because CharSet selects name mangling as well as marshalling:
+ # Unicode probes <Name>W before <Name>, Ansi probes <Name> before <Name>A.
+ # Every import names an export that exists exactly as written, so both
+ # orders arrive; matching the C# these replace keeps the metadata honest and
+ # tries the existing export first. Unicode is the default, since the calls
+ # carrying text are already spelled W.
$unicode = [System.Runtime.InteropServices.CharSet]::Unicode
$ansi = [System.Runtime.InteropServices.CharSet]::Ansi
$standard = [System.Reflection.CallingConventions]::Standard
@@ -1122,13 +1103,15 @@ exit 1
$standard, $import.Return, $import.Args, $winapi, $charSet)
$method.SetImplementationFlags(
$method.GetMethodImplementationFlags() -bor $preserveSig)
- # `out uint` in the C# this replaces, and DefinePInvokeMethod has no way to say
- # so: a by-ref type alone emits `ref`, which is In and Out unset. The value is
- # blittable and every caller initialises it first, so the marshaller pins and
- # writes back either way, but the metadata is what a reader and any future
- # marshalling change go by, so it says what the declaration said.
- # ContainsKey, not a bare property read: most imports have no Out key and reading a
- # missing one is fatal under a caller's Set-StrictMode. setup.ps1 sets none of its own.
+ # `out uint` in the C# this replaces; DefinePInvokeMethod cannot say so,
+ # since a by-ref type alone emits `ref` (In and Out unset). The value is
+ # blittable and every caller initialises it, so marshalling works either
+ # way, but the metadata is what a reader and any future marshalling
+ # change go by.
+ # ContainsKey, not a bare property read: most imports have no Out key and
+ # reading a missing one is fatal under Set-StrictMode. install.ps1 turns
+ # strict mode off for itself, studio/setup.ps1 inherits the caller's, so
+ # the guard is mirrored rather than left to one of them.
if ($import.ContainsKey("Out")) {
foreach ($position in @($import.Out)) {
if ($position) { $null = $method.DefineParameter($position, "Out", $null) }
@@ -1141,9 +1124,9 @@ exit 1
# GetFinalPathNameByHandleW is the only exact answer for a path: it follows
# junctions, symlinks and SUBST drives, expands 8.3 aliases and reports the
- # on-disk spelling, none of which GetFullPath does. Cache the answer, since
- # callers resolve dozens of paths, and where it is unavailable let
- # Get-StudioLexicalPath carry the run.
+ # on-disk spelling, none of which GetFullPath does. Cached, since callers
+ # resolve dozens of paths; where unavailable, Get-StudioLexicalPath carries the
+ # run.
$script:StudioFinalPathNativeState = $null
# Reset with the rest: under `irm | iex` these are the caller's own.
$script:StudioNativeResolveWarned = $false
@@ -1152,11 +1135,9 @@ exit 1
param([string]$Reason)
if ($script:StudioFinalPathWarned) { return }
$script:StudioFinalPathWarned = $true
- # "installation is unaffected" is what this used to promise, and it is a
- # promise this line is not in a position to keep: the same security software
- # that can stop a type being defined can also be acting on the rest of the
- # run. What is true is the narrower thing, that the installer has a way to
- # continue without it.
+ # This used to promise "installation is unaffected", which it cannot know:
+ # the same security software that blocks a type can be acting on the rest of
+ # the run. Only the narrower claim is true, that the installer can continue.
Write-StudioLine "[WARN] Could not load the native path resolver ($Reason)." -ForegroundColor Yellow
Write-StudioLine " Continuing with the PowerShell resolver, which cannot recover a path's" -ForegroundColor Yellow
Write-StudioLine " stored casing or expand an 8.3 name, so paths are compared as written." -ForegroundColor Yellow
@@ -1174,11 +1155,10 @@ exit 1
$script:StudioFinalPathNativeState = $false
$languageMode = "FullLanguage"
try { $languageMode = [string]$ExecutionContext.SessionState.LanguageMode } catch {}
- # Three reasons, because the gate now has three ways to say no and only one of
- # them is a policy. Claiming code integrity for a probe that could not be
- # spawned, or that was killed at its deadline, is a wrong answer written into
- # the log a support request is built from: it names a machine setting the user
- # would then go looking for and not find.
+ # Three reasons, because the gate has three ways to say no and only one
+ # is a policy. Blaming code integrity for a probe that could not be
+ # spawned, or was killed at its deadline, writes a machine setting into
+ # the support log that the user would go looking for and not find.
$reason = if ($languageMode -ne "FullLanguage") { "PowerShell is in $languageMode" }
elseif ($script:StudioEmitProbeOutcome -eq "blocked") {
"this host enforces user-mode code integrity"
@@ -1187,14 +1167,12 @@ exit 1
return $false
}
# Everything below the capability check is unchanged, so a host where the
- # emit fails degrades exactly as a host that could not compile already
- # did, which is a path the installer has always taken and still completes
- # on.
+ # emit fails degrades exactly as one that could not compile already did.
#
# DefinePInvokeMethod cannot ask for SetLastError, so nothing below reads
- # GetLastWin32Error. The C# it replaces threw a Win32Exception the callers
- # only ever turned back into "use the lexical answer", so returning null
- # loses a code nothing acted on.
+ # GetLastWin32Error. The C# it replaces threw a Win32Exception that callers
+ # only turned back into "use the lexical answer", so returning null loses a
+ # code nothing acted on.
try {
$null = New-StudioEmittedNativeType -TypeName "UnslothStudioFinalPathV3" -Imports @(
@{ Name = "CreateFileW"; Library = "kernel32.dll"; Return = [IntPtr]
@@ -1206,10 +1184,10 @@ exit 1
Ansi = $true }
)
} catch {
- # A throw does not always mean nothing was defined: CreateType can
- # publish the type and then fail on the way back, and the compiled
- # version this replaces checked for exactly that before giving up. If
- # the type is there it is usable, so ask before caching the negative.
+ # A throw does not mean nothing was defined: CreateType can publish the
+ # type and then fail on the way back, and the compiled version this
+ # replaces checked for that too. A published type is usable, so ask
+ # before caching the negative.
if ("UnslothStudioFinalPathV3" -as [type]) {
$script:StudioFinalPathNativeState = $true
return $true
@@ -1231,10 +1209,10 @@ exit 1
# symlinks and SUBST drives, expands 8.3 aliases and reports the on-disk
# spelling, none of which GetFullPath does.
#
- # This is what the compiled Resolve() did, moved out of C# so the imports
- # above are all the native code there is. Same flags, same two-pass buffer
- # growth. Null rather than an exception when the handle or the call fails:
- # every caller already treats "no exact answer" as "use the lexical one".
+ # What the compiled Resolve() did, moved out of C# so the imports above are all
+ # the native code there is. Same flags, same two-pass buffer growth. Null rather
+ # than an exception on failure: every caller already treats "no exact answer" as
+ # "use the lexical one".
function Get-StudioNativeFinalPath {
param([Parameter(Mandatory = $true)][string]$Path)
@@ -1243,20 +1221,15 @@ exit 1
$openExisting = [uint32]3
$backupSemantics = [uint32]0x02000000 # required to open a DIRECTORY
- # Every native call is guarded, not just the ones that can fail usefully.
- # An emitted stub binds its import on first CALL, not when the type is
- # defined, so a host missing the export raises here rather than above, and
- # the caller must see the same "no exact answer" it sees from a handle it
- # could not open. The callers already treat null that way; throwing past
- # them would be a new failure mode this change has no business inventing.
- # Acquisition sits INSIDE the region that closes the handle. The C# this
- # replaces returned a SafeFileHandle, whose finalizer was a backstop if the
- # pipeline stopped between the open and the using block; an IntPtr has
- # none, so the open and the finally are one region instead. What is left
- # is the instant between the native return and the assignment, which no
- # arrangement of PowerShell can close. It costs nothing here: the handle is
- # opened with desired access 0 and FILE_SHARE_READ|WRITE|DELETE, so even a
- # leaked one blocks no other opener, and it dies with the process.
+ # Every native call is guarded: an emitted stub binds its import on first CALL,
+ # not at definition, so a host missing the export raises here rather than above,
+ # and the caller must see the same null "no exact answer" a failed open gives.
+ # Acquisition sits INSIDE the region that closes the handle, because the
+ # SafeFileHandle the C# returned had a finalizer as a backstop and an IntPtr has
+ # none. What is left is the instant between the native return and the assignment,
+ # which no PowerShell arrangement can close and which costs nothing: opened with
+ # desired access 0 and FILE_SHARE_READ|WRITE|DELETE, even a leaked handle blocks
+ # no other opener and dies with the process.
$handle = $invalidHandle
try {
try {
@@ -1283,9 +1256,9 @@ exit 1
} catch {
return $null
} finally {
- # Guarded now that the open is inside this region: the failure paths
- # reach here with the sentinel, and closing it is a call Windows has no
- # reason to be asked to make.
+ # Guarded now that the open is inside this region: failure paths reach
+ # here with the sentinel, which there is no reason to ask Windows to
+ # close.
if ($handle -ne $invalidHandle -and $handle -ne [IntPtr]::Zero) {
try { [void][UnslothStudioFinalPathV3]::CloseHandle($handle) } catch {}
}
@@ -1475,10 +1448,10 @@ exit 1
# The helper was DEFINED and still could not answer: a path renamed
# between the Test-Path walk and CreateFileW, an access denial on a
# component, a volume with no drive letter. Falling back keeps the
- # install alive, and Exact = $false already makes the runtime lock
- # fail closed, but nothing said so out loud: the degraded warning
- # below only fires when the type itself could not be built. An operator was
- # left with a silently inexact identity on a host that looks fine.
+ # install alive and Exact = $false makes the runtime lock fail
+ # closed, but say so: the degraded warning below only fires when the
+ # type itself could not be built, leaving an operator with a
+ # silently inexact identity on a host that looks fine.
$resolved = $null
if (-not $script:StudioNativeResolveWarned) {
$script:StudioNativeResolveWarned = $true
@@ -1544,12 +1517,11 @@ exit 1
Write-StudioLine " The desktop app uses the Windows profile .unsloth\studio root." -ForegroundColor Red
Write-StudioLine " Run install.ps1 without --tauri for custom-root shell installs," -ForegroundColor Yellow
Write-StudioLine " or unset the env var for default desktop installs." -ForegroundColor Yellow
- # Belt and braces: TMP/TEMP are only redirected from
- # Initialize-StudioTempEnvironment, which now runs after this throw, so
- # there is nothing to restore here today. The call returns immediately
- # when nothing was overridden, and under `irm | iex` those variables are
- # the caller's own, so the day anything above starts redirecting them
- # again this stays correct instead of silently leaking.
+ # Belt and braces: only Initialize-StudioTempEnvironment redirects
+ # TMP/TEMP and it now runs after this throw, so there is nothing to
+ # restore today. The call is a no-op when nothing was overridden, and
+ # under `irm | iex` those variables are the caller's own, so this stays
+ # correct if anything above starts redirecting them again.
Restore-StudioTempEnvironment
throw "$envOverrideVar is not supported with --tauri."
}
@@ -1815,16 +1787,14 @@ exit 1
# so that is the path the desktop app is on.
if ($script:StudioStdoutRedirected) { return $false }
# Emitted rather than compiled, for the reason New-StudioEmittedNativeType
- # gives. -MemberDefinition runs csc.exe just as -TypeDefinition does, and
- # the guard above only keeps the desktop app off it; the console path,
- # including `irm | iex`, reached the compiler here every run. Colour is
- # cosmetic, so failure is still just a plain banner.
- # The same gate the resolver uses: colour is not worth a risk that cannot be
- # caught, and a plain banner is the answer either way.
- # The published type first, the gate only if there is nothing published. A type
- # this session already emitted is proof that emit works here, and the compiled
- # version checked for it in the same order; asking a child instead means one
- # failed probe throws away a console helper that is already loaded and usable.
+ # gives: -MemberDefinition runs csc.exe just as -TypeDefinition does, and the
+ # guard above only keeps the desktop app off it, so the console path
+ # (including `irm | iex`) reached the compiler here every run.
+ # Same gate as the resolver, since colour is not worth a risk that cannot be
+ # caught; failure is just a plain banner.
+ # The published type first, the gate only if there is nothing published: a
+ # type this session already emitted proves emit works here, and asking a
+ # child instead lets one failed probe throw away a usable console helper.
if (-not ("StudioVTNative" -as [type]) -and -not (Test-StudioCanDefineNativeTypes)) {
return $false
}
@@ -1881,11 +1851,10 @@ exit 1
}
Write-StudioLine ""
- # Here so its warning lands under the banner. The only caller now: the native
- # path resolver used to reach it first, because compiling wrote to %TEMP%, and
- # it no longer writes anything anywhere. Nothing between the banner and here
- # touches the temporary directory, so the fix-up still lands before the first
- # user of it.
+ # Here so its warning lands under the banner. The native path resolver used to
+ # reach it first, because compiling wrote to %TEMP%; it no longer writes
+ # anything. Nothing between the banner and here touches the temporary
+ # directory, so the fix-up still lands before its first user.
Initialize-StudioTempEnvironment
# ── Helper: refresh PATH from registry (deduplicating entries) ──
@@ -3180,12 +3149,11 @@ exit 0
# Per-item SHChangeNotify: the global broadcast misses a rewritten same-name .lnk.
try {
# Emitted, not compiled: -MemberDefinition runs csc.exe too.
- # Tauri returns before it gets here, so this one was only
- # ever on the console path, but that path deserves the same
- # treatment. A failure here leaves stale icons, nothing more,
- # and the enclosing catch already absorbed that.
- # Same order as everywhere else: a published type settles it, and
- # only an absent one asks the gate.
+ # Tauri returns before here, so this was only ever on the
+ # console path, which deserves the same treatment. A failure
+ # leaves stale icons and the enclosing catch absorbs it.
+ # Same order as everywhere else: a published type settles it,
+ # and only an absent one asks the gate.
if (-not ("UnslothShellIconRefresh" -as [type])) {
if (-not (Test-StudioCanDefineNativeTypes)) {
throw "native types are unavailable on this host"
@@ -3584,14 +3552,14 @@ exit 0
# QueryFullProcessImageNameW answers for processes whose MainModule is not
# readable here: PROCESS_QUERY_LIMITED_INFORMATION is granted where the
- # PROCESS_VM_READ that MainModule needs is refused, and it asks nothing of WMI.
+ # PROCESS_VM_READ that MainModule needs is refused, and it needs no WMI.
# Without it a host can find NO running processes and overwrite a venv Unsloth
# has open, so the ladder still ends at Get-Process and Win32_Process. Every
# rung reports a real executable image; a command line or working directory
# mentioning the path is never proof.
#
- # Emitted rather than compiled, like every other native declaration here. The
- # ladder itself is unchanged from before that switch.
+ # Emitted rather than compiled, like every other native declaration here; the
+ # ladder itself is unchanged.
$script:StudioProcessImageNativeState = $null
function Initialize-StudioProcessImageNativeType {
if ("UnslothStudioProcessImageV1" -as [type]) {
@@ -3599,12 +3567,11 @@ exit 0
return $true
}
if ($null -ne $script:StudioProcessImageNativeState) { return $script:StudioProcessImageNativeState }
- # A type this session already emitted outranks any probe. The compiled version
- # could not disagree with itself here, because one type carried the path helper
- # and this one; two types can, and the way they do is a session that emitted the
- # path type, was interrupted, and came back to a probe that now fails. Asking a
- # child whether emit works, in a process where emit demonstrably worked, is a
- # question with an answer already on the table.
+ # A type this session already emitted outranks any probe. The compiled
+ # version carried the path helper and this one in a single type and so could
+ # not disagree with itself; two types can, when a session emits the path type
+ # and then meets a probe that now fails. Do not ask a child whether emit
+ # works in a process where emit demonstrably worked.
$alreadyEmitted = $null -ne ("UnslothStudioFinalPathV3" -as [type])
if (-not $alreadyEmitted -and -not (Test-StudioCanDefineNativeTypes)) {
$script:StudioProcessImageNativeState = $false
@@ -3636,9 +3603,8 @@ exit 0
}
# The body of the compiled GetProcessImagePath, moved out of C# so the imports
- # above are all the native code there is. Same flags, same buffer. Null rather
- # than an exception on any failure, which is what the compiled one returned and
- # what the caller below skips on.
+ # above are all the native code there is. Same flags, same buffer. Null on any
+ # failure, as the compiled one returned and as the caller below skips on.
function Get-StudioNativeProcessImagePath {
param([Parameter(Mandatory = $true)][int]$ProcessId)
$queryLimitedInformation = [uint32]0x1000
diff --git a/studio/setup.ps1 b/studio/setup.ps1
index 7d5872b62ea..f439fab17ef 100644
--- a/studio/setup.ps1
+++ b/studio/setup.ps1
@@ -1593,19 +1593,17 @@ $Rule = [string]::new([char]0x2500, 52)
# Native declarations are emitted, never compiled. Add-Type on Windows PowerShell 5.1 has no
# in-process compiler: -TypeDefinition and -MemberDefinition alike write C# to %TEMP% and run
-# csc.exe to get a DLL back, and behavioural antivirus blocks the result, because a windowless
-# PowerShell launching a compiler and writing executable content to %TEMP% is a dropper's shape
-# whatever the code says. Reflection emit builds the same stub in memory: no compiler process, no
-# source on disk, no DLL, and an assembly whose Location is empty. install.ps1 carries the same
-# helper, and for the same reason.
+# csc.exe, and behavioural antivirus blocks the result, because a windowless PowerShell launching
+# a compiler and writing executable content to %TEMP% is a dropper's shape whatever the code says.
+# Reflection emit builds the same stub in memory: no compiler process, no source, no DLL, empty
+# assembly Location. install.ps1 carries the same helper for the same reason.
# See install.ps1: App Control's Dynamic Code Security always blocks loading unsigned
-# assemblies built with System.Reflection.Emit, and Microsoft documents the parent process as
-# usually stopped or crashing rather than raising, so this has to be a gate and not a catch.
+# System.Reflection.Emit assemblies by usually stopping or crashing the parent rather than
+# raising, so this has to be a gate and not a catch.
$script:StudioCanDefineNativeTypes = $null
# Why the last probe answered as it did, so a caller can tell "the child ran and
-# said no" from "the child never got to answer". Those are the same boolean and
-# they are not the same fact: one is a policy, the other is a process that failed
-# to start, was killed at the deadline, or lost its output.
+# said no" (a policy) from "the child never answered" (failed to start, killed
+# at the deadline, or lost its output). Same boolean, different facts.
$script:StudioEmitProbeOutcome = $null
function Test-StudioCanDefineNativeTypes {
if ($null -ne $script:StudioCanDefineNativeTypes) { return $script:StudioCanDefineNativeTypes }
@@ -1616,18 +1614,15 @@ function Test-StudioCanDefineNativeTypes {
return $false
}
# Read-and-zero is the only outcome that skips the probe. A query that threw, returned
- # nothing, or returned an object without the property is UNKNOWN, and treating unknown as
- # unrestricted lets option 19 through on a host whose CIM query failed. install.ps1
- # carries the full note.
+ # nothing, or lacked the property is UNKNOWN, and treating unknown as unrestricted lets
+ # option 19 through on a host whose CIM query failed. install.ps1 carries the full note.
$known = $false
$active = $false
try {
- # Bounded, with what that bound actually covers stated rather than implied:
- # -OperationTimeoutSec limits the CIM operation on a responsive target. It does
- # not interrupt DCOM connection setup and it does not override a provider that
- # has wedged, where the server's own timeout wins. It is worth having for the
- # slow case and it is not a hang guard. The child probe below is the part that
- # carries a real deadline.
+ # -OperationTimeoutSec bounds the CIM operation on a responsive target only:
+ # it does not interrupt DCOM connection setup, and a wedged provider's own
+ # timeout wins. Good for the slow case, not a hang guard. The child probe
+ # below carries the real deadline.
$guard = Get-CimInstance -Namespace "root\Microsoft\Windows\DeviceGuard" `
-ClassName "Win32_DeviceGuard" -OperationTimeoutSec 10 -ErrorAction Stop
# 0 off, 1 audit, 2 enforced. A null property is not a zero.
@@ -1647,12 +1642,12 @@ function Test-StudioCanDefineNativeTypes {
# an audit policy before Windows 11 24H2, while an audit policy without it emits fine. So a
# child process tries it. Same reasoning as install.ps1, which carries the full note.
$script:StudioCanDefineNativeTypes = Test-StudioEmitInChildProcess
- # One retry, and only when the first attempt never reached an answer. The compiled
- # version this replaces also tried twice before caching a negative, and without
- # that a single transient process failure is indistinguishable from a policy: it
- # is cached for the whole run and sends the installer down the lexical path, where
- # two unequal roots compare as unknown and a second lock gets taken. A child that
- # RAN and said no is not retried, so a genuinely blocked machine still pays for
+ # One retry, only when the first attempt never reached an answer (the
+ # compiled version this replaces also tried twice before caching a
+ # negative). Otherwise one transient process failure is cached for the whole
+ # run as if it were a policy, sending the installer down the lexical path
+ # where two unequal roots compare as unknown and a second lock gets taken. A
+ # child that RAN and said no is not retried, so a blocked machine pays for
# one probe.
if (-not $script:StudioCanDefineNativeTypes -and
$script:StudioEmitProbeOutcome -eq "indeterminate") {
@@ -1664,11 +1659,10 @@ function Test-StudioCanDefineNativeTypes {
# The same emit, in a process that is allowed to die. A blocked dynamic load usually stops the
# parent, so this is asked in a child; silence is refusal.
function Test-StudioEmitInChildProcess {
- # HostPath is for the tests, which have no policy to trigger the real path and cannot
- # shadow $PSHOME, since it is read-only. Production never passes it.
+ # HostPath is for the tests, which have no policy to trigger the real path and
+ # cannot shadow the read-only $PSHOME. Production never passes it.
param([string]$HostPath)
- # Until something here establishes otherwise. Every exit below either leaves this
- # alone or says what it learned.
+ # Until something below establishes otherwise.
$script:StudioEmitProbeOutcome = "indeterminate"
$probe = @'
try {
@@ -1697,10 +1691,9 @@ try {
if ('UnslothStudioEmitProbe' -as [type]) { Write-Output ('STUDIO_EMIT_OK ' + [string]$ExecutionContext.SessionState.LanguageMode); exit 0 }
exit 1
'@
- # This host, not a guessed one: a 5.1 answer does not carry to pwsh or the other
- # way, and the emit that matters is the one this interpreter will make. Both
- # spellings of the leaf, so the function is the same one a non-Windows lane can
- # execute end to end rather than a Windows-only path nothing tests.
+ # This host, not a guessed one: a 5.1 answer does not carry to pwsh or back.
+ # Both spellings of the leaf, so a non-Windows lane can execute this function
+ # end to end rather than leaving a Windows-only path untested.
$hostExe = $HostPath
if (-not $hostExe) {
try {
@@ -1713,17 +1706,15 @@ exit 1
} catch {}
}
if (-not $hostExe) { return $false }
- # Through a Process object rather than the call operator, for a deadline. The call
- # operator waits for the child forever, and "forever" is reachable: a security
- # product inspecting a freshly spawned interpreter, a wedged runtime start, a child
- # that blocks on shutdown. A probe that exists to keep the installer alive must not
- # be the thing that hangs it.
+ # A Process object rather than the call operator, for a deadline: the call
+ # operator waits forever, and forever is reachable (a security product
+ # inspecting a fresh interpreter, a wedged runtime start, a child blocking on
+ # shutdown). A probe meant to keep the installer alive must not hang it.
#
- # BOTH streams are redirected and drained asynchronously. Draining is what stops a
- # chatty child filling a pipe and deadlocking against the wait. Redirecting stderr as
- # well is the difference between a probe that is invisible and one that can write
- # into the installer's own stderr, which the desktop app reads, and which anything
- # the child spawns would inherit and hold open.
+ # BOTH streams are redirected and drained asynchronously. Draining stops a
+ # chatty child filling a pipe and deadlocking against the wait. Redirecting
+ # stderr keeps the probe out of the installer's own stderr, which the desktop
+ # app reads and anything the child spawns would inherit and hold open.
#
$info = New-Object System.Diagnostics.ProcessStartInfo
$info.FileName = $hostExe
@@ -1741,11 +1732,11 @@ exit 1
try { $child.Kill() } catch {}
return $false
}
- # Exit code AND an exact record. A marker followed by a crash is a crash: the
- # question is whether this machine can emit and live, and a child that printed
- # and then died has answered no. FullLanguage because an approved script can run
- # in FullLanguage while a fresh inline command does not, and a child restricted
- # differently from its parent has measured a different machine.
+ # Exit code AND an exact record. A marker followed by a crash is a crash:
+ # the question is whether this machine can emit and live. FullLanguage
+ # because an approved script can run in FullLanguage while a fresh inline
+ # command does not, and a child restricted differently from its parent
+ # has measured a different machine.
if ($child.ExitCode -ne 0) {
$script:StudioEmitProbeOutcome = "blocked"
return $false
@@ -1756,23 +1747,23 @@ exit 1
$script:StudioEmitProbeOutcome = "ok"
return $true
}
- # Emitted, but in a language mode this parent is not in. The child measured a
- # different machine, which is an answer rather than a missed one.
+ # Emitted, but in a language mode this parent is not in: the child
+ # measured a different machine, which is an answer, not a miss.
if ($line.Trim() -like "STUDIO_EMIT_OK *") {
$script:StudioEmitProbeOutcome = "blocked"
return $false
}
}
- # Exit 0 with no marker at all: the child cannot have got past the emit and then
- # reported nothing, so its output was lost rather than negative.
+ # Exit 0 with no marker: the child cannot have emitted and reported
+ # nothing, so its output was lost rather than negative.
return $false
} catch {
return $false
} finally {
if ($child) {
- # The read end goes first. A killed child can leave a grandchild holding the
- # write end of that pipe, and the pending async read then keeps this process
- # alive past the deadline it just enforced.
+ # The read end goes first: a killed child can leave a grandchild
+ # holding the write end, and the pending async read would then keep
+ # this process alive past the deadline it just enforced.
try { $child.StandardOutput.Close() } catch {}
try { $child.StandardError.Close() } catch {}
try { $child.Dispose() } catch {}
@@ -1782,16 +1773,16 @@ exit 1
function New-StudioDynamicAssembly {
<#
- Both spellings of "define a dynamic assembly", because the two PowerShell hosts that
- run this file are on different runtimes. The static
- AssemblyBuilder::DefineDynamicAssembly is documented for .NET Framework 4.5 through
- 4.8.1 as well as .NET Core, so Windows PowerShell 5.1 should take the first branch. It
- is tried rather than assumed because nothing here can test a .NET Framework host, and
- getting it wrong is not a visible error: the catch would cache the thunk as unavailable
- and every install would silently lose it.
-
- AppDomain.CurrentDomain.DefineDynamicAssembly is the .NET Framework spelling and is
- absent on .NET Core, so it is the fallback rather than the first try.
+ Both spellings of "define a dynamic assembly", because the two PowerShell hosts
+ that run this file are on different runtimes. The static
+ AssemblyBuilder::DefineDynamicAssembly is documented for .NET Framework 4.5
+ through 4.8.1 as well as .NET Core, so 5.1 should take the first branch; it is
+ tried rather than assumed because nothing here can test a .NET Framework host
+ and getting it wrong is invisible: the catch would cache the thunk as
+ unavailable and every install would silently lose it.
+
+ AppDomain.CurrentDomain.DefineDynamicAssembly is the .NET Framework spelling and
+ is absent on .NET Core, so it is the fallback.
#>
param([Parameter(Mandatory = $true)][System.Reflection.AssemblyName]$AssemblyName)
$access = [System.Reflection.Emit.AssemblyBuilderAccess]::Run
@@ -1801,9 +1792,9 @@ function New-StudioDynamicAssembly {
} catch [System.Management.Automation.MethodException] {
return [AppDomain]::CurrentDomain.DefineDynamicAssembly($AssemblyName, $access)
} catch [System.Management.Automation.RuntimeException] {
- # A missing static surfaces as a RuntimeException on some hosts rather than a
- # MethodException. Both mean "no such method here", and a real emit failure throws
- # from the AppDomain call too, so the caller still sees it.
+ # Some hosts surface a missing static as RuntimeException, not
+ # MethodException. Both mean "no such method here", and a real emit failure
+ # throws from the AppDomain call too, so the caller still sees it.
return [AppDomain]::CurrentDomain.DefineDynamicAssembly($AssemblyName, $access)
}
}
@@ -1820,9 +1811,10 @@ function New-StudioEmittedNativeType {
$TypeName, "Public, Class, AutoClass, AnsiClass, BeforeFieldInit")
$winapi = [System.Runtime.InteropServices.CallingConvention]::Winapi
- # Per import, because CharSet selects name mangling as well as marshalling: Unicode probes
- # <Name>W before <Name>, Ansi probes <Name> before <Name>A. Declaring the one the C# this
- # replaces declared keeps the metadata honest and puts the export that does exist first.
+ # Per import, because CharSet selects name mangling as well as marshalling:
+ # Unicode probes <Name>W before <Name>, Ansi probes <Name> before <Name>A.
+ # Matching the C# these replace keeps the metadata honest and tries the existing
+ # export first.
$unicode = [System.Runtime.InteropServices.CharSet]::Unicode
$ansi = [System.Runtime.InteropServices.CharSet]::Ansi
$standard = [System.Reflection.CallingConventions]::Standard
@@ -1836,13 +1828,15 @@ function New-StudioEmittedNativeType {
$standard, $import.Return, $import.Args, $winapi, $charSet)
$method.SetImplementationFlags(
$method.GetMethodImplementationFlags() -bor $preserveSig)
- # `out uint` in the C# this replaces, and DefinePInvokeMethod has no way to say
- # so: a by-ref type alone emits `ref`, which is In and Out unset. The value is
- # blittable and every caller initialises it first, so the marshaller pins and
- # writes back either way, but the metadata is what a reader and any future
- # marshalling change go by, so it says what the declaration said.
- # ContainsKey, not a bare property read: most imports have no Out key and reading a
- # missing one is fatal under a caller's Set-StrictMode. setup.ps1 sets none of its own.
+ # `out uint` in the C# this replaces; DefinePInvokeMethod cannot say so,
+ # since a by-ref type alone emits `ref` (In and Out unset). The value is
+ # blittable and every caller initialises it, so marshalling works either
+ # way, but the metadata is what a reader and any future marshalling
+ # change go by.
+ # ContainsKey, not a bare property read: most imports have no Out key and
+ # reading a missing one is fatal under Set-StrictMode. install.ps1 turns
+ # strict mode off for itself, studio/setup.ps1 inherits the caller's, so
+ # the guard is mirrored rather than left to one of them.
if ($import.ContainsKey("Out")) {
foreach ($position in @($import.Out)) {
if ($position) { $null = $method.DefineParameter($position, "Out", $null) }
@@ -1859,10 +1853,9 @@ function Enable-StudioVirtualTerminal {
# block below could only return $false anyway. The CLI and the desktop app both pipe us, so
# that is the path they are on.
if ($script:StudioStdoutRedirected) { return $false }
- # The published type first, the gate only if there is nothing published. A type
- # this session already emitted is proof that emit works here, and the compiled
- # version checked for it in the same order; asking a child instead means one
- # failed probe throws away a console helper that is already loaded and usable.
+ # The published type first, the gate only if there is nothing published: a
+ # type this session already emitted proves emit works here, and asking a
+ # child instead lets one failed probe throw away a usable console helper.
if (-not ("StudioVTNative" -as [type]) -and -not (Test-StudioCanDefineNativeTypes)) {
return $false
}
diff --git a/tests/python/test_windows_installer_concurrency_guard.py b/tests/python/test_windows_installer_concurrency_guard.py
index 184de5aa638..f4f507efdc3 100644
--- a/tests/python/test_windows_installer_concurrency_guard.py
+++ b/tests/python/test_windows_installer_concurrency_guard.py
@@ -43,9 +43,9 @@ def _run_powershell(shell: str, script: str, env: dict[str, str]) -> str:
Path(name).write_text(script, encoding = "utf-8-sig")
result = subprocess.run(
[shell, "-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-File", name],
- # check = False, then asserted below with the output attached. With check = True the
- # only thing a failing run reports is "exit status 1" and the PowerShell error that
- # caused it is discarded, which on a CI runner is the whole diagnosis gone.
+ # check = False, then asserted below with the output attached. check = True
+ # reports only "exit status 1" and discards the PowerShell error, which on a
+ # CI runner is the whole diagnosis.
check = False,
capture_output = True,
text = True,
diff --git a/tests/python/test_windows_installer_native_type_fallback.py b/tests/python/test_windows_installer_native_type_fallback.py
index 48e6a35c4a5..e2e608ccbb6 100644
--- a/tests/python/test_windows_installer_native_type_fallback.py
+++ b/tests/python/test_windows_installer_native_type_fallback.py
@@ -46,15 +46,14 @@ INSTALL_PS1 = REPO_ROOT / "install.ps1"
requires_pwsh = pytest.mark.skipif(shutil.which("pwsh") is None, reason = "PowerShell is unavailable")
# A compiler that always fails. install.ps1 no longer calls Add-Type at all, so this
-# now proves the absence rather than exercising a fallback: every test that installs
-# it must behave exactly as it does without it.
+# proves the absence: every test that installs it must behave as it does without it.
SABOTAGE = (
"""function Add-Type { throw "(0) : error CS2001: Source file 'a.0.cs' could not be found" }"""
)
-# What a host that cannot emit looks like. Constrained Language Mode and App
-# Control's Dynamic Code Security both land here, and Test-StudioCanDefineNativeTypes
-# is the gate they come through, so overriding it is the whole of "no native side".
+# What a host that cannot emit looks like. Constrained Language Mode and App Control's
+# Dynamic Code Security both come through Test-StudioCanDefineNativeTypes, so overriding
+# that gate is the whole of "no native side".
NO_NATIVE = """function Test-StudioCanDefineNativeTypes { return $false }"""
@@ -306,8 +305,8 @@ Write-Output "TEMP:$env:TEMP"
assert _lines(result, "TMP:") == [f"TMP:{dead}"]
assert _lines(result, "TEMP:") == [f"TEMP:{dead}"]
assert list((local_app_data / "Unsloth Studio" / "temp").glob("ust-*")) == []
- # The imports target kernel32, so off Windows the type builds and the CALL fails; it
- # has to degrade to a usable answer rather than throw.
+ # The imports target kernel32, so off Windows the type builds and the CALL fails:
+ # it must degrade to a usable answer rather than throw.
assert _lines(result, "PATH:")[0].startswith("PATH:")
assert _lines(result, "PATH:") != ["PATH:"]
@@ -1162,7 +1161,7 @@ Write-Output "CALLS:$([UnslothStudioFinalPathV3]::Calls)"
assert _lines(result, "PATH:")[0].endswith("studio")
warnings = [line for line in result.stdout.splitlines() if "native helper; continuing" in line]
assert len(warnings) == 1, warnings
- # The throwing resolver was really reached, four times, rather than the whole thing
+ # The throwing resolver was really reached, four times, rather than everything
# falling back because the type was absent.
assert _lines(result, "CALLS:") == ["CALLS:4"]
@@ -1728,12 +1727,11 @@ def _gate(source: str) -> str:
)
-# Status 0 is "no policy", and it answers without spawning anything. Anything else is a
-# policy whose OPTIONS decide the answer, and Win32_DeviceGuard does not report them:
-# option 19 Dynamic Code Security always blocks unsigned System.Reflection.Emit assemblies
-# and is enforced even in an audit policy before Windows 11 24H2, while an audit policy
-# without it emits perfectly well. So the gate asks a child process rather than guessing,
-# and what these assert is that it delegates rather than deciding.
+# Status 0 is "no policy" and answers without spawning anything. Anything else is a policy
+# whose OPTIONS decide the answer, and Win32_DeviceGuard does not report them: option 19
+# Dynamic Code Security always blocks unsigned System.Reflection.Emit assemblies and is
+# enforced even in an audit policy before Windows 11 24H2, while an audit policy without it
+# emits fine. So the gate asks a child process, and these assert that it delegates.
@requires_pwsh
@pytest.mark.parametrize("script", ["install", "setup"])
@pytest.mark.parametrize("status", ["1", "2"])
@@ -1792,10 +1790,9 @@ def test_no_policy_answers_without_spawning_a_probe(script: str):
assert _lines(result, "CALLS:") == ["CALLS:0"]
-# The runtime test below catches this by executing it, but only on a host where emit
-# succeeds. This one is the invariant itself, and it is the one a future edit trips: adding
-# a single double quote to the probe body is enough, and nothing about the resulting failure
-# points back at the quote.
+# The runtime test below catches this by executing it, but only where emit succeeds. This
+# one is the invariant itself, and the one a future edit trips: one double quote added to
+# the probe body is enough, and the resulting failure points nowhere near the quote.
@pytest.mark.parametrize("script", ["install", "setup"])
def test_the_probe_body_carries_no_double_quote(script: str):
source = (INSTALL_PS1 if script == "install" else SETUP_PS1).read_text(encoding = "utf-8")
@@ -1809,10 +1806,9 @@ def test_the_probe_body_carries_no_double_quote(script: str):
# The acceptance rules, one child at a time. $PSHOME is an ordinary variable, so a fake host
-# under a temporary one lets each rule be exercised without a policy, a Windows box or luck.
-# Every case here is a way a real child can answer badly: it printed the marker and then died,
-# it ran in a language mode its parent did not, it printed something that merely CONTAINS the
-# marker, or it never returned at all.
+# under a temporary one exercises each rule without a policy, a Windows box or luck. Each case
+# is a way a real child answers badly: printed the marker then died, ran in a language mode
+# its parent did not, printed something that merely CONTAINS the marker, or never returned.
# A #!/bin/sh host, so POSIX only: Windows CreateProcess rejects a file with no
# executable format, every case would come back false through the catch, and the
# accept case would fail while the deadline case passed without waiting.
@@ -1866,18 +1862,17 @@ def test_the_probe_only_accepts_a_clean_exact_answer(
)
assert result.returncode == 0, f"{label}: {result.stderr}"
assert _lines(result, "ANSWER:") == [f"ANSWER:{expected}"], label
- # And WHY, which is not the same fact as the boolean. A child that ran and refused is a
- # machine that cannot emit; a child that never answered is a process that failed, and
- # the gate retries only the second one and reports a different reason for it.
+ # And WHY, which is not the same fact as the boolean: a child that ran and refused is a
+ # machine that cannot emit, a child that never answered is a failed process. The gate
+ # retries only the second and reports a different reason for it.
assert _lines(result, "OUTCOME:") == [f"OUTCOME:{outcome}"], label
# One transient process failure used to be indistinguishable from a policy: cached for the
# whole run, it sends the installer down the lexical path, where two unequal roots compare as
-# unknown and a second runtime lock is taken, which is how an unrelated install elsewhere on
-# the machine turns into "the managed Unsloth environment is busy". The compiled version this
-# replaces tried twice before caching a negative. A machine that really is blocked still pays
-# for exactly one probe, so the retry costs nothing where the answer was already real.
+# unknown and a second runtime lock is taken, turning an unrelated install elsewhere into
+# "the managed Unsloth environment is busy". The compiled version tried twice before caching
+# a negative. A genuinely blocked machine still pays for exactly one probe.
@requires_pwsh
@pytest.mark.parametrize("script", ["install", "setup"])
@pytest.mark.parametrize(
@@ -1922,8 +1917,7 @@ def test_only_a_probe_that_never_answered_is_retried(script: str, outcome: str,
# The reason printed alongside the degradation. Naming a machine setting for what was really
-# a process that could not be spawned sends whoever reads the log looking for a policy that
-# is not there.
+# an unspawnable process sends whoever reads the log looking for a policy that is not there.
@requires_pwsh
@pytest.mark.parametrize(
"outcome,expected",
@@ -1961,9 +1955,9 @@ def test_the_degradation_reason_says_what_was_actually_established(outcome: str,
# The compiled version carried the path helper and the process-image helper on ONE type, so
# they could not disagree about whether this session can emit. Two types can: a session that
-# emitted the path type, was interrupted, and came back to a probe that now fails would keep
-# native path resolution and silently lose native process inspection, which is how a running
-# Studio stops being seen. A type already published here is stronger evidence than any child.
+# emitted the path type and came back to a probe that now fails would keep native path
+# resolution and silently lose native process inspection, which is how a running Studio stops
+# being seen. A type already published here outweighs any child.
@requires_pwsh
def test_an_already_emitted_type_settles_it_without_asking_a_child():
source = INSTALL_PS1.read_text(encoding = "utf-8")
@@ -2006,8 +2000,8 @@ def test_an_already_emitted_type_settles_it_without_asking_a_child():
# Same rule for the cosmetic helpers. The console one asked the gate before looking for its
-# type, so one failed probe threw away a console helper already loaded and working in this
-# very process. The compiled version checked the type first.
+# type, so one failed probe threw away a helper already loaded and working in this process.
+# The compiled version checked the type first.
@requires_pwsh
@pytest.mark.parametrize("script", ["install", "setup"])
def test_the_console_helper_keeps_a_type_it_already_has(script: str):
@@ -2077,13 +2071,13 @@ def test_a_child_that_never_returns_does_not_hang_the_installer(tmp_path: Path):
assert elapsed < 60, f"the probe took {elapsed:.0f}s, so the deadline is not bounding it"
-# Legacy is not a curiosity: it is how Windows PowerShell 5.1 ALWAYS binds a native
-# command's arguments, and 5.1 is the interpreter studio/src-tauri/src/install.rs spawns.
-# It wraps the value in quotes and appends the body verbatim without escaping the quotes
-# inside it, so a probe passed with -Command arrives as
-# `if (UnslothStudioEmitProbe -as [type])`, a command lookup that throws into the probe's
-# own catch and answers "no emit here" on every 5.1 host. pwsh can be put into that exact
-# binder with $PSNativeCommandArgumentPassing, so the case is reachable from Linux.
+# Legacy is how Windows PowerShell 5.1 ALWAYS binds a native command's arguments, and 5.1
+# is the interpreter studio/src-tauri/src/install.rs spawns. It wraps the value in quotes
+# and appends the body verbatim without escaping the quotes inside it, so a probe passed
+# with -Command arrives as `if (UnslothStudioEmitProbe -as [type])`, a command lookup that
+# throws into the probe's own catch and answers "no emit here" on every 5.1 host. pwsh can
+# be put into that binder with $PSNativeCommandArgumentPassing, so this is reachable from
+# Linux.
@requires_pwsh
@pytest.mark.parametrize("script", ["install", "setup"])
@pytest.mark.parametrize("binding", ["Legacy", "Standard"])
@@ -2104,8 +2098,8 @@ def test_the_child_probe_survives_the_5_1_argument_binder(script: str, binding:
assert result.returncode == 0, result.stderr
# Boolean either way, since the gate calls this under ErrorActionPreference Stop.
assert _lines(result, "TYPE:") == ["TYPE:Boolean"]
- # And True either way: emit works on this host, and the binder must not be able to
- # turn a working host into a refusal.
+ # And True either way: emit works on this host, and the binder must not turn a
+ # working host into a refusal.
assert _lines(result, "ANSWER:") == ["ANSWER:True"]
@@ -2152,8 +2146,8 @@ def test_an_unreadable_policy_asks_the_probe_rather_than_assuming(
# CharSet is not decoration on an emitted import: it picks the export the runtime looks for
# first. Unicode asks for <Name>W, Ansi asks for <Name>. Every one of these names an export
-# that exists exactly as written, so the wrong charset still resolves, on the second probe;
-# what it stops being is an accurate description of the C# it replaced.
+# that exists exactly as written, so a wrong charset still resolves on the second probe; it
+# just stops describing the C# it replaced.
@requires_pwsh
@pytest.mark.parametrize(
"method,charset",
@@ -2196,11 +2190,10 @@ def test_each_import_carries_the_charset_its_declaration_had(method: str, charse
assert _lines(result, "LIB:") == ["LIB:kernel32.dll"]
-# `out uint` in the C# this replaces. A by-ref type alone emits `ref`, which leaves both
-# In and Out unset, and DefinePInvokeMethod has no argument for it, so the emitter calls
-# DefineParameter. The value is blittable and the callers initialise it first, so writeback
-# works either way; what this pins is that the metadata says what the declaration said, and
-# that the Out key is wired through the emitter at all.
+# `out uint` in the C# this replaces. A by-ref type alone emits `ref`, leaving In and Out
+# unset, and DefinePInvokeMethod has no argument for it, so the emitter calls DefineParameter.
+# The value is blittable and callers initialise it first, so writeback works either way; this
+# pins that the metadata says what the declaration said and that the Out key is wired through.
@requires_pwsh
@pytest.mark.parametrize("script", ["install", "setup"])
def test_the_console_mode_parameter_is_still_declared_out(script: str):
@@ -2227,20 +2220,20 @@ def test_the_console_mode_parameter_is_still_declared_out(script: str):
assert result.returncode == 0, result.stderr
assert _lines(result, "OUT:") == ["OUT:True"]
assert _lines(result, "BYREF:") == ["BYREF:True"]
- # And the real declaration in the script carries the key, not just the emitter's ability
- # to honour it.
+ # And the real declaration in the script carries the key, not just the emitter's
+ # ability to honour it.
block = re.search(r'Name = "GetConsoleMode".*?\}', source, flags = re.DOTALL)
assert block is not None and "Out = @(2)" in block.group(
0
), "GetConsoleMode lost its Out position"
-# Almost every import declares no Out position, and reading a hashtable key that is not
-# there is fatal under Set-StrictMode -Version Latest, which raises PropertyNotFound rather
-# than returning $null. install.ps1 sets strict mode Off for itself, but setup.ps1 sets none
-# at all, so whatever the caller has is what runs, and a profile with strict mode on would
-# have taken out every emitted type. The emitter asks ContainsKey first, the way it already
-# does for the CharSet key, and this runs the whole emitter under the strictest setting.
+# Almost every import declares no Out position, and reading an absent hashtable key is fatal
+# under Set-StrictMode -Version Latest, which raises PropertyNotFound rather than returning
+# $null. install.ps1 sets strict mode Off for itself but setup.ps1 sets none, so the caller's
+# setting runs and a profile with strict mode on would have taken out every emitted type. The
+# emitter asks ContainsKey first, as it does for CharSet; this runs it under the strictest
+# setting.
@requires_pwsh
@pytest.mark.parametrize("script", ["install", "setup"])
def test_an_import_without_an_out_position_survives_strict_mode(script: str):
diff --git a/tests/studio/test_installer_av_shapes.py b/tests/studio/test_installer_av_shapes.py
index 13bbd12cf51..997bd85468f 100644
--- a/tests/studio/test_installer_av_shapes.py
+++ b/tests/studio/test_installer_av_shapes.py
@@ -111,9 +111,8 @@ def test_a_hidden_window_never_pairs_with_a_bypassed_policy(name: str) -> None:
# Every native import left in the installers, however it is declared. Both scripts define theirs through reflection
-# emit now, which costs no compile at all: install.ps1 the path resolver, the console thunk, the icon refresh and the
-# process-image lookup, studio/setup.ps1 the console thunk. A new entry still needs a reason, and a PowerShell
-# equivalent usually exists.
+# emit now, which costs no compile: install.ps1 the path resolver, console thunk, icon refresh and process-image
+# lookup, studio/setup.ps1 the console thunk. A new entry needs a reason; a PowerShell equivalent usually exists.
ALLOWED_PINVOKES = {
# Canonicalising linked ancestors of security-relevant paths.
# No PS 5.1 equivalent: ResolveLinkTarget is .NET 6+, and .Target misses a linked ancestor of a non-link leaf.
@@ -132,9 +131,9 @@ ALLOWED_PINVOKES = {
# ie4uinit.exe -show is the global broadcast, which alone does not recover a stale .lnk, so it is not a substitute.
"SHChangeNotify",
# Naming the image behind a pid, so a venv Unsloth still has open is not overwritten.
- # PROCESS_QUERY_LIMITED_INFORMATION only, and it is the rung the others cannot replace: Process.Path goes through
- # MainModule, which needs PROCESS_VM_READ and is refused across users and across bitness, and Win32_Process needs a
- # working WMI service. Without this the scan can find nothing and let the install proceed over an open venv.
+ # PROCESS_QUERY_LIMITED_INFORMATION only, and the others cannot replace it: Process.Path goes through MainModule,
+ # which needs PROCESS_VM_READ and is refused across users and bitness, and Win32_Process needs a working WMI
+ # service. Without it the scan can find nothing and proceed over an open venv.
"OpenProcess",
"QueryFullProcessImageNameW",
# Closing the handles CreateFileW and OpenProcess opened.
@@ -142,9 +141,9 @@ ALLOWED_PINVOKES = {
}
-# The two ways a native import can be declared. Add-Type takes C# and runs csc.exe; DefinePInvokeMethod builds the
-# same stub in memory. The second is invisible to a DllImport regex, so without this the inventory above would silently
-# stop covering install.ps1 the moment it stopped compiling.
+# Both ways a native import can be declared: Add-Type runs csc.exe over C#, DefinePInvokeMethod builds the same stub
+# in memory. The second is invisible to a DllImport regex, so without it the inventory above would stop covering
+# install.ps1 the moment it stopped compiling.
def _native_imports(text: str) -> set:
imported = set()
for match in re.finditer(
@@ -174,15 +173,14 @@ def test_no_new_native_imports(name: str) -> None:
def test_virtual_terminal_answers_a_redirected_stream_without_defining_a_type(name: str) -> None:
"""The answer we already know must come first, before any native work at all.
- Only the redirected case is decided early, and it is decided FALSE. A redirected stdout is
- not a console, GetConsoleMode fails on a non-console handle, and the native path could
- only have returned false too. Anything that claimed VT here would put raw escape sequences
- in the Unsloth log panel, which is a pipe.
+ Only the redirected case is decided early, and it is decided FALSE: a redirected stdout is
+ not a console, GetConsoleMode fails on a non-console handle, and the native path could only
+ have returned false too. Anything claiming VT here would put raw escape sequences in the
+ Unsloth log panel, which is a pipe.
- This used to guard an Add-Type, back when the redirect check was the only thing keeping the
- desktop app off csc.exe. Nothing here compiles now, so the ordering is no longer load-bearing
- against a scanner; it is still the cheaper answer, and getting it wrong still corrupts the
- log panel.
+ This used to guard an Add-Type, when the redirect check was all that kept the desktop app
+ off csc.exe. Nothing compiles now, so the ordering no longer matters to a scanner, but it is
+ still the cheaper answer and getting it wrong still corrupts the log panel.
"""
text = _text(name)
start = text.index("function Enable-StudioVirtualTerminal")
@@ -245,12 +243,10 @@ def test_the_installer_never_runs_the_c_sharp_compiler(name: str) -> None:
no source on disk, no DLL, nothing in %TEMP%.
Add-Type in full, not only -TypeDefinition: -MemberDefinition wraps its argument in a class
- and compiles that, so it reaches csc.exe by the same road. -AssemblyName is the exception,
- and the only one: it loads an assembly that already exists on disk and never reaches a
- compiler. Both scripts, not only the bundled one: leaving a compile anywhere means the answer
- to "does this run a compiler" depends on which entrypoint ran and whether an early return
- happened to come first, and a guard that holds only conditionally is what let this reach the
- field.
+ and compiles that too. -AssemblyName is the only exception, since it loads an assembly that
+ already exists on disk. Both scripts, because a compile left anywhere makes "does this run a
+ compiler" depend on which entrypoint ran and whether an early return came first, and a guard
+ that holds only conditionally is what let this reach the field.
"""
text = _text(name)
hits = re.findall(r"(?m)^[ \t]*Add-Type\b(?![^\r\n]*-AssemblyName).*", text)
@@ -262,10 +258,10 @@ def test_the_installer_never_runs_the_c_sharp_compiler(name: str) -> None:
assert (
"DefinePInvokeMethod" in text
), f"{name} no longer emits its native imports; update this guard"
- # The private-%TEMP% retry is gone with it. Redirecting TEMP to compile again after a block
- # cannot beat a filter driver, and "blocked writing an executable to TEMP, change TEMP, write
- # it again" is itself an evasion heuristic. Scoped to the resolver: Initialize-StudioTempEnvironment
- # legitimately redirects an unusable inherited TEMP, and that is a different thing.
+ # The private-%TEMP% retry is gone with it: redirecting TEMP to compile again cannot beat a
+ # filter driver, and "blocked writing an executable to TEMP, change TEMP, write it again" is
+ # itself an evasion heuristic. Scoped to the resolver, since Initialize-StudioTempEnvironment
+ # legitimately redirects an unusable inherited TEMP.
# Only install.ps1 has the path resolver; setup.ps1 emits the console thunk and nothing else.
if "function Initialize-StudioFinalPathNativeType" not in text:
return
@@ -279,14 +275,13 @@ def test_the_installer_never_runs_the_c_sharp_compiler(name: str) -> None:
def test_a_ci_lane_fails_when_a_compiler_actually_runs() -> None:
"""The behavioural half of the guard above.
- Reading the scripts cannot see a compile reached through a module, a dot-sourced
- file or a generated here-string, and it cannot see one a dependency performs while
- our process tree is what a scanner scores. Bitdefender scored the chain, not the
- bytes, so there has to be a lane that runs the installer and fails on the process.
+ Reading the scripts cannot see a compile reached through a module, a dot-sourced file
+ or a generated here-string, nor one a dependency performs while our process tree is
+ what a scanner scores. Bitdefender scored the chain, not the bytes, so a lane has to
+ run the installer and fail on the process.
- The positive control is the part worth asserting from here: a detector that sees
- nothing reads exactly like a clean run, and auditing can silently fail to apply. If
- the lane ever loses the control, every later green result stops meaning anything.
+ The positive control is what is worth asserting from here: a detector that sees
+ nothing reads exactly like a clean run, and auditing can silently fail to apply.
"""
workflow = REPO / ".github" / "workflows" / "windows-no-compiler-ci.yml"
assert workflow.is_file(), "the runtime guard lane is gone; the text check is alone again"
@@ -340,12 +335,10 @@ function New-FakeEvent {
def test_the_watcher_scores_the_image_that_ran_not_the_words_in_the_message(
tmp_path, image: str, command_line: str, expected: int
) -> None:
- """4688 renders the command line into the message, so a message search is not a detector.
-
- It scored `cmd.exe /c echo csc.exe` as a compile, and this job's whole output is a
- yes or no about whether a compiler ran under the installer. The record names the
- image it created in its own field; that is what gets read, and matched whole against
- the leaf name rather than as a substring.
+ """4688 renders the command line into the message, so a message search is not a detector:
+ it scored `cmd.exe /c echo csc.exe` as a compile. The record names the image it created
+ in its own field; that is what gets read, matched whole against the leaf name rather
+ than as a substring.
"""
script = tmp_path / "probe.ps1"
script.write_text(
@@ -375,8 +368,8 @@ def test_an_unreadable_security_log_is_void_rather_than_clean() -> None:
"""Get-WinEvent throws both for "nothing matched" and for "could not read".
Swallowing both made a job that could not open the Security log print "no compiler"
- and pass. The positive control does not cover it: it runs in an earlier step, and
- says nothing about whether the log was still readable during the measurement.
+ and pass. The positive control runs in an earlier step and says nothing about whether
+ the log was still readable during the measurement.
"""
body = _WATCHER.read_text(encoding = "utf-8")
assert (
@@ -386,9 +379,8 @@ def test_an_unreadable_security_log_is_void_rather_than_clean() -> None:
def test_the_native_resolver_still_has_a_lexical_fallback() -> None:
- """The point of the change is the acquisition, not the ladder. A host where emit fails has to
- degrade exactly as a host that could not compile already did, which is a path the installer
- has always taken and still completes on.
+ """The point of the change is the acquisition, not the ladder: a host where emit fails must
+ degrade exactly as one that could not compile already did.
"""
text = _text("install.ps1")
assert "Write-StudioFinalPathDegraded" in text