A GPU task manager for Windows, Linux, and WSL: per-process VRAM, GPU activity, and LLM model/KV-cache insights (Windows AMD and NVIDIA).
Python
Improve ARM64 GPU reporting and model discovery
Scott Hanselman0c0cf8feaab9
diff --git a/README.md b/README.mdindex 67cca9d..3026ce3 100644--- a/README.md+++ b/README.md@@ -16,7 +16,9 @@ Your browser? The desktop compositor? And why is so much GPU memory in system RA## Clone and runUse **64-bit Windows 10/11**, an **NVIDIA GPU with a current WDDM driver**, and a terminal-(Windows Terminal is recommended). Development and live workload checks used an RTX 4080 SUPER.+(Windows Terminal is recommended). Live workload checks have included an RTX 4080 SUPER and an+RTX Spark N1X on Windows ARM64. CUDA workloads are supported on ARM64 with a compatible NVIDIA+driver and ARM64 CUDA runtime.Install [Git for Windows](https://git-scm.com/downloads/win) and[uv](https://docs.astral.sh/uv/getting-started/installation/) if needed. With WinGet:@@ -52,12 +54,16 @@ py -m venv .venvNo PowerShell activation script or execution-policy change is needed. The pip route resolvescompatible dependencies; the uv route uses the checked-in lockfile.+If an alternate package index makes `--locked` report a stale lockfile on an unchanged checkout,+`uv run --frozen gputaskman` uses the checked-in lockfile without re-resolving it. `--frozen`+skips the consistency check, so do not use it after changing dependencies.+## What you get- **Device view:** VRAM usage, GPU activity, memory activity, PCIe traffic, temperature, power and clocks.- **Process view:** local GPU memory, GPU-accessible system RAM, estimated non-resident commit,engine activity, and detected runtimes such as CUDA, Vulkan, Direct3D, ggml and ONNX Runtime.-- **Model inspector:** which model each process holds (LLM, decision, embedding, or speech), how+- **Model inspector:** which model each process holds (LLM, decision, embedding, speech, or diffusion), howit was identified, architecture, quantization, layer count, weights, estimated KV cost, andbuffer sizes from compatible load logs.- **Live server hints:** context size, busy slots, throughput when available, and friendly Ollama@@ -106,9 +112,15 @@ uv run --locked gputaskman --helpA wide terminal (roughly 140 columns or more) makes the full table easier to read.Model/runtime discovery is throttled, so newly loaded models may take about 15 seconds to appear.-`--gpu N` is available, but **single-NVIDIA-GPU systems are the validated configuration**.-On multi-GPU systems, DXGI and NVML ordering may differ, and buffer logs can span devices;-cross-device attribution is not yet reliable.+`--gpu N` selects a detected NVIDIA graphics adapter, ordered by dedicated memory; an out-of-range+index is rejected rather than silently selecting another device. Compute-only devices such as an+NPU are not graphics adapters. **Single-NVIDIA-GPU systems are the validated configuration**.+On multi-GPU systems, adapter names may be ambiguous, DXGI and NVML ordering may differ, and buffer+logs can span devices; cross-device attribution is not yet reliable.++Unavailable utilization, temperature, power, clock and PCIe readings are displayed as `N/A` in+text and the TUI and as `null` in JSON. For example, a missing NVML power limit means that the+driver did not expose that reading; it does not by itself indicate a hardware fault.## How can it know what's inside?@@ -168,6 +180,7 @@ reading is NVML's memory-activity percentage, **not measured bandwidth in GB/s**| llama.cpp / llama-server | GGUF discovery, CLI settings, buffer logs, `/props` + `/v1/models` state; exercised live || Ollama | Bundled runners, mapped GGUF blobs named from Ollama manifests (`qwen3:14b`), parent `/api/ps`, compatible `server.log`; exercised live || LM Studio | Mapped files named from its models folder, `/api/v1/models` (falls back to `/api/v0/models`); tested with fixture responses, not yet end to end |+| stable-diffusion.cpp (`--diffusion-model`) | Diffusion, text-encoder and VAE files from launch arguments; no LLM memory breakdown || Hugging Face cache | `models--org--repo` paths are named `org/repo` || Other AI runtimes | DLL detection, plus model files installed with the app (marked *not proven loaded*) |diff --git a/src/gputaskman/app.py b/src/gputaskman/app.pyindex ca9e3bf..5864756 100644--- a/src/gputaskman/app.py+++ b/src/gputaskman/app.py@@ -19,9 +19,11 @@ from textual.worker import get_current_workerfrom .collector import Collector, ProcRow, Snapshotfrom .llm import gb+from .nvml import format_metricKIND_STYLE = {"LLM": "bold magenta", "decision": "bold orchid", "embedding": "plum2", "speech": "light_sky_blue1",+ "diffusion": "orchid","AI": "medium_purple1", "compute": "orange3", "3D/game": "green3","video": "yellow", "browser": "dodger_blue2", "app": "cyan", "compositor": "grey50","kernel": "grey50", "system": "grey50",@@ -42,6 +44,7 @@ EXPLAIN = {"per-resource names need a graphics debugger (PIX on Windows, NVIDIA Nsight Graphics).","video": "Decoded/encoded video frames and codec surfaces (NVDEC/NVENC).","compute": "CUDA allocations are opaque to the OS. For more detail trace cudaMalloc with Nsight Systems.",+ "diffusion": "Model components are identified from launch arguments; GPU allocations cannot be split by component.","AI": "ML runtime allocations (tensors, workspaces). PyTorch: torch.cuda.memory_summary(); ""ONNX Runtime/DirectML pools are opaque from outside.","app": "UI surfaces: swap chains, window back-buffers, cached bitmaps/glyphs.",@@ -55,7 +58,7 @@ HELP_MD = """\| **Device** totals, load, bandwidth, PCIe, power | NVML (`nvml.dll`) | NVIDIA || **Per-process** VRAM, spill to RAM, per-engine load | Windows WDDM counters (dxgkrnl) | every API incl. **CUDA** || **Which runtime** (CUDA, Vulkan, D3D12, ggml, ONNX, PyTorch...) | DLLs mapped in the process | everything |-| **Which model file** | memory-mapped files + command line | llama.cpp, LM Studio, Ollama, llama-cpp-python |+| **Which model file** | memory-mapped files + command line | llama.cpp, stable-diffusion.cpp, LM Studio, Ollama || **Model name & kind** (LLM, decision, embedding, speech) | GGUF header, Ollama/LM Studio/Hugging Face stores, local server APIs | GGUF family || **Weights / KV cache / compute** split | GGUF header math, llama.cpp/Ollama logs, local server APIs | GGUF/llama.cpp family || Per-texture / per-buffer names | needs a debugger (PIX, Nsight) attached to the app | games, 3D apps |@@ -184,8 +187,8 @@ class GpuTaskManager(App):self.snap = snapnv = snap.nvif nv:- self.hist_util.append(nv.util_gpu or 0)- self.hist_bw.append(nv.util_mem or 0)+ self.hist_util.append(nv.util_gpu)+ self.hist_bw.append(nv.util_mem)self.hist_vram.append(100 * snap.vram_used / snap.vram_total if snap.vram_total else 0)if not self._auto_selected:llm = next((r for r in sorted(snap.procs, key=lambda r: -r.gpu.dedicated) if r.llm), None)@@ -233,20 +236,26 @@ class GpuTaskManager(App):lines = [head, vram, legend]if nv:+ rx = nv.pcie_rx_kbs / 1024 if nv.pcie_rx_kbs is not None else None+ tx = nv.pcie_tx_kbs / 1024 if nv.pcie_tx_kbs is not None else None+ pstate = f"P{nv.pstate}" if nv.pstate is not None else "P N/A"m = Text()m.append("GPU ", style="bold")- m.append(f"{nv.util_gpu:>3}% ", style="bold green")+ m.append(f"{format_metric(nv.util_gpu, unit='%'):>4} ", style="bold green")m.append(spark(self.hist_util, 24), style="green")m.append(" MemBW ", style="bold")- m.append(f"{nv.util_mem:>3}% ", style="bold dark_orange")+ m.append(f"{format_metric(nv.util_mem, unit='%'):>4} ", style="bold dark_orange")m.append(spark(self.hist_bw, 24), style="dark_orange")- m.append(f" PCIe↓{(nv.pcie_rx_kbs or 0) / 1024:>5.0f} ↑{(nv.pcie_tx_kbs or 0) / 1024:>4.0f} MB/s")+ m.append(f" PCIe↓{format_metric(rx, '.0f'):>5} ↑{format_metric(tx, '.0f'):>4} MB/s")if nv.enc or nv.dec:- m.append(f" enc {nv.enc}% dec {nv.dec}%")+ m.append(f" enc {format_metric(nv.enc, unit='%')} dec {format_metric(nv.dec, unit='%')}")lines.append(m)- st = Text(f"{nv.temp}°C {nv.power_w or 0:.0f}/{nv.power_limit_w or 0:.0f} W P{nv.pstate} "- f"SM {nv.sm_clock}/{nv.sm_clock_max} MHz mem {nv.mem_clock} MHz "- f"PCIe gen{nv.pcie_gen} x{nv.pcie_width} CC {nv.compute_cap}", style="grey70")+ st = Text(f"{format_metric(nv.temp, unit='°C')} {format_metric(nv.power_w, '.0f')}/"+ f"{format_metric(nv.power_limit_w, '.0f')} W {pstate} "+ f"SM {format_metric(nv.sm_clock)}/{format_metric(nv.sm_clock_max)} MHz "+ f"mem {format_metric(nv.mem_clock)} MHz "+ f"PCIe gen{format_metric(nv.pcie_gen)} x{format_metric(nv.pcie_width)} "+ f"CC {nv.compute_cap or 'N/A'}", style="grey70")if nv.throttle:st.append(" throttle: " + ",".join(nv.throttle), style="yellow")lines.append(st)diff --git a/src/gputaskman/cli.py b/src/gputaskman/cli.pyindex c224cc7..59df309 100644--- a/src/gputaskman/cli.py+++ b/src/gputaskman/cli.py@@ -23,11 +23,22 @@ def main() -> None:if sys.platform != "win32":sys.exit("gputaskman currently supports Windows (WDDM) only.")+ if args.gpu:+ from .collector import enumerate_adapters, select_adapter++ try:+ select_adapter(enumerate_adapters(), args.gpu)+ except ValueError as error:+ ap.error(str(error))+if args.once or args.json:from .collector import Collectorfrom .report import dump- c = Collector(args.gpu, scan_budget=None) # one-shot: identify everything in this sample+ try:+ c = Collector(args.gpu, scan_budget=None) # one-shot: identify everything in this sample+ except ValueError as error:+ ap.error(str(error))try:time.sleep(1.0) # engine utilization is a rate; needs two samplesprint(dump(c.sample(), args.json))diff --git a/src/gputaskman/collector.py b/src/gputaskman/collector.pyindex a3a2f81..24bad58 100644--- a/src/gputaskman/collector.py+++ b/src/gputaskman/collector.py@@ -73,20 +73,42 @@ class Snapshot:return int(sum(p.gpu.dedicated for p in self.procs))+def select_adapter(adapters: list[Adapter], gpu_index: int) -> Adapter | None:+ hardware = [a for a in adapters if not a.software]+ choices = [a for a in hardware if a.vendor_id == VENDOR_NVIDIA] or hardware+ choices.sort(key=lambda a: -a.dedicated_vram)+ if gpu_index < 0:+ raise ValueError("--gpu must be a non-negative index")+ if gpu_index >= len(choices):+ if not choices and gpu_index == 0:+ return None+ raise ValueError(f"--gpu {gpu_index} is out of range; {len(choices)} GPU adapter(s) available")+ return choices[gpu_index]+++def select_nv_device(adapter: Adapter | None, devices: list[NvDevice], gpu_index: int) -> NvDevice | None:+ if adapter:+ if adapter.vendor_id != VENDOR_NVIDIA:+ return None+ matches = [d for d in devices if d.name.casefold() == adapter.name.casefold()]+ if len(matches) == 1:+ return matches[0]+ fallback = devices[gpu_index] if gpu_index < len(devices) else None+ return None if adapter and fallback and fallback.driver_model == "MCDM" else fallback++class Collector:DEEP_TOP_N = 25 # memory-map candidates: the top N VRAM users plus anything busySCAN_BUDGET = 0.15 # seconds of memory-map scanning per sample; leftovers wait for the next sampledef __init__(self, gpu_index: int = 0, scan_budget: float | None = SCAN_BUDGET) -> None:+ self.adapters = [a for a in enumerate_adapters() if not a.software]+ self.adapter = select_adapter(self.adapters, gpu_index)self.counters = GpuCounters()self.nvml = Nvml()self.inspector = ProcInspector()self.prober = ServerProber()self.scan_budget = scan_budget # None = scan every candidate now (one-shot output)- self.adapters = [a for a in enumerate_adapters() if not a.software]- nv = [a for a in self.adapters if a.vendor_id == VENDOR_NVIDIA] or self.adapters- nv.sort(key=lambda a: -a.dedicated_vram)- self.adapter = nv[min(gpu_index, len(nv) - 1)] if nv else Noneself.gpu_index = gpu_indexdef _parent_ports(self, pid: int, depth: int = 2) -> list[int]:@@ -110,7 +132,7 @@ class Collector:procs = per_luid.get(luid, {})a = adapters.get(luid)nvs = self.nvml.sample() if self.nvml.ok else []- nv = nvs[self.gpu_index] if self.gpu_index < len(nvs) else None+ nv = select_nv_device(self.adapter, nvs, self.gpu_index)total = (nv.mem_total if nv else 0) or (self.adapter.dedicated_vram if self.adapter else 0)snap = Snapshot(@@ -140,7 +162,7 @@ class Collector:ollama_runners.append(row)for row in snap.procs:- if row.kind not in AI_KINDS or not (row.detail.models or row.kind == "LLM"):+ if row.detail.is_diffusion or row.kind not in AI_KINDS or not (row.detail.models or row.kind == "LLM"):continueown = self.prober.get(row.detail.ports)host = self.prober.get(self._parent_ports(row.pid)) # Ollama / LM Studio front-end processdiff --git a/src/gputaskman/llm.py b/src/gputaskman/llm.pyindex 4968221..32daa5c 100644--- a/src/gputaskman/llm.py+++ b/src/gputaskman/llm.py@@ -301,7 +301,7 @@ def _merge_api(files: list[MappedModel], server: ServerInfo | None, host: Serverkey = os.path.normcase(os.path.abspath(server.model_path))hit = next((f for f in files if os.path.normcase(os.path.abspath(f.path)) == key), None)if hit:- hit.sources.append("api")+ hit.add_source("api")elif os.path.isfile(server.model_path): # loaded without mmap or via the router: API-only evidencefiles.insert(0, make_model(server.model_path, os.path.getsize(server.model_path), 0,is_gguf(server.model_path), "api"))@@ -310,12 +310,12 @@ def _merge_api(files: list[MappedModel], server: ServerInfo | None, host: Servernames = set((f.store_name or "").split(", "))for om in front.models:if om["name"] in names:- f.sources.append("api")+ f.add_source("api")matched.append(om)lms = host if host and host.kind == "lmstudio" else (server if server and server.kind == "lmstudio" else None)primary = [f for f in files if f.kind not in _SIDE_KINDS]if lms and len(lms.models) == 1 and len(primary) == 1: # LM Studio keys don't map to paths; only pair 1:1- primary[0].sources.append("api")+ primary[0].add_source("api")matched.append(lms.models[0])return matched@@ -368,7 +368,7 @@ def build_view(d: ProcDetail, vram: float, shared: float, server: ServerInfo | Nif cpu_w and main and cpu_w > main.token_embd_bytes * 1.5:v.notes.append(f"{gb(cpu_w)} of weights are on the CPU side (partial offload or CPU-mapped).")if total > acc:- v.segments.append(Segment("CUDA context, cuBLAS & other", total - acc, "runtime"))+ v.segments.append(Segment("runtime and other overhead", total - acc, "runtime"))else:ngl = v.flags.get("ngl") if len(v.models) == 1 else Noneweights = sum(g.gpu_weight_bytes(ngl) for g in v.models if g.kind != "LoRA adapter")@@ -390,11 +390,11 @@ def build_view(d: ProcDetail, vram: float, shared: float, server: ServerInfo | N"so the real KV cache is smaller than this estimate.")rest -= kvif rest > 0:- v.segments.append(Segment("compute buffers + CUDA context", rest, "runtime"))+ v.segments.append(Segment("compute buffers + runtime", rest, "runtime"))elif rest > 0 and no_kv:- v.segments.append(Segment("compute buffers + CUDA context", rest, "runtime"))+ v.segments.append(Segment("compute buffers + runtime", rest, "runtime"))elif rest > 0:- v.segments.append(Segment("KV cache + compute + CUDA context", rest, "unknown"))+ v.segments.append(Segment("KV cache + compute + runtime", rest, "unknown"))if not v.models and not v.log and total > 0:v.notes.append("No GGUF mapped. Model may be loaded with --no-mmap or via another format.")diff --git a/src/gputaskman/nvml.py b/src/gputaskman/nvml.pyindex 350cb82..d2f9442 100644--- a/src/gputaskman/nvml.py+++ b/src/gputaskman/nvml.py@@ -39,6 +39,10 @@ class NvDevice:compute_cap: str = ""+def format_metric(value: float | int | None, spec: str = "", unit: str = "") -> str:+ return f"{format(value, spec)}{unit}" if value is not None else "N/A"++_THROTTLE = {0x2: "app-clocks",0x4: "sw-power-cap",diff --git a/src/gputaskman/procinfo.py b/src/gputaskman/procinfo.pyindex b0ec9e8..51d6535 100644--- a/src/gputaskman/procinfo.py+++ b/src/gputaskman/procinfo.py@@ -41,7 +41,7 @@ LLM_NAMES = ("llama-", "llama.", "ollama", "lm studio", "lmstudio", "llmster", "# GGUF file kind -> process kind, in display priority orderMODEL_KINDS = {"LLM": "LLM", "decision model": "decision", "embedding model": "embedding","reranker": "embedding", "speech model": "speech"}-AI_KINDS = {"LLM", "decision", "embedding", "speech", "AI"}+AI_KINDS = {"LLM", "decision", "embedding", "speech", "diffusion", "AI"}MiB = 1 << 20INSTALLED_TTL = 300.0_SKIP_DIRS = {"ebwebview", "cache", "code cache", "gpucache", "node_modules", "logs", "crashpad", "locales", "__pycache__"}@@ -58,6 +58,10 @@ class MappedModel:store_name: str | None = None # e.g. "qwen3:14b" from an Ollama manifestkind: str = "" # GGUF kind: LLM, decision model, embedding model, ...+ def add_source(self, source: str) -> None:+ if source not in self.sources:+ self.sources.append(source)+@propertydef evidence(self) -> str:s = set(self.sources)@@ -69,6 +73,10 @@ class MappedModel:return {"mapped": "in use (mapped)", "cmdline": "launch arg", "api": "reported (api)"}.get(how, how)+def _is_diffusion_cmdline(cmd: list[str]) -> bool:+ return any(arg == "--diffusion-model" or arg.startswith("--diffusion-model=") for arg in cmd)++@dataclassclass ProcDetail:pid: int@@ -89,6 +97,10 @@ class ProcDetail:def is_ai(self) -> bool:return bool(self.models) or any(r in AI_RUNTIMES for r in self.runtimes)+ @property+ def is_diffusion(self) -> bool:+ return _is_diffusion_cmdline(self.cmdline)+def _match_runtimes(dll_names: set[str]) -> list[str]:return [tag for tag, rx in _RUNTIME_RX if any(rx.match(d) for d in dll_names)]@@ -96,12 +108,14 @@ def _match_runtimes(dll_names: set[str]) -> list[str]:def _models_from_cmdline(cmd: list[str]) -> list[str]:paths = []+ flags = {"-m", "--model", "--mmproj", "-md", "--model-draft", "--lora"}+ if _is_diffusion_cmdline(cmd):+ flags.update(("--diffusion-model", "--vae", "--llm"))for i, arg in enumerate(cmd):val = None- if arg in ("-m", "--model", "--mmproj", "-md", "--model-draft", "--lora") and i + 1 < len(cmd):- val = cmd[i + 1]- elif arg.startswith("--model="):- val = arg.split("=", 1)[1]+ key, sep, value = arg.partition("=")+ if key in flags:+ val = value if sep else (cmd[i + 1] if i + 1 < len(cmd) else None)if val and os.path.isfile(val):paths.append(os.path.abspath(val))return paths@@ -302,7 +316,7 @@ class ProcInspector:models[path.lower()] = make_model(path, size, m.rss, gg, "mapped", store)for path in _models_from_cmdline(d.cmdline):if path.lower() in models:- models[path.lower()].sources.append("cmdline")+ models[path.lower()].add_source("cmdline")else:models[path.lower()] = make_model(path, os.path.getsize(path), 0, is_gguf(path), "cmdline")d.runtimes = _match_runtimes(dlls)@@ -324,6 +338,8 @@ def classify(d: ProcDetail, engines: dict[str, float]) -> str:return "compositor"if n in ("csrss.exe", "winlogon.exe", "lsass.exe", "svchost.exe"):return "system"+ if d.is_diffusion:+ return "diffusion"kinds = {MODEL_KINDS[m.kind] for m in d.models if m.kind in MODEL_KINDS}for k in ("LLM", "decision", "embedding", "speech"):if k in kinds:diff --git a/src/gputaskman/report.py b/src/gputaskman/report.pyindex 5978bed..22ae450 100644--- a/src/gputaskman/report.py+++ b/src/gputaskman/report.py@@ -8,6 +8,7 @@ import osfrom .collector import Snapshotfrom .llm import gb+from .nvml import format_metricdef to_dict(s: Snapshot) -> dict:@@ -52,8 +53,14 @@ def to_text(s: Snapshot, limit: int = 30) -> str:out.append(f"{name} driver {s.driver} CUDA {s.cuda} {nv.driver_model if nv else ''}")out.append(f"VRAM {gb(s.vram_used)} / {gb(s.vram_total)} shared (in system RAM) {gb(s.shared_used)}")if nv:- out.append(f"GPU {nv.util_gpu}% mem-bandwidth {nv.util_mem}% {nv.temp}C {nv.power_w:.0f}/{nv.power_limit_w:.0f} W "- f"P{nv.pstate} SM {nv.sm_clock} MHz PCIe rx {(nv.pcie_rx_kbs or 0) / 1024:.0f} MB/s tx {(nv.pcie_tx_kbs or 0) / 1024:.0f} MB/s")+ rx = nv.pcie_rx_kbs / 1024 if nv.pcie_rx_kbs is not None else None+ tx = nv.pcie_tx_kbs / 1024 if nv.pcie_tx_kbs is not None else None+ pstate = f"P{nv.pstate}" if nv.pstate is not None else "P N/A"+ out.append(f"GPU {format_metric(nv.util_gpu, unit='%')} mem-bandwidth {format_metric(nv.util_mem, unit='%')} "+ f"{format_metric(nv.temp, unit='C')} {format_metric(nv.power_w, '.0f')}/"+ f"{format_metric(nv.power_limit_w, '.0f')} W {pstate} "+ f"SM {format_metric(nv.sm_clock)} MHz PCIe rx {format_metric(rx, '.0f')} MB/s "+ f"tx {format_metric(tx, '.0f')} MB/s")out.append("")out.append(f"{'PID':>7} {'PROCESS':<24} {'KIND':<10} {'RUNTIME':<24} {'VRAM':>9} {'SPILL':>8} {'PAGED':>8} {'GPU%':>5} INSIDE")for p in s.procs[:limit]:diff --git a/tests/test_core.py b/tests/test_core.pyindex 37e1283..4b07b04 100644--- a/tests/test_core.py+++ b/tests/test_core.py@@ -11,12 +11,15 @@ from unittest.mock import patchfrom gputaskman.app import GpuTaskManagerfrom gputaskman.cli import main-from gputaskman.collector import ProcRow, Snapshot+from gputaskman.collector import ProcRow, Snapshot, select_adapter, select_nv_device+from gputaskman.dxgi import Adapterfrom gputaskman.gguf import GgufInfo, read_gguffrom gputaskman.llm import (LlmView, ServerInfo, build_view, is_gpu_device, llama_server_models, lmstudio_models,parse_flags, parse_load_log, probe_server)+from gputaskman.nvml import NvDevicefrom gputaskman.pdh import GpuCounters, ProcGpu-from gputaskman.procinfo import MappedModel, ProcDetail, ProcInspector, _is_model_file, classify, installed_models+from gputaskman.procinfo import (MappedModel, ProcDetail, ProcInspector, _is_model_file, _models_from_cmdline,+ classify, installed_models)from gputaskman.report import to_dict, to_textfrom gputaskman.stores import store_namefrom tools.capture_screenshots import for_publication@@ -75,6 +78,40 @@ class CliTests(unittest.TestCase):main()collector.return_value.close.assert_called_once()+ def test_out_of_range_gpu_is_rejected(self):+ adapter = Adapter(1, "NVIDIA GPU", 0x10DE, 16 * GiB, 0, False)+ output = io.StringIO()+ with patch("sys.argv", ["gputaskman", "--gpu", "1", "--json"]), \+ patch("gputaskman.collector.enumerate_adapters", return_value=[adapter]), \+ patch("gputaskman.collector.Collector") as collector, contextlib.redirect_stderr(output):+ with self.assertRaises(SystemExit) as error:+ main()+ self.assertEqual(error.exception.code, 2)+ self.assertIn("out of range", output.getvalue())+ collector.assert_not_called()+++class DeviceSelectionTests(unittest.TestCase):+ def test_adapter_indices_do_not_clamp_to_another_gpu(self):+ gpu = Adapter(1, "NVIDIA GPU", 0x10DE, 16 * GiB, 0, False)+ software = Adapter(2, "Software", 0x1414, 0, 0, True)+ self.assertIs(select_adapter([software, gpu], 0), gpu)+ with self.assertRaisesRegex(ValueError, "out of range"):+ select_adapter([software, gpu], 1)+ self.assertIsNone(select_adapter([], 0))+ with self.assertRaisesRegex(ValueError, "out of range"):+ select_adapter([], 1)++ def test_nvml_matches_unique_adapter_name_before_index(self):+ gpu = Adapter(1, "NVIDIA RTX Spark", 0x10DE, 64 * GiB, 0, False)+ npu = NvDevice(index=0, name="NVIDIA NPU", driver_model="MCDM")+ nv_gpu = NvDevice(index=1, name="NVIDIA RTX SPARK", driver_model="WDDM")+ self.assertIs(select_nv_device(gpu, [npu, nv_gpu], 0), nv_gpu)+ self.assertIs(select_nv_device(None, [npu, nv_gpu], 0), npu)+ self.assertIsNone(select_nv_device(gpu, [npu], 0))+ intel = Adapter(2, "Intel GPU", 0x8086, 8 * GiB, 0, False)+ self.assertIsNone(select_nv_device(intel, [nv_gpu], 0))+class MemoryTests(unittest.TestCase):def test_local_counters_are_preferred(self):@@ -99,6 +136,18 @@ class MemoryTests(unittest.TestCase):self.assertIn("PAGED", text)self.assertNotIn("gen=None", text)+ def test_text_shows_unavailable_device_metrics_without_fabricating_zeroes(self):+ s = snapshot()+ s.nv = NvDevice(index=0, util_gpu=0, power_w=2.4, pcie_tx_kbs=0)+ text = to_text(s)+ self.assertIn("GPU 0%", text)+ self.assertIn("mem-bandwidth N/A", text)+ self.assertIn("2/N/A W", text)+ self.assertIn("P N/A", text)+ self.assertIn("PCIe rx N/A MB/s tx 0 MB/s", text)+ self.assertNotIn("None", text.splitlines()[2])+ self.assertIsNone(to_dict(s)["device"]["power_limit_w"])+def test_capture_does_not_change_original_or_publish_command_line(self):original = snapshot()original.procs[0].llm.models = [@@ -112,6 +161,19 @@ class MemoryTests(unittest.TestCase):class ModelTests(unittest.TestCase):+ def test_diffusion_arguments_do_not_classify_text_encoder_as_llm(self):+ with tempfile.TemporaryDirectory() as directory:+ paths = [Path(directory) / name for name in ("diffusion.gguf", "encoder.gguf", "vae.safetensors")]+ for path in paths:+ path.touch()+ cmd = ["sd-cli.exe", "--diffusion-model", str(paths[0]),+ f"--llm={paths[1]}", "--vae", str(paths[2])]+ self.assertEqual(_models_from_cmdline(cmd), [str(path) for path in paths])+ self.assertEqual(_models_from_cmdline(["sd-cli.exe", f"--diffusion-model={paths[0]}"]), [str(paths[0])])+ d = ProcDetail(1, "sd-cli.exe", cmdline=cmd, runtimes=["ggml"],+ models=[MappedModel(str(paths[1]), 0, 0, True, kind="LLM")])+ self.assertEqual(classify(d, {}), "diffusion")+def test_layer_and_kv_estimates(self):model = GgufInfo(path="model.gguf", file_size=1000, version=3,@@ -136,9 +198,10 @@ class ModelTests(unittest.TestCase):{"ctx": 8192, "ngl": -1, "ctk": "q8_0"})def test_gpu_buffers_exclude_host_and_cpu(self):- for device in ("CPU", "CPU_Mapped", "CUDA_Host"):+ for device in ("CPU", "CPU_Mapped", "CUDA_Host", "Vulkan_Host"):self.assertFalse(is_gpu_device(device))- self.assertTrue(is_gpu_device("CUDA0"))+ for device in ("CUDA0", "Vulkan0"):+ self.assertTrue(is_gpu_device(device))def test_latest_load_and_multiple_kv_buffers(self):with tempfile.TemporaryDirectory() as directory:@@ -238,6 +301,17 @@ class IdentityTests(unittest.TestCase):patch.object(ProcInspector, "_new", return_value=fresh):self.assertIs(inspector.get(42), fresh)+ def test_repeated_model_arguments_do_not_duplicate_evidence(self):+ with tempfile.TemporaryDirectory() as directory:+ path = str(Path(directory) / "model.gguf")+ Path(path).write_bytes(b"not a GGUF")+ detail = ProcDetail(6, "llama-server.exe", cmdline=["server", "--model", path, "--mmproj", path])+ with patch("gputaskman.procinfo.psutil.Process") as process:+ process.return_value.memory_maps.return_value = []+ ProcInspector().scan(detail)+ self.assertEqual(len(detail.models), 1)+ self.assertEqual(detail.models[0].sources, ["cmdline"])+def test_ambiguous_extensions_need_size_or_store(self):self.assertFalse(_is_model_file(".bin", 10 * MiB, False, None)) # e.g. a Start menu tile cacheself.assertTrue(_is_model_file(".bin", 10 * MiB, False, "org/repo"))@@ -265,9 +339,30 @@ class IdentityTests(unittest.TestCase):self.assertEqual(v.files[0].evidence, "confirmed (api+mapped)")self.assertEqual(d.models[0].sources, ["mapped"]) # cached detail is not mutatedself.assertIsNone(v.kv_per_token)- self.assertEqual([s.label for s in v.segments], ["weights", "compute buffers + CUDA context"])+ self.assertEqual([s.label for s in v.segments], ["weights", "compute buffers + runtime"])self.assertTrue(any("laya decision model" in n for n in v.notes))+ def test_vulkan_log_does_not_label_unknown_overhead_as_cuda(self):+ laya = gguf("modern-bert", **LAYA)+ detail = ProcDetail(5, "llama-server.exe", models=[+ MappedModel("laya.gguf", 1000, 0, True, kind="decision model")+ ])+ with tempfile.TemporaryDirectory() as directory:+ path = Path(directory) / "server.log"+ path.write_text(+ "load_tensors: Vulkan0 model buffer size = 10 MiB\n"+ "load_tensors: Vulkan_Host model buffer size = 3 MiB\n"+ "llama_context: Vulkan0 compute buffer size = 2 MiB\n",+ encoding="utf-8",+ )+ with patch("gputaskman.llm.read_gguf", return_value=laya):+ view = build_view(detail, 20 * MiB, 0, None, str(path))+ self.assertEqual([(s.label, s.bytes) for s in view.segments], [+ ("weights (Vulkan0)", 10 * MiB),+ ("compute buffers (Vulkan0)", 2 * MiB),+ ("runtime and other overhead", 8 * MiB),+ ])+def test_installed_models_are_candidates_only(self):with tempfile.TemporaryDirectory() as root:app = Path(root, "Speechy")@@ -305,8 +400,34 @@ class IdentityTests(unittest.TestCase):self.assertEqual(len(ollama), 1)self.assertIn("qwen3:14b", ollama[0])+ def test_runner_and_host_apis_do_not_duplicate_evidence(self):+ qwen = gguf("qwen3", block_count=1, context_length=4096)+ model = MappedModel("blob", 1000, 0, True, ["cmdline"], "qwen3:4b", "LLM")+ detail = ProcDetail(6, "llama-server.exe", models=[model])+ runner = ServerInfo("llama-server", "http://127.0.0.1:1", model_path="blob")+ host = ServerInfo("ollama", "http://127.0.0.1:2", models=[{"name": "qwen3:4b", "ctx": 4096}])+ with patch("gputaskman.llm.read_gguf", return_value=qwen):+ view = build_view(detail, GiB, 0, runner, host=host)+ self.assertEqual(view.files[0].sources, ["cmdline", "api"])+ self.assertEqual(model.sources, ["cmdline"])+class UiTests(unittest.IsolatedAsyncioTestCase):+ async def test_unavailable_device_metrics_render_as_na(self):+ with patch.object(GpuTaskManager, "poll"):+ app = GpuTaskManager()+ async with app.run_test(size=(140, 42)):+ s = snapshot()+ s.nv = NvDevice(index=0, util_mem=0, power_w=2.4, pcie_tx_kbs=0)+ app.apply(s)+ lines = "\n".join(line.plain for line in app.render_device(s).renderables)+ self.assertIn("2/N/A W", lines)+ self.assertIn("P N/A", lines)+ self.assertIn("PCIe↓ N/A", lines)+ self.assertNotIn("None", lines)+ self.assertIsNone(app.hist_util[-1])+ self.assertEqual(app.hist_bw[-1], 0)+async def test_navigation_sort_pause_and_help(self):with patch.object(GpuTaskManager, "poll"):app = GpuTaskManager()