Skip to content

A GPU task manager for Windows, Linux, and WSL: per-process VRAM, GPU activity, and LLM model/KV-cache insights (Windows AMD and NVIDIA).

Python

Improve ARM64 GPU reporting and model discovery

Scott Hanselman0c0cf8feaab9
diff --git a/README.md b/README.md
index 67cca9d..3026ce3 100644
--- a/README.md
+++ b/README.md
@@ -16,7 +16,9 @@ Your browser? The desktop compositor? And why is so much GPU memory in system RA
## Clone and run
Use **64-bit Windows 10/11**, an **NVIDIA GPU with a current WDDM driver**, and a terminal
-(Windows Terminal is recommended). Development and live workload checks used an RTX 4080 SUPER.
+(Windows Terminal is recommended). Live workload checks have included an RTX 4080 SUPER and an
+RTX Spark N1X on Windows ARM64. CUDA workloads are supported on ARM64 with a compatible NVIDIA
+driver and ARM64 CUDA runtime.
Install [Git for Windows](https://git-scm.com/downloads/win) and
[uv](https://docs.astral.sh/uv/getting-started/installation/) if needed. With WinGet:
@@ -52,12 +54,16 @@ py -m venv .venv
No PowerShell activation script or execution-policy change is needed. The pip route resolves
compatible dependencies; the uv route uses the checked-in lockfile.
+If an alternate package index makes `--locked` report a stale lockfile on an unchanged checkout,
+`uv run --frozen gputaskman` uses the checked-in lockfile without re-resolving it. `--frozen`
+skips the consistency check, so do not use it after changing dependencies.
+
## What you get
- **Device view:** VRAM usage, GPU activity, memory activity, PCIe traffic, temperature, power and clocks.
- **Process view:** local GPU memory, GPU-accessible system RAM, estimated non-resident commit,
engine activity, and detected runtimes such as CUDA, Vulkan, Direct3D, ggml and ONNX Runtime.
-- **Model inspector:** which model each process holds (LLM, decision, embedding, or speech), how
+- **Model inspector:** which model each process holds (LLM, decision, embedding, speech, or diffusion), how
it was identified, architecture, quantization, layer count, weights, estimated KV cost, and
buffer sizes from compatible load logs.
- **Live server hints:** context size, busy slots, throughput when available, and friendly Ollama
@@ -106,9 +112,15 @@ uv run --locked gputaskman --help
A wide terminal (roughly 140 columns or more) makes the full table easier to read.
Model/runtime discovery is throttled, so newly loaded models may take about 15 seconds to appear.
-`--gpu N` is available, but **single-NVIDIA-GPU systems are the validated configuration**.
-On multi-GPU systems, DXGI and NVML ordering may differ, and buffer logs can span devices;
-cross-device attribution is not yet reliable.
+`--gpu N` selects a detected NVIDIA graphics adapter, ordered by dedicated memory; an out-of-range
+index is rejected rather than silently selecting another device. Compute-only devices such as an
+NPU are not graphics adapters. **Single-NVIDIA-GPU systems are the validated configuration**.
+On multi-GPU systems, adapter names may be ambiguous, DXGI and NVML ordering may differ, and buffer
+logs can span devices; cross-device attribution is not yet reliable.
+
+Unavailable utilization, temperature, power, clock and PCIe readings are displayed as `N/A` in
+text and the TUI and as `null` in JSON. For example, a missing NVML power limit means that the
+driver did not expose that reading; it does not by itself indicate a hardware fault.
## How can it know what's inside?
@@ -168,6 +180,7 @@ reading is NVML's memory-activity percentage, **not measured bandwidth in GB/s**
| llama.cpp / llama-server | GGUF discovery, CLI settings, buffer logs, `/props` + `/v1/models` state; exercised live |
| Ollama | Bundled runners, mapped GGUF blobs named from Ollama manifests (`qwen3:14b`), parent `/api/ps`, compatible `server.log`; exercised live |
| LM Studio | Mapped files named from its models folder, `/api/v1/models` (falls back to `/api/v0/models`); tested with fixture responses, not yet end to end |
+| stable-diffusion.cpp (`--diffusion-model`) | Diffusion, text-encoder and VAE files from launch arguments; no LLM memory breakdown |
| Hugging Face cache | `models--org--repo` paths are named `org/repo` |
| Other AI runtimes | DLL detection, plus model files installed with the app (marked *not proven loaded*) |
diff --git a/src/gputaskman/app.py b/src/gputaskman/app.py
index ca9e3bf..5864756 100644
--- a/src/gputaskman/app.py
+++ b/src/gputaskman/app.py
@@ -19,9 +19,11 @@ from textual.worker import get_current_worker
from .collector import Collector, ProcRow, Snapshot
from .llm import gb
+from .nvml import format_metric
KIND_STYLE = {
"LLM": "bold magenta", "decision": "bold orchid", "embedding": "plum2", "speech": "light_sky_blue1",
+ "diffusion": "orchid",
"AI": "medium_purple1", "compute": "orange3", "3D/game": "green3",
"video": "yellow", "browser": "dodger_blue2", "app": "cyan", "compositor": "grey50",
"kernel": "grey50", "system": "grey50",
@@ -42,6 +44,7 @@ EXPLAIN = {
"per-resource names need a graphics debugger (PIX on Windows, NVIDIA Nsight Graphics).",
"video": "Decoded/encoded video frames and codec surfaces (NVDEC/NVENC).",
"compute": "CUDA allocations are opaque to the OS. For more detail trace cudaMalloc with Nsight Systems.",
+ "diffusion": "Model components are identified from launch arguments; GPU allocations cannot be split by component.",
"AI": "ML runtime allocations (tensors, workspaces). PyTorch: torch.cuda.memory_summary(); "
"ONNX Runtime/DirectML pools are opaque from outside.",
"app": "UI surfaces: swap chains, window back-buffers, cached bitmaps/glyphs.",
@@ -55,7 +58,7 @@ HELP_MD = """\
| **Device** totals, load, bandwidth, PCIe, power | NVML (`nvml.dll`) | NVIDIA |
| **Per-process** VRAM, spill to RAM, per-engine load | Windows WDDM counters (dxgkrnl) | every API incl. **CUDA** |
| **Which runtime** (CUDA, Vulkan, D3D12, ggml, ONNX, PyTorch...) | DLLs mapped in the process | everything |
-| **Which model file** | memory-mapped files + command line | llama.cpp, LM Studio, Ollama, llama-cpp-python |
+| **Which model file** | memory-mapped files + command line | llama.cpp, stable-diffusion.cpp, LM Studio, Ollama |
| **Model name & kind** (LLM, decision, embedding, speech) | GGUF header, Ollama/LM Studio/Hugging Face stores, local server APIs | GGUF family |
| **Weights / KV cache / compute** split | GGUF header math, llama.cpp/Ollama logs, local server APIs | GGUF/llama.cpp family |
| Per-texture / per-buffer names | needs a debugger (PIX, Nsight) attached to the app | games, 3D apps |
@@ -184,8 +187,8 @@ class GpuTaskManager(App):
self.snap = snap
nv = snap.nv
if nv:
- self.hist_util.append(nv.util_gpu or 0)
- self.hist_bw.append(nv.util_mem or 0)
+ self.hist_util.append(nv.util_gpu)
+ self.hist_bw.append(nv.util_mem)
self.hist_vram.append(100 * snap.vram_used / snap.vram_total if snap.vram_total else 0)
if not self._auto_selected:
llm = next((r for r in sorted(snap.procs, key=lambda r: -r.gpu.dedicated) if r.llm), None)
@@ -233,20 +236,26 @@ class GpuTaskManager(App):
lines = [head, vram, legend]
if nv:
+ rx = nv.pcie_rx_kbs / 1024 if nv.pcie_rx_kbs is not None else None
+ tx = nv.pcie_tx_kbs / 1024 if nv.pcie_tx_kbs is not None else None
+ pstate = f"P{nv.pstate}" if nv.pstate is not None else "P N/A"
m = Text()
m.append("GPU ", style="bold")
- m.append(f"{nv.util_gpu:>3}% ", style="bold green")
+ m.append(f"{format_metric(nv.util_gpu, unit='%'):>4} ", style="bold green")
m.append(spark(self.hist_util, 24), style="green")
m.append(" MemBW ", style="bold")
- m.append(f"{nv.util_mem:>3}% ", style="bold dark_orange")
+ m.append(f"{format_metric(nv.util_mem, unit='%'):>4} ", style="bold dark_orange")
m.append(spark(self.hist_bw, 24), style="dark_orange")
- m.append(f" PCIe↓{(nv.pcie_rx_kbs or 0) / 1024:>5.0f} ↑{(nv.pcie_tx_kbs or 0) / 1024:>4.0f} MB/s")
+ m.append(f" PCIe↓{format_metric(rx, '.0f'):>5} ↑{format_metric(tx, '.0f'):>4} MB/s")
if nv.enc or nv.dec:
- m.append(f" enc {nv.enc}% dec {nv.dec}%")
+ m.append(f" enc {format_metric(nv.enc, unit='%')} dec {format_metric(nv.dec, unit='%')}")
lines.append(m)
- st = Text(f"{nv.temp}°C {nv.power_w or 0:.0f}/{nv.power_limit_w or 0:.0f} W P{nv.pstate} "
- f"SM {nv.sm_clock}/{nv.sm_clock_max} MHz mem {nv.mem_clock} MHz "
- f"PCIe gen{nv.pcie_gen} x{nv.pcie_width} CC {nv.compute_cap}", style="grey70")
+ st = Text(f"{format_metric(nv.temp, unit='°C')} {format_metric(nv.power_w, '.0f')}/"
+ f"{format_metric(nv.power_limit_w, '.0f')} W {pstate} "
+ f"SM {format_metric(nv.sm_clock)}/{format_metric(nv.sm_clock_max)} MHz "
+ f"mem {format_metric(nv.mem_clock)} MHz "
+ f"PCIe gen{format_metric(nv.pcie_gen)} x{format_metric(nv.pcie_width)} "
+ f"CC {nv.compute_cap or 'N/A'}", style="grey70")
if nv.throttle:
st.append(" throttle: " + ",".join(nv.throttle), style="yellow")
lines.append(st)
diff --git a/src/gputaskman/cli.py b/src/gputaskman/cli.py
index c224cc7..59df309 100644
--- a/src/gputaskman/cli.py
+++ b/src/gputaskman/cli.py
@@ -23,11 +23,22 @@ def main() -> None:
if sys.platform != "win32":
sys.exit("gputaskman currently supports Windows (WDDM) only.")
+ if args.gpu:
+ from .collector import enumerate_adapters, select_adapter
+
+ try:
+ select_adapter(enumerate_adapters(), args.gpu)
+ except ValueError as error:
+ ap.error(str(error))
+
if args.once or args.json:
from .collector import Collector
from .report import dump
- c = Collector(args.gpu, scan_budget=None) # one-shot: identify everything in this sample
+ try:
+ c = Collector(args.gpu, scan_budget=None) # one-shot: identify everything in this sample
+ except ValueError as error:
+ ap.error(str(error))
try:
time.sleep(1.0) # engine utilization is a rate; needs two samples
print(dump(c.sample(), args.json))
diff --git a/src/gputaskman/collector.py b/src/gputaskman/collector.py
index a3a2f81..24bad58 100644
--- a/src/gputaskman/collector.py
+++ b/src/gputaskman/collector.py
@@ -73,20 +73,42 @@ class Snapshot:
return int(sum(p.gpu.dedicated for p in self.procs))
+def select_adapter(adapters: list[Adapter], gpu_index: int) -> Adapter | None:
+ hardware = [a for a in adapters if not a.software]
+ choices = [a for a in hardware if a.vendor_id == VENDOR_NVIDIA] or hardware
+ choices.sort(key=lambda a: -a.dedicated_vram)
+ if gpu_index < 0:
+ raise ValueError("--gpu must be a non-negative index")
+ if gpu_index >= len(choices):
+ if not choices and gpu_index == 0:
+ return None
+ raise ValueError(f"--gpu {gpu_index} is out of range; {len(choices)} GPU adapter(s) available")
+ return choices[gpu_index]
+
+
+def select_nv_device(adapter: Adapter | None, devices: list[NvDevice], gpu_index: int) -> NvDevice | None:
+ if adapter:
+ if adapter.vendor_id != VENDOR_NVIDIA:
+ return None
+ matches = [d for d in devices if d.name.casefold() == adapter.name.casefold()]
+ if len(matches) == 1:
+ return matches[0]
+ fallback = devices[gpu_index] if gpu_index < len(devices) else None
+ return None if adapter and fallback and fallback.driver_model == "MCDM" else fallback
+
+
class Collector:
DEEP_TOP_N = 25 # memory-map candidates: the top N VRAM users plus anything busy
SCAN_BUDGET = 0.15 # seconds of memory-map scanning per sample; leftovers wait for the next sample
def __init__(self, gpu_index: int = 0, scan_budget: float | None = SCAN_BUDGET) -> None:
+ self.adapters = [a for a in enumerate_adapters() if not a.software]
+ self.adapter = select_adapter(self.adapters, gpu_index)
self.counters = GpuCounters()
self.nvml = Nvml()
self.inspector = ProcInspector()
self.prober = ServerProber()
self.scan_budget = scan_budget # None = scan every candidate now (one-shot output)
- self.adapters = [a for a in enumerate_adapters() if not a.software]
- nv = [a for a in self.adapters if a.vendor_id == VENDOR_NVIDIA] or self.adapters
- nv.sort(key=lambda a: -a.dedicated_vram)
- self.adapter = nv[min(gpu_index, len(nv) - 1)] if nv else None
self.gpu_index = gpu_index
def _parent_ports(self, pid: int, depth: int = 2) -> list[int]:
@@ -110,7 +132,7 @@ class Collector:
procs = per_luid.get(luid, {})
a = adapters.get(luid)
nvs = self.nvml.sample() if self.nvml.ok else []
- nv = nvs[self.gpu_index] if self.gpu_index < len(nvs) else None
+ nv = select_nv_device(self.adapter, nvs, self.gpu_index)
total = (nv.mem_total if nv else 0) or (self.adapter.dedicated_vram if self.adapter else 0)
snap = Snapshot(
@@ -140,7 +162,7 @@ class Collector:
ollama_runners.append(row)
for row in snap.procs:
- if row.kind not in AI_KINDS or not (row.detail.models or row.kind == "LLM"):
+ if row.detail.is_diffusion or row.kind not in AI_KINDS or not (row.detail.models or row.kind == "LLM"):
continue
own = self.prober.get(row.detail.ports)
host = self.prober.get(self._parent_ports(row.pid)) # Ollama / LM Studio front-end process
diff --git a/src/gputaskman/llm.py b/src/gputaskman/llm.py
index 4968221..32daa5c 100644
--- a/src/gputaskman/llm.py
+++ b/src/gputaskman/llm.py
@@ -301,7 +301,7 @@ def _merge_api(files: list[MappedModel], server: ServerInfo | None, host: Server
key = os.path.normcase(os.path.abspath(server.model_path))
hit = next((f for f in files if os.path.normcase(os.path.abspath(f.path)) == key), None)
if hit:
- hit.sources.append("api")
+ hit.add_source("api")
elif os.path.isfile(server.model_path): # loaded without mmap or via the router: API-only evidence
files.insert(0, make_model(server.model_path, os.path.getsize(server.model_path), 0,
is_gguf(server.model_path), "api"))
@@ -310,12 +310,12 @@ def _merge_api(files: list[MappedModel], server: ServerInfo | None, host: Server
names = set((f.store_name or "").split(", "))
for om in front.models:
if om["name"] in names:
- f.sources.append("api")
+ f.add_source("api")
matched.append(om)
lms = host if host and host.kind == "lmstudio" else (server if server and server.kind == "lmstudio" else None)
primary = [f for f in files if f.kind not in _SIDE_KINDS]
if lms and len(lms.models) == 1 and len(primary) == 1: # LM Studio keys don't map to paths; only pair 1:1
- primary[0].sources.append("api")
+ primary[0].add_source("api")
matched.append(lms.models[0])
return matched
@@ -368,7 +368,7 @@ def build_view(d: ProcDetail, vram: float, shared: float, server: ServerInfo | N
if cpu_w and main and cpu_w > main.token_embd_bytes * 1.5:
v.notes.append(f"{gb(cpu_w)} of weights are on the CPU side (partial offload or CPU-mapped).")
if total > acc:
- v.segments.append(Segment("CUDA context, cuBLAS & other", total - acc, "runtime"))
+ v.segments.append(Segment("runtime and other overhead", total - acc, "runtime"))
else:
ngl = v.flags.get("ngl") if len(v.models) == 1 else None
weights = sum(g.gpu_weight_bytes(ngl) for g in v.models if g.kind != "LoRA adapter")
@@ -390,11 +390,11 @@ def build_view(d: ProcDetail, vram: float, shared: float, server: ServerInfo | N
"so the real KV cache is smaller than this estimate.")
rest -= kv
if rest > 0:
- v.segments.append(Segment("compute buffers + CUDA context", rest, "runtime"))
+ v.segments.append(Segment("compute buffers + runtime", rest, "runtime"))
elif rest > 0 and no_kv:
- v.segments.append(Segment("compute buffers + CUDA context", rest, "runtime"))
+ v.segments.append(Segment("compute buffers + runtime", rest, "runtime"))
elif rest > 0:
- v.segments.append(Segment("KV cache + compute + CUDA context", rest, "unknown"))
+ v.segments.append(Segment("KV cache + compute + runtime", rest, "unknown"))
if not v.models and not v.log and total > 0:
v.notes.append("No GGUF mapped. Model may be loaded with --no-mmap or via another format.")
diff --git a/src/gputaskman/nvml.py b/src/gputaskman/nvml.py
index 350cb82..d2f9442 100644
--- a/src/gputaskman/nvml.py
+++ b/src/gputaskman/nvml.py
@@ -39,6 +39,10 @@ class NvDevice:
compute_cap: str = ""
+def format_metric(value: float | int | None, spec: str = "", unit: str = "") -> str:
+ return f"{format(value, spec)}{unit}" if value is not None else "N/A"
+
+
_THROTTLE = {
0x2: "app-clocks",
0x4: "sw-power-cap",
diff --git a/src/gputaskman/procinfo.py b/src/gputaskman/procinfo.py
index b0ec9e8..51d6535 100644
--- a/src/gputaskman/procinfo.py
+++ b/src/gputaskman/procinfo.py
@@ -41,7 +41,7 @@ LLM_NAMES = ("llama-", "llama.", "ollama", "lm studio", "lmstudio", "llmster", "
# GGUF file kind -> process kind, in display priority order
MODEL_KINDS = {"LLM": "LLM", "decision model": "decision", "embedding model": "embedding",
"reranker": "embedding", "speech model": "speech"}
-AI_KINDS = {"LLM", "decision", "embedding", "speech", "AI"}
+AI_KINDS = {"LLM", "decision", "embedding", "speech", "diffusion", "AI"}
MiB = 1 << 20
INSTALLED_TTL = 300.0
_SKIP_DIRS = {"ebwebview", "cache", "code cache", "gpucache", "node_modules", "logs", "crashpad", "locales", "__pycache__"}
@@ -58,6 +58,10 @@ class MappedModel:
store_name: str | None = None # e.g. "qwen3:14b" from an Ollama manifest
kind: str = "" # GGUF kind: LLM, decision model, embedding model, ...
+ def add_source(self, source: str) -> None:
+ if source not in self.sources:
+ self.sources.append(source)
+
@property
def evidence(self) -> str:
s = set(self.sources)
@@ -69,6 +73,10 @@ class MappedModel:
return {"mapped": "in use (mapped)", "cmdline": "launch arg", "api": "reported (api)"}.get(how, how)
+def _is_diffusion_cmdline(cmd: list[str]) -> bool:
+ return any(arg == "--diffusion-model" or arg.startswith("--diffusion-model=") for arg in cmd)
+
+
@dataclass
class ProcDetail:
pid: int
@@ -89,6 +97,10 @@ class ProcDetail:
def is_ai(self) -> bool:
return bool(self.models) or any(r in AI_RUNTIMES for r in self.runtimes)
+ @property
+ def is_diffusion(self) -> bool:
+ return _is_diffusion_cmdline(self.cmdline)
+
def _match_runtimes(dll_names: set[str]) -> list[str]:
return [tag for tag, rx in _RUNTIME_RX if any(rx.match(d) for d in dll_names)]
@@ -96,12 +108,14 @@ def _match_runtimes(dll_names: set[str]) -> list[str]:
def _models_from_cmdline(cmd: list[str]) -> list[str]:
paths = []
+ flags = {"-m", "--model", "--mmproj", "-md", "--model-draft", "--lora"}
+ if _is_diffusion_cmdline(cmd):
+ flags.update(("--diffusion-model", "--vae", "--llm"))
for i, arg in enumerate(cmd):
val = None
- if arg in ("-m", "--model", "--mmproj", "-md", "--model-draft", "--lora") and i + 1 < len(cmd):
- val = cmd[i + 1]
- elif arg.startswith("--model="):
- val = arg.split("=", 1)[1]
+ key, sep, value = arg.partition("=")
+ if key in flags:
+ val = value if sep else (cmd[i + 1] if i + 1 < len(cmd) else None)
if val and os.path.isfile(val):
paths.append(os.path.abspath(val))
return paths
@@ -302,7 +316,7 @@ class ProcInspector:
models[path.lower()] = make_model(path, size, m.rss, gg, "mapped", store)
for path in _models_from_cmdline(d.cmdline):
if path.lower() in models:
- models[path.lower()].sources.append("cmdline")
+ models[path.lower()].add_source("cmdline")
else:
models[path.lower()] = make_model(path, os.path.getsize(path), 0, is_gguf(path), "cmdline")
d.runtimes = _match_runtimes(dlls)
@@ -324,6 +338,8 @@ def classify(d: ProcDetail, engines: dict[str, float]) -> str:
return "compositor"
if n in ("csrss.exe", "winlogon.exe", "lsass.exe", "svchost.exe"):
return "system"
+ if d.is_diffusion:
+ return "diffusion"
kinds = {MODEL_KINDS[m.kind] for m in d.models if m.kind in MODEL_KINDS}
for k in ("LLM", "decision", "embedding", "speech"):
if k in kinds:
diff --git a/src/gputaskman/report.py b/src/gputaskman/report.py
index 5978bed..22ae450 100644
--- a/src/gputaskman/report.py
+++ b/src/gputaskman/report.py
@@ -8,6 +8,7 @@ import os
from .collector import Snapshot
from .llm import gb
+from .nvml import format_metric
def to_dict(s: Snapshot) -> dict:
@@ -52,8 +53,14 @@ def to_text(s: Snapshot, limit: int = 30) -> str:
out.append(f"{name} driver {s.driver} CUDA {s.cuda} {nv.driver_model if nv else ''}")
out.append(f"VRAM {gb(s.vram_used)} / {gb(s.vram_total)} shared (in system RAM) {gb(s.shared_used)}")
if nv:
- out.append(f"GPU {nv.util_gpu}% mem-bandwidth {nv.util_mem}% {nv.temp}C {nv.power_w:.0f}/{nv.power_limit_w:.0f} W "
- f"P{nv.pstate} SM {nv.sm_clock} MHz PCIe rx {(nv.pcie_rx_kbs or 0) / 1024:.0f} MB/s tx {(nv.pcie_tx_kbs or 0) / 1024:.0f} MB/s")
+ rx = nv.pcie_rx_kbs / 1024 if nv.pcie_rx_kbs is not None else None
+ tx = nv.pcie_tx_kbs / 1024 if nv.pcie_tx_kbs is not None else None
+ pstate = f"P{nv.pstate}" if nv.pstate is not None else "P N/A"
+ out.append(f"GPU {format_metric(nv.util_gpu, unit='%')} mem-bandwidth {format_metric(nv.util_mem, unit='%')} "
+ f"{format_metric(nv.temp, unit='C')} {format_metric(nv.power_w, '.0f')}/"
+ f"{format_metric(nv.power_limit_w, '.0f')} W {pstate} "
+ f"SM {format_metric(nv.sm_clock)} MHz PCIe rx {format_metric(rx, '.0f')} MB/s "
+ f"tx {format_metric(tx, '.0f')} MB/s")
out.append("")
out.append(f"{'PID':>7} {'PROCESS':<24} {'KIND':<10} {'RUNTIME':<24} {'VRAM':>9} {'SPILL':>8} {'PAGED':>8} {'GPU%':>5} INSIDE")
for p in s.procs[:limit]:
diff --git a/tests/test_core.py b/tests/test_core.py
index 37e1283..4b07b04 100644
--- a/tests/test_core.py
+++ b/tests/test_core.py
@@ -11,12 +11,15 @@ from unittest.mock import patch
from gputaskman.app import GpuTaskManager
from gputaskman.cli import main
-from gputaskman.collector import ProcRow, Snapshot
+from gputaskman.collector import ProcRow, Snapshot, select_adapter, select_nv_device
+from gputaskman.dxgi import Adapter
from gputaskman.gguf import GgufInfo, read_gguf
from gputaskman.llm import (LlmView, ServerInfo, build_view, is_gpu_device, llama_server_models, lmstudio_models,
parse_flags, parse_load_log, probe_server)
+from gputaskman.nvml import NvDevice
from gputaskman.pdh import GpuCounters, ProcGpu
-from gputaskman.procinfo import MappedModel, ProcDetail, ProcInspector, _is_model_file, classify, installed_models
+from gputaskman.procinfo import (MappedModel, ProcDetail, ProcInspector, _is_model_file, _models_from_cmdline,
+ classify, installed_models)
from gputaskman.report import to_dict, to_text
from gputaskman.stores import store_name
from tools.capture_screenshots import for_publication
@@ -75,6 +78,40 @@ class CliTests(unittest.TestCase):
main()
collector.return_value.close.assert_called_once()
+ def test_out_of_range_gpu_is_rejected(self):
+ adapter = Adapter(1, "NVIDIA GPU", 0x10DE, 16 * GiB, 0, False)
+ output = io.StringIO()
+ with patch("sys.argv", ["gputaskman", "--gpu", "1", "--json"]), \
+ patch("gputaskman.collector.enumerate_adapters", return_value=[adapter]), \
+ patch("gputaskman.collector.Collector") as collector, contextlib.redirect_stderr(output):
+ with self.assertRaises(SystemExit) as error:
+ main()
+ self.assertEqual(error.exception.code, 2)
+ self.assertIn("out of range", output.getvalue())
+ collector.assert_not_called()
+
+
+class DeviceSelectionTests(unittest.TestCase):
+ def test_adapter_indices_do_not_clamp_to_another_gpu(self):
+ gpu = Adapter(1, "NVIDIA GPU", 0x10DE, 16 * GiB, 0, False)
+ software = Adapter(2, "Software", 0x1414, 0, 0, True)
+ self.assertIs(select_adapter([software, gpu], 0), gpu)
+ with self.assertRaisesRegex(ValueError, "out of range"):
+ select_adapter([software, gpu], 1)
+ self.assertIsNone(select_adapter([], 0))
+ with self.assertRaisesRegex(ValueError, "out of range"):
+ select_adapter([], 1)
+
+ def test_nvml_matches_unique_adapter_name_before_index(self):
+ gpu = Adapter(1, "NVIDIA RTX Spark", 0x10DE, 64 * GiB, 0, False)
+ npu = NvDevice(index=0, name="NVIDIA NPU", driver_model="MCDM")
+ nv_gpu = NvDevice(index=1, name="NVIDIA RTX SPARK", driver_model="WDDM")
+ self.assertIs(select_nv_device(gpu, [npu, nv_gpu], 0), nv_gpu)
+ self.assertIs(select_nv_device(None, [npu, nv_gpu], 0), npu)
+ self.assertIsNone(select_nv_device(gpu, [npu], 0))
+ intel = Adapter(2, "Intel GPU", 0x8086, 8 * GiB, 0, False)
+ self.assertIsNone(select_nv_device(intel, [nv_gpu], 0))
+
class MemoryTests(unittest.TestCase):
def test_local_counters_are_preferred(self):
@@ -99,6 +136,18 @@ class MemoryTests(unittest.TestCase):
self.assertIn("PAGED", text)
self.assertNotIn("gen=None", text)
+ def test_text_shows_unavailable_device_metrics_without_fabricating_zeroes(self):
+ s = snapshot()
+ s.nv = NvDevice(index=0, util_gpu=0, power_w=2.4, pcie_tx_kbs=0)
+ text = to_text(s)
+ self.assertIn("GPU 0%", text)
+ self.assertIn("mem-bandwidth N/A", text)
+ self.assertIn("2/N/A W", text)
+ self.assertIn("P N/A", text)
+ self.assertIn("PCIe rx N/A MB/s tx 0 MB/s", text)
+ self.assertNotIn("None", text.splitlines()[2])
+ self.assertIsNone(to_dict(s)["device"]["power_limit_w"])
+
def test_capture_does_not_change_original_or_publish_command_line(self):
original = snapshot()
original.procs[0].llm.models = [
@@ -112,6 +161,19 @@ class MemoryTests(unittest.TestCase):
class ModelTests(unittest.TestCase):
+ def test_diffusion_arguments_do_not_classify_text_encoder_as_llm(self):
+ with tempfile.TemporaryDirectory() as directory:
+ paths = [Path(directory) / name for name in ("diffusion.gguf", "encoder.gguf", "vae.safetensors")]
+ for path in paths:
+ path.touch()
+ cmd = ["sd-cli.exe", "--diffusion-model", str(paths[0]),
+ f"--llm={paths[1]}", "--vae", str(paths[2])]
+ self.assertEqual(_models_from_cmdline(cmd), [str(path) for path in paths])
+ self.assertEqual(_models_from_cmdline(["sd-cli.exe", f"--diffusion-model={paths[0]}"]), [str(paths[0])])
+ d = ProcDetail(1, "sd-cli.exe", cmdline=cmd, runtimes=["ggml"],
+ models=[MappedModel(str(paths[1]), 0, 0, True, kind="LLM")])
+ self.assertEqual(classify(d, {}), "diffusion")
+
def test_layer_and_kv_estimates(self):
model = GgufInfo(
path="model.gguf", file_size=1000, version=3,
@@ -136,9 +198,10 @@ class ModelTests(unittest.TestCase):
{"ctx": 8192, "ngl": -1, "ctk": "q8_0"})
def test_gpu_buffers_exclude_host_and_cpu(self):
- for device in ("CPU", "CPU_Mapped", "CUDA_Host"):
+ for device in ("CPU", "CPU_Mapped", "CUDA_Host", "Vulkan_Host"):
self.assertFalse(is_gpu_device(device))
- self.assertTrue(is_gpu_device("CUDA0"))
+ for device in ("CUDA0", "Vulkan0"):
+ self.assertTrue(is_gpu_device(device))
def test_latest_load_and_multiple_kv_buffers(self):
with tempfile.TemporaryDirectory() as directory:
@@ -238,6 +301,17 @@ class IdentityTests(unittest.TestCase):
patch.object(ProcInspector, "_new", return_value=fresh):
self.assertIs(inspector.get(42), fresh)
+ def test_repeated_model_arguments_do_not_duplicate_evidence(self):
+ with tempfile.TemporaryDirectory() as directory:
+ path = str(Path(directory) / "model.gguf")
+ Path(path).write_bytes(b"not a GGUF")
+ detail = ProcDetail(6, "llama-server.exe", cmdline=["server", "--model", path, "--mmproj", path])
+ with patch("gputaskman.procinfo.psutil.Process") as process:
+ process.return_value.memory_maps.return_value = []
+ ProcInspector().scan(detail)
+ self.assertEqual(len(detail.models), 1)
+ self.assertEqual(detail.models[0].sources, ["cmdline"])
+
def test_ambiguous_extensions_need_size_or_store(self):
self.assertFalse(_is_model_file(".bin", 10 * MiB, False, None)) # e.g. a Start menu tile cache
self.assertTrue(_is_model_file(".bin", 10 * MiB, False, "org/repo"))
@@ -265,9 +339,30 @@ class IdentityTests(unittest.TestCase):
self.assertEqual(v.files[0].evidence, "confirmed (api+mapped)")
self.assertEqual(d.models[0].sources, ["mapped"]) # cached detail is not mutated
self.assertIsNone(v.kv_per_token)
- self.assertEqual([s.label for s in v.segments], ["weights", "compute buffers + CUDA context"])
+ self.assertEqual([s.label for s in v.segments], ["weights", "compute buffers + runtime"])
self.assertTrue(any("laya decision model" in n for n in v.notes))
+ def test_vulkan_log_does_not_label_unknown_overhead_as_cuda(self):
+ laya = gguf("modern-bert", **LAYA)
+ detail = ProcDetail(5, "llama-server.exe", models=[
+ MappedModel("laya.gguf", 1000, 0, True, kind="decision model")
+ ])
+ with tempfile.TemporaryDirectory() as directory:
+ path = Path(directory) / "server.log"
+ path.write_text(
+ "load_tensors: Vulkan0 model buffer size = 10 MiB\n"
+ "load_tensors: Vulkan_Host model buffer size = 3 MiB\n"
+ "llama_context: Vulkan0 compute buffer size = 2 MiB\n",
+ encoding="utf-8",
+ )
+ with patch("gputaskman.llm.read_gguf", return_value=laya):
+ view = build_view(detail, 20 * MiB, 0, None, str(path))
+ self.assertEqual([(s.label, s.bytes) for s in view.segments], [
+ ("weights (Vulkan0)", 10 * MiB),
+ ("compute buffers (Vulkan0)", 2 * MiB),
+ ("runtime and other overhead", 8 * MiB),
+ ])
+
def test_installed_models_are_candidates_only(self):
with tempfile.TemporaryDirectory() as root:
app = Path(root, "Speechy")
@@ -305,8 +400,34 @@ class IdentityTests(unittest.TestCase):
self.assertEqual(len(ollama), 1)
self.assertIn("qwen3:14b", ollama[0])
+ def test_runner_and_host_apis_do_not_duplicate_evidence(self):
+ qwen = gguf("qwen3", block_count=1, context_length=4096)
+ model = MappedModel("blob", 1000, 0, True, ["cmdline"], "qwen3:4b", "LLM")
+ detail = ProcDetail(6, "llama-server.exe", models=[model])
+ runner = ServerInfo("llama-server", "http://127.0.0.1:1", model_path="blob")
+ host = ServerInfo("ollama", "http://127.0.0.1:2", models=[{"name": "qwen3:4b", "ctx": 4096}])
+ with patch("gputaskman.llm.read_gguf", return_value=qwen):
+ view = build_view(detail, GiB, 0, runner, host=host)
+ self.assertEqual(view.files[0].sources, ["cmdline", "api"])
+ self.assertEqual(model.sources, ["cmdline"])
+
class UiTests(unittest.IsolatedAsyncioTestCase):
+ async def test_unavailable_device_metrics_render_as_na(self):
+ with patch.object(GpuTaskManager, "poll"):
+ app = GpuTaskManager()
+ async with app.run_test(size=(140, 42)):
+ s = snapshot()
+ s.nv = NvDevice(index=0, util_mem=0, power_w=2.4, pcie_tx_kbs=0)
+ app.apply(s)
+ lines = "\n".join(line.plain for line in app.render_device(s).renderables)
+ self.assertIn("2/N/A W", lines)
+ self.assertIn("P N/A", lines)
+ self.assertIn("PCIe↓ N/A", lines)
+ self.assertNotIn("None", lines)
+ self.assertIsNone(app.hist_util[-1])
+ self.assertEqual(app.hist_bw[-1], 0)
+
async def test_navigation_sort_pause_and_help(self):
with patch.object(GpuTaskManager, "poll"):
app = GpuTaskManager()