A GPU task manager for Windows, Linux, and WSL: per-process VRAM, GPU activity, and LLM model/KV-cache insights (Windows AMD and NVIDIA).
Python
Show Handy's selected model and runtime-reported model events with explicit provenance
Scott Hanselman0c1f757db566
diff --git a/README.md b/README.mdindex 919b64c..999f9fd 100644--- a/README.md+++ b/README.md@@ -190,7 +190,7 @@ reading is NVML's memory-activity percentage, **not measured bandwidth in GB/s**| stable-diffusion.cpp (`--diffusion-model`) | Diffusion, text-encoder and VAE files from launch arguments; no LLM memory breakdown || Hugging Face cache | `models--org--repo` paths are named `org/repo` || Other AI runtimes | DLL detection, plus model files installed with the app (marked *not proven loaded*) |-| Handy | Local speech workloads are in scope; runtime DLLs/mapped files and installed candidates are inspected. A selected/downloaded speech model alone is not proof of GPU residency or use |+| Handy | Reads selected model from settings and bounded load/start/preview/unload events from `handy.log`; selection is not load evidence and log events are historical, not proof of current residency || WinML / local helper processes | GPU-backed execution is in scope; generic process counters and runtime detection only. No WinML-specific model/session or client-to-helper attribution yet || Windows Studio Effects | Only any observable GPU portion is in scope. NPU inference is out of scope; do not interpret GPU camera/render activity as proof of model execution |@@ -215,9 +215,11 @@ Every model row shows its **evidence**, strongest first:| `mapped` | Windows shows the file memory-mapped into the process right now || `cmdline` | The file was a launch argument (`-m model.gguf`) || `installed` | Found in the app's install or data folder; nothing proves it is loaded |+| Handy settings / log | Settings show selection; timestamped log events show what Handy reported, not current residency |-Two sources including `api` or `mapped` show as **confirmed**. Model-store folders only supply a-friendly name. The authoritative model *kind* comes from GGUF header keys: `decision.type` marks a+Two sources including `api` or `mapped` show as **confirmed**. Runtime logs can report model events,+but are displayed as timestamped history and do not prove current residency. Model-store folders+only supply a friendly name. The authoritative model *kind* comes from GGUF header keys: `decision.type` marks adecision model (for example [Laya](https://huggingface.co/ggml-org/Laya-GGUF)), `pooling_type`an embedding model or reranker, `stt.*` a speech model, and a causal model with layers and atrained context an LLM. llama-server's `/v1/models` `output_modalities` (`text`, `decisions`)@@ -238,10 +240,23 @@ Neither a type hint nor an API/file match proves which model produced the procesSome runtimes (ONNX Runtime, whisper.cpp) read a model into memory and close the file. Windows thenhas no record of which file is loaded, so the app's installed models are listed as candidates only.-For Handy, the configured `selected_model` can refer to a different model in a separate cache;-the current installed-folder fallback does not read that setting. Even reading it would establish-selection, not current GPU residency or execution. Do not assume the largest installed speech-model is the one in use.+For Handy, gputaskman reads the configured `selected_model` and only model-load, streaming-start,+preview, and unload events from its local log. Settings prove selection; a fresh log event proves+Handy reported that event after the current process started. Neither proves continuous GPU residency.+Log parsing is bounded to the last 512 KiB, cached until the file changes, and ignores transcription+content. Handy's selected Hugging Face model can live outside its Roaming `models` folder. Handy+logs in UTC; JSON keeps that UTC timestamp and the TUI/text output show local time.++**Other speech and vision apps.** Handy is the first *app adapter*: a read-only recipe for where an+app keeps its settings and which log lines mark a model load or unload. Most whisper-style apps+(whisper.cpp, faster-whisper, sherpa-onnx, ONNX Runtime) read a model and close the file, so the OS+keeps no record of it. Adapters give exact names when an app logs them, but each app needs its own+recipe, and a log-format change can break it. The runtime-agnostic answer is to see the file being+opened during the load: Windows kernel file-I/O ETW events record which process opened which model+file. Combined with that process's VRAM jump, this identifies the model without app-specific code.+It needs elevation and must already be running when the load happens, so it is planned as an+opt-in mode. Until then, unknown apps show their runtime DLLs, VRAM, engine activity, and installed+candidates. Reading model names out of another process's memory is deliberately not done.Identification is designed to stay cheap. Measured on one Windows machine:@@ -349,6 +364,9 @@ downloads are bundled in this repository.- Explicit provenance and availability for every counter and estimate.- Reliable multi-GPU matching and per-device model placement.- Live throughput deltas, LM Studio validation, and per-model context accounting.+- Opt-in ETW file-open tracing to name models loaded by read-and-close runtimes (any whisper or ONNX+ app), correlated with the process's VRAM jump at load time.+- Data-driven app adapters (settings path plus load/unload log patterns) once a second app needs one.- Optional ETW / runtime instrumentation for allocation lifetime and residency detail.## Licensediff --git a/src/gputaskman/app.py b/src/gputaskman/app.pyindex 78c7bc7..c6a1016 100644--- a/src/gputaskman/app.py+++ b/src/gputaskman/app.py@@ -329,6 +329,14 @@ class GpuTaskManager(App):"usage. Estimated non-residency, not proof of eviction or disk paging.",style="yellow"))+ if r.handy:+ if r.handy.selected_model:+ out.append(Text(f"Handy selected (settings; not load proof): {r.handy.selected_model}", style="grey70"))+ if r.handy.last_event:+ out.append(Text(f"Handy last event: {r.handy.last_event} {r.handy.last_model or ''} "+ f"at {r.handy.last_event_local} (runtime-reported history, not current GPU residency)",+ style="green"))+v = r.llmif v:width = max(min(self.size.width - 8, 100), 20)diff --git a/src/gputaskman/collector.py b/src/gputaskman/collector.pyindex dfcaab0..216b4ae 100644--- a/src/gputaskman/collector.py+++ b/src/gputaskman/collector.py@@ -9,6 +9,7 @@ from dataclasses import dataclass, fieldimport psutilfrom .dxgi import VENDOR_NVIDIA, Adapter, enumerate_adapters+from .handy import HandyEvidence, inspect_handyfrom .llm import LlmView, ServerProber, build_view, gb, ollama_log_pathfrom .nvml import Nvml, NvDevicefrom .pdh import GpuCounters, ProcGpu@@ -25,6 +26,7 @@ class ProcRow:gpu: ProcGpudetail: ProcDetailllm: LlmView | None = None+ handy: HandyEvidence | None = None@propertydef runtime(self) -> str:@@ -33,6 +35,11 @@ class ProcRow:@propertydef inside(self) -> str:"""One-line 'what's in there' summary for the process table."""+ if self.handy:+ if self.handy.last_model:+ return f"Handy last log ({self.handy.last_event}): {self.handy.last_model.rsplit('/', 1)[-1]}"+ if self.handy.selected_model:+ return f"Handy selected (not load proof): {self.handy.selected_model.rsplit('/', 1)[-1]}"if self.llm and self.llm.models:parts = []for i, g in enumerate(self.llm.models):@@ -164,6 +171,8 @@ class Collector:ollama_runners.append(row)for row in snap.procs:+ if os.path.basename(row.detail.exe).lower() == "handy.exe":+ row.handy = inspect_handy(row.detail.create_time)if row.detail.is_diffusion or row.kind not in AI_KINDS or not (row.detail.models or row.kind == "LLM"):continueown = self.prober.get(row.detail.ports)diff --git a/src/gputaskman/handy.py b/src/gputaskman/handy.pynew file mode 100644index 0000000..14380ea--- /dev/null+++ b/src/gputaskman/handy.py@@ -0,0 +1,134 @@+"""Small, read-only Handy settings and model-event probe."""++from __future__ import annotations++import json+import os+import re+from dataclasses import dataclass+from datetime import datetime, timezone++_MAX_SETTINGS_BYTES = 256 << 10+_MAX_LOG_BYTES = 512 << 10+_settings_cache: dict[str, tuple[tuple[int, int], str | None]] = {}+_log_cache: dict[tuple[str, float], tuple[tuple[int, int], tuple[str, str, str] | None]] = {}++_TIMESTAMP = re.compile(r"^\[(\d{4}-\d{2}-\d{2})\]\[(\d{2}:\d{2}:\d{2})\]")+_MODEL_EVENT = re.compile(+ r"(?:Loaded whisper model |Live preview: model |Live streaming transcription started \(model )'([^']+)'"+)+_UNLOAD_EVENT = re.compile(r"Model unloaded due to inactivity|Model idle for \d+s.*unloading", re.I)+++@dataclass(frozen=True)+class HandyEvidence:+ selected_model: str | None = None+ last_model: str | None = None+ last_event: str | None = None+ last_event_at: str | None = None # UTC ISO 8601, as Handy logs it++ @property+ def last_event_local(self) -> str:+ if not self.last_event_at:+ return ""+ try:+ return datetime.fromisoformat(self.last_event_at).astimezone().strftime("%Y-%m-%d %H:%M:%S")+ except ValueError:+ return self.last_event_at+++def _signature(path: str) -> tuple[int, int] | None:+ try:+ stat = os.stat(path)+ return stat.st_mtime_ns, stat.st_size+ except OSError:+ return None+++def _selected_model(path: str) -> str | None:+ signature = _signature(path)+ if signature is None:+ return None+ hit = _settings_cache.get(path)+ if hit and hit[0] == signature:+ return hit[1]+ value = None+ if signature[1] <= _MAX_SETTINGS_BYTES:+ try:+ with open(path, "rb") as f:+ data = json.loads(f.read(_MAX_SETTINGS_BYTES + 1))+ selected = (data.get("settings") or {}).get("selected_model")+ value = selected if isinstance(selected, str) and selected else None+ except (OSError, ValueError, AttributeError):+ pass+ _settings_cache[path] = (signature, value)+ return value+++def _last_model_event(path: str, process_started: float) -> tuple[str, str, str] | None:+ signature = _signature(path)+ if signature is None:+ return None+ key = (path, process_started)+ hit = _log_cache.get(key)+ if hit and hit[0] == signature:+ return hit[1]+ latest: tuple[str, str, str] | None = None+ try:+ with open(path, "rb") as f:+ f.seek(0, os.SEEK_END)+ end = f.tell()+ f.seek(max(0, end - _MAX_LOG_BYTES))+ lines = f.read(_MAX_LOG_BYTES).decode("utf-8", "replace").splitlines()+ except OSError:+ lines = []+ for line in lines:+ match = _TIMESTAMP.match(line)+ if not match:+ continue+ try:+ occurred = datetime.strptime(+ f"{match[1]} {match[2]}", "%Y-%m-%d %H:%M:%S"+ ).replace(tzinfo=timezone.utc)+ except ValueError:+ continue+ if process_started and occurred.timestamp() < process_started - 2:+ continue+ event = None+ model_match = _MODEL_EVENT.search(line)+ if model_match:+ if "Loaded whisper model " in line:+ event = "loaded"+ elif "Live streaming transcription started " in line:+ event = "streaming started"+ else:+ event = "preview"+ latest = (event, model_match[1], occurred.isoformat())+ elif _UNLOAD_EVENT.search(line):+ event = "unloaded" if "unloaded" in line.lower() else "unloading"+ model = latest[1] if latest else ""+ latest = (event, model, occurred.isoformat())+ _log_cache[key] = (signature, latest)+ return latest+++def inspect_handy(+ process_started: float,+ appdata: str | None = None,+ localappdata: str | None = None,+) -> HandyEvidence | None:+ """Read Handy's selected model and latest model event after this process started."""+ appdata = os.environ.get("APPDATA", "") if appdata is None else appdata+ localappdata = os.environ.get("LOCALAPPDATA", "") if localappdata is None else localappdata+ settings_path = os.path.join(appdata, "com.pais.handy", "settings_store.json")+ log_path = os.path.join(localappdata, "com.pais.handy", "logs", "handy.log")+ selected = _selected_model(settings_path)+ event = _last_model_event(log_path, process_started)+ if not selected and not event:+ return None+ return HandyEvidence(+ selected_model=selected,+ last_model=event[1] or None if event else None,+ last_event=event[0] if event else None,+ last_event_at=event[2] if event else None,+ )diff --git a/src/gputaskman/report.py b/src/gputaskman/report.pyindex 7030ea6..846b469 100644--- a/src/gputaskman/report.py+++ b/src/gputaskman/report.py@@ -24,6 +24,8 @@ def to_dict(s: Snapshot) -> dict:"model_files": [dataclasses.asdict(m) for m in p.detail.models],"installed_models": [dataclasses.asdict(m) for m in p.detail.candidates],}+ if p.handy:+ item["handy"] = dataclasses.asdict(p.handy)if p.llm:item["llm"] = {"models": [{"path": g.path, "name": g.name, "label": p.llm.label(i), "arch": g.arch, "kind": g.kind,@@ -71,6 +73,13 @@ def to_text(s: Snapshot, limit: int = 30) -> str:out.append(f"{p.pid:>7} {p.name[:24]:<24} {p.kind:<10} {p.runtime[:24]:<24} {gb(p.gpu.dedicated):>9} {spill:>8} "f"{paged:>8} {p.gpu.gpu_pct:>5.0f} {p.inside}")for p in s.procs:+ if p.handy:+ out.append("")+ if p.handy.selected_model:+ out.append(f"== {p.name} ({p.pid}) Handy selected (settings, not load proof): {p.handy.selected_model}")+ if p.handy.last_event:+ out.append(f" last Handy event: {p.handy.last_event} {p.handy.last_model or ''} "+ f"at {p.handy.last_event_local} (runtime-reported history, not current GPU residency)")if not p.llm:continueout.append("")diff --git a/tests/test_core.py b/tests/test_core.pyindex 74131ab..cbda729 100644--- a/tests/test_core.py+++ b/tests/test_core.py@@ -7,6 +7,7 @@ import osimport tempfileimport unittestimport xml.etree.ElementTree as ET+from datetime import datetime, timezonefrom pathlib import Pathfrom unittest.mock import patch@@ -15,6 +16,7 @@ from gputaskman.cli import mainfrom gputaskman.collector import ProcRow, Snapshot, select_adapter, select_nv_devicefrom gputaskman.dxgi import Adapterfrom gputaskman.gguf import GgufInfo, read_gguf+from gputaskman.handy import HandyEvidence, inspect_handy, _log_cache, _settings_cachefrom gputaskman.llm import (LlmView, ServerInfo, build_view, is_gpu_device, llama_server_models, lmstudio_models,parse_flags, parse_load_log, probe_server)from gputaskman.nvml import NvDevice@@ -161,6 +163,74 @@ class MemoryTests(unittest.TestCase):self.assertIn("Users", original.procs[0].llm.models[0].path)+class HandyTests(unittest.TestCase):+ def setUp(self):+ _log_cache.clear()+ _settings_cache.clear()++ def test_selected_model_is_distinct_from_runtime_load_and_unload_events(self):+ with tempfile.TemporaryDirectory() as directory:+ appdata, localappdata = Path(directory) / "Roaming", Path(directory) / "Local"+ settings_dir = appdata / "com.pais.handy"+ log_dir = localappdata / "com.pais.handy" / "logs"+ settings_dir.mkdir(parents=True)+ log_dir.mkdir(parents=True)+ selected = "handy-computer/nemotron/model-Q8_0.gguf"+ (settings_dir / "settings_store.json").write_text(+ json.dumps({"settings": {"selected_model": selected}}), encoding="utf-8"+ )+ (log_dir / "handy.log").write_text(+ "[2026-11-02][22:59:00][transcription][INFO] Loaded whisper model 'old.gguf'\n"+ "[2026-11-02][23:01:00][transcription][INFO] Loaded whisper model 'handy-computer/nemotron/model-Q8_0.gguf' (backend 'Vulkan0')\n"+ "[2026-11-02][23:01:01][transcription][INFO] Live streaming transcription started (model 'handy-computer/nemotron/model-Q8_0.gguf', backend 'Vulkan0')\n"+ "[2026-11-02][23:01:02][transcription][INFO] Model idle for 3606s (limit: 3600s), unloading\n"+ "[2026-11-02][23:01:03][transcription][INFO] Model unloaded due to inactivity (took 20ms)\n"+ "[2026-11-02][23:01:04][transcription][INFO] Partial transcription: private dictation text\n",+ encoding="utf-8",+ )+ started = datetime(2026, 11, 2, 23, 0, tzinfo=timezone.utc).timestamp()+ evidence = inspect_handy(started, str(appdata), str(localappdata))+ self.assertEqual(evidence.selected_model, selected)+ self.assertEqual(evidence.last_model, "handy-computer/nemotron/model-Q8_0.gguf")+ self.assertEqual(evidence.last_event, "unloaded")+ self.assertTrue(evidence.last_event_at.endswith("+00:00"))+ local = datetime(2026, 11, 2, 23, 1, 3, tzinfo=timezone.utc).astimezone()+ self.assertEqual(evidence.last_event_local, local.strftime("%Y-%m-%d %H:%M:%S"))+ self.assertNotIn("private dictation text", repr(evidence))++ def test_selected_model_without_fresh_log_is_not_reported_as_loaded(self):+ with tempfile.TemporaryDirectory() as directory:+ appdata, localappdata = Path(directory) / "Roaming", Path(directory) / "Local"+ settings_dir = appdata / "com.pais.handy"+ log_dir = localappdata / "com.pais.handy" / "logs"+ settings_dir.mkdir(parents=True)+ log_dir.mkdir(parents=True)+ selected = "handy-computer/nemotron/model-Q8_0.gguf"+ (settings_dir / "settings_store.json").write_text(+ json.dumps({"settings": {"selected_model": selected}}), encoding="utf-8"+ )+ (log_dir / "handy.log").write_text(+ "[2026-11-02][22:59:00][transcription][INFO] Loaded whisper model 'old.gguf'\n",+ encoding="utf-8",+ )+ started = datetime(2026, 11, 2, 23, 0, tzinfo=timezone.utc).timestamp()+ evidence = inspect_handy(started, str(appdata), str(localappdata))+ self.assertEqual(evidence.selected_model, selected)+ self.assertIsNone(evidence.last_model)+ self.assertIsNone(evidence.last_event)++ def test_handy_evidence_is_visible_in_text_and_json(self):+ s = snapshot()+ s.procs[0] = ProcRow(+ 10, "handy.exe", "AI", ProcGpu(dedicated=GiB), ProcDetail(10, "handy.exe"),+ handy=HandyEvidence("handy-computer/nemotron/model.gguf", "model.gguf", "loaded",+ "2026-11-02T23:01:00+00:00"),+ )+ self.assertIn("Handy last log (loaded): model.gguf", s.procs[0].inside)+ self.assertIn("not current GPU residency", to_text(s))+ self.assertEqual(to_dict(s)["processes"][0]["handy"]["last_event"], "loaded")++class ModelTests(unittest.TestCase):def test_diffusion_arguments_do_not_classify_text_encoder_as_llm(self):with tempfile.TemporaryDirectory() as directory:@@ -532,6 +602,24 @@ class UiTests(unittest.IsolatedAsyncioTestCase):self.assertIsNone(app.hist_util[-1])self.assertEqual(app.hist_bw[-1], 0)+ async def test_handy_evidence_is_explicit_in_inspector(self):+ with patch.object(GpuTaskManager, "poll"):+ app = GpuTaskManager()+ async with app.run_test(size=(180, 42)) as pilot:+ s = snapshot()+ s.procs[0] = ProcRow(+ 10, "handy.exe", "AI", ProcGpu(dedicated=GiB), ProcDetail(10, "handy.exe"),+ handy=HandyEvidence("handy-computer/nemotron/model.gguf", "model.gguf", "loaded",+ "2026-11-02T23:01:00+00:00"),+ )+ app.apply(s)+ await pilot.pause()+ await pilot.press("down")+ await pilot.pause()+ text = " ".join(ET.fromstring(app.export_screenshot()).itertext()).replace("\u00a0", " ")+ self.assertIn("Handy selected (settings; not load proof)", text)+ self.assertIn("runtime-reported history, not current GPU residency", text)+async def test_likely_kind_is_explicit_in_process_and_inspector(self):g = gguf("qwen35", block_count=2, context_length=4096, **{"general.name": "Snapshot_Cfg"})with patch.object(GpuTaskManager, "poll"):