Skip to content

A GPU task manager for Windows, Linux, and WSL: per-process VRAM, GPU activity, and LLM model/KV-cache insights (Windows AMD and NVIDIA).

Python

Show Handy's selected model and runtime-reported model events with explicit provenance

Scott Hanselman0c1f757db566
diff --git a/README.md b/README.md
index 919b64c..999f9fd 100644
--- a/README.md
+++ b/README.md
@@ -190,7 +190,7 @@ reading is NVML's memory-activity percentage, **not measured bandwidth in GB/s**
| stable-diffusion.cpp (`--diffusion-model`) | Diffusion, text-encoder and VAE files from launch arguments; no LLM memory breakdown |
| Hugging Face cache | `models--org--repo` paths are named `org/repo` |
| Other AI runtimes | DLL detection, plus model files installed with the app (marked *not proven loaded*) |
-| Handy | Local speech workloads are in scope; runtime DLLs/mapped files and installed candidates are inspected. A selected/downloaded speech model alone is not proof of GPU residency or use |
+| Handy | Reads selected model from settings and bounded load/start/preview/unload events from `handy.log`; selection is not load evidence and log events are historical, not proof of current residency |
| WinML / local helper processes | GPU-backed execution is in scope; generic process counters and runtime detection only. No WinML-specific model/session or client-to-helper attribution yet |
| Windows Studio Effects | Only any observable GPU portion is in scope. NPU inference is out of scope; do not interpret GPU camera/render activity as proof of model execution |
@@ -215,9 +215,11 @@ Every model row shows its **evidence**, strongest first:
| `mapped` | Windows shows the file memory-mapped into the process right now |
| `cmdline` | The file was a launch argument (`-m model.gguf`) |
| `installed` | Found in the app's install or data folder; nothing proves it is loaded |
+| Handy settings / log | Settings show selection; timestamped log events show what Handy reported, not current residency |
-Two sources including `api` or `mapped` show as **confirmed**. Model-store folders only supply a
-friendly name. The authoritative model *kind* comes from GGUF header keys: `decision.type` marks a
+Two sources including `api` or `mapped` show as **confirmed**. Runtime logs can report model events,
+but are displayed as timestamped history and do not prove current residency. Model-store folders
+only supply a friendly name. The authoritative model *kind* comes from GGUF header keys: `decision.type` marks a
decision model (for example [Laya](https://huggingface.co/ggml-org/Laya-GGUF)), `pooling_type`
an embedding model or reranker, `stt.*` a speech model, and a causal model with layers and a
trained context an LLM. llama-server's `/v1/models` `output_modalities` (`text`, `decisions`)
@@ -238,10 +240,23 @@ Neither a type hint nor an API/file match proves which model produced the proces
Some runtimes (ONNX Runtime, whisper.cpp) read a model into memory and close the file. Windows then
has no record of which file is loaded, so the app's installed models are listed as candidates only.
-For Handy, the configured `selected_model` can refer to a different model in a separate cache;
-the current installed-folder fallback does not read that setting. Even reading it would establish
-selection, not current GPU residency or execution. Do not assume the largest installed speech
-model is the one in use.
+For Handy, gputaskman reads the configured `selected_model` and only model-load, streaming-start,
+preview, and unload events from its local log. Settings prove selection; a fresh log event proves
+Handy reported that event after the current process started. Neither proves continuous GPU residency.
+Log parsing is bounded to the last 512 KiB, cached until the file changes, and ignores transcription
+content. Handy's selected Hugging Face model can live outside its Roaming `models` folder. Handy
+logs in UTC; JSON keeps that UTC timestamp and the TUI/text output show local time.
+
+**Other speech and vision apps.** Handy is the first *app adapter*: a read-only recipe for where an
+app keeps its settings and which log lines mark a model load or unload. Most whisper-style apps
+(whisper.cpp, faster-whisper, sherpa-onnx, ONNX Runtime) read a model and close the file, so the OS
+keeps no record of it. Adapters give exact names when an app logs them, but each app needs its own
+recipe, and a log-format change can break it. The runtime-agnostic answer is to see the file being
+opened during the load: Windows kernel file-I/O ETW events record which process opened which model
+file. Combined with that process's VRAM jump, this identifies the model without app-specific code.
+It needs elevation and must already be running when the load happens, so it is planned as an
+opt-in mode. Until then, unknown apps show their runtime DLLs, VRAM, engine activity, and installed
+candidates. Reading model names out of another process's memory is deliberately not done.
Identification is designed to stay cheap. Measured on one Windows machine:
@@ -349,6 +364,9 @@ downloads are bundled in this repository.
- Explicit provenance and availability for every counter and estimate.
- Reliable multi-GPU matching and per-device model placement.
- Live throughput deltas, LM Studio validation, and per-model context accounting.
+- Opt-in ETW file-open tracing to name models loaded by read-and-close runtimes (any whisper or ONNX
+ app), correlated with the process's VRAM jump at load time.
+- Data-driven app adapters (settings path plus load/unload log patterns) once a second app needs one.
- Optional ETW / runtime instrumentation for allocation lifetime and residency detail.
## License
diff --git a/src/gputaskman/app.py b/src/gputaskman/app.py
index 78c7bc7..c6a1016 100644
--- a/src/gputaskman/app.py
+++ b/src/gputaskman/app.py
@@ -329,6 +329,14 @@ class GpuTaskManager(App):
"usage. Estimated non-residency, not proof of eviction or disk paging.",
style="yellow"))
+ if r.handy:
+ if r.handy.selected_model:
+ out.append(Text(f"Handy selected (settings; not load proof): {r.handy.selected_model}", style="grey70"))
+ if r.handy.last_event:
+ out.append(Text(f"Handy last event: {r.handy.last_event} {r.handy.last_model or ''} "
+ f"at {r.handy.last_event_local} (runtime-reported history, not current GPU residency)",
+ style="green"))
+
v = r.llm
if v:
width = max(min(self.size.width - 8, 100), 20)
diff --git a/src/gputaskman/collector.py b/src/gputaskman/collector.py
index dfcaab0..216b4ae 100644
--- a/src/gputaskman/collector.py
+++ b/src/gputaskman/collector.py
@@ -9,6 +9,7 @@ from dataclasses import dataclass, field
import psutil
from .dxgi import VENDOR_NVIDIA, Adapter, enumerate_adapters
+from .handy import HandyEvidence, inspect_handy
from .llm import LlmView, ServerProber, build_view, gb, ollama_log_path
from .nvml import Nvml, NvDevice
from .pdh import GpuCounters, ProcGpu
@@ -25,6 +26,7 @@ class ProcRow:
gpu: ProcGpu
detail: ProcDetail
llm: LlmView | None = None
+ handy: HandyEvidence | None = None
@property
def runtime(self) -> str:
@@ -33,6 +35,11 @@ class ProcRow:
@property
def inside(self) -> str:
"""One-line 'what's in there' summary for the process table."""
+ if self.handy:
+ if self.handy.last_model:
+ return f"Handy last log ({self.handy.last_event}): {self.handy.last_model.rsplit('/', 1)[-1]}"
+ if self.handy.selected_model:
+ return f"Handy selected (not load proof): {self.handy.selected_model.rsplit('/', 1)[-1]}"
if self.llm and self.llm.models:
parts = []
for i, g in enumerate(self.llm.models):
@@ -164,6 +171,8 @@ class Collector:
ollama_runners.append(row)
for row in snap.procs:
+ if os.path.basename(row.detail.exe).lower() == "handy.exe":
+ row.handy = inspect_handy(row.detail.create_time)
if row.detail.is_diffusion or row.kind not in AI_KINDS or not (row.detail.models or row.kind == "LLM"):
continue
own = self.prober.get(row.detail.ports)
diff --git a/src/gputaskman/handy.py b/src/gputaskman/handy.py
new file mode 100644
index 0000000..14380ea
--- /dev/null
+++ b/src/gputaskman/handy.py
@@ -0,0 +1,134 @@
+"""Small, read-only Handy settings and model-event probe."""
+
+from __future__ import annotations
+
+import json
+import os
+import re
+from dataclasses import dataclass
+from datetime import datetime, timezone
+
+_MAX_SETTINGS_BYTES = 256 << 10
+_MAX_LOG_BYTES = 512 << 10
+_settings_cache: dict[str, tuple[tuple[int, int], str | None]] = {}
+_log_cache: dict[tuple[str, float], tuple[tuple[int, int], tuple[str, str, str] | None]] = {}
+
+_TIMESTAMP = re.compile(r"^\[(\d{4}-\d{2}-\d{2})\]\[(\d{2}:\d{2}:\d{2})\]")
+_MODEL_EVENT = re.compile(
+ r"(?:Loaded whisper model |Live preview: model |Live streaming transcription started \(model )'([^']+)'"
+)
+_UNLOAD_EVENT = re.compile(r"Model unloaded due to inactivity|Model idle for \d+s.*unloading", re.I)
+
+
+@dataclass(frozen=True)
+class HandyEvidence:
+ selected_model: str | None = None
+ last_model: str | None = None
+ last_event: str | None = None
+ last_event_at: str | None = None # UTC ISO 8601, as Handy logs it
+
+ @property
+ def last_event_local(self) -> str:
+ if not self.last_event_at:
+ return ""
+ try:
+ return datetime.fromisoformat(self.last_event_at).astimezone().strftime("%Y-%m-%d %H:%M:%S")
+ except ValueError:
+ return self.last_event_at
+
+
+def _signature(path: str) -> tuple[int, int] | None:
+ try:
+ stat = os.stat(path)
+ return stat.st_mtime_ns, stat.st_size
+ except OSError:
+ return None
+
+
+def _selected_model(path: str) -> str | None:
+ signature = _signature(path)
+ if signature is None:
+ return None
+ hit = _settings_cache.get(path)
+ if hit and hit[0] == signature:
+ return hit[1]
+ value = None
+ if signature[1] <= _MAX_SETTINGS_BYTES:
+ try:
+ with open(path, "rb") as f:
+ data = json.loads(f.read(_MAX_SETTINGS_BYTES + 1))
+ selected = (data.get("settings") or {}).get("selected_model")
+ value = selected if isinstance(selected, str) and selected else None
+ except (OSError, ValueError, AttributeError):
+ pass
+ _settings_cache[path] = (signature, value)
+ return value
+
+
+def _last_model_event(path: str, process_started: float) -> tuple[str, str, str] | None:
+ signature = _signature(path)
+ if signature is None:
+ return None
+ key = (path, process_started)
+ hit = _log_cache.get(key)
+ if hit and hit[0] == signature:
+ return hit[1]
+ latest: tuple[str, str, str] | None = None
+ try:
+ with open(path, "rb") as f:
+ f.seek(0, os.SEEK_END)
+ end = f.tell()
+ f.seek(max(0, end - _MAX_LOG_BYTES))
+ lines = f.read(_MAX_LOG_BYTES).decode("utf-8", "replace").splitlines()
+ except OSError:
+ lines = []
+ for line in lines:
+ match = _TIMESTAMP.match(line)
+ if not match:
+ continue
+ try:
+ occurred = datetime.strptime(
+ f"{match[1]} {match[2]}", "%Y-%m-%d %H:%M:%S"
+ ).replace(tzinfo=timezone.utc)
+ except ValueError:
+ continue
+ if process_started and occurred.timestamp() < process_started - 2:
+ continue
+ event = None
+ model_match = _MODEL_EVENT.search(line)
+ if model_match:
+ if "Loaded whisper model " in line:
+ event = "loaded"
+ elif "Live streaming transcription started " in line:
+ event = "streaming started"
+ else:
+ event = "preview"
+ latest = (event, model_match[1], occurred.isoformat())
+ elif _UNLOAD_EVENT.search(line):
+ event = "unloaded" if "unloaded" in line.lower() else "unloading"
+ model = latest[1] if latest else ""
+ latest = (event, model, occurred.isoformat())
+ _log_cache[key] = (signature, latest)
+ return latest
+
+
+def inspect_handy(
+ process_started: float,
+ appdata: str | None = None,
+ localappdata: str | None = None,
+) -> HandyEvidence | None:
+ """Read Handy's selected model and latest model event after this process started."""
+ appdata = os.environ.get("APPDATA", "") if appdata is None else appdata
+ localappdata = os.environ.get("LOCALAPPDATA", "") if localappdata is None else localappdata
+ settings_path = os.path.join(appdata, "com.pais.handy", "settings_store.json")
+ log_path = os.path.join(localappdata, "com.pais.handy", "logs", "handy.log")
+ selected = _selected_model(settings_path)
+ event = _last_model_event(log_path, process_started)
+ if not selected and not event:
+ return None
+ return HandyEvidence(
+ selected_model=selected,
+ last_model=event[1] or None if event else None,
+ last_event=event[0] if event else None,
+ last_event_at=event[2] if event else None,
+ )
diff --git a/src/gputaskman/report.py b/src/gputaskman/report.py
index 7030ea6..846b469 100644
--- a/src/gputaskman/report.py
+++ b/src/gputaskman/report.py
@@ -24,6 +24,8 @@ def to_dict(s: Snapshot) -> dict:
"model_files": [dataclasses.asdict(m) for m in p.detail.models],
"installed_models": [dataclasses.asdict(m) for m in p.detail.candidates],
}
+ if p.handy:
+ item["handy"] = dataclasses.asdict(p.handy)
if p.llm:
item["llm"] = {
"models": [{"path": g.path, "name": g.name, "label": p.llm.label(i), "arch": g.arch, "kind": g.kind,
@@ -71,6 +73,13 @@ def to_text(s: Snapshot, limit: int = 30) -> str:
out.append(f"{p.pid:>7} {p.name[:24]:<24} {p.kind:<10} {p.runtime[:24]:<24} {gb(p.gpu.dedicated):>9} {spill:>8} "
f"{paged:>8} {p.gpu.gpu_pct:>5.0f} {p.inside}")
for p in s.procs:
+ if p.handy:
+ out.append("")
+ if p.handy.selected_model:
+ out.append(f"== {p.name} ({p.pid}) Handy selected (settings, not load proof): {p.handy.selected_model}")
+ if p.handy.last_event:
+ out.append(f" last Handy event: {p.handy.last_event} {p.handy.last_model or ''} "
+ f"at {p.handy.last_event_local} (runtime-reported history, not current GPU residency)")
if not p.llm:
continue
out.append("")
diff --git a/tests/test_core.py b/tests/test_core.py
index 74131ab..cbda729 100644
--- a/tests/test_core.py
+++ b/tests/test_core.py
@@ -7,6 +7,7 @@ import os
import tempfile
import unittest
import xml.etree.ElementTree as ET
+from datetime import datetime, timezone
from pathlib import Path
from unittest.mock import patch
@@ -15,6 +16,7 @@ from gputaskman.cli import main
from gputaskman.collector import ProcRow, Snapshot, select_adapter, select_nv_device
from gputaskman.dxgi import Adapter
from gputaskman.gguf import GgufInfo, read_gguf
+from gputaskman.handy import HandyEvidence, inspect_handy, _log_cache, _settings_cache
from gputaskman.llm import (LlmView, ServerInfo, build_view, is_gpu_device, llama_server_models, lmstudio_models,
parse_flags, parse_load_log, probe_server)
from gputaskman.nvml import NvDevice
@@ -161,6 +163,74 @@ class MemoryTests(unittest.TestCase):
self.assertIn("Users", original.procs[0].llm.models[0].path)
+class HandyTests(unittest.TestCase):
+ def setUp(self):
+ _log_cache.clear()
+ _settings_cache.clear()
+
+ def test_selected_model_is_distinct_from_runtime_load_and_unload_events(self):
+ with tempfile.TemporaryDirectory() as directory:
+ appdata, localappdata = Path(directory) / "Roaming", Path(directory) / "Local"
+ settings_dir = appdata / "com.pais.handy"
+ log_dir = localappdata / "com.pais.handy" / "logs"
+ settings_dir.mkdir(parents=True)
+ log_dir.mkdir(parents=True)
+ selected = "handy-computer/nemotron/model-Q8_0.gguf"
+ (settings_dir / "settings_store.json").write_text(
+ json.dumps({"settings": {"selected_model": selected}}), encoding="utf-8"
+ )
+ (log_dir / "handy.log").write_text(
+ "[2026-11-02][22:59:00][transcription][INFO] Loaded whisper model 'old.gguf'\n"
+ "[2026-11-02][23:01:00][transcription][INFO] Loaded whisper model 'handy-computer/nemotron/model-Q8_0.gguf' (backend 'Vulkan0')\n"
+ "[2026-11-02][23:01:01][transcription][INFO] Live streaming transcription started (model 'handy-computer/nemotron/model-Q8_0.gguf', backend 'Vulkan0')\n"
+ "[2026-11-02][23:01:02][transcription][INFO] Model idle for 3606s (limit: 3600s), unloading\n"
+ "[2026-11-02][23:01:03][transcription][INFO] Model unloaded due to inactivity (took 20ms)\n"
+ "[2026-11-02][23:01:04][transcription][INFO] Partial transcription: private dictation text\n",
+ encoding="utf-8",
+ )
+ started = datetime(2026, 11, 2, 23, 0, tzinfo=timezone.utc).timestamp()
+ evidence = inspect_handy(started, str(appdata), str(localappdata))
+ self.assertEqual(evidence.selected_model, selected)
+ self.assertEqual(evidence.last_model, "handy-computer/nemotron/model-Q8_0.gguf")
+ self.assertEqual(evidence.last_event, "unloaded")
+ self.assertTrue(evidence.last_event_at.endswith("+00:00"))
+ local = datetime(2026, 11, 2, 23, 1, 3, tzinfo=timezone.utc).astimezone()
+ self.assertEqual(evidence.last_event_local, local.strftime("%Y-%m-%d %H:%M:%S"))
+ self.assertNotIn("private dictation text", repr(evidence))
+
+ def test_selected_model_without_fresh_log_is_not_reported_as_loaded(self):
+ with tempfile.TemporaryDirectory() as directory:
+ appdata, localappdata = Path(directory) / "Roaming", Path(directory) / "Local"
+ settings_dir = appdata / "com.pais.handy"
+ log_dir = localappdata / "com.pais.handy" / "logs"
+ settings_dir.mkdir(parents=True)
+ log_dir.mkdir(parents=True)
+ selected = "handy-computer/nemotron/model-Q8_0.gguf"
+ (settings_dir / "settings_store.json").write_text(
+ json.dumps({"settings": {"selected_model": selected}}), encoding="utf-8"
+ )
+ (log_dir / "handy.log").write_text(
+ "[2026-11-02][22:59:00][transcription][INFO] Loaded whisper model 'old.gguf'\n",
+ encoding="utf-8",
+ )
+ started = datetime(2026, 11, 2, 23, 0, tzinfo=timezone.utc).timestamp()
+ evidence = inspect_handy(started, str(appdata), str(localappdata))
+ self.assertEqual(evidence.selected_model, selected)
+ self.assertIsNone(evidence.last_model)
+ self.assertIsNone(evidence.last_event)
+
+ def test_handy_evidence_is_visible_in_text_and_json(self):
+ s = snapshot()
+ s.procs[0] = ProcRow(
+ 10, "handy.exe", "AI", ProcGpu(dedicated=GiB), ProcDetail(10, "handy.exe"),
+ handy=HandyEvidence("handy-computer/nemotron/model.gguf", "model.gguf", "loaded",
+ "2026-11-02T23:01:00+00:00"),
+ )
+ self.assertIn("Handy last log (loaded): model.gguf", s.procs[0].inside)
+ self.assertIn("not current GPU residency", to_text(s))
+ self.assertEqual(to_dict(s)["processes"][0]["handy"]["last_event"], "loaded")
+
+
class ModelTests(unittest.TestCase):
def test_diffusion_arguments_do_not_classify_text_encoder_as_llm(self):
with tempfile.TemporaryDirectory() as directory:
@@ -532,6 +602,24 @@ class UiTests(unittest.IsolatedAsyncioTestCase):
self.assertIsNone(app.hist_util[-1])
self.assertEqual(app.hist_bw[-1], 0)
+ async def test_handy_evidence_is_explicit_in_inspector(self):
+ with patch.object(GpuTaskManager, "poll"):
+ app = GpuTaskManager()
+ async with app.run_test(size=(180, 42)) as pilot:
+ s = snapshot()
+ s.procs[0] = ProcRow(
+ 10, "handy.exe", "AI", ProcGpu(dedicated=GiB), ProcDetail(10, "handy.exe"),
+ handy=HandyEvidence("handy-computer/nemotron/model.gguf", "model.gguf", "loaded",
+ "2026-11-02T23:01:00+00:00"),
+ )
+ app.apply(s)
+ await pilot.pause()
+ await pilot.press("down")
+ await pilot.pause()
+ text = " ".join(ET.fromstring(app.export_screenshot()).itertext()).replace("\u00a0", " ")
+ self.assertIn("Handy selected (settings; not load proof)", text)
+ self.assertIn("runtime-reported history, not current GPU residency", text)
+
async def test_likely_kind_is_explicit_in_process_and_inspector(self):
g = gguf("qwen35", block_count=2, context_length=4096, **{"general.name": "Snapshot_Cfg"})
with patch.object(GpuTaskManager, "poll"):