Skip to content

A GPU task manager for Windows, Linux, and WSL: per-process VRAM, GPU activity, and LLM model/KV-cache insights (Windows AMD and NVIDIA).

Python

Make Intel memory figures honest about what they are

Scott Hanselmanb496d851fad7
diff --git a/AGENTS.md b/AGENTS.md
index 3a9ad54..0dc492b 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -49,9 +49,9 @@ these rules:
```
cli.py ──► Collector.sample() (collector.py)
├─ adapters dxgi.py (Windows) / linux.enumerate_adapters / macos.enumerate_adapters
- ├─ counters pdh.py GpuCounters (Windows PDH) / linux.py LinuxCounters (NVML, + drm.py for amdgpu) /
+ ├─ counters pdh.py GpuCounters (Windows PDH) / linux.py LinuxCounters (NVML, + drm.py for amdgpu and Intel) /
│ macos.py MacCounters (IOKit Metal clients + footprint poller)
- ├─ device metrics nvml.py; drm.py read_device (AMD, Linux) (macOS: MacCounters.device from IOKit)
+ ├─ device metrics nvml.py; drm.py read_device (AMD and Intel, Linux) (macOS: MacCounters.device from IOKit)
├─ WSL host wslhost.py: HostStream (powershell PDH via interop), ContextTracker
├─ processes procinfo.py: runtime DLLs, mapped files, cmdline, classify()
├─ model naming stores.py (Ollama, LM Studio, Hugging Face), gguf.py (header, kind, sizes)
@@ -63,7 +63,9 @@ cli.py diff ──► diff.py: compares two saved --json snapshots (no collectio
The backend is chosen at import time in `collector.py`: `BACKEND` is `wddm` on Windows, `nvml` on
native Linux, `wsl` under WSL, and `metal` on macOS. On native Linux, `Collector.backend` becomes `drm`
-when the selected GPU is an amdgpu card (device readings from sysfs, processes from DRM fdinfo). One-shot
+when the selected GPU is an amdgpu or Intel i915 card (device readings from sysfs, processes from DRM
+fdinfo). An Intel card has no device-wide memory counter: its VRAM used is a sum over visible clients and
+its total is a PCI window size, so both carry `report.INTEL_MEMORY_NOTE`, and a total of 0 is unknown. One-shot
output (`--once`, `--json`) uses `scan_budget=None` to
identify everything immediately; the TUI spreads scans across refreshes.
@@ -136,5 +138,8 @@ workflow `release.yml`, and environment `pypi`. Only the isolated publishing job
Validated live: Windows 11 with an RTX 4080 SUPER (WDDM, plus WSL2), a Linux container with an
RTX 3060 Ti, macOS 26.3.1 on an M4 Mac mini (Metal; own-user processes, not run as root), and
bare-metal Linux with an amdgpu Radeon RX 9070-series card plus a Radeon 610M (DRM path).
-Not yet validated on hardware: bare-metal Linux with NVIDIA, multi-GPU systems, and GPU-backed WinML on
-Copilot+ PCs. Planned work is listed under **Next** in the README.
+Intel i915 (an Arc A770 with resizable BAR plus a UHD 730) was validated by a contributor, not repeated by
+the maintainer.
+Not yet validated on hardware: bare-metal Linux with NVIDIA, multi-GPU systems, Intel `xe` and Arc cards
+without resizable BAR, and GPU-backed WinML on Copilot+ PCs. Planned work is listed under **Next** in the
+README.
diff --git a/README.md b/README.md
index eb7e55f..d901e08 100644
--- a/README.md
+++ b/README.md
@@ -238,13 +238,15 @@ covered by unit tests only and **not yet validated on AMD hardware**.
| | Source |
|---|---|
| Device temperature, power, GT clock | `hwmon` and `gt/gt0` under `/sys/class/drm/card*`. Power is the rate of the cumulative energy counter between two refreshes, so it is `n/a` on the first |
-| Device VRAM total | Size of the local-memory BAR (`resource2`). A card whose window is under 1 GiB (an integrated GPU, or a discrete one without resizable BAR) is treated as integrated and has no total |
-| Device VRAM used | The sum of its clients' resident local memory: i915 has no device-wide counter, so buffers shared between processes can be counted twice and processes gputaskman can't inspect are missing (the UI and `--once` say so). `n/a` when any client's local memory is unknown |
-| Per-process VRAM and SPILL | `drm-resident-local0` and `drm-resident-system0`, plus their `stolen-` regions, under the same private-client rules as AMD. Both `drm-shared-local0` and `drm-shared-system0` must be present to charge a process, so an integrated GPU (no `local0`) shows `n/a` (`legacy`) |
+| Device VRAM total | The size of the local-memory PCI window (`resource2`), an **estimate**: kernel-owned memory is missing from the sysfs view, resizable BAR rounds to a power of two, and a partial window can be smaller than the card, so the total can differ from the card's memory. A window under 1 GiB (an integrated GPU, or a discrete card without resizable BAR) shows `N/A` (`null` in JSON). Free memory is never shown for Intel |
+| Device VRAM used | The sum of its clients' resident local memory: i915 has no device-wide counter, so buffers shared between processes can be counted twice, and kernel-owned memory and processes gputaskman can't inspect are missing. The UI, `--once`, `--json` (`memory_note`) and `gputaskman diff` all say so. `n/a` when any client's local memory is unknown |
+| Integrated or discrete | An iGPU has no `local0`, so a card with a local-memory window of 1 GiB or more is discrete, and so is any card whose fdinfo reports local memory, even when its window is smaller |
+| Per-process VRAM and SPILL | `drm-resident-local0` and `drm-resident-system0`, plus their `stolen-` regions (the firmware-reserved part of each), under the same private-client rules as AMD. `system0` is memory the GPU's buffers currently sit in, including buffers placed there on purpose, so SPILL here (like AMD's GTT) is where they are, not proof they were pushed out of VRAM. Both `drm-shared-local0` and `drm-shared-system0` must be present to charge a process, so an integrated GPU (no `local0`) shows `n/a` (`legacy`). A region value that can't be parsed makes that figure `n/a`, not a smaller number |
| Per-process engine load (3d, compute, copy, video) | `drm-engine-render`, `-compute`, `-copy`, `-video`, `-video-enhance` busy time, sampled twice. `n/a` where the driver reports none (before Gen 8). A class with several identical engines (`drm-engine-capacity-*`, such as video on an Arc) is not divided by its capacity, so one busy engine reads 100% |
Device GPU load, memory bandwidth, and the PCIe link are `n/a`: i915 has no device-wide busy counter, and its
-sysfs link fields can report the power-on default instead of the live link. The GT clock is read but not shown yet.
+sysfs link fields can report the power-on default instead of the live link. The GT clock is read but not shown yet, and
+what is read (`rps_cur_freq_mhz`) is the frequency the driver last *requested*; the kernel's actual frequency is `rps_act_freq_mhz`.
`xe` cards are detected, but its fdinfo uses other keys (`drm-cycles-*`, `drm-resident-vram0`/`-gtt`), so its
per-process rows show `n/a` until they are decoded on hardware.
diff --git a/src/gputaskman/app.py b/src/gputaskman/app.py
index 34da39b..4a91b3f 100644
--- a/src/gputaskman/app.py
+++ b/src/gputaskman/app.py
@@ -22,7 +22,7 @@ from .llm import gb
from .nvml import NvDevice, format_metric
from .recovery import LiveCollectors
from .report import (DRM_RESIDENT_NOTE, METAL_NOTE, WSL_EMPTY, drm_note, has_engine_counters, has_spill_counters, has_wddm_counters,
- inferred_note, intel_memory_note, vram_label, wsl_note)
+ inferred_note, intel_memory_note, vram_label, vram_total_label, wsl_note)
KIND_STYLE = {
"LLM": "bold magenta", "decision": "bold orchid", "decision?": "orchid",
@@ -94,8 +94,9 @@ opened through a render node, and no shared buffers. Otherwise they show `n/a` a
the inspector still shows the resident bytes it can reach (shared buffers included, so not added up). **Intel GPUs
on Linux** (`i915`) take the same path: sysfs for the device, fdinfo for engine load and, on a discrete Arc, local-memory
VRAM. An integrated GPU shows engine load only (VRAM `n/a`), `xe` cards are detected but their per-process fdinfo isn't
-decoded yet (rows show `n/a`), and with no device-wide memory counter the device's VRAM used is a sum over the processes
-it can inspect: shared buffers can be counted twice and uninspectable processes are missing. Only processes you can inspect are
+decoded yet (rows show `n/a`). With no device-wide memory counter, an Intel card's VRAM used is a sum over the processes
+it can inspect (shared buffers can be counted twice; kernel-owned memory and uninspectable processes are missing), its total is
+the size of the card's local-memory PCI window (an estimate), and free memory is not shown. Only processes you can inspect are
listed, and the header says how many it couldn't; run with `sudo` to see other users'. Under **WSL**, Windows
charges all Linux GPU memory to the VM (`vmwp.exe`), so VRAM and load are device-wide only; rows
are Linux processes with `/dev/dxg` open and show `n/a` rather than a made-up share. When WSL
@@ -428,7 +429,7 @@ class GpuTaskManager(App):
vram = Text("GPU mem " if s.backend == "metal" else "VRAM ")
vram.append_text(stacked_bar(parts, s.vram_total, width))
used_label = gb(s.vram_used) if s.vram_used is not None else "N/A"
- vram.append(f" {used_label} / {gb(s.vram_total)}", style="bold")
+ vram.append(f" {used_label} / {vram_total_label(s)}", style="bold")
legend = Text(" ")
for k, v in order[:7]:
legend.append("■ ", style=KIND_STYLE.get(k, "white"))
@@ -436,7 +437,7 @@ class GpuTaskManager(App):
if other > 32 << 20:
legend.append("■ ", style="grey35")
legend.append(f"unattributed {gb(other)} ")
- if s.vram_used is not None:
+ if s.vram_used is not None and s.vram_total and not intel_memory_note(s):
legend.append(f"free {gb(max(s.vram_total - s.vram_used, 0))}", style="grey62")
if s.shared_used is not None and s.shared_used > 64 << 20:
legend.append(f" shared(RAM) {gb(s.shared_used)}", style="yellow")
diff --git a/src/gputaskman/diff.py b/src/gputaskman/diff.py
index 0c2062f..aec01d6 100644
--- a/src/gputaskman/diff.py
+++ b/src/gputaskman/diff.py
@@ -155,6 +155,7 @@ def diff(before: dict, after: dict) -> dict:
taken_b, taken_a = before.get("taken"), after.get("taken")
seconds = round(taken_a - taken_b, 1) if isinstance(taken_b, (int, float)) and isinstance(taken_a, (int, float)) else None
return {"gpu": after.get("gpu"), "backend": after.get("backend"), "seconds": seconds,
+ "memory_note": after.get("memory_note") or before.get("memory_note"),
"device": device, "processes": procs}
@@ -209,6 +210,8 @@ def to_text(d: dict) -> str:
for key, label in DEVICE:
c = d["device"][key]
out.append(f"{label:<26}{_gb(c['before']):>11}{_gb(c['after']):>11}{_signed(c['delta']):>11}")
+ if d.get("memory_note"):
+ out.append(d["memory_note"])
procs = [p for p in d["processes"] if _changed(p)]
out += ["", f"{'PROCESSES (dedicated)':<22}{'PID':>14} {'MATCH':<11}{'BEFORE':>11}{'AFTER':>11}"
diff --git a/src/gputaskman/drm.py b/src/gputaskman/drm.py
index 2649812..e1e03ec 100644
--- a/src/gputaskman/drm.py
+++ b/src/gputaskman/drm.py
@@ -58,7 +58,8 @@ class DrmCard:
# so its busy time is video work of either kind, not encode.
unified_video: bool = False
# Intel (i915/xe): no device-wide memory sysfs exists. vram_total holds the local-memory (resizable
- # BAR) window for discrete cards, 0 for integrated ones; per-process bytes still come from fdinfo.
+ # BAR) window, an estimate of the card's memory; 0 means unknown (an integrated GPU, or a discrete card
+ # whose window is under 1 GiB). Per-process bytes still come from fdinfo.
intel: bool = False
# Intel only: used local memory, stamped in each DrmCounters.sample() as the sum of the card's
# clients' local bytes (shared buffers can double-count); None = no readable client, unknown.
@@ -116,13 +117,15 @@ def parse_fdinfo(text: str) -> dict | None:
return parse_size(fields.get(resident, fields.get(legacy)))
def intel_resident(*keys: str) -> int | None:
- """Resident bytes of an Intel logical region: the sum of its memory placements. A placement key
- that is absent counts as 0 (older kernels don't report stolen); all keys absent means the card
- has no such region (an iGPU has no local0) and returns None -- unknown, not 0."""
- values = [parse_size(fields.get(k)) for k in keys]
- if all(v is None for v in values):
+ """Resident bytes of an Intel memory type: the sum of its regions. i915 prints a line for every region the
+ device has, so an absent key has nothing to add and no key at all means the device has none (an iGPU has
+ no local0): None, unknown, not 0. A key that won't parse makes the whole figure unknown rather than a
+ smaller number."""
+ present = [k for k in keys if k in fields]
+ if not present:
return None
- return sum(v for v in values if v is not None)
+ values = [parse_size(fields[k]) for k in present]
+ return None if None in values else sum(values)
# drm-shared-<region> counts buffers with more than one handle by their *preferred* region, resident or not,
# while drm-resident-<region> counts by where buffers are now, so the two can't be subtracted. A shared
@@ -149,10 +152,11 @@ def parse_fdinfo(text: str) -> dict | None:
# xe reports drm-cycles-* instead and gen7 i915 nothing. No such line means the load is unknown, not 0
# (AGENTS.md rule 2).
engines = None
- # Intel's logical regions are local0 (device memory; the local/stolen pair covers memory that
- # shrunk out of local into system) and system0 (GPU-placed system memory, the GTT equivalent).
- # There is no separate CPU-only region: system0 feeds the shared/spill slot instead. These are
- # i915's names; xe calls them vram0/gtt/system, so its memory stays unknown until validated.
+ # i915's regions: local0 is device memory and system0 is system memory the GPU's buffers currently
+ # sit in (the GTT slot); stolen-local0 and stolen-system0 are the firmware-reserved parts of each.
+ # system0 includes buffers placed there on purpose, not only memory that spilled from local0.
+ # There is no separate CPU-only region. These are i915's names; xe calls them vram0/gtt/system, so
+ # its memory stays unknown until validated.
return {
"driver": fields["drm-driver"],
"client_id": client_id,
@@ -290,13 +294,17 @@ def cards(root: str = "/sys/class/drm", pci_ids: str | None = None) -> list[DrmC
if intel and not vram_total:
# i915/xe expose no VRAM sysfs. The resource2 file IS the local-memory BAR, so its size is
# the window (the file is binary -- its contents are not a number): >= 1 GiB marks a
- # discrete card's local VRAM, ~256 MB is an iGPU's system-memory window.
- try:
- bar = os.path.getsize(os.path.join(device, "resource2_wc"))
- if bar <= 0:
- bar = os.path.getsize(os.path.join(device, "resource2"))
- except OSError:
- bar = 0
+ # discrete card's local VRAM, ~256 MB is an iGPU's system-memory window. The window is an
+ # estimate of the card's memory, not a reading of it: resizable BAR rounds to a power of two and
+ # a partial BAR can be smaller than the card (report.INTEL_MEMORY_NOTE says so).
+ bar = 0
+ for bar_file in ("resource2_wc", "resource2"):
+ try:
+ bar = os.path.getsize(os.path.join(device, bar_file))
+ except OSError:
+ continue
+ if bar > 0:
+ break
vram_total = bar if bar >= _LOCAL_BAR_MIN else 0
found.append(DrmCard(
key=pci_key(pdev),
@@ -308,7 +316,8 @@ def cards(root: str = "/sys/class/drm", pci_ids: str | None = None) -> list[DrmC
name=_card_name(device, vendor_id, device_id, pci_ids),
vram_total=vram_total,
# amdgpu reports the VRAM chips' vendor only for a discrete card; an APU's "VRAM" is a RAM
- # carve-out. Intel reports no vendor node at all: a card with a local-memory BAR is discrete.
+ # carve-out. Intel reports no vendor node at all: a card with a local-memory BAR is discrete,
+ # and fdinfo showing local memory (see _stamp_intel_memory) overrides a small window.
integrated=(vram_total == 0) if intel else not os.path.exists(os.path.join(device, "mem_info_vram_vendor")),
unified_video=_vcn_major(device) >= 4,
intel=intel,
@@ -324,8 +333,8 @@ def adapters(found: list[DrmCard]) -> list[Adapter]:
luid=c.key,
name=c.name,
vendor_id=c.vendor_id,
- # Intel: the local-memory BAR for discrete cards, 0 for an integrated GPU (its carve-out size
- # is not exposed; totals stay unknown rather than invented).
+ # Intel: the local-memory BAR window (an estimate), 0 (unknown) when it is under 1 GiB: an
+ # integrated GPU's carve-out size is not exposed, and totals stay unknown rather than invented.
dedicated_vram=c.vram_total,
shared_system=_int(os.path.join(c.sysfs, "mem_info_gtt_total")) or 0 if not c.intel else 0,
software=False,
@@ -425,7 +434,9 @@ def read_device(card: DrmCard, index: int) -> DeviceTelemetry:
gt = os.path.join(os.path.dirname(d), "gt", "gt0")
cur = _int(os.path.join(gt, "rps_cur_freq_mhz"))
if cur is not None:
- dev.gfx_clock = cur # engine frequency from the gt node (hwmon has no sclk on Intel)
+ # rps_cur_freq_mhz is the frequency the driver last requested (rps_act_freq_mhz is the actual one,
+ # per the kernel's intel_gt_sysfs_pm.c); hwmon has no sclk on Intel. Not displayed yet.
+ dev.gfx_clock = cur
dev.sm_clock_max = _int(os.path.join(gt, "rps_RP0_freq_mhz"))
# sysfs current_link_* on i915/xe can report the power-on default instead of the live link
# (2.5 GT/s x1 shown while a measured DMA transfer ran at ~19 GB/s), so both stay unset:
@@ -575,6 +586,10 @@ class DrmCounters:
continue
used = [c[1]["vram"] for _key, c in clients.items() if c[1]["pdev"] == card.pdev]
card.vram_used = sum(used) if used and None not in used else None
+ if any(v is not None for v in used):
+ # A client reports local memory, so the card has some: discrete, whatever its BAR window says
+ # (a card without resizable BAR has a small window and no known total).
+ card.integrated = False
def _start_time(self, pid: int) -> str | None:
"""Field 22 of /proc/<pid>/stat: tells a reused pid from the process cached under it."""
diff --git a/src/gputaskman/report.py b/src/gputaskman/report.py
index 4820cd8..ed796a4 100644
--- a/src/gputaskman/report.py
+++ b/src/gputaskman/report.py
@@ -91,13 +91,21 @@ METAL_NOTE = ("macOS: unified memory, so GPU memory is part of system RAM. Rows
"memory needs root.")
INTEL_MEMORY_NOTE = ("Intel VRAM used is the sum of local memory resident in the processes gputaskman can inspect "
"(the driver has no device-wide counter): buffers shared between processes can be counted "
- "twice, and processes it can't inspect are missing.")
+ "twice, and kernel-owned memory and processes it can't inspect are missing. The total is the "
+ "size of the card's local-memory PCI window, an estimate that can differ from its memory, so "
+ "free memory is not shown.")
def intel_memory_note(s: Snapshot) -> str:
- """Intel's device VRAM used is a sum over visible DRM clients, not a device counter."""
+ """Intel's device VRAM used is a sum over visible DRM clients and its total a PCI window size: neither is a
+ device counter, so the output says so whenever it shows either."""
return INTEL_MEMORY_NOTE if (s.backend == "drm" and s.nv and s.nv.source == "Intel sysfs"
- and s.vram_used is not None) else ""
+ and (s.vram_used is not None or s.vram_total)) else ""
+
+
+def vram_total_label(s: Snapshot) -> str:
+ """A total of 0 means unknown (an Intel iGPU, or a card whose memory window can't be read), not 0 bytes."""
+ return gb(s.vram_total) if s.vram_total else "N/A"
def to_dict(s: Snapshot) -> dict:
@@ -147,10 +155,11 @@ def to_dict(s: Snapshot) -> dict:
return {
"taken": s.taken, "gpu": s.adapter.name if s.adapter else None, "driver": s.driver, "cuda": s.cuda,
"adapter": dataclasses.asdict(s.adapter) if s.adapter else None,
- "vram_total": s.vram_total, "vram_used": s.vram_used, "shared_used": s.shared_used,
+ "vram_total": s.vram_total or None, "vram_used": s.vram_used, "shared_used": s.shared_used,
"attributed": s.attributed, "backend": s.backend, "shared_is_spill": s.shared_is_spill,
"host_status": s.host_status or None,
"process_status": s.process_status or None,
+ "memory_note": intel_memory_note(s) or None,
"counter_status": s.counter_status or None, "counter_adapter_matched": s.counter_adapter_matched,
"wsl_split": s.wsl_split or None, "device": nv, "processes": procs,
}
@@ -174,7 +183,7 @@ def to_text(s: Snapshot, limit: int = 30) -> str:
out.append(f"{name}{info} {nv.driver_model if nv else ''}")
used = gb(s.vram_used) if s.vram_used is not None else "N/A"
shared = gb(s.shared_used) if s.shared_used is not None else "N/A"
- memory = f"VRAM {used} / {gb(s.vram_total)}"
+ memory = f"VRAM {used} / {vram_total_label(s)}"
if s.backend in ("wddm", "drm") or s.host_data:
memory += f" shared (in system RAM) {shared}"
out.append(memory)
diff --git a/tests/test_diff.py b/tests/test_diff.py
index b9dbabe..aa0a685 100644
--- a/tests/test_diff.py
+++ b/tests/test_diff.py
@@ -104,6 +104,18 @@ class DiffTests(unittest.TestCase):
with self.assertRaisesRegex(ValueError, "different GPUs"):
diff(snap([], 0, 0, gpu="A"), snap([], 0, 0, gpu="B"))
+ def test_memory_note_follows_the_snapshots_into_the_comparison(self):
+ note = "Intel VRAM used is a sum over the processes gputaskman can inspect."
+ noted = {**snap([], 3 * GiB, 0, taken=130.0, backend="drm", gpu="Arc"), "memory_note": note}
+ older = snap([], 2 * GiB, 0, backend="drm", gpu="Arc") # saved by a version that had no note
+ for before, after in ((older, noted), (noted, older), (noted, noted)):
+ d = diff(before, after)
+ self.assertEqual(d["memory_note"], note)
+ self.assertIn(note, to_text(d))
+ plain = diff(snap([], 2 * GiB, 0), snap([], 3 * GiB, 0)) # every other GPU: nothing to add
+ self.assertIsNone(plain["memory_note"])
+ self.assertNotIn(note, to_text(plain))
+
def test_remainder_segments_follow_the_noise_threshold(self):
def server(runtime):
view = {"n_ctx": 4096, "segments": [{"label": "rest", "bytes": runtime, "kind": "runtime", "exact": False}]}
diff --git a/tests/test_drm.py b/tests/test_drm.py
index 2abe286..9086362 100644
--- a/tests/test_drm.py
+++ b/tests/test_drm.py
@@ -169,6 +169,19 @@ def _make_card(root: Path, name, pdev, vendor, device, vram, driver="amdgpu", gt
return dev
+def _make_intel_card(root: Path, name, pdev, device, driver="i915") -> Path:
+ """An Intel card's sysfs: there are no mem_info_* files, the local-memory BAR is the resource2 file."""
+ dev = root / "drm" / name / "device"
+ dev.mkdir(parents=True)
+ (dev / "vendor").write_text("0x8086\n")
+ (dev / "device").write_text(f"0x{device:04x}\n")
+ driver_dir = root / "drivers" / driver
+ driver_dir.mkdir(parents=True, exist_ok=True)
+ os.symlink(driver_dir, dev / "driver")
+ (dev / "uevent").write_text(f"DRIVER={driver}\nPCI_SLOT_NAME={pdev}\n")
+ return dev
+
+
def _card(dev, pdev, name="AMD GPU", vram=16 * GiB, vendor=0x1002, device=0x7550) -> drm.DrmCard:
return drm.DrmCard(
key=drm.pci_key(pdev), pdev=pdev, sysfs=str(dev), driver="amdgpu",
@@ -1035,7 +1048,7 @@ class IntelFdinfoTests(unittest.TestCase):
def test_i915_discrete_client(self):
info = drm.parse_fdinfo(I915_FDINFO)
self.assertEqual((info["driver"], info["client_id"], info["pdev"]), ("i915", 42, "0000:03:00.0"))
- self.assertEqual(info["vram"], 2048 * MiB + 64 * 1024) # local0 plus the stolen-local0 it shrank into
+ self.assertEqual(info["vram"], 2048 * MiB + 64 * 1024) # local0 plus stolen-local0 (the fixture's 64 KiB is synthetic)
self.assertEqual(info["gtt"], 6 * MiB) # system0
self.assertEqual(info["shared"], 0) # both shared regions are reported, and empty
# Only the drm-engine-<class> counters are engines; the capacity lines are not.
@@ -1054,6 +1067,20 @@ class IntelFdinfoTests(unittest.TestCase):
self.assertIsNone(info["engines"]) # unknown, not an empty (zero) load
self.assertEqual(info["gtt"], 4 * MiB) # the memory is still read
+ def test_i915_unparseable_region_makes_that_figure_unknown(self):
+ # Treating a region that won't parse as 0 would show a smaller number than the process holds.
+ info = drm.parse_fdinfo(I915_FDINFO.replace("drm-resident-local0:\t2048 MiB", "drm-resident-local0:\tlots"))
+ self.assertIsNone(info["vram"]) # stolen-local0 still parses, but the local0 half is unknown
+ self.assertEqual(info["gtt"], 6 * MiB) # the other figure is unaffected
+ info = drm.parse_fdinfo(I915_FDINFO.replace("drm-resident-system0:\t6 MiB", "drm-resident-system0:\t?"))
+ self.assertIsNone(info["gtt"])
+ self.assertEqual(info["vram"], 2048 * MiB + 64 * 1024)
+
+ def test_i915_region_it_does_not_print_adds_nothing(self):
+ # i915 prints a line for every region the device has, so a missing stolen-local0 line is a region it lacks.
+ text = "\n".join(line for line in I915_FDINFO.splitlines() if "stolen-local0" not in line)
+ self.assertEqual(drm.parse_fdinfo(text)["vram"], 2048 * MiB)
+
def test_xe_client_reports_neither_memory_nor_load(self):
info = drm.parse_fdinfo(XE_FDINFO)
self.assertEqual((info["driver"], info["client_id"], info["pdev"]), ("xe", 3, "0000:03:00.0"))
@@ -1063,21 +1090,91 @@ class IntelFdinfoTests(unittest.TestCase):
self.assertIsNone(info[field])
+class IntelBarTests(unittest.TestCase):
+ """cards() takes an Intel card's total from the size of its local-memory BAR file. The sizes are faked: a real
+ BAR is GiBs of address space, not data."""
+
+ def _cards(self, bars: dict[str, int], driver="i915"):
+ real_getsize = os.path.getsize
+
+ def getsize(path):
+ name = os.path.basename(path)
+ if name not in ("resource2_wc", "resource2"):
+ return real_getsize(path)
+ if name not in bars:
+ raise FileNotFoundError(path)
+ return bars[name]
+
+ with tempfile.TemporaryDirectory() as tmp:
+ root = Path(tmp)
+ _make_intel_card(root, "card0", "0000:03:00.0", 0x56A0, driver)
+ with patch.object(os.path, "getsize", getsize):
+ return drm.cards(str(root / "drm"), str(root / "pci.ids"))
+
+ def test_resizable_bar_marks_a_discrete_card_and_sizes_it(self):
+ (card,) = self._cards({"resource2_wc": 16 * GiB})
+ self.assertTrue(card.intel)
+ self.assertEqual((card.vram_total, card.integrated), (16 * GiB, False))
+
+ def test_resource2_alone_is_enough(self):
+ (card,) = self._cards({"resource2": 8 * GiB}) # no write-combining alias to read
+ self.assertEqual((card.vram_total, card.integrated), (8 * GiB, False))
+
+ def test_empty_resource2_wc_falls_back_to_resource2(self):
+ (card,) = self._cards({"resource2_wc": 0, "resource2": 8 * GiB})
+ self.assertEqual(card.vram_total, 8 * GiB)
+
+ def test_small_or_missing_window_is_an_unknown_total_not_a_size(self):
+ (card,) = self._cards({"resource2_wc": 256 * MiB}) # an iGPU's system-memory window
+ self.assertEqual((card.vram_total, card.integrated), (0, True))
+ (card,) = self._cards({}) # no BAR file at all
+ self.assertEqual((card.vram_total, card.integrated), (0, True))
+
+ def test_xe_card_is_sized_the_same_way(self):
+ (card,) = self._cards({"resource2_wc": 16 * GiB}, driver="xe")
+ self.assertEqual((card.driver, card.intel, card.vram_total), ("xe", True, 16 * GiB))
+
+
class IntelScanTests(unittest.TestCase):
"""DrmCounters.sample() over Intel clients in a fake /proc. Each test builds its own card, because a sample
stamps the card's used memory."""
PDEV = "0000:03:00.0"
- def _setup(self, tmp, driver="i915"):
+ def _setup(self, tmp, driver="i915", vram_total=16 * GiB, integrated=False):
proc = Path(tmp) / "proc"
proc.mkdir()
card = drm.DrmCard(key=drm.pci_key(self.PDEV), pdev=self.PDEV, sysfs=str(Path(tmp) / "card1" / "device"),
driver=driver, vendor_id=0x8086, device_id=0x56A0, name="Intel Arc A770",
- vram_total=16 * GiB, integrated=False, intel=True)
+ vram_total=vram_total, integrated=integrated, intel=True)
self.clock = Clock()
return card, proc, drm.DrmCounters(found=[card], proc=str(proc), clock=self.clock)
+ def test_local_memory_in_fdinfo_makes_a_card_with_a_small_window_discrete(self):
+ # An Arc without resizable BAR has a window under 1 GiB, so cards() calls it integrated; only a card with
+ # local memory prints local0, and that is what shows it is discrete.
+ with tempfile.TemporaryDirectory() as tmp:
+ card, proc, counters = self._setup(tmp, vram_total=0, integrated=True)
+ counters.sample()
+ self.assertTrue(card.integrated) # no client yet: nothing shows otherwise
+ _write_fd(proc, 100, 5, _i915_fdinfo(1, self.PDEV, local_mib=64))
+ counters.sample()
+ self.assertFalse(card.integrated)
+ self.assertEqual(card.vram_total, 0) # and its total is still unknown, not invented
+
+ def test_integrated_gpu_stays_integrated_when_its_clients_print_no_local_memory(self):
+ pdev = "0000:00:02.0"
+ with tempfile.TemporaryDirectory() as tmp:
+ proc = Path(tmp) / "proc"
+ proc.mkdir()
+ card = drm.DrmCard(key=drm.pci_key(pdev), pdev=pdev, sysfs=str(Path(tmp) / "card0" / "device"),
+ driver="i915", vendor_id=0x8086, device_id=0xA782, name="Intel UHD Graphics",
+ vram_total=0, integrated=True, intel=True)
+ counters = drm.DrmCounters(found=[card], proc=str(proc), clock=Clock())
+ _write_fd(proc, 100, 5, I915_IGPU_FDINFO)
+ counters.sample()
+ self.assertTrue(card.integrated)
+
def test_render_and_video_load_are_measured_between_two_samples(self):
with tempfile.TemporaryDirectory() as tmp:
card, proc, counters = self._setup(tmp)
@@ -1155,33 +1252,64 @@ class IntelScanTests(unittest.TestCase):
class IntelNoteTests(unittest.TestCase):
- """Intel's device VRAM-used is a sum over the clients gputaskman can inspect, and the output says so."""
+ """Intel's device VRAM used is a sum over the clients gputaskman can inspect and its total is a PCI window size,
+ and the output says so."""
- def _snapshot(self, source, vram_used=4 * GiB):
+ def _snapshot(self, source, vram_used=4 * GiB, vram_total=16 * GiB):
from gputaskman.collector import Snapshot
from gputaskman.telemetry import DeviceTelemetry
return Snapshot(taken=1, adapter=None, adapters=[], nv=DeviceTelemetry(index=0, source=source), driver="",
- cuda="", vram_total=16 * GiB, vram_used=vram_used, shared_used=None, backend="drm",
+ cuda="", vram_total=vram_total, vram_used=vram_used, shared_used=None, backend="drm",
procs=[])
+ def _panel(self, snapshot):
+ from gputaskman.app import GpuTaskManager
+
+ return "\n".join(item.plain for item in GpuTaskManager().render_device(snapshot).renderables)
+
def test_text_carries_the_note_for_intel_only(self):
from gputaskman.report import INTEL_MEMORY_NOTE, to_text
self.assertIn(INTEL_MEMORY_NOTE, to_text(self._snapshot("Intel sysfs")))
self.assertNotIn(INTEL_MEMORY_NOTE, to_text(self._snapshot("AMD sysfs"))) # amdgpu has a device counter
- self.assertNotIn(INTEL_MEMORY_NOTE, to_text(self._snapshot("Intel sysfs", vram_used=None))) # N/A, no caveat
+ # Nothing derived from the client sum or the window is shown (N/A used, unknown total): no caveat to make.
+ self.assertNotIn(INTEL_MEMORY_NOTE, to_text(self._snapshot("Intel sysfs", vram_used=None, vram_total=0)))
def test_device_panel_carries_the_note_for_intel_only(self):
- from gputaskman.app import GpuTaskManager
from gputaskman.report import INTEL_MEMORY_NOTE
- def panel(snapshot):
- return "\n".join(item.plain for item in GpuTaskManager().render_device(snapshot).renderables)
-
- self.assertIn(INTEL_MEMORY_NOTE, panel(self._snapshot("Intel sysfs")))
- self.assertNotIn(INTEL_MEMORY_NOTE, panel(self._snapshot("AMD sysfs")))
- self.assertNotIn(INTEL_MEMORY_NOTE, panel(self._snapshot("Intel sysfs", vram_used=None)))
+ self.assertIn(INTEL_MEMORY_NOTE, self._panel(self._snapshot("Intel sysfs")))
+ self.assertNotIn(INTEL_MEMORY_NOTE, self._panel(self._snapshot("AMD sysfs")))
+ self.assertNotIn(INTEL_MEMORY_NOTE, self._panel(self._snapshot("Intel sysfs", vram_used=None, vram_total=0)))
+
+ def test_unknown_total_reads_n_a_and_free_is_left_out(self):
+ from gputaskman.report import to_text
+
+ snapshot = self._snapshot("Intel sysfs", vram_used=3 * GiB, vram_total=0)
+ self.assertIn("VRAM 3.00 GB / N/A", to_text(snapshot))
+ panel = self._panel(snapshot)
+ self.assertIn("3.00 GB / N/A", panel)
+ self.assertNotIn("/ 0 MB", panel)
+ self.assertNotRegex(panel, r"free \d") # a free figure needs a total (the note's own "free memory" wording is not one)
+
+ def test_window_total_is_not_turned_into_a_free_figure(self):
+ panel = self._panel(self._snapshot("Intel sysfs")) # 4 GB used of a 16 GB window
+ self.assertIn("4.00 GB / 16.00 GB", panel)
+ self.assertNotRegex(panel, r"free \d")
+ self.assertIn("free 12.00 GB", self._panel(self._snapshot("AMD sysfs"))) # AMD's total is a device counter
+
+ def test_json_says_unknown_and_carries_the_note(self):
+ from gputaskman.report import INTEL_MEMORY_NOTE, to_dict
+
+ unknown = to_dict(self._snapshot("Intel sysfs", vram_used=3 * GiB, vram_total=0))
+ self.assertIsNone(unknown["vram_total"]) # unknown, not 0 bytes
+ self.assertEqual(unknown["vram_used"], 3 * GiB)
+ self.assertEqual(unknown["memory_note"], INTEL_MEMORY_NOTE)
+ window = to_dict(self._snapshot("Intel sysfs"))
+ self.assertEqual((window["vram_total"], window["memory_note"]), (16 * GiB, INTEL_MEMORY_NOTE))
+ amd = to_dict(self._snapshot("AMD sysfs"))
+ self.assertEqual((amd["vram_total"], amd["memory_note"]), (16 * GiB, None))
if __name__ == "__main__":