Skip to content

A GPU task manager for Windows, Linux, and WSL: per-process VRAM, GPU activity, and LLM model/KV-cache insights (Windows AMD and NVIDIA).

Python

Identify vLLM model IDs through conservative local API discovery

Scott Hanselmand1d871181177
diff --git a/README.md b/README.md
index 39ff1b1..d36b8b7 100644
--- a/README.md
+++ b/README.md
@@ -752,6 +752,7 @@ reading is NVML's memory-activity percentage, **not measured bandwidth in GB/s**
| llama.cpp / llama-server | GGUF discovery, CLI settings, buffer logs, `/props` + `/v1/models` state; exercised live |
| Ollama | Bundled runners, mapped GGUF blobs named from Ollama manifests (`qwen3:14b`), parent `/api/ps`, compatible `server.log`; exercised live |
| LM Studio | Mapped files named from its models folder, `/api/v1/models` (falls back to `/api/v0/models`); tested with fixture responses, not yet end to end |
+| vLLM | Recognized launch/worker signatures; `/v1/models` identifiers from an own listener or a verified parent, labelled API-reported, never worker residency or memory placement. Fixture-tested, including a real lightweight child process and HTTP listener; live Linux single-GPU and TP hardware **not validated** |
| stable-diffusion.cpp (`--diffusion-model`) | Diffusion, text-encoder and VAE files from launch arguments; no LLM memory breakdown |
| Hugging Face cache | `models--org--repo` paths are named `org/repo` |
| Other AI runtimes | DLL detection, plus model files installed with the app (marked *not proven loaded*) |
@@ -761,9 +762,23 @@ reading is NVML's memory-activity percentage, **not measured bandwidth in GB/s**
WinML can select different execution backends; a local helper/service is still **local**, not
cloud-hosted. The GPU counters belong to whichever process holds/uses the GPU resources.
-Parent-PID lookup currently helps find Ollama/LM Studio APIs; it is not a general cross-process
+Parent-PID lookup currently helps find Ollama/LM Studio and local vLLM APIs; it is not a general cross-process
ownership graph for shared Windows services.
+For vLLM, `vllm serve`, `python -m vllm.entrypoints.openai.api_server`, `VLLM::EngineCore`
+and `VLLM::Worker_TP<n>` are detection hints, not loading evidence. An inherited launch command
+also works without process titles. Worker discovery checks at most two live ancestors, their
+creation times and their signatures. Only one exclusively owned listening port on one recognized
+server is accepted; shared/multiple listeners, unrelated or inaccessible ancestry and explicit
+distributed/data-parallel options give no association. There is no port scanning or matching by
+model name. Processes outside the visible PID namespace are not linked to local servers.
+
+The table, inspector and text label these identifiers as API-reported, with **parent process**
+when applicable. JSON has `llm.server_source` (`"process"`, `"parent"`, or `null`) alongside
+`llm.server.models[].name`. A reported ID is not a model file or evidence that a worker holds
+that model; it adds no weights, KV-cache, context-size or GPU-placement estimate. Measured memory
+and attributed totals are unchanged.
+
GPU-backed WinML and Studio Effects paths have **not been validated on a Copilot+ PC**. A future
hardware check should compare idle versus active workload samples, record the actual GPU adapter
and resource-owning PID (including helpers), and verify the execution provider using runtime
@@ -833,6 +848,7 @@ Identification is designed to stay cheap. Measured on one Windows machine:
| Memory-map scan (DLLs + mapped files) | 1-50 ms per process | New processes, a 64 MiB / 10% memory change, or every 2 min; at most 150 ms per refresh |
| GGUF header, model-store manifests | ms, cached | Once per file / every 60 s |
| Local API probes | ms, local only | llama-server `/v1/models` only when its model path changes |
+| vLLM `/v1/models` | 1.32 ms median, 21.36 ms max in 30 loopback fixture GETs; cached lookup ~0.0004 ms (Windows 11 build 28000, ARM64; no vLLM runtime/GPU workload) | One GET, 0.5 s timeout and 4 MiB response cap; success cached 3 s, misses/401 cached 30 s. No redirects; actual Linux/vLLM latency unmeasured |
| `psutil.open_files()` | 0.6-2.5 s per process | Never |
| AMD and Intel on Linux: DRM fdinfo scan (`readlink` every fd, read fdinfo of `/dev/dri` ones) | ~9 ms for ~8,000 fds (AMD, Linux desktop; Intel not measured) | Every refresh, only when an amdgpu or Intel card is shown; listing fds is bounded to 50 ms in the TUI and resumes next refresh, known GPU fds are still read |
| macOS: `ioreg` device stats + Metal clients | ~10 ms | Every refresh |
@@ -867,6 +883,12 @@ gputaskman does not open a path taken from another process's command line, and a
can't be seen from outside the process. In that case the probe stops at the first 401, so the server logs
one rejected request every 30 seconds, and the inspector says why server details are missing.
+vLLM uses only `/v1/models`, with the same request bounds and cache intervals. Parent-discovered
+ports are **always unauthenticated**, even when parent and worker launch arguments match:
+the worker's key is never sent there. A 401 is shown as **Parent-process API locked**.
+Probe caches separate own/parent origins, process creation times, runtime and credentials, so
+an authenticated own-port result cannot silently satisfy a parent request.
+
Ollama's local `server.log` is considered when one Ollama runner is detected. Log formats,
rotation, and stale load blocks can affect attribution, so compare the reported model and
process when interpreting logs. Log sizes describe allocations at load time, not necessarily
diff --git a/src/gputaskman/app.py b/src/gputaskman/app.py
index 5bf00d8..e73ae80 100644
--- a/src/gputaskman/app.py
+++ b/src/gputaskman/app.py
@@ -857,6 +857,8 @@ class GpuTaskManager(App):
if v.server:
sv = v.server
st = Text.assemble(("server ", "bold"), (f"{sv.kind} {sv.url}", "cyan"))
+ if v.server_source == "parent":
+ st.append(" (parent process; API report, not worker residency)")
if sv.n_ctx:
st.append(f" ctx {sv.n_ctx:,}")
if sv.slots_total is not None:
@@ -868,6 +870,8 @@ class GpuTaskManager(App):
for m in sv.models[:4]:
if sv.kind == "lmstudio":
st.append(f"\n loaded: {m['name']} {m.get('type') or ''} {m.get('quant') or ''} ctx {m.get('ctx') or '?'}")
+ elif sv.kind == "vllm":
+ st.append(f"\n API-reported model ID: {m['name']}")
out.append(st)
for n in v.notes:
out.append(Text("• " + n, style="bold red" if n.startswith("!") else "grey70"))
diff --git a/src/gputaskman/collector.py b/src/gputaskman/collector.py
index 38d4c20..9fc6cdf 100644
--- a/src/gputaskman/collector.py
+++ b/src/gputaskman/collector.py
@@ -79,6 +79,9 @@ class ProcRow:
return f"Handy last log ({self.handy.last_event}): {self.handy.last_model.rsplit('/', 1)[-1]}"
if self.handy.selected_model:
return f"Handy selected (not load proof): {self.handy.selected_model.rsplit('/', 1)[-1]}"
+ if self.llm and self.llm.server and self.llm.server.kind == "vllm" and self.llm.server.models:
+ source = "parent API" if self.llm.server_source == "parent" else "API"
+ return f"reported ({source}): " + ", ".join(m["name"] for m in self.llm.server.models)
if self.llm and self.llm.models:
parts = []
for i, g in enumerate(self.llm.models):
@@ -246,8 +249,41 @@ class Collector:
self._own_start = time.time() # our NVML context is created now, not when the process started
self._host_restarted = time.time()
- def _parent_ports(self, pid: int, depth: int = 2) -> list[int]:
+ def _parent_ports(self, pid: int, depth: int = 2, *, vllm: bool = False) -> list[int]:
d = self.inspector.get(pid)
+ if vllm:
+ d.parents = []
+ candidates = []
+ try:
+ p = psutil.Process(pid)
+ if not d.create_time or p.create_time() != d.create_time or d.vllm_ambiguous:
+ return []
+ child_start = d.create_time
+ seen = {pid}
+ for _ in range(depth):
+ p = p.parent()
+ if p is None:
+ break
+ parent = self.inspector.get(p.pid)
+ if p.pid in seen or not parent.create_time or parent.create_time != p.create_time():
+ return []
+ if parent.create_time > child_start:
+ return []
+ if not parent.vllm_role:
+ break
+ if parent.vllm_ambiguous:
+ return []
+ seen.add(p.pid)
+ d.parents.append(p.pid)
+ child_start = parent.create_time
+ ports = self.inspector.exclusive_ports(p.pid)
+ if parent.ports and set(parent.ports) != set(ports):
+ return [] # a shared listener cannot identify a server
+ if parent.vllm_role == "server":
+ candidates.extend(ports)
+ return candidates if len(candidates) == 1 else []
+ except (psutil.Error, OSError):
+ return []
if d.parents is None:
d.parents = []
try:
@@ -349,20 +385,35 @@ class Collector:
continue
ports = self.inspector.exclusive_ports(row.pid)
key = api_key(row.detail.cmdline)
- own = self.prober.get(ports, key)
- host = self.prober.get(self._parent_ports(row.pid)) # Ollama / LM Studio front-end process
+ vllm = bool(row.detail.vllm_role)
+ kind = "vllm" if vllm else None
+ own_origin = ("process", row.pid, row.detail.create_time)
+ if vllm and (row.detail.vllm_ambiguous or len(ports) > 1 or
+ set(ports) != set(row.detail.ports)):
+ ports = []
+ parent_ports = []
+ else:
+ parent_ports = self._parent_ports(row.pid, vllm=vllm)
+ parent_origin = ("parent", tuple((pid, self.inspector.get(pid).create_time)
+ for pid in row.detail.parents or []))
+ own = self.prober.get(ports, key, kind=kind, origin=own_origin)
+ host = self.prober.get(parent_ports, kind=kind, origin=parent_origin)
log = ollama_log_path() if row in ollama_runners and len(ollama_runners) == 1 else None
# On an APU, GTT is the GPU's ordinary memory (often most of it), not spill: count it as held memory.
held = (row.gpu.shown_vram or 0) + (0 if snap.shared_is_spill else (row.gpu.shared or 0))
row.llm = build_view(row.detail, held, row.gpu.shared if snap.shared_is_spill else 0,
own or host, log, host=host if own else None,
- attributed=row.gpu.shown_vram is not None)
- locked = None if own else self.prober.locked_port(ports)
+ attributed=row.gpu.shown_vram is not None,
+ server_source="process" if own else "parent" if host else None)
+ locked = None if own else self.prober.locked_port(ports, key, kind=kind, origin=own_origin)
if locked:
why = ("rejected the key from the launch arguments" if key else
"wants an API key, and none was found in the launch arguments (keys from LLAMA_API_KEY "
"or --api-key-file are not read)")
row.llm.notes.append(f"Local API on port {locked} {why}. Server details are skipped.")
+ if not host and self.prober.locked_port(parent_ports, kind=kind, origin=parent_origin):
+ row.llm.notes.append("Parent-process API locked (401). Parent probes never send the worker's "
+ "API key; parent server details are skipped.")
self.inspector.prune(set(procs))
return snap
diff --git a/src/gputaskman/llm.py b/src/gputaskman/llm.py
index 0ef2a07..e14f4d4 100644
--- a/src/gputaskman/llm.py
+++ b/src/gputaskman/llm.py
@@ -11,6 +11,7 @@ The process's real VRAM total always comes from Windows, so the remainder is nev
from __future__ import annotations
import dataclasses
+import hashlib
import json
import os
import re
@@ -169,12 +170,12 @@ def redact_cmdline(cmd: list[str]) -> list[str]:
return out
-def _get(url: str, timeout: float = 0.5, key: str | None = None):
+def _get(url: str, timeout: float = 0.5, key: str | None = None, redirects: bool = True):
headers = {"Accept": "application/json"}
if key:
headers["Authorization"] = f"Bearer {key}"
try:
- with (_keyed_opener if key else _opener).open(urllib.request.Request(url, headers=headers),
+ with (_keyed_opener if key or not redirects else _opener).open(urllib.request.Request(url, headers=headers),
timeout=timeout) as r:
if r.status != 200:
return None
@@ -191,7 +192,7 @@ def _get(url: str, timeout: float = 0.5, key: str | None = None):
@dataclass
class ServerInfo:
- kind: str # llama-server | ollama | lmstudio
+ kind: str # llama-server | ollama | lmstudio | vllm
url: str
n_ctx: int | None = None
slots_total: int | None = None
@@ -251,13 +252,30 @@ def lmstudio_models(v1: dict | None, v0: dict | None) -> list[dict] | None:
return None
-def probe_server(port: int, prev: ServerInfo | None = None, key: str | None = None) -> ServerInfo | None:
+def vllm_models(response) -> list[dict] | None:
+ """Model identifiers only: the OpenAI list says nothing about bytes or worker placement."""
+ if not isinstance(response, dict) or not isinstance(response.get("data"), list):
+ return None
+ entries = response["data"]
+ if any(not isinstance(m, dict) or not isinstance(m.get("id"), str) or not m["id"].strip() for m in entries):
+ return None
+ return [_model_entry(m["id"]) for m in entries]
+
+
+def probe_server(port: int, prev: ServerInfo | None = None, key: str | None = None,
+ kind: str | None = None) -> ServerInfo | None:
"""Read-only GETs. Never calls inference, load or download endpoints.
Returns UNAUTHORIZED when the port answers 401 and no key (or a wrong one) was sent. Probing stops there,
so a locked server logs one rejected request per probe instead of one per endpoint.
"""
base = f"http://127.0.0.1:{port}"
+ if kind == "vllm":
+ response = _get(base + "/v1/models", key=key, redirects=False)
+ if response is UNAUTHORIZED:
+ return UNAUTHORIZED
+ models = vllm_models(response)
+ return ServerInfo("vllm", base, models=models) if models is not None else None
props = _get(base + "/props", key=key)
if props is UNAUTHORIZED:
return UNAUTHORIZED
@@ -298,33 +316,44 @@ class ServerProber:
TTL_HIT, TTL_MISS = 3.0, 30.0
def __init__(self) -> None:
- self._cache: dict[int, tuple[float, ServerInfo | None]] = {}
- self._last: dict[int, ServerInfo] = {}
+ self._cache: dict[tuple, tuple[float, ServerInfo | None]] = {}
+ self._last: dict[tuple, ServerInfo] = {}
+ self._locked: set[tuple] = set()
self.locked: set[int] = set() # ports whose last probe was refused with 401
- def get(self, ports: list[int], key: str | None = None) -> ServerInfo | None:
+ @staticmethod
+ def _identity(port: int, key: str | None, kind: str | None, origin: object) -> tuple:
+ # Never retain the key itself; authenticated own-port results cannot satisfy a parent probe.
+ return port, kind, origin, hashlib.sha256(key.encode()).digest() if key else None
+
+ def get(self, ports: list[int], key: str | None = None, *, kind: str | None = None,
+ origin: object = "process") -> ServerInfo | None:
now = time.time()
for port in ports:
- hit = self._cache.get(port)
+ identity = self._identity(port, key, kind, origin)
+ hit = self._cache.get(identity)
if hit and now - hit[0] < (self.TTL_HIT if hit[1] else self.TTL_MISS):
info = hit[1]
else:
- info = probe_server(port, self._last.get(port), key)
+ info = probe_server(port, self._last.get(identity), key, kind=kind)
if info is UNAUTHORIZED:
+ self._locked.add(identity)
self.locked.add(port)
info = None
else:
+ self._locked.discard(identity)
self.locked.discard(port)
- self._cache[port] = (now, info)
+ self._cache[identity] = (now, info)
if info:
- self._last[port] = info
+ self._last[identity] = info
if info:
return info
return None
- def locked_port(self, ports: list[int]) -> int | None:
+ def locked_port(self, ports: list[int], key: str | None = None, *, kind: str | None = None,
+ origin: object = "process") -> int | None:
"""First of these ports whose last probe was refused for a missing or wrong API key."""
- return next((p for p in ports if p in self.locked), None)
+ return next((p for p in ports if self._identity(p, key, kind, origin) in self._locked), None)
# ---------------------------------------------------------------- the breakdown
@@ -346,6 +375,7 @@ class LlmView:
flags: dict = field(default_factory=dict)
server: ServerInfo | None = None
host: ServerInfo | None = None # managing front-end (Ollama / LM Studio) when the runner has its own server
+ server_source: str | None = None # process | parent; an API report, not proof of worker residency
log_source: str | None = None
log: dict | None = None
segments: list[Segment] = field(default_factory=list)
@@ -406,8 +436,16 @@ def _merge_api(files: list[MappedModel], server: ServerInfo | None, host: Server
def build_view(d: ProcDetail, vram: float, shared: float, server: ServerInfo | None,
- log_path: str | None = None, host: ServerInfo | None = None, attributed: bool = True) -> LlmView:
- v = LlmView(flags=parse_flags(d.cmdline), server=server, host=host)
+ log_path: str | None = None, host: ServerInfo | None = None, attributed: bool = True,
+ server_source: str | None = None) -> LlmView:
+ v = LlmView(flags=parse_flags(d.cmdline), server=server, host=host,
+ server_source=(server_source or "process") if server else None)
+ if server and server.kind == "vllm":
+ source = "parent process" if v.server_source == "parent" else "process's own listener"
+ v.notes.append(f"vLLM API reported via {source}; model IDs do not prove this process's "
+ "model residency, weights, KV cache or GPU placement.")
+ elif d.vllm_role and not server:
+ v.notes.append("vLLM process signature only; no unambiguous local API association was established.")
files = [dataclasses.replace(m, sources=list(m.sources)) for m in d.models] # cached details stay untouched
matched = _merge_api(files, server, host)
for m in files:
diff --git a/src/gputaskman/procinfo.py b/src/gputaskman/procinfo.py
index 53542e6..3d83fba 100644
--- a/src/gputaskman/procinfo.py
+++ b/src/gputaskman/procinfo.py
@@ -69,6 +69,7 @@ MODEL_KINDS = {"LLM": "LLM", "decision model": "decision", "embedding model": "e
"reranker": "embedding", "speech model": "speech"}
AI_KINDS = {"LLM", "decision", "decision?", "embedding", "speech", "diffusion", "AI"}
_DECISION_NAME = re.compile(r"(?<![a-z0-9])(?:openjev|laya|julia-1|lev|kev)(?![a-z0-9])", re.I)
+_VLLM_WORKER = re.compile(r"VLLM::(?:EngineCore|Worker_TP[0-9]+)")
MiB = 1 << 20
INSTALLED_TTL = 300.0
_SKIP_DIRS = {"ebwebview", "cache", "code cache", "gpucache", "node_modules", "logs", "crashpad", "locales", "__pycache__"}
@@ -133,6 +134,33 @@ class ProcDetail:
def is_diffusion(self) -> bool:
return _is_diffusion_cmdline(self.cmdline)
+ @property
+ def vllm_role(self) -> str:
+ """A process signature, not evidence that a model is resident in this process."""
+ if any(_VLLM_WORKER.fullmatch(s) for s in (self.name, *self.cmdline[:1])):
+ return "worker"
+ cmd = self.cmdline
+ names = [arg.replace("\\", "/").rsplit("/", 1)[-1] for arg in cmd[:2]]
+ if len(cmd) >= 2 and names[0] in ("vllm", "vllm.exe") and cmd[1] == "serve":
+ return "server"
+ if names and re.fullmatch(r"python(?:[0-9]+(?:\.[0-9]+)*)?(?:\.exe)?", names[0]):
+ if len(cmd) >= 3 and names[1] in ("vllm", "vllm.exe") and cmd[2] == "serve":
+ return "server"
+ args = cmd[1:]
+ while args and args[0] in ("-u", "-B", "-E", "-s", "-S", "-I", "-O", "-OO"):
+ args = args[1:]
+ if args[:2] == ["-m", "vllm.entrypoints.openai.api_server"]:
+ return "server"
+ return ""
+
+ @property
+ def vllm_ambiguous(self) -> bool:
+ # Distributed/data-parallel layouts need an ownership graph, not just two ancestors.
+ return any(arg.split("=", 1)[0].startswith("--data-parallel") or
+ arg.split("=", 1)[0] in ("--distributed-executor-backend", "--nnodes", "--node-rank",
+ "--headless", "--api-server-count")
+ for arg in self.cmdline)
+
def _lib_name(base: str) -> str | None:
"""Lower-cased library file name with any Linux version suffix removed ('libcudart.so.12.4' ->
@@ -447,7 +475,7 @@ def classify(d: ProcDetail, engines: dict[str, float]) -> str:
return k
if any(m.kind_hint == "decision model" for m in d.models):
return "decision?"
- if "llama.cpp" in d.runtimes or n.startswith(LLM_NAMES):
+ if d.vllm_role or "llama.cpp" in d.runtimes or n.startswith(LLM_NAMES):
return "LLM"
video = engines.get("decode", 0) + engines.get("encode", 0) + engines.get("video", 0) > 1
if d.is_ai and not video:
diff --git a/src/gputaskman/report.py b/src/gputaskman/report.py
index a123e9f..8680b20 100644
--- a/src/gputaskman/report.py
+++ b/src/gputaskman/report.py
@@ -151,6 +151,7 @@ def to_dict(s: Snapshot) -> dict:
"segments": [dataclasses.asdict(x) for x in p.llm.segments],
"kv_bytes_per_token": p.llm.kv_per_token, "n_ctx": p.llm.n_ctx, "flags": p.llm.flags,
"server": dataclasses.asdict(p.llm.server) if p.llm.server else None,
+ "server_source": p.llm.server_source,
"log_source": p.llm.log_source, "notes": p.llm.notes,
}
procs.append(item)
@@ -270,6 +271,11 @@ def to_text(s: Snapshot, limit: int = 30) -> str:
sv = p.llm.server
out.append(f" server: {sv.kind} {sv.url} ctx={sv.n_ctx} slots={sv.slots_busy}/{sv.slots_total}"
+ (f" gen={sv.gen_tok_s:.1f} tok/s" if sv.gen_tok_s is not None else ""))
+ if p.llm.server_source == "parent":
+ out.append(" server source: parent process (API report, not worker residency)")
+ if sv.kind == "vllm":
+ for m in sv.models:
+ out.append(f" API-reported model ID: {m['name']}")
if p.llm.log_source:
out.append(f" exact sizes from log: {p.llm.log_source}")
for n in p.llm.notes:
diff --git a/tests/test_vllm.py b/tests/test_vllm.py
new file mode 100644
index 0000000..0f55377
--- /dev/null
+++ b/tests/test_vllm.py
@@ -0,0 +1,314 @@
+from __future__ import annotations
+
+import contextlib
+import http.server
+import json
+import os
+import subprocess
+import sys
+import threading
+import time
+import unittest
+from types import SimpleNamespace
+from unittest.mock import Mock, patch
+
+import psutil
+from textual.widgets import Static
+
+from gputaskman.app import GpuTaskManager
+from gputaskman.collector import Collector
+from gputaskman.llm import ServerProber, probe_server, vllm_models
+from gputaskman.pdh import AdapterGpu, ProcGpu
+from gputaskman.procinfo import ProcDetail, ProcInspector, classify
+from gputaskman.report import to_dict, to_text
+
+MiB = 1 << 20
+LAUNCH = ["vllm", "serve", "org/model"]
+
+
+def listener(*, locked=False, redirect=False):
+ seen = []
+
+ class Handler(http.server.BaseHTTPRequestHandler):
+ protocol_version = "HTTP/1.1"
+
+ def do_GET(self):
+ seen.append((self.path, self.headers.get("Authorization")))
+ body = json.dumps({"data": [{"id": "org/model", "root": "/not/a/model/file",
+ "max_model_len": 8192, "size": 999999}]}).encode()
+ status = 401 if locked and not self.headers.get("Authorization") else 302 if redirect else 200
+ self.send_response(status)
+ self.send_header("Content-Length", str(len(body) if status == 200 else 0))
+ if redirect:
+ self.send_header("Location", "/elsewhere")
+ self.end_headers()
+ if status == 200:
+ self.wfile.write(body)
+
+ def log_message(self, *args):
+ pass
+
+ server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)
+ thread = threading.Thread(target=lambda: server.serve_forever(poll_interval=0.01), daemon=True)
+ thread.start()
+ try:
+ yield server.server_address[1], seen
+ finally:
+ server.shutdown()
+ server.server_close()
+ thread.join(timeout=2)
+
+
+def collector(details, ports):
+ c = Collector.__new__(Collector)
+ c.adapter, c.adapters = None, []
+ c.gpu_index, c.scan_budget, c.host, c.host_status = 0, None, None, ""
+ c.nvml = Mock(ok=False)
+ c.counters = Mock(outside={})
+ c.counters.sample.return_value = (
+ {0: {10: ProcGpu(dedicated=512 * MiB, shared=0, engines={"compute": 10})}},
+ {0: AdapterGpu(dedicated=512 * MiB, shared=0)})
+ c.inspector = ProcInspector()
+ c.inspector._cache = details
+ c.inspector._ports = ports
+ c.inspector._ports_time = float("inf")
+ c.inspector.needs_scan = Mock(return_value=False)
+ c.inspector.prune = Mock()
+ c.prober = ServerProber()
+ return c
+
+
+def chain(details, parents):
+ nodes = {pid: Mock(pid=pid) for pid in details}
+ for pid, node in nodes.items():
+ node.create_time.return_value = details[pid].create_time
+ node.parent.return_value = nodes.get(parents.get(pid))
+ return nodes
+
+
+class VllmTests(unittest.TestCase):
+ def test_signatures_are_hints_even_without_mapped_models_or_titles(self):
+ cases = [
+ ("vllm", LAUNCH, "server"),
+ ("python3", ["/env/bin/python3", "/env/bin/vllm", "serve", "org/model"], "server"),
+ ("python", ["python", "-u", "-m", "vllm.entrypoints.openai.api_server", "--model", "org/model"], "server"),
+ ("VLLM::EngineCore", [], "worker"),
+ ("VLLM::Worker_TP12", [], "worker"),
+ ("python", ["VLLM::Worker_TP0"], "worker"),
+ ("python", LAUNCH, "server"), # inherited command line when setproctitle is absent
+ ("vllm", ["vllm", "bench", "serve"], ""),
+ ("python", ["python", "script.py", "-m", "vllm.entrypoints.openai.api_server"], ""),
+ ("echo", ["echo", "-m", "vllm.entrypoints.openai.api_server"], ""),
+ ("VLLM::Worker_TP0-extra", [], ""),
+ ("VLLM::EngineCore_DP0", [], ""),
+ ("vllm-helper", [], ""),
+ ]
+ for name, cmd, role in cases:
+ with self.subTest(name=name, cmd=cmd):
+ d = ProcDetail(10, name, cmdline=cmd)
+ self.assertEqual(d.vllm_role, role)
+ self.assertEqual(classify(d, {}), "LLM" if role else "app")
+
+ def test_models_are_full_identifiers_only(self):
+ for bad in (None, [], {}, {"data": None}, {"data": {}}, {"data": [None]},
+ {"data": [{"id": 5}]}, {"data": [{"id": ""}]}, {"data": [{"id": " "}]}):
+ with self.subTest(response=bad):
+ self.assertIsNone(vllm_models(bad))
+ self.assertEqual(vllm_models({"data": []}), [])
+ model = vllm_models({"data": [{"id": "org/model", "size": 100, "max_model_len": 200}]})[0]
+ self.assertEqual(model["name"], "org/model")
+ self.assertTrue(all(value is None for key, value in model.items() if key != "name"))
+
+ def test_one_endpoint_cache_origin_and_credential_isolation(self):
+ with listener(locked=True) as (port, seen):
+ p = ServerProber()
+ own = ("process", 1, 100)
+ parent = ("parent", ((1, 100),))
+ self.assertIsNotNone(p.get([port], "private-key", kind="vllm", origin=own))
+ self.assertIsNotNone(p.get([port], "private-key", kind="vllm", origin=own))
+ self.assertEqual(len(seen), 1)
+ self.assertIsNone(p.get([port], kind="vllm", origin=parent))
+ self.assertIsNone(p.get([port], kind="vllm", origin=parent))
+ self.assertEqual(p.locked_port([port], kind="vllm", origin=parent), port)
+ self.assertIsNone(p.locked_port([port], "private-key", kind="vllm", origin=own))
+ self.assertEqual(len(seen), 2)
+ self.assertIsNone(seen[-1][1])
+ self.assertIsNotNone(p.get([port], "different-key", kind="vllm", origin=own))
+ self.assertEqual(len(seen), 3) # changed keys do not use the previous cache
+ self.assertIsNotNone(p.get([port], "private-key", kind="vllm", origin=("process", 1, 101)))
+ self.assertEqual(len(seen), 4) # reused PID/port does not use old identity
+ self.assertEqual({path for path, _ in seen}, {"/v1/models"})
+ self.assertNotIn("private-key", repr(vars(p)))
+ with patch("gputaskman.llm.time.time", return_value=time.time() + 31):
+ self.assertIsNone(p.get([port], kind="vllm", origin=parent))
+ self.assertEqual(len(seen), 5)
+ self.assertEqual(p.locked_port([port], kind="vllm", origin=parent), port)
+
+ def test_unauthenticated_parent_redirect_is_not_followed(self):
+ with listener(redirect=True) as (port, seen):
+ self.assertIsNone(probe_server(port, kind="vllm"))
+ self.assertEqual(seen, [("/v1/models", None)])
+
+ def sample(self, details, ports, parents, *, nodes=None, prober=None):
+ c = collector(details, ports)
+ if prober is not None:
+ c.prober = prober
+ nodes = nodes or chain(details, parents)
+ with patch("gputaskman.collector.BACKEND", "nvml"), \
+ patch("gputaskman.collector.psutil.Process", side_effect=lambda pid: nodes[pid]), \
+ patch("gputaskman.procinfo._create_time", side_effect=lambda pid: details[pid].create_time):
+ return c, c.sample()
+
+ def test_collector_two_hops_no_files_no_invented_memory(self):
+ with listener() as (port, seen):
+ details = {10: ProcDetail(10, "VLLM::Worker_TP0", create_time=3),
+ 20: ProcDetail(20, "VLLM::EngineCore", create_time=2),
+ 30: ProcDetail(30, "python", cmdline=LAUNCH, create_time=1)}
+ _, snap = self.sample(details, {30: [port]}, {10: 20, 20: 30})
+ row = snap.procs[0]
+ self.assertEqual(row.kind, "LLM")
+ self.assertEqual(row.llm.server_source, "parent")
+ self.assertEqual(row.llm.server.models[0]["name"], "org/model")
+ self.assertIn("reported (parent API): org/model", row.inside)
+ self.assertEqual(seen, [("/v1/models", None)])
+ self.assertEqual(row.gpu.dedicated, 512 * MiB)
+ self.assertEqual(snap.attributed, 512 * MiB)
+ self.assertEqual([(s.kind, s.bytes) for s in row.llm.segments], [("unknown", 512 * MiB)])
+ self.assertEqual(row.llm.models, [])
+ self.assertEqual(row.llm.files, [])
+ self.assertIsNone(row.llm.kv_per_token)
+ self.assertIsNone(row.llm.n_ctx)
+ data = to_dict(snap)["processes"][0]["llm"]
+ self.assertEqual(data["server_source"], "parent")
+ self.assertEqual(data["server"]["models"][0]["name"], "org/model")
+ text = to_text(snap)
+ self.assertIn("parent process", text)
+ self.assertIn("API-reported model ID: org/model", text)
+ self.assertIn("do not prove this process's model residency", text)
+
+ def test_unclear_ancestry_or_listeners_never_probe(self):
+ for reason in ("two servers", "shared port", "two ports", "unrelated parent",
+ "newer parent", "reused parent", "reused worker", "unknown start", "cycle",
+ "distributed parent", "distributed worker", "protected parent", "no parent"):
+ with self.subTest(reason=reason):
+ details = {10: ProcDetail(10, "VLLM::Worker_TP0", create_time=3),
+ 20: ProcDetail(20, "python", cmdline=list(LAUNCH), create_time=2),
+ 30: ProcDetail(30, "python", cmdline=list(LAUNCH), create_time=1)}
+ ports, parents = {20: [8000]}, {10: 20}
+ if reason == "two servers":
+ ports[30], parents[20] = [8001], 30
+ elif reason == "shared port":
+ ports[30] = [8000]
+ elif reason == "two ports":
+ ports[20] = [8000, 8001]
+ elif reason == "unrelated parent":
+ details[20].cmdline = ["python", "unrelated.py"]
+ elif reason == "newer parent":
+ details[20].create_time = 4
+ elif reason == "unknown start":
+ details[20].create_time = 0
+ elif reason == "cycle":
+ parents[20] = 10
+ elif reason.startswith("distributed"):
+ details[20 if reason.endswith("parent") else 10].cmdline += ["--data-parallel-size", "2"]
+ elif reason == "no parent":
+ parents = {}
+ nodes = chain(details, parents)
+ if reason in ("reused parent", "reused worker"):
+ nodes[20 if reason.endswith("parent") else 10].create_time.return_value = 99
+ if reason == "protected parent":
+ nodes[10].parent.side_effect = psutil.AccessDenied(10)
+ with patch("gputaskman.llm.probe_server") as probe:
+ _, snap = self.sample(details, ports, parents, nodes=nodes)
+ probe.assert_not_called()
+ self.assertIsNone(snap.procs[0].llm.server)
+
+ def test_parent_401_does_not_receive_worker_key_or_use_authenticated_cache(self):
+ with listener(locked=True) as (port, seen):
+ details = {10: ProcDetail(10, "VLLM::EngineCore",
+ cmdline=LAUNCH + ["--api-key", "private-worker-key"], create_time=2),
+ 20: ProcDetail(20, "python", cmdline=LAUNCH + ["--api-key", "private-worker-key"],
+ create_time=1)}
+ prober = ServerProber()
+ self.assertIsNotNone(prober.get([port], "private-worker-key", kind="vllm", origin=("process", 20, 1)))
+ c, snap = self.sample(details, {20: [port]}, {10: 20}, prober=prober)
+ self.assertIsNone(snap.procs[0].llm.server)
+ self.assertIn("Parent-process API locked (401)", " ".join(snap.procs[0].llm.notes))
+ self.assertEqual([path for path, _ in seen], ["/v1/models", "/v1/models"])
+ self.assertIsNotNone(seen[0][1])
+ self.assertIsNone(seen[1][1])
+ self.assertNotIn("private-worker-key", json.dumps(to_dict(snap)))
+ self.assertNotIn("private-worker-key", to_text(snap))
+ self.assertNotIn("private-worker-key", repr(vars(c.prober)))
+
+ def test_own_listener_keeps_own_provenance(self):
+ with listener() as (port, seen):
+ details = {10: ProcDetail(10, "python", cmdline=LAUNCH, create_time=1)}
+ _, snap = self.sample(details, {10: [port]}, {})
+ self.assertEqual(snap.procs[0].llm.server_source, "process")
+ self.assertIn("reported (API): org/model", snap.procs[0].inside)
+ self.assertEqual(seen, [("/v1/models", None)])
+
+ def test_api_unavailable_does_not_change_measured_memory(self):
+ for response in (None, {"error": "disabled"}, {"data": [None]}):
+ with self.subTest(response=response):
+ details = {10: ProcDetail(10, "VLLM::EngineCore", create_time=2),
+ 20: ProcDetail(20, "python", cmdline=LAUNCH, create_time=1)}
+ with patch("gputaskman.llm._get", return_value=response):
+ _, snap = self.sample(details, {20: [8000]}, {10: 20})
+ self.assertIsNone(snap.procs[0].llm.server)
+ self.assertEqual(snap.attributed, 512 * MiB)
+
+ def test_real_child_ancestry_and_http_through_collector(self):
+ """Real cheap processes/listener, not a vLLM runtime or a synthetic ancestry lookup."""
+ with listener() as (port, seen):
+ child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(30)"])
+ try:
+ c = collector({}, {})
+ c.inspector = ProcInspector()
+ original_new = c.inspector._new
+
+ def details(pid):
+ d = original_new(pid)
+ if pid == os.getpid():
+ d.cmdline = LAUNCH
+ elif pid == child.pid:
+ d.cmdline = LAUNCH # no setproctitle; inherited launch signature
+ return d
+
+ c.inspector._new = details
+ c.inspector.needs_scan = Mock(return_value=False)
+ c.counters.sample.return_value = (
+ {0: {child.pid: ProcGpu(dedicated=512 * MiB, shared=0)}},
+ {0: AdapterGpu(dedicated=512 * MiB, shared=0)})
+ connections = [SimpleNamespace(status=psutil.CONN_LISTEN, pid=os.getpid(),
+ laddr=SimpleNamespace(port=port))]
+ with patch("gputaskman.collector.BACKEND", "nvml"), \
+ patch("gputaskman.procinfo.psutil.net_connections", return_value=connections):
+ snap = c.sample()
+ again = c.sample()
+ self.assertEqual(snap.procs[0].llm.server_source, "parent")
+ self.assertEqual(again.procs[0].llm.server.models[0]["name"], "org/model")
+ self.assertEqual(seen, [("/v1/models", None)])
+ finally:
+ child.terminate()
+ child.wait(timeout=5)
+
+
+class VllmUiTests(unittest.IsolatedAsyncioTestCase):
+ async def test_parent_report_in_table_inspector_and_json(self):
+ with listener() as (port, _):
+ details = {10: ProcDetail(10, "VLLM::EngineCore", create_time=2),
+ 20: ProcDetail(20, "python", cmdline=LAUNCH, create_time=1)}
+ _, snap = VllmTests().sample(details, {20: [port]}, {10: 20})
+ with patch.object(GpuTaskManager, "poll"), patch("gputaskman.app.BACKEND", "nvml"):
+ app = GpuTaskManager()
+ app.selected_key = "10"
+ async with app.run_test(size=(180, 50)) as pilot:
+ app.apply(snap)
+ await pilot.pause()
+ text = "\n".join(getattr(line, "plain", "") for line in
+ app.query_one("#inspector", Static).content.renderables)
+ self.assertIn("parent process; API report, not worker residency", text)
+ self.assertIn("API-reported model ID: org/model", text)