Router that virtually distributes inference across connected devices in the home.
nvpair-node-info
A Go service that exposes this machine's hardware inventory (GPUs, CPU, physical memory) over a small HTTP API at /v1/node-info. It advertises nothing over mDNS itself — its parent (the broker) registers its ni port with the nvpair-node-scanner discovery daemon, which folds it into this node's single _nvpair-node record and fetches /v1/node-info to enrich the node for peers.
Communication
Two surfaces:
- HTTP(S) — serves the node inventory at
/v1/node-info. Plaintext HTTP on:14318by default; optional HTTPS (with optional mTLS) on:14319when a cert/key pair is supplied. - stdio JSON-RPC 2.0 — newline-delimited, used for lifecycle/control (
log/set-level,nodeinfo:set-cluster-identity) and shutdown via stdin EOF. The service is normally launched as a subprocess by the broker.
It does not advertise itself over mDNS. Discovery is centralized in the nvpair-node-scanner daemon: the broker registers this service's ni port with the daemon, which carries it on the node's one _nvpair-node record and fetches /v1/node-info over plain HTTP to enrich each node.
CLI Flags
| Flag | Default | Description |
|---|---|---|
--port <n> |
14318 |
HTTP port to listen on |
--tls-port <n> |
14319 |
HTTPS port (only used when --cert and --key are set) |
--cert <path> |
(none) | TLS server certificate (PEM); enables HTTPS when set together with --key |
--key <path> |
(none) | TLS server private key (PEM); enables HTTPS when set together with --cert |
--client-ca <path> |
(none) | PEM bundle of CAs trusted to sign client certificates; enables mTLS when set |
--accept-http |
false |
Keep the plaintext HTTP listener alive even when TLS is enabled (default: HTTPS-only once --cert is set) |
--cluster-dir <path> |
(none) | Cluster config dir (node.crt/node.key + trusted/); when set, the primary port carries one cluster-gated listener. While this node is a cluster member, /v1/node-info is served only over cluster-scoped pin-based mTLS, to pinned cluster peers — or this node itself via self-trust — and a plaintext caller is refused with 403; while it is not a member the plaintext inventory is served as usual. Membership is re-read per request. Takes precedence over the --cert path |
--version |
Print version and exit |
--log-level is also accepted (registered by nvpair-shared/applog).
TLS / mTLS configuration
The cert flags follow a bring-your-own contract, validated at startup:
--certand--keymust be supplied together; either alone is rejected.--client-carequires--certand--key. When set, the server requires and verifies a client certificate (RequireAndVerifyClientCert); connections without a trusted client cert are dropped at handshake time.- TLS minimum version is pinned to TLS 1.2.
When TLS is enabled, the HTTP listener is dropped unless --accept-http is passed. Both listeners share the same handler, but bind separate ports so clients can migrate independently.
--cluster-dir replaces both with a single cluster-gated listener on the primary port (see the flag table): one bound port, both personalities, and the choice between them follows live cluster membership rather than being fixed when the listener was bound.
HTTP API
GET /v1/node-info
Returns the merged static identity (collected once at startup) and the latest dynamic stats snapshot.
{
"GPUs": [
{
"name": "NVIDIA GeForce RTX 3080",
"vram_bytes": 10737418240,
"vram_used_bytes": 2147483648,
"utilization_percent": 42
}
],
"telemetryValid": true,
"msSince": 137,
"cpu": {
"name": "AMD Ryzen 9 5900X 12-Core Processor",
"cores": 12,
"utilization_percent": 7
},
"memory": {
"total_bytes": 34359738368,
"used_bytes": 12884901888
},
"hostUuid": "8661676a-0d1c-4bd3-ac5e-4d370e6f1a9c",
"clusterUuid": ""
}
Field notes:
- Dynamic fields (
vram_used_bytes,utilization_percent,cpu.utilization_percent,memory.used_bytes) are populated by the Windows, Linux, and macOS per-tick stats collectors. telemetryValidandmsSincedescribe the node-wide GPU utilization snapshot.telemetryValidistrueafter the collector has produced a usable GPU sample;msSinceis that sample's age in milliseconds at response time. A failed collection retains the last usable sample and lets its age increase. Before the first usable sample, and on platforms without dynamic GPU telemetry, the response reportstelemetryValid:falseandmsSince:0; consumers must ignore the age while validity is false.clusterUuidis the cluster principal this node currently holds. It has three distinct states on the wire: absent means unknown, present and empty means this node belongs to no cluster, and a value is that principal. A consumer must not read absent as unclustered — that is how a node too old to report the field answers, and also how this node answers before its parent has told it anything, so acting on it would clear a correct annotation elsewhere in the fleet.- Under the broker,
clusterUuidis pushed in over stdin (nodeinfo:set-cluster-identity) because node-info is spawned with no cluster dir and so cannot read membership itself; the field stays absent until the first push arrives. Standalone with--cluster-dir, it reads membership from the trust store per request instead and is therefore always known. The two sources are mutually exclusive by deployment, not a fallback chain. clusterUuidexists so a peer can learn this node's membership without its mDNS record. Membership otherwise travels only as thecluster-uuid=TXT key, which a consumer reads once per record change; a consumer that misses that change keeps the previous value indefinitely, and one still holding a departed node's principal will suppress the invite that would bring it back.- All dynamic fields and the
cpu/memoryobjects useomitempty: a value the service couldn't read is dropped from the JSON entirely rather than reported as a misleading literal zero. A genuinely idle CPU renders the same as "unknown" — that ambiguity is intentional and benign. vram_bytesis reported through DXGI on Windows,nvidia-smion Linux, and IORegistry on macOS. On a unified-memory NVIDIA GPU such as DGX Spark, Linux uses total physical system memory forvram_bytesand the independently sampled system-memory usage forvram_used_bytes. On Apple Silicon,vram_bytesis total physical unified memory andvram_used_bytesis the GPU driver's mapped allocation (Alloc system memory), not whole-system RAM usage or the momentarily active subset.
Discovery
The service no longer advertises itself over mDNS. Its parent (the broker) registers the ni port with the nvpair-node-scanner discovery daemon, which carries it on this node's single _nvpair-node record; a peer's daemon then fetches /v1/node-info over plain HTTP to enrich the node. Transport is derived from the shared per-service policy (nvpair-shared/noderec): node-info is served plain on the broker path. The standalone BYO-TLS / --cluster-dir mTLS serving above is retained for running node-info on its own, but is unused under the broker.
Platform Notes
- Windows (first-class): GPU inventory comes from DXGI (vendor-agnostic, includes VRAM). Dynamic CPU / VRAM-used / utilization / memory-used numbers come from a persistent PDH query plus
GlobalMemoryStatusEx. - Linux (first-class): NVIDIA GPU inventory, dedicated VRAM usage, and utilization come from
nvidia-smi; CPU and system-memory usage come from/proc. Unified-memory GPUs use the/proc/meminfosystem-memory snapshot even when dynamicnvidia-smicollection is unavailable. Non-NVIDIA adapters fall back to names fromghwwithout dynamic GPU stats. - macOS: CPU and system-memory usage come from Mach through gopsutil's purego bindings. GPU identity, mapped memory, and utilization come from the built-in, unprivileged
/usr/sbin/ioregcommand'sIOAcceleratorPerformanceStatistics; no sudo or private framework binding is required. Apple Silicon is supported directly. Intel/AMD fields are best-effort when their drivers expose the same dedicated-memory counters. The performance keys are undocumented and may change across macOS releases; a missing or changed key leaves only that metric out and does not stop CPU or memory collection. - Other platforms: GPU names come from
ghw; VRAM and dynamic stats are not reported.
Shutdown
The service shuts down on:
- stdin EOF (parent process closed the pipe)
SIGINT/SIGTERM
On shutdown it gracefully stops the HTTP/HTTPS listeners (3 s timeout).