Router that virtually distributes inference across connected devices in the home.
nvpair-manual-nodes
A Go service for managing manually configured nodes on networks where mDNS discovery is unavailable. Accepts node addresses via JSON-RPC, probes each for Ollama, LM Studio, and node-info, and emits status events.
Communication
Uses bidirectional newline-delimited JSON-RPC 2.0. By default, communication is over stdin/stdout. An alternative IPC transport (Unix domain socket or Windows named pipe) can be specified with --ipc <path>.
CLI Flags
| Flag | Default | Description |
|---|---|---|
--ipc <path> |
(stdio) | IPC endpoint: Unix socket or Windows named pipe |
--client-cert <path> |
(none) | PEM client certificate to present when probing TLS-enabled manual nodes (requires --client-key) |
--client-key <path> |
(none) | PEM client private key matching --client-cert |
--ca-bundle <path> |
(none) | PEM bundle of CAs to trust when verifying server certificates (additive to the system trust store) |
--cluster-dir <path> |
(none) | Cluster config dir; when set, a TLS manual node's node-info is probed over cluster mTLS, presenting this node's leaf and accepting any currently-pinned server cert |
--log-level <level> |
info |
error / warn / info / debug; falls back to $NVPAIR_LOG_LEVEL. Also changeable at runtime via the log/set-level method |
--version |
Print version and exit |
JSON-RPC Events (notifications, manager → caller)
ready
Emitted once on startup.
{"jsonrpc":"2.0","method":"ready","params":{"version":"0.1.0"}}
node/discovered
Emitted when a manually added node has been probed and its initial status determined.
{
"jsonrpc":"2.0",
"method":"node/discovered",
"params":{
"id":"manual:10.0.1.50",
"address":"10.0.1.50",
"ollama_up":true,
"ollama_port":11434,
"ollama_models":["llama3.2:latest","gemma3:4b"],
"lmstudio_up":true,
"lmstudio_port":1234,
"lmstudio_models":["qwen2.5-7b-instruct"],
"node_info_up":true,
"node_info_port":14318,
"gpus":[{"name":"NVIDIA GeForce RTX 3080","utilization_percent":37}],
"telemetryValid":true,
"msSince":120,
"hostUuid":"stable-node-uuid"
}
}
Each node is probed for both inference engines: Ollama on its default :11434 (GET / + /api/tags) and LM Studio on its default :1234 (GET /v1/models, which doubles as the liveness check and the model list). lmstudio_up / lmstudio_port / lmstudio_models mirror the ollama_* fields and let a supervising broker bridge the node into that engine's nvpair-proxy instance the same way it bridges Ollama into its own. A node can run either engine, both, or neither.
node/updated
Emitted when a periodic probe detects a change (service going up/down, models,
GPU values, telemetry validity, or sample age changed). telemetryValid and
msSince preserve node-info's distinction between an idle 0% sample and missing
telemetry so the broker can feed manual nodes into the same scheduler cache.
node/removed
Emitted when a node is explicitly removed via node/remove.
errors:report / errors:clear
Emitted so the supervising broker can forward them into the nvpair-errors pipeline. A node whose probes fail probeFailThreshold consecutive times (3 probes, so roughly 30 s of unreachability) reports under the id manual-nodes:probe-failed:<node-id>; a subsequent successful probe clears the same id.
JSON-RPC Methods (caller → manager)
node/add
Add a node by address. The manager immediately probes it and emits a node/discovered event.
A hostname is preferred over an IP literal: probe clients disable keep-alives specifically so every probe re-resolves the name, which lets a node that gets a new address recover on its own. An IP-literal entry is dead once the device is renumbered. Supply the address on its own — a host:port string is not parsed, because ports are appended to it, so such an entry reads permanently down.
{"jsonrpc":"2.0","id":1,"method":"node/add","params":{"address":"10.0.1.50","name":"my-server"}}
| Param | Required | Description |
|---|---|---|
address |
Yes | IP address or hostname of the node, with no port |
name |
No | Friendly name (used as node ID; defaults to manual:<address>) |
tls_port |
No | Probe node-info over HTTPS on this port instead of plain HTTP on 14318. Echoed back as tls_enabled |
mtls |
No | Stored and echoed back as mtls_required. The probe transport itself is chosen by tls_port and live cluster membership, so this field records intent rather than driving it |
Response: the initial node status object.
node/remove
Remove a previously added manual node.
{"jsonrpc":"2.0","id":2,"method":"node/remove","params":{"id":"my-server"}}
nodes/list
Returns all currently tracked manual nodes with their latest probe status.
{"jsonrpc":"2.0","id":3,"method":"nodes/list"}
shutdown
Gracefully shuts down the manager.
{"jsonrpc":"2.0","id":4,"method":"shutdown"}
log/set-level
Changes the log level at runtime. Accepted as either a request (answered with {"level":"<resolved>"}) or a fire-and-forget notification.
{"jsonrpc":"2.0","id":5,"method":"log/set-level","params":{"level":"debug"}}
Probing
Each manual node is probed every 10 seconds, with a 3-second timeout per leg, for:
- Ollama on port 11434: health check (
GET /) and model list (GET /api/tags) - LM Studio on port 1234:
GET /v1/models, which doubles as the liveness check and the model list - Node Info on port 14318, or
tls_portover HTTPS: hardware inventory and identity (GET /v1/node-info)
A node can have any combination of these, or none if the target is unreachable. Status changes trigger node/updated events. Because change detection compares CPU, memory, and GPU values, a node running node-info emits a node/updated on most probe cycles as utilization moves.
The three engine ports are compiled in: only the node-info leg's port can be moved, via tls_port. A remote engine on a non-default port is not discovered.
Shutdown
The manager shuts down on:
- stdin EOF (parent process closed the pipe)
SIGINT/SIGTERMshutdownJSON-RPC request