Skip to content

Router that virtually distributes inference across connected devices in the home.

Go
3.4 KB
Raw Blame

NVIDIA Personal AI Router — Linux install

This tarball is the portable "extract anywhere" backend bundle. The shipped distribution places its graphical UI alongside this bundle in the same installation directory. The UI launches nvpair-ui-broker, which drives the API and supervises the workers under bin/.

1. Extract

tar xf NVIDIA-Personal-AI-Router-<version>-linux-amd64.tar.gz   # or -linux-arm64
cd NVIDIA-Personal-AI-Router-<version>

You should end up with this layout:

NVIDIA-Personal-AI-Router-<version>/
├── bin/
│   ├── nvpair-ui-broker              # primary entry point (JSON-RPC over stdio / IPC)
│   ├── nvpair-proxy
│   ├── nvpair-node-info
│   ├── nvpair-node-scanner
│   ├── nvpair-manual-nodes
│   ├── nvpair-workload-manager
│   ├── nvpair-errors
│   ├── nvpair-engine-manager
│   ├── nvpair-node-settings
│   ├── nvpair-cluster-manager
│   ├── nvpair-job-scheduler
│   └── nvpair-tui                    # terminal interface; starts the broker beside it
└── INSTALL.md                     # this file

2. Run

The bundled UI normally launches the broker from the same installation directory. For backend-only development or direct API access, run it yourself. It supervises the other workers (which it expects as siblings in the same bin/ directory) and speaks newline-delimited JSON-RPC 2.0 over stdio (or a Unix socket via --ipc):

./bin/nvpair-ui-broker

The bundled UI connects to the broker over this contract. Other clients can use the same API; see the nvpair-ui-broker README for its JSON-RPC surface.

3. Uninstall

Since nothing was installed by a package manager, removal is just deleting the extracted directory:

rm -rf /path/to/NVIDIA-Personal-AI-Router-<version>

The workers store configuration in ~/.config/Nvidia Corporation/Personal AI Router/ (manual-node list, log-level preference, cluster identity/pins, etc.). Remove that directory too if you want a fully clean slate.

Downloaded models are not in either directory. Each engine keeps them in its own store — ~/.ollama, ~/.lmstudio/models, and ~/.llamacpp — so deleting the bundle or the configuration directory leaves them in place.

Requirements

  • 64-bit Linux on x86_64 or arm64 (aarch64). Use the archive whose name matches your machine.
  • mDNS on UDP 5353. The workers run their own — a custom per-interface responder (nvpair-shared/mdns) plus a grandcat/zeroconf browser (nvpair-shared/discovery), bound with SO_REUSEADDR — and coexist fine with a system responder like avahi-daemon; no configuration needed.

Troubleshooting

  • No nodes discovered: confirm UDP 5353 isn't blocked by your firewall and that other machines on the LAN are actually advertising. Run the broker with --log-level debug to see live discovery logs on stderr.
  • "Bind: address already in use": the proxies listen on 11434 (Ollama), 1234 (LM Studio), and 8080 (llama.cpp); another instance of the proxy or another process is holding one of them. Find the offender with ss -tlnp | grep -E ':(11434|1234|8080)\b' and stop or kill it.

Report issues at the project's tracker. Include the broker's --log-level debug output and your distro / kernel version.