Router that virtually distributes inference across connected devices in the home.
Go
Reconcile documentation with the engine settings and scheduler changes
Tervedb67146362a4
diff --git a/README.md b/README.mdindex 2ccba4d..757c3f3 100644--- a/README.md+++ b/README.md@@ -287,7 +287,7 @@ feedback and contributions will help shape priorities.- [ ] Requester-only clients that send work without hosting an inference engine.- [ ] Launch PAIR automatically on system startup.-- [ ] Configure engine launch options and environment variables from PAIR.+- [x] Configure engine launch options and environment variables from PAIR.- [ ] Stability and reliability improvements informed by real-world use.Have a feature request or a workflow you want PAIR to support? Open andiff --git a/docs/known-issues.mdx b/docs/known-issues.mdxindex 6853f0c..59aa0cd 100644--- a/docs/known-issues.mdx+++ b/docs/known-issues.mdx@@ -26,18 +26,24 @@ memory total on that architecture, so PAIR substitutes system memory instead.The other two cases are the gap.**The impact is display, not routing.** These numbers appear on a node's card-and in the GPU chart. Routing does not consider VRAM at all, because the-scheduler counts workloads. A wrong figure does not send work to the wrong place.+and in the GPU chart. Routing does not consider VRAM at all — the scheduler+weighs queued work and GPU utilization, never memory — so a wrong figure does+not send work to the wrong place.It misleads *you* when you decide which node should hold a large model, so onthose systems judge capacity from the machine's own tools rather than from PAIR.-## Scheduling Only Counts Jobs+## Scheduling Sees Load, Not CapacityThe single scheduling policy ranks nodes by how much work is already queued on-them. It does not consider GPU model, available memory, current utilization,-measured latency, whether a model is already loaded, or how expensive a request-looks. On a cluster of similar machines this behaves reasonably. On a mixed one,-expect work to land on a slower node about as often as a faster one.+them together with coarse GPU load, which it derives from smoothed utilization.+It does not consider GPU model, available memory, measured latency, whether a+model is already loaded in memory, or how expensive a request looks. Telemetry+that is missing or more than ten seconds old counts as middling load rather than+idle.++On a cluster of similar machines this behaves reasonably. On a mixed one, the+same utilization percentage means the same thing on a small GPU and a large one,+so work can still land on the slower machine.The full list of gaps is in[Scheduler Limitations](architecture.mdx#scheduler-limitations).