Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Grid harness: find, start and verify models on any machine and engine

Timd97257fc7af1
diff --git a/store/agents/autonomous-grid/AGENTS.md b/store/agents/autonomous-grid/AGENTS.md
index df79304e6..ba92c40b3 100644
--- a/store/agents/autonomous-grid/AGENTS.md
+++ b/store/agents/autonomous-grid/AGENTS.md
@@ -26,8 +26,72 @@ paragraph is not a stop. **Talk the way Harness talks:** short, declarative, sec
is true and what happens next; no "I'd be happy to", no exclamation marks; a thing that is not set
up is said plainly and stopped at, the way a failed build is reported.
-Read `skills/grid-operations/SKILL.md` from this package (or its workspace skill link) before operating
-the fleet. `$GRID_FLEET` is the workspace-aware command runner; `$GRID_CLI` is the selected Grid CLI.
+**Starting a model on this computer — the whole flow.** Do it in this order; open a skill only when a
+step names one. The commands below are verified: run them as written, without reading `--help` first —
+each extra step costs a minute. Technical choices (file, engine, context, port, flags) are yours, from
+measurements; the person is asked only what the model is for, and about trade-offs they can feel. When
+the person names an engine, that engine is the plan: if it has no suitable model, ask one question —
+*Use <file on disk> with Grid's engine now · Get a model for <engine> (N GB download)* — never switch
+engines silently.
+
+1. `"$GRID_FLEET" models --summary` — one short table: this machine (chip, GPUs and whether each is
+ active, engines installed, what it can run, ports in use), memory and swap now, engines already
+ answering with their `--at` URL, grids this computer is joined to (`joined`), and one row per model file on disk of every format (size, context,
+ cache at 64K, tool calls, vision, which engines read it). Read the table; do not filter it yourself.
+ Drop `--summary` for the full JSON only when a field you need is not in the table. A request that
+ does not say what the model is for ("I have some models, can we run one?") gets the purpose question
+ right after this table — question tool, before reading any skill: a person waited 6 minutes for
+ one [run]. Everything this flow needs is on this page; open a skill only where a step names it.
+2. Which grid: "on this computer only", "not online", "just here" → a **local grid** (below); otherwise
+ the one the person named, else `personalGrid` (never a shared grid unasked — below). `"$GRID_FLEET" run -- models GRID --json` — already served
+ there? Then no start. This computer already joined to it = the `joined` line of step 1, never the
+ relay's engine list (an asleep grid lists none). Since Grid 0.3.53 a join there **adds** a model beside
+ what this computer already serves (each with its own port and alias) [run]; adding or removing a
+ `--serve` model restarts the others for a few seconds, so say so first. If memory will not hold both,
+ ask *Replace it · Keep it* (Replace = `leave GRID --engine ALIAS`, then join).
+3. Pick a file already on disk: context ≥ 64K (128K for coding when it fits); weights + context × cache
+ per token + 0.5 GB within free memory (+3 GB) and the GPU ceiling; tool calls for coding. Nothing
+ fits: `skills/grid-operations` step 4 (catalog). An engine installed with nothing to serve:
+ `"$GRID_FLEET" candidates mlx` on a Mac, recipes on a GPU box (`skills/run-local-model`).
+4. Engine by format: GGUF → Grid's own engine; an engine already answering → `join --at http://127.0.0.1:PORT/v1
+ -m ID --advertise-as ALIAS` (the `/v1` is required; without the alias the picker shows the raw id);
+ engines already answering that belong to a `joined` grid are not free to reuse; MLX folder → `skills/engine-mlx-lm`; safetensors on an NVIDIA/AMD box →
+ `skills/engine-vllm` or `engine-sglang`; Ollama or LM Studio → their `skills/engine-*`.
+5. GGUF start: `"$GRID_FLEET" link FILE NAME.gguf` (Grid serves only from `~/.grid/models`; a link, no copy), then
+ `"$GRID_FLEET" run --thinking off -- join GRID --serve NAME.gguf --advertise-as ALIAS --name MACHINE-ALIAS
+ --ctx-size CTX --endpoint-port PORT --reasoning-budget 0` plus `--max-concurrency 1` on a remote grid or
+ `--parallel 1` on a local one (`--max-concurrency` is remote-only), PORT not in `listeningPorts`.
+6. `"$GRID_FLEET" verify --at http://127.0.0.1:PORT/v1 --model ALIAS --kind llama.cpp --grid GRID --alias ALIAS`
+ — `--alias` is the `--advertise-as` name; `--model` is what the engine itself lists. Run it right after
+ the start, without `| tail` or `| head` — it waits by itself and narrates; pass its progress on. Every
+ engine the same way (Ollama, LM Studio, mlx-lm…: `--kind` is that engine, and after its `join --at`,
+ `verify --grid GRID --alias ALIAS` proves the relay half). This is the only check: never `sleep`,
+ `stats`, `grid chat` or a log in `~/.grid` to find out whether it works [run: 2.5 minutes lost that way]. Report only what passed — an engine that answers but is
+ not joined is "running here, not in your model picker yet", never "done" — then `fleet models` once more
+ (swap still climbing a minute later = too big: stop, take a smaller file), `refresh`, and the hand-off.
+
+**Another machine in `grid-fleet.json`** runs the same flow with `--machine M`: `"$GRID_FLEET" models
+--machine M --summary` (an SSH machine, read with its own Node; drop `--summary` for file paths),
+`"$GRID_FLEET" link --machine M FILE NAME.gguf` — it prints `NOT READY` when that machine lacks Grid's
+engine: ask, then `"$GRID_FLEET" run --machine M -- engine install llama.cpp` before any join (without it
+the join still says "starting" and the engine dies at once) —, then
+`"$GRID_FLEET" run --machine M --thinking off -- join GRID --serve NAME.gguf --advertise-as ALIAS --name M
+--ctx-size CTX --max-concurrency 1 --reasoning-budget 0`, where ALIAS is a name no other machine serves on
+that grid (`run -- models GRID --json`) — a shared name cannot show which machine answered — and then
+`"$GRID_FLEET" verify --grid GRID --alias ALIAS` (through the relay only).
+Only Grid's own engine can be started there; a Harness-linked machine's files cannot be listed, so its
+model comes from the catalog.
+
+"On this computer only", "not online", "just here" mean a **local grid** — the personal grid is online
+and is not local. Kept on this computer only: `"$GRID_CLI" --local start NAME --port P --host 127.0.0.1 --advertise-host
+127.0.0.1`, then `"$GRID_FLEET" connect --mode local --grid NAME`, and every `join` also takes
+`--advertise-host 127.0.0.1` — without it the engine registers this computer's LAN address [run]. **Never run `grid mode`** — it switches
+the CLI's mode for every workspace and app on the machine — and never `start` a hosted grid unasked.
+A Grid refusal names its own fix: read the whole message and follow it; never open `~/.grid`.
+
+The skills hold the detail: `skills/grid-operations/SKILL.md` (grids and access, catalog downloads,
+vision, change, stop, move), `skills/run-local-model/SKILL.md` (sizing, choosing, reading official
+sources when unsure), `skills/engine-*/SKILL.md` (one per engine). `$GRID_FLEET` is the workspace-aware command runner; `$GRID_CLI` is the selected Grid CLI.
Use the runner for operations so progress appears in the viewer. The terminal beside the viewer is
the conversation; do not build another chat UI or run a second background agent.
@@ -36,15 +100,24 @@ viewer's published observations without network access. For questions about runn
machines, use a fresh, live snapshot and its observation time; do not start a second network poll.
Downloaded files and catalog entries are not proof of serving models. The user's own grid is
`personalGrid` in `grid-fleet.json`, named by Harness from the signed-in account: "my grid" means
-that exact name, never asked for. Which grid to use otherwise: `grid`, the workspace's selected
-fleet; never assume one named `home`. Add `--remember` when the user wants that fleet reused by future Grid workspaces.
+that exact name, never asked for. **A model started on the person's own machine joins `personalGrid`
+by default** — "run a model on my Mac" names no grid, and that is not consent to serve a shared one:
+`grid` (the workspace's selected fleet) can be a team or company grid where other people's requests
+would land on this machine [run: a model went to a company grid that way]. Use `grid`, or any other
+grid, only when the person names it or asks to share with that team; never assume one named `home`.
+Add `--remember` when the user wants that fleet reused by future Grid workspaces.
**This harness runs without Codex's sandbox**, the way a terminal does: `join` starts an engine
with the GPU, and every Grid command reaches the network. So run fleet commands directly
(`"$GRID_FLEET" run -- …`) — never request escalation, and never treat a failure as a sandbox
problem. Every fleet operation goes through the runner or `$GRID_CLI`: never `ps`, `pgrep`, `lsof`,
`sysctl`, `kill`, llama logs or hand-written scripts to start, stop, inspect or diagnose an engine;
-the command's own output is the evidence.
+the command's own output is the evidence. Two written exceptions: `"$GRID_FLEET" models`, `recipe`
+and `model-facts` read this machine and the engines' official sources for you; and an engine other
+than Grid's own (Ollama, LM Studio, mlx-lm, vLLM, SGLang) is started with `"$GRID_FLEET" serve NAME -- …`
+and stopped with `"$GRID_FLEET" stop NAME`, using the command in its `skills/engine-*/SKILL.md`. A plain
+`nohup … &` dies when your command returns [run]; `serve` gives the engine its own session and records
+its PID, and `stop` kills only that PID.
**Two failed attempts at the same step is the limit.** Report what failed in one line, with the
command's own error, and wait — never improvise a workaround. If the viewer observation is stale,
diff --git a/store/agents/autonomous-grid/lib/candidates.mjs b/store/agents/autonomous-grid/lib/candidates.mjs
new file mode 100644
index 000000000..4b3fbc650
--- /dev/null
+++ b/store/agents/autonomous-grid/lib/candidates.mjs
@@ -0,0 +1,69 @@
+// Models to download for an engine that is installed but has nothing to serve. MLX: the mlx-community
+// organisation on Hugging Face, sized from each repo's own safetensors and config — nothing is downloaded
+// here. Every candidate carries its size, its cache cost at the context asked, and whether it fits.
+const HUB = 'https://huggingface.co';
+const CHAT_TASKS = new Set(['text-generation', 'image-text-to-text']);
+
+/**
+ * A first estimate for filtering only: the Hub counts a quantized repo's parameters unpacked (a 4B model at
+ * 4 bits reports 4.0e9 "U32"), so size ≈ parameters × bits ÷ 8, plus ~15% for scales. The real size comes
+ * from the files (`repoBytes`).
+ */
+export function estimateBytes(safetensors, bits) {
+ return Math.round((safetensors?.total ?? 0) * (bits / 8) * 1.15);
+}
+
+/** The exact size of a repo's weight files, from the Hub's file listing. */
+export async function repoBytes(id, fetchImpl) {
+ const info = await fetchImpl(`${HUB}/api/models/${id}?blobs=true`, { signal: AbortSignal.timeout(20_000) }).then(r => (r.ok ? r.json() : null)).catch(() => null);
+ if (!info) return null;
+ return (info.siblings ?? []).filter(file => file.rfilename.endsWith('.safetensors')).reduce((sum, file) => sum + (file.size ?? 0), 0) || null;
+}
+
+/** f16/bf16 KV-cache bytes per token from a Hugging Face config, counting only layers that keep a cache. */
+export function kvBytesFromConfig(config) {
+ const text = config.text_config ?? config;
+ const layers = text.num_hidden_layers, heads = text.num_attention_heads;
+ const kvHeads = text.num_key_value_heads ?? heads;
+ const headDim = text.head_dim ?? (text.hidden_size && heads ? text.hidden_size / heads : undefined);
+ if (!layers || !kvHeads || !headDim) return null;
+ let cached = layers;
+ if (Array.isArray(text.layer_types)) cached = text.layer_types.filter(type => /full|global/.test(type) || type === 'attention').length;
+ else if (text.full_attention_interval > 1) cached = Math.floor(layers / text.full_attention_interval);
+ return cached * kvHeads * headDim * 2 * 2;
+}
+
+/**
+ * MLX models for this Mac: chat-capable, quantized, public, newest-first within popularity, each checked
+ * against `budgetBytes` at `contextTokens`. `search` narrows by name (e.g. a family the person asked for).
+ */
+const SORTS = { downloads: 'downloads', trending: 'trendingScore', recent: 'lastModified' };
+
+export async function mlxCandidates({ budgetBytes, contextTokens = 65536, search = '', sort = 'downloads', limit = 12, fetchImpl = globalThis.fetch } = {}) {
+ if (!SORTS[sort]) throw new Error('sort is downloads, trending or recent');
+ const query = new URLSearchParams({ author: 'mlx-community', sort: SORTS[sort], direction: '-1', limit: '300' });
+ if (search) query.set('search', search);
+ for (const field of ['safetensors', 'config', 'downloads', 'pipeline_tag', 'lastModified', 'gated']) query.append('expand[]', field);
+ const response = await fetchImpl(`${HUB}/api/models?${query}`, { signal: AbortSignal.timeout(30_000) });
+ if (!response.ok) throw new Error(`huggingface.co answered HTTP ${response.status}`);
+ // Weights alone must fit before the config is worth reading.
+ const listed = (await response.json()).filter(row => CHAT_TASKS.has(row.pipeline_tag) && !row.gated && row.config?.quantization_config?.bits
+ && (!budgetBytes || estimateBytes(row.safetensors, row.config.quantization_config.bits) < budgetBytes));
+ const out = [];
+ for (const row of listed.slice(0, limit)) {
+ const bytes = await repoBytes(row.id, fetchImpl) ?? estimateBytes(row.safetensors, row.config.quantization_config.bits);
+ const config = await fetchImpl(`${HUB}/${row.id}/raw/main/config.json`, { signal: AbortSignal.timeout(20_000) })
+ .then(r => (r.ok ? r.json() : null)).catch(() => null);
+ const kv = config ? kvBytesFromConfig(config) : null;
+ const contextMax = config ? (config.text_config ?? config).max_position_embeddings ?? null : null;
+ const need = kv === null ? null : bytes + kv * contextTokens + 512 * 1024 ** 2;
+ out.push({
+ id: row.id, downloads: row.downloads, lastModified: row.lastModified, task: row.pipeline_tag,
+ bits: row.config.quantization_config.bits, architecture: row.config.architectures?.[0] ?? null,
+ bytes, kvBytesPerToken: kv, contextMax, needBytesAtContext: need,
+ fits: need === null || !budgetBytes ? null : need <= budgetBytes && (contextMax ?? contextTokens) >= contextTokens,
+ toolCalls: /\btools\b/.test(row.config.tokenizer_config?.chat_template ?? '') || null,
+ });
+ }
+ return { source: `${HUB}/models?author=mlx-community`, contextTokens, budgetBytes: budgetBytes ?? null, candidates: out };
+}
diff --git a/store/agents/autonomous-grid/lib/finderMain.mjs b/store/agents/autonomous-grid/lib/finderMain.mjs
new file mode 100644
index 000000000..a3bee380a
--- /dev/null
+++ b/store/agents/autonomous-grid/lib/finderMain.mjs
@@ -0,0 +1,7 @@
+// Entry point for `fleet models --machine ID`: copied to that machine beside inventory.mjs and
+// candidates.mjs and run by its own Node, so the table describes that machine — its disk, its engines,
+// its memory — not the controller's.
+import { inventory, summarize } from './inventory.mjs';
+
+const found = await inventory();
+console.log(process.argv.includes('--summary') ? summarize(found) : JSON.stringify(found));
diff --git a/store/agents/autonomous-grid/lib/inventory.mjs b/store/agents/autonomous-grid/lib/inventory.mjs
new file mode 100644
index 000000000..e984909df
--- /dev/null
+++ b/store/agents/autonomous-grid/lib/inventory.mjs
@@ -0,0 +1,579 @@
+// Read-only inventory of this computer for local models: model files other apps already downloaded,
+// engines answering on loopback, and the memory the OS reports right now. Nothing here starts, stops,
+// moves or downloads anything; every probe is a file read, a GET, or a read-only system command.
+import { execFile } from 'node:child_process';
+import { open, readdir, readFile, realpath, stat } from 'node:fs/promises';
+import { arch, cpus, freemem, homedir, platform, totalmem } from 'node:os';
+import { basename, dirname, join, relative, sep } from 'node:path';
+import { promisify } from 'node:util';
+
+import { kvBytesFromConfig } from './candidates.mjs';
+
+const exec = promisify(execFile);
+const SCALAR_BYTES = { 0: 1, 1: 1, 2: 2, 3: 2, 4: 4, 5: 4, 6: 4, 7: 1, 10: 8, 11: 8, 12: 8 };
+// Grid's llama.cpp refuses higher tensor types: "invalid ggml type 143. should be in [0, 43)".
+export const GGML_TYPE_LIMIT = 43;
+const SKIP_DIRS = new Set(['node_modules', '.git', '__pycache__', '.Trash']);
+const KNOWN_PORTS = [11434, 1234, 8000, 8080, 8081, 30000];
+
+class HeaderReader {
+ constructor(handle) { this.handle = handle; this.buffer = Buffer.alloc(0); this.offset = 0; this.next = 0; }
+ async need(bytes) {
+ if (this.buffer.length - this.offset >= bytes) return;
+ const kept = this.buffer.subarray(this.offset);
+ const chunk = Buffer.alloc(Math.max(bytes - kept.length, 1 << 20));
+ const { bytesRead } = await this.handle.read(chunk, 0, chunk.length, this.next);
+ this.next += bytesRead;
+ this.buffer = Buffer.concat([kept, chunk.subarray(0, bytesRead)]);
+ this.offset = 0;
+ if (this.buffer.length < bytes) throw new Error('truncated GGUF header');
+ }
+ skip(bytes) {
+ const buffered = this.buffer.length - this.offset;
+ if (bytes <= buffered) { this.offset += bytes; return; }
+ this.next += bytes - buffered; this.buffer = Buffer.alloc(0); this.offset = 0;
+ }
+ async number(type) {
+ const size = SCALAR_BYTES[type];
+ await this.need(size);
+ const b = this.buffer, o = this.offset;
+ this.offset += size;
+ switch (type) {
+ case 0: return b.readUInt8(o); case 1: return b.readInt8(o);
+ case 2: return b.readUInt16LE(o); case 3: return b.readInt16LE(o);
+ case 4: return b.readUInt32LE(o); case 5: return b.readInt32LE(o);
+ case 6: return b.readFloatLE(o); case 7: return b.readUInt8(o) !== 0;
+ case 10: return Number(b.readBigUInt64LE(o)); case 11: return Number(b.readBigInt64LE(o));
+ default: return b.readDoubleLE(o);
+ }
+ }
+ async string(limit = 1 << 20) {
+ const length = await this.number(10);
+ if (length > limit) { this.skip(length); return null; }
+ await this.need(length);
+ const text = this.buffer.toString('utf8', this.offset, this.offset + length);
+ this.offset += length;
+ return text;
+ }
+ async value(type, keep) {
+ if (type === 8) return this.string();
+ if (type in SCALAR_BYTES) return this.number(type);
+ if (type !== 9) throw new Error(`unsupported GGUF value type ${type}`);
+ const itemType = await this.number(4), length = await this.number(10);
+ if (itemType in SCALAR_BYTES) {
+ if (!keep || length > 4096) { this.skip(length * SCALAR_BYTES[itemType]); return undefined; }
+ const items = [];
+ for (let i = 0; i < length; i++) items.push(await this.number(itemType));
+ return items;
+ }
+ if (itemType !== 8) throw new Error(`unsupported GGUF array type ${itemType}`);
+ for (let i = 0; i < length; i++) this.skip(await this.number(10));
+ return undefined;
+ }
+}
+
+/** The metadata and tensor types of a GGUF file, or null when the file is not GGUF. */
+export async function readGguf(path) {
+ const handle = await open(path, 'r');
+ try {
+ const reader = new HeaderReader(handle);
+ await reader.need(4);
+ if (reader.buffer.toString('latin1', 0, 4) !== 'GGUF') return null;
+ reader.offset = 4;
+ const version = await reader.number(4);
+ const tensorCount = await reader.number(10), keyCount = await reader.number(10);
+ if (version < 2 || keyCount > 100_000 || tensorCount > 1_000_000) throw new Error('unsupported GGUF header');
+ const meta = {};
+ for (let i = 0; i < keyCount; i++) {
+ const key = await reader.string(4096), type = await reader.number(4);
+ const keep = !key?.startsWith('tokenizer.') || key === 'tokenizer.chat_template';
+ const value = await reader.value(type, keep);
+ if (key && keep && value !== undefined) meta[key] = value;
+ }
+ const tensorTypes = new Set();
+ for (let i = 0; i < tensorCount; i++) {
+ reader.skip(await reader.number(10));
+ reader.skip((await reader.number(4)) * 8);
+ tensorTypes.add(await reader.number(4));
+ reader.skip(8);
+ }
+ return { meta, tensorTypes: [...tensorTypes].sort((a, b) => a - b) };
+ } finally { await handle.close(); }
+}
+
+/**
+ * What an agent needs from GGUF metadata to size a start. The KV figure is f16 cache bytes per token of
+ * context; llama.cpp reported exactly this (1024 MiB for 32768 cells, 8 of 32 layers) for Qwen3.5-4B.
+ */
+export function summarizeGguf({ meta, tensorTypes }) {
+ const architecture = meta['general.architecture'];
+ const get = key => meta[`${architecture}.${key}`];
+ const layers = get('block_count'), heads = get('attention.head_count');
+ const kvHeads = get('attention.head_count_kv') ?? heads;
+ const widest = Array.isArray(heads) ? Math.max(...heads) : heads;
+ const keyLength = get('attention.key_length') ?? (get('embedding_length') && widest ? get('embedding_length') / widest : undefined);
+ const valueLength = get('attention.value_length') ?? keyLength;
+ let kvBytesPerToken = null, kvBasis = null;
+ if (get('attention.kv_lora_rank') !== undefined) kvBasis = 'latent attention: not estimated';
+ else if (layers && keyLength && kvHeads !== undefined) {
+ const perLayer = Array.isArray(kvHeads) ? kvHeads.slice(0, layers) : Array(layers).fill(kvHeads);
+ const interval = get('full_attention_interval');
+ const cached = perLayer.map((n, i) => (interval > 1 && (i + 1) % interval !== 0 ? 0 : n));
+ kvBytesPerToken = cached.reduce((sum, n) => sum + n, 0) * (keyLength + valueLength) * 2;
+ kvBasis = get('attention.sliding_window') ? 'upper bound (sliding-window layers hold less)' : 'f16 cache';
+ }
+ const template = meta['tokenizer.chat_template'] || '';
+ return {
+ architecture: architecture ?? null, name: meta['general.name'] ?? null, parameters: meta['general.size_label'] ?? null,
+ contextLength: get('context_length') ?? null, layers: layers ?? null,
+ experts: get('expert_count') ? { total: get('expert_count'), active: get('expert_used_count') ?? null } : null,
+ kvBytesPerToken, kvBasis,
+ toolCalls: /\btools\b/.test(template), thinking: /enable_thinking|<think>/.test(template),
+ unsupportedTensorTypes: tensorTypes.filter(type => type >= GGML_TYPE_LIMIT),
+ };
+}
+
+export function defaultRoots(env = process.env, home = homedir()) {
+ const hub = env.HF_HUB_CACHE || (env.HF_HOME ? join(env.HF_HOME, 'hub') : join(env.XDG_CACHE_HOME || join(home, '.cache'), 'huggingface', 'hub'));
+ return [
+ { source: 'grid', path: join(env.GRID_HOME || join(home, '.grid'), 'models') },
+ { source: 'ollama', path: env.OLLAMA_MODELS || join(home, '.ollama', 'models') },
+ { source: 'ollama', path: '/usr/share/ollama/.ollama/models' },
+ { source: 'lm-studio', path: join(home, '.lmstudio', 'models') },
+ { source: 'lm-studio', path: join(home, '.cache', 'lm-studio', 'models') },
+ { source: 'huggingface', path: hub },
+ { source: 'folder', path: join(home, 'models') },
+ { source: 'folder', path: join(home, 'Models') },
+ { source: 'folder', path: join(home, 'Downloads'), depth: 3 },
+ ];
+}
+
+const QUANT = /(?:^|[-_.])((?:UD-)?(?:I?Q\d(?:_[A-Z0-9]+)*|BF16|F16|F32|MXFP4|NVFP4))(?=[-_.]|$)/i;
+const READERS = { gguf: ['llama.cpp', 'ollama', 'lm-studio'], mlx: ['mlx-lm', 'lm-studio'], safetensors: ['vllm', 'sglang', 'mlx-lm'] };
+
+async function walk(dir, depth, visit, budget) {
+ if (depth < 0 || budget.dirs-- <= 0) return;
+ let entries;
+ try { entries = await readdir(dir, { withFileTypes: true }); } catch { return; }
+ if (entries.some(e => e.name === 'config.json') && entries.some(e => e.name.endsWith('.safetensors'))) await visit('folder', dir);
+ for (const entry of entries) {
+ const path = join(dir, entry.name);
+ if (entry.isDirectory()) { if (!SKIP_DIRS.has(entry.name) && entry.name !== 'blobs') await walk(path, depth - 1, visit, budget); }
+ else if (/\.gguf$/i.test(entry.name) && (entry.isFile() || entry.isSymbolicLink())) await visit('gguf', path);
+ }
+}
+
+async function listFiles(dir, depth) {
+ if (depth < 0) return [];
+ let entries;
+ try { entries = await readdir(dir, { withFileTypes: true }); } catch { return []; }
+ const nested = await Promise.all(entries.map(e => (e.isDirectory() ? listFiles(join(dir, e.name), depth - 1) : [join(dir, e.name)])));
+ return nested.flat();
+}
+
+async function ollamaEntries(root) {
+ const manifests = join(root, 'manifests'), entries = [];
+ for (const file of await listFiles(manifests, 4)) {
+ const [host, namespace, model, tag] = relative(manifests, file).split(sep);
+ if (!tag) continue;
+ let layers;
+ try { ({ layers = [] } = JSON.parse(await readFile(file, 'utf8'))); } catch { continue; }
+ const blob = mediaType => layers.find(layer => layer.mediaType === `application/vnd.ollama.image.${mediaType}`)?.digest;
+ const weights = blob('model'), projector = blob('projector');
+ if (!weights) continue;
+ const short = namespace === 'library' ? `${model}:${tag}` : `${namespace}/${model}:${tag}`;
+ entries.push({
+ kind: 'gguf', path: join(root, 'blobs', weights.replace(':', '-')),
+ name: host === 'registry.ollama.ai' ? short : `${host}/${short}`,
+ projector: projector ? join(root, 'blobs', projector.replace(':', '-')) : null,
+ });
+ }
+ return entries;
+}
+
+function hubName(path) {
+ const repo = path.split(sep).find(part => part.startsWith('models--'));
+ return repo ? repo.slice('models--'.length).replace('--', '/') : null;
+}
+
+async function describeFolder(dir) {
+ const config = JSON.parse(await readFile(join(dir, 'config.json'), 'utf8'));
+ let bytes = 0;
+ for (const name of await readdir(dir)) if (name.endsWith('.safetensors')) bytes += (await stat(join(dir, name))).size;
+ const mlx = Boolean(config.quantization?.bits) || /mlx/i.test(dir);
+ const template = await readFile(join(dir, 'chat_template.jinja'), 'utf8')
+ .catch(async () => JSON.parse(await readFile(join(dir, 'tokenizer_config.json'), 'utf8')).chat_template ?? '')
+ .catch(() => '');
+ return {
+ format: mlx ? 'mlx' : 'safetensors', bytes,
+ config: {
+ architecture: config.architectures?.[0] ?? config.model_type ?? null,
+ contextLength: config.max_position_embeddings ?? config.text_config?.max_position_embeddings ?? null,
+ quantization: config.quantization?.bits ? `${config.quantization.bits}-bit` : config.quantization_config?.quant_method ?? null,
+ kvBytesPerToken: kvBytesFromConfig(config),
+ toolCalls: typeof template === 'string' && template ? /\btools\b/.test(template) : null,
+ },
+ };
+}
+
+/** Every model file under the roots, one entry per real file; later sightings of a file become `alsoAt`. */
+export async function scanModels({ roots = defaultRoots(), minBytes = 50 * 1024 ** 2 } = {}) {
+ const byRealPath = new Map(), projectors = [], seenRoots = new Set(), scanned = [];
+ const add = async (source, found) => {
+ let realPath, info;
+ try { realPath = await realpath(found.path); info = await stat(realPath); } catch { return; }
+ const sighting = { source, path: found.path, name: found.name ?? null };
+ if (byRealPath.has(realPath)) { byRealPath.get(realPath).alsoAt.push(sighting); return; }
+ const entry = { name: found.name ?? null, source, path: found.path, realPath, format: found.kind, bytes: info.size,
+ quant: null, projector: found.projector ?? null, alsoAt: [], readableBy: [], gguf: null, config: null, error: null };
+ try {
+ if (found.kind === 'folder') {
+ Object.assign(entry, await describeFolder(realPath));
+ if (entry.bytes < minBytes) return;
+ } else {
+ if (info.size < minBytes) return;
+ const header = await readGguf(realPath);
+ if (!header) return;
+ entry.gguf = summarizeGguf(header);
+ entry.quant = basename(found.path).match(QUANT)?.[1]?.toUpperCase() ?? null;
+ }
+ } catch (error) { entry.error = error.message; }
+ entry.name ??= hubName(found.path) ?? basename(found.path).replace(/\.gguf$/i, '');
+ entry.readableBy = READERS[entry.format] ?? [];
+ byRealPath.set(realPath, entry);
+ };
+ for (const root of roots) {
+ let real;
+ try { real = await realpath(root.path); } catch { continue; }
+ if (seenRoots.has(real)) continue;
+ seenRoots.add(real); scanned.push(root.path);
+ if (root.source === 'ollama') { for (const found of await ollamaEntries(real)) await add('ollama', found); continue; }
+ await walk(root.path, root.depth ?? 6, async (kind, path) => {
+ if (kind === 'gguf' && /mmproj/i.test(basename(path))) projectors.push(path);
+ else await add(root.source, { kind, path });
+ }, { dirs: 20_000 });
+ }
+ const models = [...byRealPath.values()];
+ for (const path of projectors) {
+ const siblings = models.filter(m => m.format === 'gguf' && dirname(m.path) === dirname(path));
+ const stem = basename(path).split('.mmproj')[0];
+ const owner = siblings.find(m => basename(m.path).replace(/\.gguf$/i, '') === stem) ?? (siblings.length === 1 ? siblings[0] : null);
+ if (owner && !owner.projector) owner.projector = path;
+ }
+ return { models: models.sort((a, b) => b.bytes - a.bytes), scannedRoots: scanned };
+}
+
+export function parseListeningPorts(lsofOutput) {
+ const ports = new Set();
+ for (const line of lsofOutput.split('\n')) if (line.startsWith('n')) {
+ const port = Number(line.slice(line.lastIndexOf(':') + 1));
+ if (Number.isInteger(port) && port > 0) ports.add(port);
+ }
+ return [...ports].sort((a, b) => a - b);
+}
+
+/**
+ * Linux listening TCP ports from `/proc/net/tcp` and `/proc/net/tcp6` (state `0A`, port in hex after the
+ * last colon). A file read, so it works where `lsof` is missing — slim containers and many servers [run].
+ */
+export function parseProcNetTcp(text) {
+ const ports = new Set();
+ for (const line of text.split('\n')) {
+ const cols = line.trim().split(/\s+/);
+ if (cols[3] !== '0A') continue;
+ const port = parseInt(cols[1]?.slice(cols[1].lastIndexOf(':') + 1) ?? '', 16);
+ if (Number.isInteger(port) && port > 0) ports.add(port);
+ }
+ return [...ports].sort((a, b) => a - b);
+}
+
+/** Windows `netstat -ano` lines such as ` TCP [::]:11434 [::]:0 LISTENING 4812`. */
+export function parseNetstatListening(text) {
+ const ports = new Set();
+ for (const [, port] of text.matchAll(/^\s*TCP\s+\S*:(\d+)\s+\S+\s+LISTENING\b/gim)) ports.add(Number(port));
+ return [...ports].sort((a, b) => a - b);
+}
+
+async function listeningPorts(os) {
+ if (os === 'linux') {
+ const tables = await Promise.all(['/proc/net/tcp', '/proc/net/tcp6'].map(file => readFile(file, 'utf8').catch(() => '')));
+ return parseProcNetTcp(tables.join('\n'));
+ }
+ if (os === 'win32') return parseNetstatListening(await output('netstat', ['-ano']));
+ return parseListeningPorts(await output('lsof', ['-nP', '-iTCP', '-sTCP:LISTEN', '-Fn']));
+}
+
+function engineKind(body, ownedBy) {
+ if (Array.isArray(body.models) && body.data) return 'llama.cpp';
+ return { vllm: 'vllm', sglang: 'sglang', llamacpp: 'llama.cpp', organization_owner: 'lm-studio', library: 'ollama' }[ownedBy] ?? 'openai-compatible';
+}
+
+/** One GET per port. An answer with a model list is an engine; `url` is the exact `--at` form. */
+export async function probeEngines(ports, { host = '127.0.0.1', timeoutMs = 800, fetchImpl = globalThis.fetch } = {}) {
+ const probe = async port => {
+ const root = `http://${host}:${port}`;
+ try {
+ const response = await fetchImpl(`${root}/v1/models`, { signal: AbortSignal.timeout(timeoutMs) });
+ if (!response.ok) return null;
+ const body = await response.json();
+ const rows = Array.isArray(body.data) ? body.data : null;
+ if (!rows) return null;
+ let kind = engineKind(body, rows[0]?.owned_by);
+ if (kind === 'openai-compatible' || kind === 'ollama') {
+ const version = await fetchImpl(`${root}/api/version`, { signal: AbortSignal.timeout(timeoutMs) })
+ .then(r => (r.ok ? r.json() : null)).catch(() => null);
+ if (typeof version?.version === 'string') kind = 'ollama';
+ }
+ return { kind, port, url: `${root}/v1`, models: rows.map(row => row.id).filter(Boolean) };
+ } catch { return null; }
+ };
+ return (await Promise.all([...new Set(ports)].map(probe))).filter(Boolean);
+}
+
+const pidAlive = pid => { try { process.kill(pid, 0); return true; } catch (error) { return error.code === 'EPERM'; } };
+
+/**
+ * Grids this computer is joined to, from Grid's own run files rather than the relay: an asleep grid lists
+ * no engines, so an agent that read the relay once joined over a live engine [run]. Grid itself refuses a
+ * second `--serve` model per grid while its run record exists; an engine whose record is gone is invisible
+ * here too, and a join displaces it.
+ */
+export async function joinedGrids(gridHome, { alive = pidAlive } = {}) {
+ const dir = join(gridHome, 'run', 'engines');
+ const rows = [];
+ for (const entry of await readdir(dir, { withFileTypes: true }).catch(() => [])) {
+ if (!entry.isDirectory()) continue;
+ const state = await readFile(join(dir, entry.name, 'remote.json'), 'utf8').then(JSON.parse).catch(() => null);
+ if (!Number.isInteger(state?.pid) || !alive(state.pid)) continue;
+ rows.push({
+ gridId: state.grid_id ?? entry.name, grid: null, pid: state.pid, startedAt: state.started_at ?? null,
+ models: state.advertise_as?.length ? state.advertise_as : state.models ?? [],
+ at: state.engines?.find(engine => engine.endpoint_url)?.endpoint_url ?? state.endpoint_url ?? null,
+ });
+ }
+ return rows;
+}
+
+export function parseVmStat(text) {
+ const pageSize = Number(text.match(/page size of (\d+) bytes/)?.[1] || 4096);
+ const pages = label => Number(text.match(new RegExp(`Pages ${label}:\\s+(\\d+)`))?.[1] || 0);
+ return (pages('free') + pages('inactive') + pages('speculative') + pages('purgeable')) * pageSize;
+}
+
+export function parseSwapUsage(text) {
+ const mb = label => Number(text.match(new RegExp(`${label} = ([\\d.]+)M`))?.[1] || 0) * 1024 ** 2;
+ return { total: mb('total'), used: mb('used') };
+}
+
+export function parseMeminfo(text) {
+ const kb = label => Number(text.match(new RegExp(`^${label}:\\s+(\\d+) kB`, 'm'))?.[1] || 0) * 1024;
+ return { total: kb('MemTotal'), available: kb('MemAvailable'), swapUsed: kb('SwapTotal') - kb('SwapFree') };
+}
+
+/** `llama-server --list-devices`: each accelerator's memory budget as llama.cpp will use it. */
+export function parseListDevices(text) {
+ return [...text.matchAll(/^\s*(\w+): (.+?) \((\d+) MiB, (\d+) MiB free\)/gm)]
+ .filter(([, id, , total]) => Number(total) > 0 && !/^(CPU|BLAS)/.test(id))
+ .map(([, id, name, total, free]) => ({ id, name, totalBytes: Number(total) * 1024 ** 2, freeBytes: Number(free) * 1024 ** 2 }));
+}
+
+async function output(file, args, timeout = 10_000) {
+ try { return (await exec(file, args, { timeout, maxBuffer: 4 * 1024 * 1024 })).stdout; } catch { return ''; }
+}
+
+/** stdout and whether the command itself answered: a present binary that errors is "not active". */
+async function probe(file, args, timeout = 10_000) {
+ try { const { stdout, stderr } = await exec(file, args, { timeout, maxBuffer: 4 * 1024 * 1024 }); return { ran: true, ok: true, out: `${stdout}${stderr}` }; }
+ catch (error) { return { ran: error.code !== 'ENOENT', ok: false, out: `${error.stdout ?? ''}${error.stderr ?? ''}`.trim() || error.message }; }
+}
+
+export function parseNvidiaSmi(text) {
+ return text.split('\n').map(line => line.split(',').map(cell => cell.trim())).filter(cells => cells.length >= 6 && /^\d+$/.test(cells[0]))
+ .map(([index, name, total, used, utilization, driver]) => ({
+ vendor: 'nvidia', id: `CUDA${index}`, name, totalBytes: Number(total) * 1024 ** 2, usedBytes: Number(used) * 1024 ** 2,
+ utilizationPct: Number(utilization), driver, active: true,
+ }));
+}
+
+/**
+ * The accelerators that actually answer. Apple silicon: llama.cpp's Metal device list. NVIDIA: nvidia-smi
+ * must run and list the card; a card that lspci sees while nvidia-smi fails is reported inactive, with why.
+ */
+async function accelerators(llamaServer, os) {
+ let listed = parseListDevices((await probe(llamaServer, ['--list-devices'], 30_000)).out);
+ // Seen once on a Mac under memory pressure: an empty list that the next call fills. One retry, then report what is there.
+ if (!listed.length && (await stat(llamaServer).catch(() => null))) {
+ await new Promise(resolve => setTimeout(resolve, 1_000));
+ listed = parseListDevices((await probe(llamaServer, ['--list-devices'], 30_000)).out);
+ }
+ const found = listed
+ .map(device => ({ vendor: /^MTL/.test(device.id) ? 'apple' : /^CUDA/.test(device.id) ? 'nvidia' : /^(ROCm|HIP)/i.test(device.id) ? 'amd' : 'other', ...device, active: true, source: 'llama-server --list-devices' }));
+ const smi = await probe('nvidia-smi', ['--query-gpu=index,name,memory.total,memory.used,utilization.gpu,driver_version', '--format=csv,noheader,nounits']);
+ if (smi.ok) {
+ for (const gpu of parseNvidiaSmi(smi.out)) if (!found.some(d => d.vendor === 'nvidia' && d.name === gpu.name)) found.push({ ...gpu, source: 'nvidia-smi' });
+ } else if (smi.ran) found.push({ vendor: 'nvidia', active: false, source: 'nvidia-smi', error: smi.out.split('\n')[0].slice(0, 200) });
+ const rocm = await probe('rocm-smi', ['--showproductname', '--showmeminfo', 'vram', '--json']);
+ if (rocm.ok) found.push({ vendor: 'amd', active: true, source: 'rocm-smi', detail: rocm.out.slice(0, 400) });
+ else if (rocm.ran) found.push({ vendor: 'amd', active: false, source: 'rocm-smi', error: rocm.out.split('\n')[0].slice(0, 200) });
+ if (os === 'linux' && !found.some(d => d.vendor === 'nvidia')) {
+ const pci = (await probe('lspci', [])).out;
+ if (/nvidia/i.test(pci)) found.push({ vendor: 'nvidia', active: false, source: 'lspci', error: 'card present, nvidia-smi missing: driver not installed or not loaded' });
+ }
+ return found;
+}
+
+const firstLine = text => text.split('\n').map(line => line.trim()).find(Boolean) ?? null;
+
+/** Engines installed here (on PATH or in their standard install place), with the version each reports. */
+async function installedEngines(llamaServer, home) {
+ const engines = [];
+ const add = (kind, path, version, note) => engines.push({ kind, path, version: version || null, ...(note ? { note } : {}) });
+ const grid = await probe(llamaServer, ['--version']);
+ if (grid.ok) add('llama.cpp', llamaServer, firstLine(grid.out), "Grid's own engine");
+ // PATH first, then ~/.grid/envs/<engine>/bin, where the engine skills install Python engines.
+ const envs = join(home, '.grid', 'envs');
+ const windows = platform() === 'win32';
+ const onPath = async name => (windows
+ ? (await output('where', [name])).split(/\r?\n/)[0] // Windows has no `sh`; `where` prints each match
+ : await output('sh', ['-c', `command -v ${name}`])).trim();
+ const inEnv = async (name, env) => {
+ if (!env) return null;
+ const file = windows ? join(envs, env, 'Scripts', `${name}.exe`) : join(envs, env, 'bin', name); // a venv's layout
+ return (await stat(file).catch(() => null)) ? file : null;
+ };
+ const exists = async file => ((await stat(file).catch(() => null)) ? file : null);
+ const which = async (name, env) => (await onPath(name)) || inEnv(name, env);
+ const own = await which('llama-server');
+ if (own && own !== llamaServer) {
+ const answer = await probe(own, ['--version']);
+ add('llama.cpp', own, answer.ok ? firstLine(answer.out) : null, answer.ok ? null : 'on PATH but did not answer --version');
+ }
+ const ollama = (await which('ollama'))
+ || ((await exists('/Applications/Ollama.app')) ? '/Applications/Ollama.app/Contents/Resources/ollama' : null)
+ // Windows installs per user under %LOCALAPPDATA%\Programs\Ollama and adds it to PATH [doc: docs.ollama.com/windows].
+ || (process.env.LOCALAPPDATA ? await exists(join(process.env.LOCALAPPDATA, 'Programs', 'Ollama', 'ollama.exe')) : null);
+ if (ollama) add('ollama', ollama, (await probe(ollama, ['--version'])).out.match(/version is ([\w.-]+)/)?.[1]);
+ const lms = (await which('lms')) || (await exists(join(home, '.lmstudio', 'bin', 'lms')))
+ || (windows ? await exists(join(home, '.lmstudio', 'bin', 'lms.exe')) : null);
+ const studioApp = await stat('/Applications/LM Studio.app').catch(() => null);
+ if (lms || studioApp) add('lm-studio', lms ?? '/Applications/LM Studio.app', null, lms ? null : 'app installed; lms appears after its first launch');
+ const mlx = await which('mlx_lm.server', 'mlx-lm');
+ if (mlx) {
+ const python = (await readFile(mlx, 'utf8').catch(() => '')).match(/^#!(\S+)/)?.[1];
+ add('mlx-lm', mlx, python ? firstLine((await probe(python, ['-c', 'import mlx_lm; print(mlx_lm.__version__)'])).out) : null);
+ }
+ const vllm = await which('vllm', 'vllm');
+ if (vllm) add('vllm', vllm, firstLine((await probe(vllm, ['--version'], 60_000)).out));
+ const sglang = await which('sglang', 'sglang');
+ if (sglang) add('sglang', sglang, null);
+ return engines;
+}
+
+/** Which engines this hardware can run at all, from each engine's documented platforms. */
+export function runnableEngines(os, cpu, gpus) {
+ const active = vendor => gpus.some(gpu => gpu.vendor === vendor && gpu.active);
+ if (os === 'darwin' && cpu === 'arm64') return ['llama.cpp', 'mlx-lm', 'ollama', 'lm-studio'];
+ if (os === 'linux' && (active('nvidia') || active('amd'))) return ['llama.cpp', 'ollama', 'vllm', 'sglang'];
+ return ['llama.cpp', 'ollama'];
+}
+
+export async function machineState({ env = process.env, home = homedir() } = {}) {
+ const machine = { platform: platform(), arch: arch(), chip: null, memory: null, accelerators: [], engines: [], canRun: [], listeningPorts: [] };
+ machine.chip = (machine.platform === 'darwin'
+ ? (await output('sysctl', ['-n', 'machdep.cpu.brand_string'])).trim()
+ : (await readFile('/proc/cpuinfo', 'utf8').catch(() => '')).match(/^model name\s*:\s*(.+)$/m)?.[1])
+ // ARM Linux has no "model name" line and Windows no /proc; Node's own CPU list covers both — except
+ // on ARM Linux, where Node itself answers the word "unknown" [run].
+ || [cpus()[0]?.model?.trim()].find(model => model && model !== 'unknown') || null;
+ if (machine.platform === 'darwin') {
+ const total = Number((await output('sysctl', ['-n', 'hw.memsize'])).trim()) || null;
+ const swap = parseSwapUsage(await output('sysctl', ['-n', 'vm.swapusage']));
+ machine.memory = { totalBytes: total, availableBytes: parseVmStat(await output('vm_stat', [])), swapUsedBytes: swap.used,
+ note: 'available = free + inactive + speculative + purgeable pages; Metal and Grid do not see other apps, this does' };
+ } else if (machine.platform === 'linux') {
+ const info = parseMeminfo(await readFile('/proc/meminfo', 'utf8').catch(() => ''));
+ machine.memory = { totalBytes: info.total, availableBytes: info.available, swapUsedBytes: info.swapUsed };
+ }
+ if (!machine.memory?.totalBytes) {
+ // Windows and anything else: the OS's own figures through Node. Swap is not reported there.
+ machine.memory = { totalBytes: totalmem(), availableBytes: freemem(), swapUsedBytes: null, note: 'as the OS reports it' };
+ }
+ const llamaServer = join(env.GRID_HOME || join(home, '.grid'), 'bin', machine.platform === 'win32' ? 'llama-server.exe' : 'llama-server');
+ [machine.accelerators, machine.engines] = await Promise.all([accelerators(llamaServer, machine.platform), installedEngines(llamaServer, home)]);
+ if (machine.platform === 'darwin' && machine.arch === 'arm64' && !machine.accelerators.some(gpu => gpu.vendor === 'apple')) {
+ machine.accelerators.push({ vendor: 'apple', name: machine.chip, active: true, source: 'Apple silicon GPU (built in; budget unknown until llama.cpp is installed)' });
+ }
+ machine.canRun = runnableEngines(machine.platform, machine.arch, machine.accelerators);
+ machine.listeningPorts = await listeningPorts(machine.platform);
+ return machine;
+}
+
+const gb = bytes => (bytes === null || bytes === undefined ? '—' : `${(bytes / 1e9).toFixed(1)} GB`);
+
+/**
+ * One model's identity across formats and publishers: `Qwen3.5-9B-UD-Q4_K_XL` (GGUF) and
+ * `mlx-community/Qwen3.5-9B-4bit` (MLX) are both `qwen3.5-9b`. Only the packaging words go —
+ * quantization, bit width, the format — so a variant such as `-MTP` or `-Coder` stays its own model.
+ */
+export function modelFamily(name) {
+ return String(name).split('/').pop().toLowerCase()
+ .replace(/\.gguf$/, '')
+ .split(/[-_.](?=(?:ud|i?q\d[a-z0-9_]*|\d+bit|bf16|f16|f32|fp16|mxfp4|nvfp4|gguf|mlx|awq|gptq)(?:[-_.]|$))/)[0];
+}
+
+/**
+ * The inventory as a short table an agent reads directly, so it never writes its own filter over the
+ * JSON (an agent's hand-written `jq` once dropped every MLX model and reported none on disk).
+ */
+export function summarize({ machine, engines, models, joined = [] }, contextTokens = 65536) {
+ const mem = machine.memory ?? {};
+ const gpus = machine.accelerators.map(g => `${g.vendor}${g.name ? ` ${g.name}` : ''} ${g.active ? 'active' : `INACTIVE (${g.error ?? 'no answer'})`}${g.totalBytes ? ` ${gb(g.totalBytes)}` : ''}`);
+ const lines = [
+ `machine ${machine.chip ?? '?'} · ${machine.platform}/${machine.arch} · memory ${gb(mem.totalBytes)}, available ${gb(mem.availableBytes)}, swap used ${gb(mem.swapUsedBytes)}`,
+ `gpu ${gpus.join(' · ') || 'none'}`,
+ `installed ${machine.engines.map(e => `${e.kind}${e.version ? ` ${e.version}` : ''}${e.note ? ` (${e.note})` : ''}`).join(' · ') || 'none'}`,
+ `can run ${machine.canRun.join(', ')}`,
+ `answering ${engines.map(e => `${e.kind} ${e.url} [${e.models.slice(0, 3).join(', ')}]`).join(' · ') || 'none'}`,
+ `joined ${joined.map(j => `${j.grid ?? j.gridId} serving [${j.models.join(', ')}]${j.at ? ` at ${j.at}` : ''}`).join(' · ') || 'none'}${joined.length ? ' — a join to that grid adds to what it serves (Grid 0.3.53+)' : ''}`,
+ `ports in use ${machine.listeningPorts.join(' ')}`,
+ '',
+ `FORMAT SIZE CONTEXT CACHE@${contextTokens / 1024}K TOOLS VISION READ BY MODEL (where)`,
+ ];
+ // On Apple silicon with an MLX engine installed, a GGUF that also exists as an MLX folder says so on
+ // its own row: an agent handed the rule "prefer the MLX copy" still served the GGUF, because nothing
+ // told it the two names were one model [run].
+ const mlxHere = machine.canRun.includes('mlx-lm') && machine.engines.some(e => e.kind === 'mlx-lm' || e.kind === 'lm-studio');
+ const mlxCopies = new Map();
+ if (mlxHere) for (const m of models) if (m.format === 'mlx') mlxCopies.set(modelFamily(m.name), m.name);
+ for (const m of models) {
+ const facts = m.gguf ?? m.config ?? {};
+ const kv = facts.kvBytesPerToken;
+ const runnable = m.readableBy.some(reader => machine.canRun.includes(reader));
+ const mlxCopy = m.format === 'gguf' ? mlxCopies.get(modelFamily(m.name)) : null;
+ const note = [runnable ? '' : `no engine this machine can run reads ${m.format}${m.format === 'mlx' ? ' (Apple silicon only)' : ''}`,
+ mlxCopy ? `an MLX copy is on disk (${mlxCopy}): serve that one on this Mac unless vision is needed` : '',
+ m.gguf?.unsupportedTensorTypes?.length ? `llama.cpp cannot load (tensor type ${m.gguf.unsupportedTensorTypes.join(', ')})` : '',
+ facts.contextLength && facts.contextLength < contextTokens ? `context ${facts.contextLength} < ${contextTokens}` : '', m.error ?? ''].filter(Boolean).join('; ');
+ lines.push([
+ m.format.padEnd(11), gb(m.bytes).padStart(8), String(facts.contextLength ?? '—').padStart(8),
+ (kv ? gb(kv * contextTokens) : '—').padStart(10), String(facts.toolCalls ?? '?').padEnd(5), (m.projector ? 'yes' : 'no').padEnd(6),
+ m.readableBy.join(',').padEnd(27), `${m.name} (${m.source}${m.alsoAt.length ? `, also ${m.alsoAt.map(a => a.source).join('/')}` : ''})${note ? ` ! ${note}` : ''}`,
+ ].join(' '));
+ }
+ return lines.join('\n');
+}
+
+/** The whole picture for one start decision: memory now, engines answering, and models on disk. */
+export async function inventory({ extraRoots = [], extraPorts = [], env = process.env, home = homedir() } = {}) {
+ const machine = await machineState({ env, home });
+ const roots = [...defaultRoots(env, home), ...extraRoots.map(path => ({ source: 'folder', path }))];
+ const ports = [...KNOWN_PORTS, ...machine.listeningPorts.filter(port => port >= 1024), ...extraPorts];
+ const [engines, disk, joined] = await Promise.all([probeEngines(ports), scanModels({ roots }), joinedGrids(env.GRID_HOME || join(home, '.grid'))]);
+ if (joined.length) {
+ // Run files carry the grid id; the name the agent uses comes from the CLI's own list.
+ let grids = [];
+ try { grids = JSON.parse(await output(env.GRID_CLI || 'grid', ['ls', '--json'], 8_000)); } catch {}
+ for (const row of joined) row.grid = (Array.isArray(grids) ? grids : []).find(g => g.id === row.gridId)?.grid ?? null;
+ }
+ return { machine, engines, joined, ...disk };
+}
diff --git a/store/agents/autonomous-grid/lib/modelFacts.mjs b/store/agents/autonomous-grid/lib/modelFacts.mjs
new file mode 100644
index 000000000..11e0e472b
--- /dev/null
+++ b/store/agents/autonomous-grid/lib/modelFacts.mjs
@@ -0,0 +1,98 @@
+// What a model says about itself, for the case no engine recipe covers it: the files every Hugging Face
+// model ships (config, generation defaults, chat template, model card), read from the Hub or a local
+// folder, plus whether each engine's official supported-model list names its architecture. Facts only.
+import { readFile, stat } from 'node:fs/promises';
+import { join } from 'node:path';
+
+const HUB = 'https://huggingface.co';
+const FILES = ['config.json', 'generation_config.json', 'tokenizer_config.json', 'chat_template.jinja', 'chat_template.json', 'README.md'];
+const VLLM_SUPPORTED = 'https://raw.githubusercontent.com/vllm-project/vllm/main/docs/models/supported_models.md';
+// SGLang's docs name families, not architectures; its model registry is one file per model_type with an EntryClass.
+const SGLANG_MODELS = 'https://raw.githubusercontent.com/sgl-project/sglang/main/python/sglang/srt/models';
+const SERVE = /\b(vllm serve|sglang serve|sglang\.launch_server|llama-server|mlx_lm\.server|ollama (run|create|pull)|lms (get|load))\b/;
+const SAMPLING = ['temperature', 'top_p', 'top_k', 'min_p', 'repetition_penalty', 'presence_penalty'];
+
+const json = text => { try { return JSON.parse(text); } catch { return null; } };
+
+async function hubReader(hfId, { fetchImpl, token }) {
+ const headers = token ? { authorization: `Bearer ${token}` } : {};
+ const response = await fetchImpl(`${HUB}/api/models/${hfId}`, { headers, signal: AbortSignal.timeout(20_000) });
+ if (response.status === 401 || response.status === 404) return { missing: true };
+ if (!response.ok) throw new Error(`huggingface.co answered HTTP ${response.status} for ${hfId}`);
+ const info = await response.json();
+ const present = new Set((info.siblings ?? []).map(file => file.rfilename));
+ return {
+ gated: Boolean(info.gated), present,
+ read: async name => {
+ if (!present.has(name)) return null;
+ const file = await fetchImpl(`${HUB}/${hfId}/raw/main/${name}`, { headers, signal: AbortSignal.timeout(20_000) });
+ return file.ok ? file.text() : null;
+ },
+ };
+}
+
+function localReader(dir) {
+ return { gated: false, read: name => readFile(join(dir, name), 'utf8').catch(() => null) };
+}
+
+/** A few lines of the chat template around its tool-call markup: compare them with the engine's parser docs. */
+function toolSyntax(template) {
+ const at = template.search(/<tool_call>|\[TOOL_CALLS\]|<\|python_tag\|>|tool_calls|<function|functools|tool▁call/);
+ if (at < 0) return null;
+ return template.slice(Math.max(0, at - 160), at + 320).trim();
+}
+
+function codeBlocks(markdown) {
+ return [...markdown.matchAll(/```[^\n]*\n([\s\S]*?)```/g)].map(match => match[1].trim());
+}
+
+/** `listed`: true = named by the engine's own list, false = its full list omits it, null = this check cannot tell. */
+async function support(architecture, modelType, fetchImpl) {
+ const text = url => fetchImpl(url, { signal: AbortSignal.timeout(20_000) }).then(r => (r.ok ? r.text() : null)).catch(() => null);
+ const vllmList = architecture ? await text(VLLM_SUPPORTED) : null;
+ const vllm = vllmList === null
+ ? { listed: null, source: VLLM_SUPPORTED, note: architecture ? 'list unreachable' : 'config.json names no architecture' }
+ : { listed: vllmList.split('\n').some(line => line.includes(`\`${architecture}\``)), source: VLLM_SUPPORTED,
+ lines: vllmList.split('\n').filter(line => line.includes(`\`${architecture}\``)).map(line => line.trim().slice(0, 240)).slice(0, 3) };
+ const registryFile = modelType && /^[\w.-]+$/.test(modelType) ? `${SGLANG_MODELS}/${modelType}.py` : null;
+ const registry = registryFile && architecture ? await text(registryFile) : null;
+ const sglang = registry?.includes(architecture)
+ ? { listed: true, source: registryFile }
+ : { listed: null, source: registryFile ?? SGLANG_MODELS,
+ note: 'not found in the registry file for this model_type; it may live in another file, or run through --model-impl transformers' };
+ return { vllm, sglang };
+}
+
+export async function modelFacts(target, { fetchImpl = globalThis.fetch, token = process.env.HF_TOKEN, checkSupport = true } = {}) {
+ const isLocal = await stat(target).then(info => info.isDirectory()).catch(() => false);
+ if (!isLocal && !/^[\w.-]+\/[\w.-]+$/.test(target || '')) throw new Error('Pass a Hugging Face id (ORG/NAME) or a model folder');
+ const reader = isLocal ? localReader(target) : await hubReader(target, { fetchImpl, token });
+ const source = isLocal ? target : `${HUB}/${target}`;
+ if (reader.missing) return { source, found: false, note: 'Hugging Face has no such public model, or it is private: check the id; a private model needs HF_TOKEN.' };
+ const files = Object.fromEntries(await Promise.all(FILES.map(async name => [name, await reader.read(name)])));
+ if (reader.gated && !files['config.json']) {
+ return { source, found: true, gated: true, note: 'Gated model: the person must accept its license on Hugging Face and provide HF_TOKEN. Ask; do not work around it.' };
+ }
+ const config = json(files['config.json']) ?? {}, text = config.text_config ?? {};
+ const generation = json(files['generation_config.json']) ?? {};
+ const tokenizer = json(files['tokenizer_config.json']) ?? {};
+ const template = files['chat_template.jinja'] ?? json(files['chat_template.json'])?.chat_template
+ ?? (typeof tokenizer.chat_template === 'string' ? tokenizer.chat_template : tokenizer.chat_template?.map?.(t => t.template).join('\n')) ?? '';
+ const card = files['README.md'] ?? '';
+ const architecture = config.architectures?.[0] ?? null, modelType = config.model_type ?? null;
+ const flagValues = flag => [...new Set([...card.matchAll(new RegExp(`--${flag}[ =]+([A-Za-z0-9_.-]+)`, 'g'))].map(match => match[1]))];
+ return {
+ source, found: true, gated: reader.gated,
+ architecture, modelType,
+ contextLength: config.max_position_embeddings ?? text.max_position_embeddings ?? null,
+ dtype: config.torch_dtype ?? config.dtype ?? text.torch_dtype ?? null,
+ quantization: config.quantization_config?.quant_method ?? (config.quantization?.bits ? `mlx ${config.quantization.bits}-bit` : null),
+ samplingDefaults: Object.fromEntries(SAMPLING.filter(key => generation[key] !== undefined).map(key => [key, generation[key]])),
+ chatTemplate: template ? { toolCalls: /\btools\b/.test(template), toolSyntax: toolSyntax(template), thinking: /<think>|enable_thinking/.test(template) } : null,
+ modelCard: {
+ serveCommands: codeBlocks(card).filter(block => SERVE.test(block)).slice(0, 8),
+ toolCallParsers: flagValues('tool-call-parser'), reasoningParsers: flagValues('reasoning-parser'),
+ },
+ support: checkSupport ? await support(architecture, modelType, fetchImpl) : null,
+ };
+}
diff --git a/store/agents/autonomous-grid/lib/recipes.mjs b/store/agents/autonomous-grid/lib/recipes.mjs
new file mode 100644
index 000000000..cb43eb39e
--- /dev/null
+++ b/store/agents/autonomous-grid/lib/recipes.mjs
@@ -0,0 +1,95 @@
+// Official serving recipes, read live from the engines' own sites. When a site has no entry for the
+// model this throws NoRecipe: the caller says so and never invents flags.
+const VLLM_SITE = 'https://recipes.vllm.ai';
+const SGLANG_INDEX = 'https://docs.sglang.io/llms.txt';
+
+export class NoRecipe extends Error {
+ constructor(message) { super(message); this.code = 'NO_RECIPE'; }
+}
+
+const squash = text => text.toLowerCase().replace(/[^a-z0-9]/g, '');
+
+async function get(fetchImpl, url, as) {
+ const response = await fetchImpl(url, { signal: AbortSignal.timeout(20_000) });
+ if (response.status === 404) return null;
+ if (!response.ok) throw new Error(`${new URL(url).host} answered HTTP ${response.status} for ${url}`);
+ return as === 'json' ? response.json() : response.text();
+}
+
+function checkId(hfId) {
+ if (!/^[\w.-]+\/[\w.-]+$/.test(hfId || '')) throw new Error('Pass the Hugging Face id: ORG/NAME');
+}
+
+/** vLLM's per-model recipe: minimum version, always-on args, features (tool calls, reasoning, opt-ins), variants. */
+export async function vllmRecipe(hfId, { fetchImpl = globalThis.fetch } = {}) {
+ checkId(hfId);
+ const catalogue = await get(fetchImpl, `${VLLM_SITE}/models.json`, 'json');
+ if (!Array.isArray(catalogue)) throw new Error('recipes.vllm.ai/models.json did not return a list');
+ const entry = catalogue.find(row => row.hf_id?.toLowerCase() === hfId.toLowerCase());
+ if (!entry) throw new NoRecipe(`recipes.vllm.ai has no recipe for ${hfId}.`);
+ const recipe = await get(fetchImpl, `${VLLM_SITE}${entry.json}`, 'json');
+ if (!recipe) throw new NoRecipe(`recipes.vllm.ai lists ${entry.hf_id} but its recipe file is missing.`);
+ const model = recipe.model ?? {}, optIn = new Set(recipe.opt_in_features ?? []);
+ const recommended = recipe.recommended_command ?? {};
+ return {
+ engine: 'vllm', source: `${VLLM_SITE}${entry.url}`, hfId: entry.hf_id,
+ minVersion: model.min_vllm_version ?? null, contextLength: Number(model.context_length) || null,
+ architecture: model.architecture ?? null, parameters: model.parameter_count ?? null, activeParameters: model.active_parameters ?? null,
+ baseArgs: model.base_args ?? [], baseEnv: model.base_env ?? {},
+ features: Object.fromEntries(Object.entries(recipe.features ?? {}).map(([name, feature]) =>
+ [name, { optIn: optIn.has(name), args: feature.args ?? [], env: feature.env ?? {}, description: feature.description ?? '' }])),
+ variants: Object.fromEntries(Object.entries(recipe.variants ?? {}).map(([name, variant]) => [name, {
+ modelId: variant.model_id ?? model.model_id ?? entry.hf_id, precision: variant.precision ?? null,
+ vramMinimumGb: variant.vram_minimum_gb ?? null, extraArgs: variant.extra_args ?? [], description: variant.description ?? '',
+ }])),
+ install: model.install ?? null,
+ recommended: { hardware: recommended.hardware ?? null, variant: recommended.variant ?? null, command: recommended.command ?? null, dockerImage: recommended.docker_image ?? null },
+ guide: recipe.guide ?? '',
+ };
+}
+
+/** A heading's text up to the next heading of the same or a higher level; `#` lines inside code fences are not headings. */
+function section(markdown, heading) {
+ const lines = markdown.split('\n'), wanted = new RegExp(`^(#{2,3}) .*${heading}`, 'i');
+ let fenced = false, level = 0;
+ const kept = [];
+ for (const line of lines) {
+ if (/^```/.test(line)) fenced = !fenced;
+ const depth = !fenced && !/^```/.test(line) ? line.match(/^(#{1,6}) /)?.[1].length : undefined;
+ if (!level) { if (depth && wanted.test(line)) { level = depth; kept.push(line); } continue; }
+ if (depth && depth <= level) break;
+ kept.push(line);
+ }
+ return kept.join('\n').trim();
+}
+
+/** SGLang's cookbook page for the model's family: its serve commands, parsers, install and tuning notes. */
+export async function sglangCookbook(hfId, { fetchImpl = globalThis.fetch } = {}) {
+ checkId(hfId);
+ const index = await get(fetchImpl, SGLANG_INDEX, 'text');
+ if (!index) throw new Error('docs.sglang.io/llms.txt is unavailable');
+ const pages = [...index.matchAll(/\]\((https:\/\/docs\.sglang\.io\/cookbook\/autoregressive\/([^/)]+)\/([^/)]+)\.md)\)/g)]
+ .map(([, url, org, family]) => ({ url, org, family }));
+ const name = squash(hfId.split('/')[1]);
+ const fits = pages.filter(page => squash(page.family) && name.startsWith(squash(page.family)));
+ const longest = Math.max(0, ...fits.map(page => squash(page.family).length));
+ const best = fits.filter(page => squash(page.family).length === longest);
+ if (!best.length) throw new NoRecipe(`The SGLang cookbook has no page for ${hfId}.`);
+ const page = best[0];
+ const markdown = await get(fetchImpl, page.url, 'text');
+ if (!markdown) throw new NoRecipe(`${page.url} is listed but missing.`);
+ const blocks = [...markdown.matchAll(/```[^\n]*\n([\s\S]*?)```/g)].map(match => match[1].trim());
+ const values = flag => [...new Set([...markdown.matchAll(new RegExp(`--${flag}[ =]+([A-Za-z0-9_.-]+)`, 'g'))].map(match => match[1]))];
+ return {
+ engine: 'sglang', source: page.url, family: page.family, alsoMatching: best.slice(1).map(other => other.url),
+ toolCallParsers: values('tool-call-parser'), reasoningParsers: values('reasoning-parser'),
+ serveCommands: blocks.filter(block => /sglang serve|sglang\.launch_server/.test(block)).slice(0, 6),
+ installation: section(markdown, 'Installation'), configurationTips: section(markdown, 'Configuration Tips'),
+ };
+}
+
+export async function recipe(engine, hfId, options) {
+ if (engine === 'vllm') return vllmRecipe(hfId, options);
+ if (engine === 'sglang') return sglangCookbook(hfId, options);
+ throw new Error('Use: fleet recipe vllm|sglang ORG/NAME');
+}
diff --git a/store/agents/autonomous-grid/lib/remoteModels.mjs b/store/agents/autonomous-grid/lib/remoteModels.mjs
new file mode 100644
index 000000000..b168d4faf
--- /dev/null
+++ b/store/agents/autonomous-grid/lib/remoteModels.mjs
@@ -0,0 +1,130 @@
+// `fleet models` for a machine other than this one. The finder reads a machine's own disk and asks its
+// own loopback ports, so it has to run there: over SSH it is copied to the machine and run by that
+// machine's Node. Without Node 18+ there, a plain `find` still lists the model files, and says what is
+// missing. A Harness link runs Grid commands only, so it cannot run the finder at all.
+import { spawn } from 'node:child_process';
+import { mkdir, readFile, readlink, stat, symlink } from 'node:fs/promises';
+import { homedir } from 'node:os';
+import { dirname, isAbsolute, join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+import { shellQuote } from './fleet.mjs';
+
+const HERE = dirname(fileURLToPath(import.meta.url));
+export const FINDER_FILES = ['inventory.mjs', 'candidates.mjs', 'finderMain.mjs'];
+const DIR = '.cache/grid-harness/finder'; // relative to the remote home, where an SSH command starts
+const NO_NODE = '__GRID_FINDER_NO_NODE__';
+
+// The same roots the finder scans (inventory.mjs `defaultRoots`), for the listing without Node. Sizes
+// in kilobytes: BusyBox `find` (Alpine) has no `M` suffix [run].
+const FALLBACK = [
+ 'for d in ~/.grid/models ~/.ollama/models /usr/share/ollama/.ollama/models ~/.lmstudio/models',
+ '~/.cache/lm-studio/models "${HF_HUB_CACHE:-${HF_HOME:-$HOME/.cache/huggingface}/hub}" ~/models ~/Models ~/Downloads;',
+ 'do [ -d "$d" ] && find -L "$d" -maxdepth 8 \\( -iname "*.gguf" -size +51200k -o -name config.json',
+ '-o -path "*/manifests/*" -type f \\) 2>/dev/null; done | head -300',
+].join(' ');
+
+/** SSH options every call shares: the user's existing trust, never a prompt (as `invocation` in fleet.mjs). */
+export function sshBase(machine) {
+ return ['-o', 'BatchMode=yes', '-o', 'StrictHostKeyChecking=yes', '-o', 'ConnectTimeout=8',
+ ...(machine.port ? ['-p', String(machine.port)] : []), machine.host];
+}
+
+function sshRun(machine, remote, { input = null, timeoutMs = 90_000, spawnImpl = spawn } = {}) {
+ return new Promise(resolve => {
+ let stdout = '', stderr = '', done = false;
+ const child = spawnImpl('ssh', [...sshBase(machine), remote], { stdio: ['pipe', 'pipe', 'pipe'] });
+ const finish = result => { if (!done) { done = true; clearTimeout(timer); resolve({ stdout, stderr, ...result }); } };
+ const timer = setTimeout(() => { child.kill('SIGTERM'); finish({ code: 124, error: `no answer from ${machine.host} within ${timeoutMs / 1000}s` }); }, timeoutMs);
+ child.stdout.setEncoding('utf8').on('data', chunk => { stdout += chunk; });
+ child.stderr.setEncoding('utf8').on('data', chunk => { stderr += chunk; });
+ child.once('error', error => finish({ code: 127, error: error.code === 'ENOENT' ? 'ssh is not installed here' : error.message }));
+ child.once('close', code => finish({ code: code ?? 1, error: code === 0 ? null : stderr.trim().split('\n').pop() || `ssh exited ${code}` }));
+ child.stdin.end(input ?? '');
+ });
+}
+
+/**
+ * The finder's output for `machine`: `{ text, full }`, where `full` says whether it is the finder's own
+ * table (Node there) or only a file list. Throws with the reason when the machine cannot be read.
+ */
+export async function remoteModels(machine, { summary = true, spawnImpl = spawn } = {}) {
+ if (machine.transport === 'harness') {
+ throw new Error(`${machine.name || machine.id} is linked through Harness, which runs Grid commands only, so its model `
+ + `files cannot be listed from here. Its hardware: fleet run --machine ${machine.id} -- device-info --json`);
+ }
+ if (machine.transport !== 'ssh') throw new Error(`fleet models --machine needs an SSH machine; ${machine.id} is ${machine.transport}.`);
+ const call = (remote, options = {}) => sshRun(machine, remote, { spawnImpl, ...options });
+ for (const file of FINDER_FILES) {
+ const sent = await call(`mkdir -p ${DIR} && cat > ${DIR}/${file}`, { input: await readFile(join(HERE, file), 'utf8'), timeoutMs: 30_000 });
+ if (sent.code !== 0) throw new Error(`Could not copy the finder to ${machine.host}: ${sent.error}`);
+ }
+ // A number, not a boolean: newer Node throws on `process.exit(true)`, which would read as "no Node".
+ const node = 'command -v node >/dev/null 2>&1 && node -e "process.exit(Number(+process.versions.node.split(\'.\')[0] < 18))"';
+ const run = await call(`if ${node}; then exec node ${DIR}/finderMain.mjs${summary ? ' --summary' : ''}; `
+ + `else echo ${NO_NODE}; uname -sm; ${FALLBACK}; fi`);
+ if (run.code !== 0) throw new Error(`The finder failed on ${machine.host}: ${run.error}`);
+ if (!run.stdout.startsWith(NO_NODE)) return { text: run.stdout.trimEnd(), full: true };
+ const [, system, ...files] = run.stdout.trimEnd().split('\n');
+ return {
+ full: false,
+ text: [`machine ${system} (${machine.host}) — Node 18+ is not installed there, so this is a file list only:`,
+ 'no context, memory fit, tool calls, engines or ports. Install Node there for the full table.', '',
+ ...(files.length ? files : ['(no model files found)'])].join('\n'),
+ };
+}
+
+const GGUF_NAME = /^[A-Za-z0-9][A-Za-z0-9._-]{0,200}\.gguf$/;
+
+/**
+ * Make a GGUF already on a machine servable by Grid there, without copying it: `join --serve` reads only
+ * `~/.grid/models` and keeps just the file name [run], so the file is linked in. A projector is linked as
+ * `<name>.mmproj.gguf`, the name `--serve` finds it by [run]. An existing name is never replaced.
+ */
+export async function linkModel(machine, file, name, { projector, env = process.env, spawnImpl = spawn } = {}) {
+ if (!GGUF_NAME.test(name)) throw new Error('NAME is a plain file name ending in .gguf.');
+ const pairs = [[file, name], ...(projector ? [[projector, name.replace(/\.gguf$/, '.mmproj.gguf')]] : [])];
+ if (pairs.some(([source]) => !isAbsolute(source))) throw new Error('Give FILE (and --projector) as full paths on that machine, as `fleet models` prints them.');
+ if (machine.transport === 'harness') throw new Error(`${machine.name || machine.id} is linked through Harness, which runs Grid commands only; a model for it comes from the catalog (pull).`);
+ if (machine.transport === 'local') {
+ const dir = join(env.GRID_HOME || join(homedir(), '.grid'), 'models');
+ await mkdir(dir, { recursive: true });
+ const lines = [];
+ for (const [source, target] of pairs) {
+ if (!(await stat(source).catch(() => null))?.isFile()) throw new Error(`No file at ${source}.`);
+ const at = join(dir, target);
+ const current = await readlink(at).catch(error => (error.code === 'ENOENT' ? null : '(not a link)'));
+ if (current === source) { lines.push(`already linked ${at}`); continue; }
+ if (current !== null) throw new Error(`${at} already exists and is not a link to ${source}; choose another NAME.`);
+ await symlink(source, at);
+ lines.push(`linked ${at} -> ${source}`);
+ }
+ const engine = join(env.GRID_HOME || join(homedir(), '.grid'), 'bin', process.platform === 'win32' ? 'llama-server.exe' : 'llama-server');
+ if (!(await stat(engine).catch(() => null))) lines.push(noEngine(machine.id).replace(` --machine ${machine.id}`, ''));
+ return { text: lines.join('\n') };
+ }
+ if (machine.transport !== 'ssh') throw new Error(`Unknown transport for ${machine.id}.`);
+ const script = [
+ ...(machine.gridHome ? [`GRID_HOME=${shellQuote(machine.gridHome)}; export GRID_HOME;`] : []),
+ 'd="${GRID_HOME:-$HOME/.grid}/models"; mkdir -p "$d" || exit 5;',
+ 'link() { if [ ! -f "$1" ]; then echo "No file at $1." >&2; exit 3; fi;',
+ 'if [ -L "$d/$2" ] && [ "$(readlink "$d/$2")" = "$1" ]; then echo "already linked $d/$2";',
+ 'elif [ -e "$d/$2" ] || [ -L "$d/$2" ]; then echo "$d/$2 already exists and is not a link to $1; choose another NAME." >&2; exit 4;',
+ 'else ln -s "$1" "$d/$2" && echo "linked $d/$2 -> $1"; fi; };',
+ ...pairs.map(([source, target]) => `link ${shellQuote(source)} ${shellQuote(target)} || exit $?;`),
+ // The join after this needs Grid's own engine there, and needs it to RUN: without it the join still
+ // says "starting" and the engine dies at once in a log on that machine — "llama-server not found", or
+ // "libgomp.so.1: cannot open shared object file" on a minimal Ubuntu [run].
+ 'e="${GRID_HOME:-$HOME/.grid}/bin/llama-server"; [ -x "$e" ] || e="$(command -v llama-server 2>/dev/null)";',
+ `if [ -z "$e" ]; then echo ${shellQuote(noEngine(machine.id))};`,
+ 'elif ! err="$("$e" --version 2>&1)"; then',
+ `echo "NOT READY: Grid's engine is installed on this machine but does not start: $(printf %s "$err" | head -1)";`,
+ `echo " A missing lib*.so is a system package there (libgomp.so.1 = libgomp1); tell the person which one — installing it is theirs to do."; fi;`,
+ ].join(' ');
+ const result = await sshRun(machine, script, { spawnImpl, timeoutMs: 30_000 });
+ if (result.code !== 0) throw new Error(result.stderr.trim() || `Could not link on ${machine.host}: ${result.error}`);
+ return { text: result.stdout.trimEnd() };
+}
+
+const noEngine = id => `NOT READY: Grid's engine (llama.cpp) is not installed on this machine, so a join here would die at start. `
+ + `Ask first (a download), then: "$GRID_FLEET" run --machine ${id} -- engine install llama.cpp`;
diff --git a/store/agents/autonomous-grid/lib/serve.mjs b/store/agents/autonomous-grid/lib/serve.mjs
new file mode 100644
index 000000000..54c3b26ed
--- /dev/null
+++ b/store/agents/autonomous-grid/lib/serve.mjs
@@ -0,0 +1,56 @@
+// Engines other than Grid's own, started so they outlive the command that started them. An agent's
+// shell tears down what it spawned when the command returns — a `nohup … &` mlx-lm died that way with an
+// empty log — so the engine runs as the leader of its own session, logging to run/NAME.log, with its PID
+// in run/NAME.pid. Stop kills only a PID this file recorded.
+import { spawn } from 'node:child_process';
+import { mkdir, open, readFile, rm, writeFile } from 'node:fs/promises';
+import { join } from 'node:path';
+
+const NAME = /^[a-z0-9][a-z0-9._-]{0,62}$/;
+
+export function runFiles(workspace, name) {
+ if (!NAME.test(name || '')) throw new Error('The engine name is lowercase letters, digits, dot, dash or underscore.');
+ const dir = join(workspace, 'run');
+ return { dir, log: join(dir, `${name}.log`), pid: join(dir, `${name}.pid`) };
+}
+
+const alive = pid => { try { process.kill(pid, 0); return true; } catch (error) { return error.code === 'EPERM'; } };
+
+export async function recordedPid(workspace, name) {
+ const pid = Number((await readFile(runFiles(workspace, name).pid, 'utf8').catch(() => '')).trim());
+ return Number.isInteger(pid) && pid > 1 ? pid : null;
+}
+
+/** Start `argv` detached: own session and process group, stdio to the log, PID recorded. */
+export async function serve(workspace, name, argv, { env = {}, spawnImpl = spawn } = {}) {
+ if (!Array.isArray(argv) || !argv.length) throw new Error('Give the engine command after --.');
+ const files = runFiles(workspace, name);
+ const running = await recordedPid(workspace, name);
+ if (running && alive(running)) throw new Error(`${name} is already running as PID ${running}; stop it first.`);
+ await mkdir(files.dir, { recursive: true });
+ const log = await open(files.log, 'a');
+ try {
+ await log.write(`\n=== ${new Date().toISOString()} ${argv.join(' ')}\n`);
+ const child = spawnImpl(argv[0], argv.slice(1), {
+ cwd: workspace, detached: true, stdio: ['ignore', log.fd, log.fd], env: { ...process.env, ...env },
+ });
+ await new Promise((resolve, reject) => { child.once('spawn', resolve); child.once('error', reject); });
+ child.unref();
+ await writeFile(files.pid, `${child.pid}\n`);
+ return { name, pid: child.pid, log: files.log };
+ } finally { await log.close(); }
+}
+
+/** Stop the PID recorded for NAME: TERM, then KILL after `graceMs`; never any other process. */
+export async function stop(workspace, name, { graceMs = 10_000 } = {}) {
+ const files = runFiles(workspace, name);
+ const pid = await recordedPid(workspace, name);
+ if (!pid) return { name, stopped: false, note: `no PID recorded for ${name}` };
+ if (!alive(pid)) { await rm(files.pid, { force: true }); return { name, pid, stopped: true, note: 'already gone' }; }
+ process.kill(-pid, 'SIGTERM');
+ const deadline = Date.now() + graceMs;
+ while (alive(pid) && Date.now() < deadline) await new Promise(resolve => setTimeout(resolve, 250));
+ if (alive(pid)) process.kill(-pid, 'SIGKILL');
+ await rm(files.pid, { force: true });
+ return { name, pid, stopped: true };
+}
diff --git a/store/agents/autonomous-grid/lib/verify.mjs b/store/agents/autonomous-grid/lib/verify.mjs
index d6ab8b40c..248308eac 100644
--- a/store/agents/autonomous-grid/lib/verify.mjs
+++ b/store/agents/autonomous-grid/lib/verify.mjs
@@ -47,3 +47,133 @@ export async function verifyModel(workspace, machine, mode, grid, model, { run =
/^(Could not read|The model|The model endpoint)/.test(error.message) ? error.message : 'The model check was interrupted. Ask Model Manager to retry.');
} finally { clearInterval(heartbeat); }
}
+
+// Acceptance for a served model, step by step, every step bounded and every wait narrated: the engine
+// (ready, listed, answers, calls a tool, speed) and then, with a grid, the relay (listed, answers).
+// Nothing here waits without a deadline, and nothing is reported as working that was not seen working.
+const TOOL = { type: 'function', function: { name: 'read_file', description: 'Read a text file from the project', parameters: { type: 'object', properties: { path: { type: 'string' } }, required: ['path'] } } };
+// chat_template_kwargs is read by llama.cpp, mlx-lm, vLLM and SGLang; Ollama documents reasoning_effort "none".
+const noThinking = kind => (kind === 'ollama' ? { reasoning_effort: 'none' } : { chat_template_kwargs: { enable_thinking: false } });
+
+export function createLog(write = line => process.stderr.write(`${line}\n`), clock = Date.now) {
+ const start = clock();
+ return message => write(`[+${String(Math.round((clock() - start) / 1000)).padStart(4)}s] ${message}`);
+}
+
+async function call(fetchImpl, url, { body, headers = {}, timeoutMs, log, label }) {
+ const started = Date.now();
+ const heartbeat = setInterval(() => log(`${label}: still waiting (${Math.round((Date.now() - started) / 1000)}s of ${Math.round(timeoutMs / 1000)}s)`), 15_000);
+ try {
+ const response = await fetchImpl(url, {
+ method: body ? 'POST' : 'GET', headers: { 'content-type': 'application/json', ...headers },
+ body: body ? JSON.stringify(body) : undefined, signal: AbortSignal.timeout(timeoutMs),
+ });
+ const text = await response.text();
+ let json = null;
+ try { json = JSON.parse(text); } catch {}
+ return { status: response.status, json, text: text.slice(0, 300), seconds: (Date.now() - started) / 1000 };
+ } catch (error) {
+ return { status: 0, json: null, text: error.name === 'TimeoutError' ? `no answer within ${Math.round(timeoutMs / 1000)}s` : error.message, seconds: (Date.now() - started) / 1000 };
+ } finally { clearInterval(heartbeat); }
+}
+
+const message = json => json?.choices?.[0]?.message ?? {};
+const reasoningOf = json => message(json).reasoning_content || message(json).reasoning || '';
+const THINKING_ROOM = 1024;
+
+/**
+ * The one bounded "does it answer?" request. Some engines keep a model thinking whatever the request
+ * says (LM Studio with a reasoning MLX model ignored every switch [run]), so 16 tokens come back as
+ * reasoning only. That model still works — it answers after thinking — so ask once more with room for
+ * the thinking, and say that it stays on, instead of failing a working engine.
+ */
+async function answerOnce(fetchImpl, url, { body, log, label, timeoutMs, headers }) {
+ const first = await call(fetchImpl, url, { timeoutMs, log, label, headers, body });
+ const content = (message(first.json).content ?? '').trim();
+ if (content || !reasoningOf(first.json)) return { answer: first, content, thinkingStaysOn: false };
+ log(` ${label}: only thinking came back in ${body.max_tokens} tokens; asking once more with room for it (${THINKING_ROOM} tokens)`);
+ const second = await call(fetchImpl, url, { timeoutMs, log, label, headers, body: { ...body, max_tokens: THINKING_ROOM } });
+ return { answer: second, content: (message(second.json).content ?? '').trim(), thinkingStaysOn: true };
+}
+
+/** The engine itself, at its own `/v1` URL. */
+const wait = ms => new Promise(resolve => setTimeout(resolve, ms));
+
+export async function verifyEngine({ url, model, kind, readyMs = 180_000, answerMs = 300_000, tools = true, fetchImpl = globalThis.fetch, log = createLog(), sleep = wait }) {
+ const base = url.replace(/\/+$/, '');
+ const steps = [];
+ const record = (name, ok, detail) => { steps.push({ name, ok, ...detail }); log(`${ok ? 'PASS' : 'FAIL'} ${name}${detail.note ? ` — ${detail.note}` : ''}`); return ok; };
+ if (!/\/v1$/.test(base)) return { ok: false, steps: [{ name: 'url', ok: false, note: 'the URL must end in /v1' }] };
+
+ log(`step 1/4 ready: GET ${base}/models (up to ${readyMs / 1000}s, every 3s)`);
+ const deadline = Date.now() + readyMs;
+ let listed = null, attempt = 0;
+ while (Date.now() < deadline) {
+ attempt++;
+ const models = await call(fetchImpl, `${base}/models`, { timeoutMs: 5_000, log, label: 'models' });
+ const ids = (models.json?.data ?? []).map(row => row.id);
+ if (models.status === 200) { listed = ids; break; }
+ if (attempt === 1 || attempt % 5 === 0) log(` attempt ${attempt}: ${models.status || models.text}`);
+ await sleep(3_000);
+ }
+ if (!record('ready', listed !== null, { note: listed === null ? `no 200 from /models within ${readyMs / 1000}s` : `${listed.length} model(s) listed` })) return { ok: false, steps };
+ if (!record('listed', listed.includes(model), { note: listed.includes(model) ? model : `${model} not in ${JSON.stringify(listed.slice(0, 8))}` })) return { ok: false, steps };
+
+ log(`step 2/4 answer: one request, max_tokens 16, thinking off (up to ${answerMs / 1000}s)`);
+ const { answer, content, thinkingStaysOn } = await answerOnce(fetchImpl, `${base}/chat/completions`, { timeoutMs: answerMs, log, label: 'answer', body: {
+ model, max_tokens: 16, temperature: 0, messages: [{ role: 'user', content: 'Reply with the single word: ok' }], ...noThinking(kind) } });
+ const thinkingOnly = !content && reasoningOf(answer.json);
+ if (!record('answer', Boolean(content), { seconds: answer.seconds, note: content
+ ? `${JSON.stringify(content.slice(0, 40))}${thinkingStaysOn ? ` — thinking stays on here: it answers after thinking, slower and with more tokens` : ''}`
+ : thinkingOnly ? `only reasoning text came back, even with ${THINKING_ROOM} tokens: thinking is still on` : `HTTP ${answer.status}: ${answer.text}` })) return { ok: false, steps };
+
+ if (tools) {
+ log('step 3/4 tool call: a read_file tool, "Open the file README.md."');
+ const tool = await call(fetchImpl, `${base}/chat/completions`, { timeoutMs: answerMs, log, label: 'tool call', body: {
+ model, max_tokens: 256, temperature: 0, tools: [TOOL], messages: [{ role: 'user', content: 'Open the file README.md.' }], ...noThinking(kind) } });
+ const called = message(tool.json).tool_calls?.[0]?.function;
+ if (!record('tool call', called?.name === 'read_file', { seconds: tool.seconds, note: called ? `${called.name}(${called.arguments})` : `no tool_calls; HTTP ${tool.status}` })) return { ok: false, steps };
+ } else log('step 3/4 tool call: skipped');
+
+ log('step 4/4 speed: 128 tokens');
+ const speed = await call(fetchImpl, `${base}/chat/completions`, { timeoutMs: answerMs, log, label: 'speed', body: {
+ model, max_tokens: 128, temperature: 0, messages: [{ role: 'user', content: 'Count from 1 to 60 in words.' }], ...noThinking(kind) } });
+ const reported = speed.json?.timings?.predicted_per_second;
+ const tokens = speed.json?.usage?.completion_tokens;
+ const perSecond = reported ?? (tokens ? tokens / speed.seconds : null);
+ record('speed', perSecond !== null, { seconds: speed.seconds, tokensPerSecond: perSecond && Math.round(perSecond * 10) / 10,
+ note: perSecond ? `${perSecond.toFixed(1)} tokens/s${reported ? ' (engine timing)' : ' (wall clock, includes reading the prompt)'}` : `HTTP ${speed.status}` });
+ return { ok: steps.every(step => step.ok), steps };
+}
+
+/** After a join: the relay must list the alias, then answer through the grid's own endpoint. */
+export async function verifyRelay({ alias, engineModel, listModels, relay, listMs = 300_000, answerMs = 420_000, fetchImpl = globalThis.fetch, log = createLog(), sleep = wait }) {
+ const steps = [];
+ const record = (name, ok, detail) => { steps.push({ name, ok, ...detail }); log(`${ok ? 'PASS' : 'FAIL'} ${name}${detail.note ? ` — ${detail.note}` : ''}`); return ok; };
+ log(`relay 1/2 listed: grid models, every 10s, up to ${listMs / 1000}s`);
+ const deadline = Date.now() + listMs;
+ let seen = false, attempt = 0, last = '';
+ // The relay changes a name's case (an alias given in lower case came back upper case, and the other
+ // way round), so names match whatever their case [run: 5 minutes lost waiting on a case difference].
+ const same = (a, b) => typeof a === 'string' && typeof b === 'string' && a.toLowerCase() === b.toLowerCase();
+ while (Date.now() < deadline) {
+ attempt++;
+ const listed = await listModels();
+ const found = listed.ok && listed.models.find(name => same(name, alias));
+ if (found) { seen = true; alias = found; break; }
+ // Registered under the engine's own id: waiting cannot fix a missing --advertise-as [run: 5 minutes lost].
+ const raw = listed.ok && engineModel && listed.models.find(name => same(name, engineModel));
+ if (raw) return (record('relay listed', false, { note: `listed as ${raw}, not ${alias}: join again with --advertise-as ${alias}` }), { ok: false, steps });
+ last = listed.ok ? `${listed.models.length} model(s), not yet ${alias}: ${listed.models.slice(0, 5).map(name => String(name).slice(0, 60)).join(', ') || 'none'}` : listed.error;
+ log(` attempt ${attempt}: ${last}`);
+ await sleep(10_000);
+ }
+ if (!record('relay listed', seen, { note: seen ? alias : `not listed within ${listMs / 1000}s (${last})` })) return { ok: false, steps };
+ log(`relay 2/2 answer through the grid (up to ${answerMs / 1000}s: a new engine is probed first and may hold its slot for minutes)`);
+ const { answer, content, thinkingStaysOn } = await answerOnce(fetchImpl, `${relay.baseUrl.replace(/\/+$/, '')}/chat/completions`, {
+ timeoutMs: answerMs, log, label: 'relay answer', headers: relay.apiKey ? { authorization: `Bearer ${relay.apiKey}` } : {},
+ body: { model: alias, max_tokens: 16, messages: [{ role: 'user', content: 'Reply with the single word: ok' }], ...noThinking() } });
+ record('relay answer', Boolean(content), { seconds: answer.seconds, note: content
+ ? `${JSON.stringify(content.slice(0, 40))}${thinkingStaysOn ? ' — thinking stays on' : ''}` : `HTTP ${answer.status}: ${answer.text}` });
+ return { ok: steps.every(step => step.ok), steps };
+}
diff --git a/store/agents/autonomous-grid/skills/engine-lm-studio/SKILL.md b/store/agents/autonomous-grid/skills/engine-lm-studio/SKILL.md
new file mode 100644
index 000000000..66581a59e
--- /dev/null
+++ b/store/agents/autonomous-grid/skills/engine-lm-studio/SKILL.md
@@ -0,0 +1,105 @@
+---
+name: engine-lm-studio
+description: "Reuse what LM Studio already has on this computer through its `lms` CLI: adopt its running server into the person's fleet, serve its MLX models, or serve its GGUF files without it. Load before touching LM Studio, its models or its server."
+---
+
+# LM Studio (`lms` CLI, no clicks)
+
+Official docs, read 2026-09-29 (source files in github.com/lmstudio-ai/docs):
+[App, llmster and lms](https://lmstudio.ai/docs/app/basics/lmstudio-vs-llmster-vs-lms) ·
+[Headless](https://lmstudio.ai/docs/developer/core/headless) · [lms server start](https://lmstudio.ai/docs/cli/serve/server-start) ·
+[lms server stop](https://lmstudio.ai/docs/cli/serve/server-stop) · [lms load](https://lmstudio.ai/docs/cli/local-models/load) ·
+[lms import](https://lmstudio.ai/docs/cli/local-models/import) · [lms daemon up/down](https://lmstudio.ai/docs/cli/daemon/daemon-up) ·
+[Idle TTL](https://lmstudio.ai/docs/developer/core/ttl-and-auto-evict) · [OpenAI compatibility](https://lmstudio.ai/docs/developer/openai-compat) ·
+[Tools](https://lmstudio.ai/docs/developer/openai-compat/tools) · [Structured output](https://lmstudio.ai/docs/developer/openai-compat/structured-output) ·
+[Import models](https://lmstudio.ai/docs/app/advanced/import-model) · [Server settings](https://lmstudio.ai/docs/developer/core/server/settings)
+Tested: LM Studio 0.4.25 (`brew install --cask lm-studio`), MacBook Pro M1 Pro 32 GB, macOS 26.6, 2026-09-29.
+Tags: `[doc]` official page above, `[run]` seen on the tested machine, `[?]` unverified.
+
+## When to use it
+
+- **Its server is answering** (`fleet models` → `kind: lm-studio`): adopt it, but load the model
+ yourself with a 64K+ context (below) — a model it loads on demand gets 8192 [run], too small for an agent.
+- **An MLX model sits in LM Studio's folder and mlx-lm is not installed**: LM Studio runs MLX on Apple
+ silicon (its structured output uses Outlines for MLX [doc]), so use it instead of installing anything.
+- **A GGUF in LM Studio's folder**: Grid's own engine serves it in place; LM Studio need not run.
+- Never install LM Studio just to serve a model: Grid's engine covers GGUF, mlx-lm covers MLX.
+
+## Where its models live / how to list them
+
+- `~/.lmstudio/models/<publisher>/<model>/<file>` — it keeps Hugging Face's layout [doc]. The folder can be
+ moved in the app's My Models tab [doc]; no documented setting file or env var [?]. Older versions used
+ `~/.cache/lm-studio/models` [?]. `fleet models` scans both.
+- `lms ls --json` → `modelKey`, `path`, `format` (gguf/mlx), `sizeBytes` [run]. Keys prefix-match: a short
+ key matched two downloaded models and loaded "the first one" [run] — always pass the exact key.
+- A fresh install holds only an embedding model, no chat model [run].
+
+## Installed? Running?
+
+- App: `/Applications/LM Studio.app`. `lms` lives in `~/.lmstudio/bin` and appears only after the app ran
+ once [run]. Headless servers use **llmster** instead of the app: `curl -fsSL https://lmstudio.ai/install.sh | bash` [doc].
+- `lms server status` → "The server is not running." or its port [run]. Launching the app does not
+ start the server [run]. `fleet models` lists a running one as `kind: lm-studio` (its `owned_by` is
+ `organization_owner` [run]).
+
+## Start an already-downloaded model
+
+Port P from outside `machine.listeningPorts`. Always `--port`: without it the last used port is reused [doc].
+
+ export PATH="$HOME/.lmstudio/bin:$PATH"
+ lms server start --port P --bind 127.0.0.1 # starts LM Studio in the background if needed [doc][run]
+ lms load <exact key> --estimate-only --context-length 65536 --gpu max -y # fits? [doc][run]
+ lms load <exact key> --context-length 65536 --gpu max --identifier <id> -y
+
+- With the app closed, `server start` printed "Waking up LM Studio service..." and was up in 4 s; it runs
+ the app as `LM Studio --run-as-service`, with no window [run]. It returns at once (not a foreground process).
+- `--estimate-only` prints GPU and total memory for that context and whether guardrails allow it [doc][run].
+- A model loaded with `lms load` has no idle timer and stays until unloaded [doc]; on-demand loads unload
+ after 60 minutes idle [doc]. Nothing downloads on load [run].
+
+## Ready means
+
+Run `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <id> --kind lm-studio` — it performs the
+request checks with deadlines and prints each one. Also:
+
+1. `lms server status` shows port P [run]. 2. `GET http://127.0.0.1:P/v1/models` lists `<id>` [doc][run].
+3. `lms ps` shows `<id>` with CONTEXT ≥ 65536 [run]. 4. One bounded `/v1/chat/completions` request with
+`model: <id>`, `max_tokens` 16 → non-empty `content` [run].
+A 401 means the person turned on "Require Authentication" [doc]: do not change their setting; say so and
+serve the file with Grid's engine instead.
+
+## Join Harness Compute
+
+ "$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <id> --advertise-as ALIAS --name NAME
+
+`/v1` is required: with the bare root, Grid's capability probe records structured output as unsupported [run].
+Grid's own detector finds LM Studio only on 1234 [run].
+
+## Tool calls, JSON output, thinking
+
+- Tools: `tools` on `/v1/chat/completions` and `/v1/responses`, OpenAI format [doc]. Grid's probe: tools yes [run].
+- Structured output: `response_format` with `json_schema` [doc]; GGUF uses llama.cpp grammars, MLX uses
+ Outlines [doc]. Grid's probe: json_schema yes, json_object no [run].
+- Thinking control through the OpenAI endpoint: not documented [?]. Check `content` in the ready request.
+
+## Memory and speed knobs
+
+`--context-length` (≥ 65536 here), `--gpu max|off|0–1` (automatic when omitted) [doc], `--ttl <seconds>` [doc],
+`--parallel <n>` (default 4 on this build [run]), `--speculative-draft-mtp` for models with MTP layers [run: in help].
+
+## Stop
+
+- Your model: `lms unload <id>` [doc]. Your server: `lms server stop` [doc] — the background service keeps running [run].
+- The service you woke: `lms daemon down` stops llmster only, not the app [doc]; the app service ends with
+ `osascript -e 'quit app "LM Studio"'` [run].
+- If LM Studio was running before you came, unload only your model and leave the rest as it was.
+
+## Known failures → what to do
+
+| Sign | Do |
+|---|---|
+| "2 models match the provided model key" | use the exact `modelKey` from `lms ls --json` [run] |
+| CONTEXT 8192 in `lms ps` | the model was loaded on demand; unload it and `lms load` with `--context-length 65536` [run] |
+| `lms get … -y` shows a spinner forever | it downloads in the background [run]; never start a download without the go-ahead |
+| a foreign GGUF must appear in LM Studio | `lms import -l -y --user-repo <user>/<repo> FILE.gguf`: **without `-l` it moves the file** [doc]; the file name must end in `.gguf` and the link must point at the real file, not a temp copy [run] |
+| 401 from `/v1/models` | authentication is on in the person's settings [doc]; do not turn it off |
diff --git a/store/agents/autonomous-grid/skills/engine-mlx-lm/SKILL.md b/store/agents/autonomous-grid/skills/engine-mlx-lm/SKILL.md
new file mode 100644
index 000000000..b497a721e
--- /dev/null
+++ b/store/agents/autonomous-grid/skills/engine-mlx-lm/SKILL.md
@@ -0,0 +1,121 @@
+---
+name: engine-mlx-lm
+description: "Serve an MLX or Hugging Face safetensors model already on this Mac with mlx-lm's server and join it to the person's fleet. Load before installing, starting or stopping mlx-lm, or when `fleet models` lists an `mlx` or `safetensors` model on Apple silicon."
+---
+
+# mlx-lm server (Apple silicon)
+
+Official docs, read 2026-09-29:
+[SERVER.md](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md) ·
+[server.py (flags, routes)](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/server.py) ·
+[README](https://github.com/ml-explore/mlx-lm/blob/main/README.md) ·
+[Hugging Face cache variables](https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables) ·
+[Hugging Face cache layout](https://huggingface.co/docs/huggingface_hub/guides/manage-cache)
+Tested: mlx-lm 0.31.3 (the latest on PyPI that day), uv venv with Python 3.12, MacBook Pro M1 Pro 32 GB,
+macOS 26.6, 2026-09-29. Tags: `[doc]` official source above, `[run]` seen on the tested machine, `[?]` unverified.
+
+## When to use it
+
+- `fleet models` lists a model with `format: mlx` (an MLX-converted folder: `config.json` with a `quantization` block) on
+ a Mac: this is its engine. Grid's own engine reads only GGUF.
+- `format: safetensors` (a plain Hugging Face model) on a Mac: mlx-lm served one directly [run]; vLLM and
+ SGLang are for NVIDIA/AMD servers.
+- If LM Studio is installed and already holds the MLX model, LM Studio can serve it with nothing new
+ installed (`engine-lm-studio`). If a GGUF of the same model exists, prefer Grid's engine.
+- The docs call the server "not recommended for production" (basic security only) [doc]: bind 127.0.0.1.
+- A GGUF of the same model already on disk is served by Grid's engine; do not download an MLX copy of it.
+- mlx-lm installed and no MLX model yet: `"$GRID_FLEET" candidates mlx` (see `run-local-model`).
+- Verified end to end [run]: started offline on a local snapshot, `fleet verify` passed ready, answer,
+ tool call and speed; `grid join … --at http://127.0.0.1:P/v1 -m <snapshot path> --advertise-as NAME`
+ joined in 6 s and the relay answered through the grid.
+
+## Where its models live / how to list them
+
+- Hugging Face cache: `$HF_HUB_CACHE`, else `$HF_HOME/hub`, else `~/.cache/huggingface/hub` [doc].
+ Layout `models--<org>--<name>/snapshots/<commit>/` with files linked into `blobs/` [doc].
+- An MLX folder has `config.json` with a `quantization` block (`bits`, `group_size`) or sits in an
+ `mlx-community` repo; a plain model has `config.json` and `*.safetensors` without it. `fleet models`
+ reports both, including folders outside the cache (`~/models`, LM Studio's folder).
+- Server side: `GET /v1/models` lists every cached repo as `org/name` plus the served path [doc][run].
+
+## Installed? Running?
+
+- Installed: `fleet models` lists `mlx-lm` under installed engines — on `PATH` or in `~/.grid/envs/mlx-lm`.
+ Install there and nowhere else, never into the system Python (a slow step: ask for the go-ahead first):
+ `uv venv --python 3.12 ~/.grid/envs/mlx-lm && uv pip install --python ~/.grid/envs/mlx-lm/bin/python mlx-lm`
+ [run] (`pip install mlx-lm` or `conda install -c conda-forge mlx-lm` [doc]). ENV below is that folder.
+- Running: `fleet models` → an engine on 8080 labelled `openai-compatible` (it has no `owned_by`) whose
+ models are Hugging Face ids [run].
+
+## Start an already-downloaded model (never downloads)
+
+Port P from outside `machine.listeningPorts` (8090 was taken by another app during testing [run]):
+
+ "$GRID_FLEET" serve mlx-P --env HF_HUB_OFFLINE=1 -- ENV/bin/mlx_lm.server \
+ --model <snapshot dir or cached org/name> --host 127.0.0.1 --port P --max-tokens 32768 \
+ --chat-template-args '{"enable_thinking":false}'
+
+- `fleet serve` starts it in its own session, so it outlives your shell; log in `run/mlx-P.log`, PID in
+ `run/mlx-P.pid`. Under the agent's shell a plain `nohup … &` died with an empty log when the command
+ returned, while a `fleet serve` process was still alive in a later command [run]. Never `launchd`.
+
+- `HF_HUB_OFFLINE=1`: no HTTP calls, cached files only, an error if missing [doc]. Without it an uncached
+ model is downloaded from Hugging Face [doc].
+- `--max-tokens` defaults to 512 [doc]: a request without its own limit would stop mid-answer.
+- Sampling defaults are greedy (`--temp 0.0`, `--top-p 1.0`) [doc]; set `--temp`/`--top-p`/`--top-k` from
+ the model card's recommended settings.
+- Ready log: `Starting httpd at 127.0.0.1 on port P...` [run].
+- There is no context size to set: the cache grows with the conversation, up to the model's
+ `max_position_embeddings`. Check that it is ≥ 65536 and that weights + 64K of cache fit before starting.
+
+## Ready means
+
+Run `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <path or id> --kind mlx-lm` right after
+`serve` — it waits for loading itself; no `sleep`, `curl` or log reading first. What it checks:
+
+1. The PID is alive. 2. `GET /health` → `{"status": "ok"}` [run] (in the source, not in SERVER.md [doc]).
+3. `GET /v1/models` → 200 [doc]. 4. One bounded `/v1/chat/completions` request, `max_tokens` 16, `model`
+ = the path or id you started with → non-empty `content` [run].
+
+## Join Harness Compute
+
+ "$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <path or id you started with> --advertise-as ALIAS --name NAME
+
+`/v1` is required: `/models` without it answers 404 (only chat accepts both) [doc][run]. Without
+`--advertise-as` the picker shows the whole snapshot path, and `verify --alias` waits five minutes for a
+name that never appears [run]. Grid's detector labels anything on 8080 as `mlx`, whatever it is [run].
+
+## Tool calls, JSON output, thinking
+
+- Tools: passed to the chat template; when the tokenizer has no tool-calling support the server logs
+ "Received tools but model does not support tool calling" and answers without calls [doc]. Grid's probe
+ reported no tool calls for a 0.5B 4-bit model whose template does mention tools [run]: run the tool
+ check before offering a model for coding.
+- Thinking: `--chat-template-args '{"enable_thinking":false}'` at start, or `chat_template_kwargs` per
+ request [doc]. Reasoning text comes back in `message.reasoning` [doc].
+- JSON: Grid's probe reported JSON object and schema modes as supported [run].
+
+## Memory and speed knobs
+
+| Flag | Default | Use |
+|---|---|---|
+| `--decode-concurrency` | 32 | requests decoded together [doc] |
+| `--prompt-concurrency` | 8 | prompts prefilled together [doc] |
+| `--prefill-step-size` | 2048 | lower it if prefill spikes memory [doc] |
+| `--prompt-cache-size` / `--prompt-cache-bytes` | 10 caches / unlimited | cap memory kept for reuse [doc] |
+| `--kv-bits 4\|8` | off | smaller cache for long context, but one request at a time [doc]; newer than 0.31.3 [run: absent from its help] |
+| `--draft-model`, `--num-draft-tokens` | none, 3 | speculative decoding [doc] |
+
+## Stop
+
+`"$GRID_FLEET" stop mlx-P`, then confirm P is gone from `fleet models --summary` (ports in use).
+
+## Known failures → what to do
+
+| Sign | Do |
+|---|---|
+| `OfflineModeIsEnabled` or file-not-found at start | the model is not fully cached; offer the download as a slow step [doc] |
+| answers stop around 512 tokens | start with `--max-tokens` [doc] |
+| empty `content`, text in `reasoning` | disable thinking in the chat template [doc] |
+| "Received tools but model does not support tool calling" | not a coding model; pick another [doc] |
+| address already in use | pick another port; never stop the other process [run] |
diff --git a/store/agents/autonomous-grid/skills/engine-ollama/SKILL.md b/store/agents/autonomous-grid/skills/engine-ollama/SKILL.md
new file mode 100644
index 000000000..32a7fbfcb
--- /dev/null
+++ b/store/agents/autonomous-grid/skills/engine-ollama/SKILL.md
@@ -0,0 +1,131 @@
+---
+name: engine-ollama
+description: "Reuse what Ollama already has on this computer: adopt a running Ollama server into the person's fleet, or serve an Ollama-downloaded model without Ollama. Load before touching Ollama, its models or its settings."
+---
+
+# Ollama
+
+Official docs, read 2026-09-29 (source files in github.com/ollama/ollama/tree/main/docs):
+[FAQ](https://docs.ollama.com/faq) · [Context length](https://docs.ollama.com/context-length) ·
+[OpenAI compatibility](https://docs.ollama.com/api/openai-compatibility) ·
+[Thinking](https://docs.ollama.com/capabilities/thinking) · [CLI](https://docs.ollama.com/cli) ·
+[macOS](https://docs.ollama.com/macos) · [Linux](https://docs.ollama.com/linux) · [Import](https://docs.ollama.com/import) ·
+[API reference](https://github.com/ollama/ollama/blob/main/docs/openapi.yaml)
+Tested: Ollama 0.34.4 (`brew install ollama`), MacBook Pro M1 Pro 32 GB, macOS 26.6, 2026-09-29.
+Tags: `[doc]` official page above, `[run]` seen on the tested machine, `[?]` unverified.
+
+## When to use it
+
+- **Ollama is already answering** (`fleet models` lists `kind: ollama`): adopt it as it is; it is the
+ person's app, so never restart or reconfigure it without asking.
+- **Ollama is installed but stopped, or its settings do not fit the job**: do not start it. Its models
+ are GGUF files that Grid's engine serves in place — link the blob into `~/.grid/models` and
+ `join --serve` [run]. That also gives the context you choose instead of Ollama's default.
+- Start Ollama yourself only when Grid's engine refuses a blob Ollama runs (Ollama ships its own engine
+ for some new architectures [?]); then use the start below.
+- Never `ollama create` to reuse a file from another app: it copies the whole file into Ollama's blobs
+ (+624 MB for a 640 MB GGUF) [run]. Never `ollama pull` without the person's go-ahead: it downloads.
+
+## MLX inside Ollama (Apple silicon)
+
+Since 0.19 Ollama has its own MLX engine on Apple silicon, announced as a preview on 2026-03-30
+([blog](https://ollama.com/blog/mlx)) [doc]. It runs on MLX only the architectures registered in its
+source — read the list live, never from memory:
+`https://raw.githubusercontent.com/ollama/ollama/main/mlxrunner/model/architectures/architectures.go`
+(one import per architecture folder under `mlxrunner/model/`) [doc]. Compare with `model_type` from
+`fleet model-facts`. Everything else keeps running on Ollama's GGML engine. The announced models are
+Ollama-library tags in their own quantization, and the post asks for more than 32 GB of unified memory [doc];
+importing other MLX models is not yet documented [?].
+
+## Where its models live / how to list them
+
+- Default dir: macOS `~/.ollama/models`, Linux service `/usr/share/ollama/.ollama/models`, Windows
+ `C:\Users\%username%\.ollama\models`; `OLLAMA_MODELS` moves it [doc].
+- Layout: `manifests/registry.ollama.ai/library/<model>/<tag>` (a JSON file) and `blobs/sha256-<hex>`;
+ the manifest layer `application/vnd.ollama.image.model` names the weights blob, a GGUF [run]. Model id is
+ `<model>:<tag>`; other namespaces are `<user>/<model>:<tag>` [?].
+- `fleet models` already reads the manifests: entries with `source: ollama`, the blob path, and
+ `alsoAt` when another app links the same file [run].
+- From a running server: `GET /api/tags` (downloaded), `GET /api/ps` (loaded, with `context_length` and
+ `size_vram`) [doc], `GET /v1/models` (`owned_by` is the Ollama user, `library` by default) [doc][run].
+ `POST /api/show {"model":ID}` lists `capabilities` (completion, tools, thinking, vision) [doc].
+
+## Installed? Running?
+
+- Installed: `command -v ollama` or `/Applications/Ollama.app`. `ollama --version` with no server prints
+ "Warning: could not connect to a running Ollama instance" and the client version [run].
+- The macOS app starts the server at login (a login item) [doc]; `brew install ollama` starts nothing [run];
+ the Linux installer creates a systemd service `ollama` [doc].
+- Running: `fleet models` → `engines[]` with `kind: ollama`. Default bind is 127.0.0.1:11434 [doc].
+
+## Start an already-downloaded model (only when Grid's engine refused it)
+
+Port P from outside `machine.listeningPorts`; the port is set only through `OLLAMA_HOST` (no `--port`) [doc]:
+
+ "$GRID_FLEET" serve ollama-P --env OLLAMA_HOST=127.0.0.1:P --env OLLAMA_CONTEXT_LENGTH=65536 -- ollama serve
+
+(`fleet serve` keeps it alive after your shell returns; log `run/ollama-P.log`, PID `run/ollama-P.pid`.)
+
+- Foreground process; the log says `Listening on 127.0.0.1:P (version …)` when ready [run].
+- Every other `ollama` command must carry the same `OLLAMA_HOST`, or it talks to 11434 [run].
+- Nothing downloads unless you pull; the model loads on its first request [run].
+- Context: the default depends on GPU memory — 4K below 24 GiB, 32K at 24–48 GiB, 256K at 48 GiB or
+ more [doc] (the FAQ still says 4096 [doc]). Agents need at least 64000 [doc], and every model here is
+ used by an agent: always set `OLLAMA_CONTEXT_LENGTH` to 65536 or more. The OpenAI API cannot set
+ context per request [doc].
+
+## Ready means
+
+Run `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <model>:<tag> --kind ollama` — it performs
+these checks with deadlines and prints each one. What it checks:
+
+1. The PID is alive. 2. `GET /` answers "Ollama is running" [run]. 3. `GET /api/version` → 200 [run].
+4. One bounded request through `/v1/chat/completions` with the model id, `max_tokens` 16 and
+ `"reasoning_effort":"none"` for a thinking model → non-empty `content` [doc][run].
+Then `GET /api/ps` must show the model with `context_length` ≥ 65536 [doc]; less is not started.
+
+## Join Harness Compute
+
+ "$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <model>:<tag> --advertise-as ALIAS --name NAME
+
+`/v1` is required: without it `/models` and `/chat/completions` answer 404, and Grid's capability probe
+records JSON output as unsupported without any error [run]. Grid's own detector finds Ollama only on
+11434 [run].
+
+## Tool calls, JSON output, thinking
+
+- `/v1/chat/completions` supports `tools`, `response_format` and `reasoning_effort` [doc].
+ `reasoning_effort: "none"` asks for no thinking; the native API uses `"think": false` [doc].
+- A model's thinking values and default: `/api/show` → `thinking.values`, `thinking.default` [doc].
+- Tool support is per model: check `capabilities` contains `tools` before offering it for coding [doc].
+
+## Memory and speed knobs (server environment, apply to all models)
+
+| Variable | Default | Effect |
+|---|---|---|
+| `OLLAMA_CONTEXT_LENGTH` | by GPU memory (above) | context for every model [doc] |
+| `OLLAMA_NUM_PARALLEL` | 1 | requests at once per model; memory scales with parallel × context [doc] |
+| `OLLAMA_KV_CACHE_TYPE` | `f16` | `q8_0` ≈ half the KV memory, `q4_0` ≈ a quarter; needs flash attention [doc] |
+| `OLLAMA_FLASH_ATTENTION` | automatic | `1` forces on, `0` off [doc] |
+| `OLLAMA_KEEP_ALIVE` | 5m | how long an idle model stays loaded [doc] |
+| `OLLAMA_MAX_LOADED_MODELS` | 3 × GPUs (3 on CPU) | models loaded at once [doc] |
+
+`brew services start ollama` sets `OLLAMA_FLASH_ATTENTION=1` and `OLLAMA_KV_CACHE_TYPE=q8_0` [run: brew caveat].
+`ollama ps` → `PROCESSOR` must read `100% GPU`; a CPU/GPU split is slow [doc].
+
+## Stop
+
+- A server you started: `"$GRID_FLEET" stop ollama-P`.
+- Unload one model but keep the server: `OLLAMA_HOST=… ollama stop <model>` [doc].
+- The person's app or service: ask first. macOS app: quit from the menu bar; Linux: `sudo systemctl stop ollama` [doc].
+
+## Known failures → what to do
+
+| Sign | Do |
+|---|---|
+| 404 on `/models` or `/chat/completions` | the URL lacks `/v1` [run] |
+| `content` empty, `thinking` full | add `reasoning_effort: "none"` [doc] |
+| `/api/ps` context below 65536 | the person's Ollama runs its default (4K or 32K); serve the blob with Grid's engine instead of changing their app |
+| 503 "server is overloaded" | queue full (`OLLAMA_MAX_QUEUE`, default 512) [doc]; wait, do not retry in a loop |
+| `PROCESSOR` shows CPU share | model plus context does not fit the GPU; smaller context or model [doc] |
+| model not found | not downloaded; offer the download as a slow step, never pull silently |
diff --git a/store/agents/autonomous-grid/skills/engine-sglang/SKILL.md b/store/agents/autonomous-grid/skills/engine-sglang/SKILL.md
new file mode 100644
index 000000000..e8f42ff19
--- /dev/null
+++ b/store/agents/autonomous-grid/skills/engine-sglang/SKILL.md
@@ -0,0 +1,95 @@
+---
+name: engine-sglang
+description: "Serve a Hugging Face model with SGLang on a Linux machine with an NVIDIA or AMD GPU, configured from the model's SGLang cookbook page — or, when it has none, from the model's own files — and join it to the person's fleet. Load before installing, configuring, starting or stopping SGLang."
+---
+
+# SGLang (Linux + NVIDIA CUDA or AMD ROCm)
+
+Official sources, read 2026-09-29. Agent index: [docs.sglang.io/llms.txt](https://docs.sglang.io/llms.txt)
+(every page is available as `.md`). [Cookbook](https://docs.sglang.io/cookbook/intro) ·
+[Models](https://www.sglang.io/models) · [Quickstart](https://docs.sglang.io/docs/get-started/quickstart.md) ·
+[Server arguments](https://docs.sglang.io/docs/advanced_features/server_arguments.md) ·
+[Tool parser](https://docs.sglang.io/docs/advanced_features/tool_parser.md) ·
+[Separate reasoning](https://docs.sglang.io/docs/advanced_features/separate_reasoning.md) ·
+[Native API](https://docs.sglang.io/docs/basic_usage/native_api.md) ·
+[Transformers fallback](https://docs.sglang.io/docs/supported-models/transformers_fallback.md) ·
+source: [github.com/sgl-project/sglang](https://github.com/sgl-project/sglang)
+Tested: not runnable on macOS (below); no GPU run yet. Tags: `[doc]` official source, `[run]` seen on a machine, `[?]` unverified.
+
+## When to use it
+
+- Linux with an NVIDIA GPU of compute capability 8.0 or newer (A10, A100, L4, L40S, H100), Python ≥ 3.10 [doc];
+ AMD GPUs, Intel Xeon CPUs and TPUs have their own guides [doc]. Hugging Face models, many requests at once,
+ several GPUs.
+- **Never on macOS.** The current release ships Linux wheels only and requires CUDA builds of its kernels;
+ on a Mac the installer falls back to an old release whose server does not import [run].
+- A GGUF file for one person: Grid's own engine serves it with nothing to install.
+
+## 1. The cookbook page — `"$GRID_FLEET" recipe sglang ORG/NAME`
+
+Finds the model family's page through `llms.txt` (the most specific family name the model name starts
+with) and prints: `source` (the page), `installation` (its version requirement — some families need a
+release or the main branch — and install commands), `serveCommands` (the page's own launch commands per
+GPU type), `toolCallParsers`, `reasoningParsers`, and `configurationTips`. Read the page itself for
+anything else. Exit code 3 means no page: go to step 2.
+
+## 2. No cookbook page — read the model: `"$GRID_FLEET" model-facts ORG/NAME`
+
+1. `support.sglang.listed: true` means SGLang's model registry has the architecture; `null` means this
+ check cannot tell. SGLang can run most decoder models through `--model-impl transformers` [doc] — a test,
+ not a promise.
+2. `modelCard.serveCommands` and parsers: the model authors' own SGLang command. Use it when present.
+3. Otherwise `--tool-call-parser auto` and `--reasoning-parser auto`: SGLang detects both from the model's
+ chat template [doc]. Then the tool-call acceptance check decides; if it fails, say so and stop — do not
+ cycle through parser names.
+4. `contextLength` must be ≥ 65536.
+
+## Install and version
+
+- Slow step, ask first. `uv pip install --prerelease=allow sglang` [doc]; a family whose cookbook page asks
+ for the main branch: `uv pip install --prerelease=allow 'git+https://github.com/sgl-project/sglang.git#subdirectory=python'` [doc].
+ `OSError: CUDA_HOME environment variable is not set` → `export CUDA_HOME=/usr/local/cuda-<version>` [doc].
+- Docker: `lmsysorg/sglang:latest` (NVIDIA), `lmsysorg/sglang-rocm:<tag>` (AMD, tag per GPU generation on
+ the cookbook page) [doc], run with `--gpus all --shm-size 32g --ipc=host -p 127.0.0.1:P:30000
+ -v ~/.cache/huggingface:/root/.cache/huggingface` [doc].
+
+## Common configs (the cookbook's flags win)
+
+Each adds the parsers from step 1 or 2, `--host 127.0.0.1 --port P` and `--context-length` of at least 65536
+(the default is the model's own maximum [doc]).
+
+| Case | Add |
+|---|---|
+| One GPU, one agent | `--max-running-requests 4` |
+| One GPU, many people | leave `--max-running-requests` to SGLang |
+| Several GPUs, one machine | `--tp-size <GPU count>` [doc] |
+| Out of memory | lower `--mem-fraction-static` (weights plus KV pool share) [doc]; `--kv-cache-dtype fp8_e4m3` [doc] |
+
+## Start, ready, join
+
+ "$GRID_FLEET" serve sglang-P --env HF_HUB_OFFLINE=1 -- ~/.grid/envs/sglang/bin/sglang serve \
+ --model-path <model id or snapshot dir> --served-model-name <id> --host 127.0.0.1 --port P <flags>
+
+(`python3 -m sglang.launch_server` takes the same arguments [doc].) Defaults are 127.0.0.1 and port 30000 [doc].
+Install into `~/.grid/envs/sglang` so `fleet models` finds it; `fleet serve` keeps it alive after your shell
+returns (log `run/sglang-P.log`).
+
+- Verify: `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <id> --kind sglang` (bounded, narrated).
+- Ready: the log says `The server is fired up and ready to roll!` [doc]; `GET /health` → 200;
+ `GET /health_generate` generates one token [doc]; `GET /v1/models` lists `<id>`; one bounded answer; one tool call.
+- Join: `"$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <id> --advertise-as ALIAS --name NAME`. Grid's detector has
+ no SGLang probe and would mislabel it on 8000 or 8080 [run], so always `--at`, always with `/v1`.
+- Thinking off per request: `"chat_template_kwargs": {"enable_thinking": false}` where the cookbook page shows it [doc].
+
+## Stop
+
+`"$GRID_FLEET" stop sglang-P` (or `docker stop` your container); confirm the port is free.
+
+## Known failures → what to do
+
+| Sign | Do |
+|---|---|
+| out of memory while serving | lower `--mem-fraction-static` [doc] |
+| no `tool_calls` | the cookbook's `--tool-call-parser`, else `auto` [doc]; still none → report |
+| a CUDA crash, a multi-GPU hang, a production incident | read SGLang's own maintainer skills: `.agents/skills/debug-cuda-crash`, `debug-distributed-hang`, `sglang-prod-incident-triage` (`SKILL.md` in each) at `https://raw.githubusercontent.com/sgl-project/sglang/main/` |
+| 404 on `/models` | the `--at` URL lacks `/v1` |
diff --git a/store/agents/autonomous-grid/skills/engine-vllm/SKILL.md b/store/agents/autonomous-grid/skills/engine-vllm/SKILL.md
new file mode 100644
index 000000000..7968bdb49
--- /dev/null
+++ b/store/agents/autonomous-grid/skills/engine-vllm/SKILL.md
@@ -0,0 +1,117 @@
+---
+name: engine-vllm
+description: "Serve a Hugging Face model with vLLM on a Linux machine with an NVIDIA or AMD GPU, configured from the model's official vLLM recipe — or, when it has none, from the model's own files — and join it to the person's fleet. Load before installing, configuring, starting or stopping vLLM."
+---
+
+# vLLM (Linux + NVIDIA CUDA or AMD ROCm)
+
+Official sources, read 2026-09-29. Agent index: [recipes.vllm.ai/llms.txt](https://recipes.vllm.ai/llms.txt).
+Reference docs as raw markdown under `https://raw.githubusercontent.com/vllm-project/vllm/main/docs/`:
+[OpenAI-compatible server](https://docs.vllm.ai/en/latest/serving/online_serving/openai_compatible_server/) (`serving/online_serving/openai_compatible_server.md`) ·
+[Tool calling](https://docs.vllm.ai/en/latest/features/tool_calling/) (`features/tool_calling.md`) ·
+[Reasoning outputs](https://docs.vllm.ai/en/latest/features/reasoning_outputs/) (`features/reasoning_outputs.md`) ·
+[Supported models](https://docs.vllm.ai/en/latest/models/supported_models/) (`models/supported_models.md`) ·
+[GPU install](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/) · [Docker](https://docs.vllm.ai/en/latest/deployment/docker/) ·
+[Optimization](https://docs.vllm.ai/en/latest/configuration/optimization/) · [Conserving memory](https://docs.vllm.ai/en/latest/configuration/conserving_memory/)
+Tested: only vLLM 0.11.0 on an Apple-silicon Mac (CPU backend), 2026-09-29 — no GPU run yet.
+Tags: `[doc]` official source above, `[run]` seen on the tested machine, `[?]` unverified.
+
+## When to use it
+
+- Linux with an NVIDIA GPU of compute capability 7.5 or newer [doc], or an AMD GPU with ROCm [doc], and a
+ Hugging Face model (safetensors, FP8, AWQ…). vLLM is built for many requests at once and for models split
+ across several GPUs.
+- A GGUF file for one person on one GPU: Grid's own engine serves it with nothing to install.
+- **Not on a Mac.** There it runs on the CPU only, slowly (26 tokens in 2.3 s for a 0.5B model), and needed
+ pinned `transformers<5`, `--dtype float32` for JSON output and a batched-token limit to start [run].
+
+## 1. The recipe — `"$GRID_FLEET" recipe vllm ORG/NAME`
+
+Reads [recipes.vllm.ai](https://recipes.vllm.ai/) live (`models.json`, then `<ORG>/<NAME>.json`) and prints:
+
+| Field | Use |
+|---|---|
+| `minVersion` | the installed `vllm --version` must be at least this |
+| `baseArgs`, `baseEnv` | always |
+| `features.tool_calling.args`, `features.reasoning.args` | always for an agent |
+| `features.*` with `optIn: true` | only on purpose (faster decoding, text only to free memory, longer context) |
+| `variants.<name>` | pick by `vramMinimumGb` against the GPU; a variant can carry its own `modelId` and `extraArgs` |
+| `recommended` | the site's reference command and Docker image for its reference hardware |
+| `install` | the recipe's own install steps, including any pinned extras |
+| `guide` | prose; its **Troubleshooting** holds model-specific fixes |
+
+The recipe beats the generic docs: a family's generic parser entry and a model's recipe can name different
+parsers [doc]. Exit code 3 means there is no recipe: go to step 2.
+
+## 2. No recipe — read the model: `"$GRID_FLEET" model-facts ORG/NAME`
+
+1. `support.vllm.listed` must be `true` (its architecture is in vLLM's supported-models list). `false`:
+ the Transformers backend (`--model-impl transformers`) may still run it [doc] — a test, not a promise.
+2. `modelCard.serveCommands` / `toolCallParsers` / `reasoningParsers`: the model authors' own vLLM command.
+ Use it when present.
+3. Otherwise match `chatTemplate.toolSyntax` against the parser descriptions in `features/tool_calling.md`
+ (each parser documents the output format it reads) and, when `chatTemplate.thinking`, pick the reasoning
+ parser from the table in `features/reasoning_outputs.md` [doc]. Quote the doc line you matched.
+4. `contextLength` must be ≥ 65536. Sampling defaults come from the model's `generation_config.json`
+ automatically [doc]; leave them.
+5. No match in the docs: say so and stop — do not cycle through parsers until one passes.
+
+## Install and version
+
+- Slow step, ask first. Venv: `uv venv && uv pip install -U vllm --torch-backend=auto` [doc]; pin with
+ `"vllm==X.Y.Z"` at or above the recipe's `minVersion`. Wheels target CUDA 12.9 by default, builds for 12.8
+ and 13.0 exist, Blackwell needs CUDA ≥ 12.8 [doc]. AMD: `--extra-index-url https://wheels.vllm.ai/rocm`
+ (Python 3.12, ROCm 7.0, glibc ≥ 2.35) [doc]. Check the GPU and driver with `nvidia-smi`.
+- Docker: `vllm/vllm-openai` (NVIDIA), `vllm/vllm-openai-rocm` (AMD) [doc], with
+ `--gpus all --ipc=host -p 127.0.0.1:P:8000 -v ~/.cache/huggingface:/root/.cache/huggingface` [doc].
+
+## Common configs (the recipe's flags win)
+
+Each adds the recipe's (or step 2's) tool and reasoning flags, `--host 127.0.0.1 --port P` and a
+`--max-model-len` of at least 65536.
+
+| Case | Add |
+|---|---|
+| One GPU, one agent | `--max-model-len 131072 --max-num-seqs 4 --enable-prefix-caching` |
+| One GPU, many people | `--max-model-len 65536 --enable-prefix-caching` (`max-num-seqs` left to vLLM) |
+| Several GPUs, one machine | `--tensor-parallel-size <GPU count>` [doc] |
+| Memory is tight | the recipe's FP8 variant, `--kv-cache-dtype fp8`, fewer `--max-num-seqs` [doc] |
+
+## Start, ready, join
+
+ "$GRID_FLEET" serve vllm-P --env HF_HUB_OFFLINE=1 -- ~/.grid/envs/vllm/bin/vllm serve <model id or snapshot dir> \
+ --served-model-name <id> <flags>
+
+(Install into `~/.grid/envs/vllm` so `fleet models` finds it; `fleet serve` keeps it alive after your shell
+returns; log `run/vllm-P.log`.)
+
+- Verify: `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <id> --kind vllm` (bounded, narrated).
+- Ready: PID alive; log shows `Application startup complete.` [run] (minutes for a big model); `GET /health`
+ → 200 and `GET /v1/models` lists `<id>` [run]; one bounded answer; one tool call.
+- Join: `"$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <id> --advertise-as ALIAS --name NAME`. `/v1` is required —
+ without it `/models` and `/chat/completions` answer 404 [run]. `--api-key` guards only `/v1` routes [doc].
+- Thinking off for everyday use: the recipe's guide names the flag when there is one
+ (`--default-chat-template-kwargs '{"enable_thinking": false}'` in the recipes that document it [doc]).
+
+## Knobs
+
+`--gpu-memory-utilization` (share of GPU memory pre-allocated for weights plus cache), `--max-model-len`,
+`--max-num-seqs`, `--max-num-batched-tokens`, `--kv-cache-dtype fp8`, `--enforce-eager` (fastest start,
+slower serving), `--tensor-parallel-size` [doc].
+
+## Stop
+
+`"$GRID_FLEET" stop vllm-P` (or `docker stop` the container you started); confirm the port is free.
+
+## Known failures → what to do
+
+| Sign | Do |
+|---|---|
+| "preempted … not enough KV cache space" | raise `--gpu-memory-utilization`, lower `--max-num-seqs`, or more tensor parallel [doc] |
+| CUDA out of memory at start | FP8 variant, `--kv-cache-dtype fp8`, fewer sequences — context stays ≥ 65536 |
+| "max_num_batched_tokens … smaller than max_model_len" | raise `--max-num-batched-tokens` to at least the context [doc][run] |
+| an error the recipe's guide names | apply the guide's fix exactly [doc] |
+| tokenizer `AttributeError` after install | transformers newer than that vLLM supports (0.11.0 needed `transformers<5` [run]); use a vLLM at or above the recipe minimum |
+| no `tool_calls` | `--enable-auto-tool-choice` plus the recipe's parser [doc][run] |
+| 404 on `/models` | the `--at` URL lacks `/v1` [run] |
+| "CUDA error: an illegal memory access" | read vLLM's own maintainer skill `.agents/skills/debug-ima/SKILL.md` at `https://raw.githubusercontent.com/vllm-project/vllm/main/` |
diff --git a/store/agents/autonomous-grid/skills/grid-operations/SKILL.md b/store/agents/autonomous-grid/skills/grid-operations/SKILL.md
index 7a45c7e40..cdb86692f 100644
--- a/store/agents/autonomous-grid/skills/grid-operations/SKILL.md
+++ b/store/agents/autonomous-grid/skills/grid-operations/SKILL.md
@@ -40,7 +40,8 @@ company grid") goes to that row's exact `grid` name; a request that names none g
(also `$HARNESS_PRIVATE_GRID`). Harness put it there from the account that is signed in, so it is
the answer, not a guess: pass that exact name to `join`, `leave`, `engines` and `models`, and do not
ask the person which grid is theirs. It can differ from `grid` (the workspace's selected fleet) —
-a request that says "my grid" goes to `personalGrid`, whatever is selected. The `type` column of
+a request that says "my grid" goes to `personalGrid`, whatever is selected, and so does a request that
+names no grid at all ("run a model on my Mac"). The `type` column of
`ls --json` is what kind of grid a row is, not a permission to ask about: `permissioned-public` is
a person's own grid, `private-domain` a company's, `domain-restricted` a team's, `os-community` a
public one. Only when `personalGrid` is null: say in one line that Harness has not named this
@@ -88,9 +89,22 @@ success, say **"Your model is running. Select it from the model picker in a sess
Slow steps are a real stop: a download or an engine build is asked about through the question tool
and runs in the **next** turn, never in the message that asks. Every model you offer comes from
-something you looked up — the host's disk, the catalog, or Hugging Face — never from memory. The
-pick is made once; that is the model through the download and the start, unless it fails (won't
-fit, won't pull, won't answer) or the person asks for a different one.
+something you looked up — the grid, a machine's disk (step 0), the catalog, or Hugging Face — never
+from memory. The pick is made once; that is the model through the download and the start, unless it
+fails (won't fit, won't pull, won't answer) or the person asks for a different one.
+
+**0. Scan first — what the grid serves, and what every machine already holds.**
+
+- *The grid:* `"$GRID_FLEET" run -- models GRID --json` for the grid it will go on. A model already
+ served there needs no start at all — say "use it from the model picker now".
+- *This computer:* `skills/run-local-model/SKILL.md` — it takes stock (`fleet models`), sizes, picks
+ file and engine, starts and verifies. This skill adds only the grid parts below.
+- *An SSH machine in `grid-fleet.json`:* `"$GRID_FLEET" models --machine M --summary` runs the same
+ finder on that machine with its own Node — its disk, engines, ports and memory [run]. Without Node
+ 18+ there it prints a file list only and says so.
+- *A Harness-linked machine:* its disk cannot be listed from here yet. Read its hardware
+ (`run --machine M -- device-info --json`) and say in one line that its downloaded files are not
+ visible, so a model for it comes from the catalog.
**1. What they need — decided for them, not asked.** Harness is a coding tool: the model is for a
coding agent unless the person says otherwise. Do **not** ask them to choose an engine,
@@ -116,18 +130,16 @@ before the start say "the model takes W GB, and its context uses the rest of the
can give models" (`usable_bytes` from `device-info`); after the start, give Grid's measured figure
when `engines --json` reports one. Never invent a number.
-**2. What the host has.** `device-info --json` on the intended host first: `usable_bytes` is the
-real ceiling for weights plus context. Per-machine, never summed across hosts. The engine check is
-the binary, not a status command: `~/.grid/bin/llama-server --version` printing a version line means
-llama.cpp is installed — go on without a word. ⚠️ `engine status` reports the **media** engine
-(ComfyUI) and says `Installed: no` on a host whose llama.cpp is fine; it sent an agent asking to
-install what was already built. Only when the binary is missing, ask (build from source or prebuilt)
-and run `engine install llama.cpp [--from-source]` in the next turn.
+**2. Another machine's hardware.** `run --machine M -- device-info --json`: `usable_bytes` is its
+ceiling, per machine, never summed. Its llama.cpp is installed when `~/.grid/bin/llama-server --version`
+prints a version. ⚠️ `engine status` reports the **media** engine (ComfyUI) and says `Installed: no`
+on a host whose llama.cpp is fine. Only when the binary is missing, ask and run
+`engine install llama.cpp [--from-source]` in the next turn.
-**3. What is already on the host's disk comes first.** `ls ~/.grid/models/*.gguf` minus the
-`.mmproj.gguf` sidecars; `ctx FILE --json` says the most each was trained to hold — a file under
-64K is skipped; a `<stem>.mmproj.gguf` beside a file means it reads images. A suitable file already
-here is the pick ("already on this computer, no download"); fetch something new only when none is.
+**3. What is already on disk comes first.** Step 0's table lists every model file on that machine,
+from every app, with its context and whether it reads images — a file under 64K is skipped. A
+suitable file already there is the pick ("already on this computer, no download"); fetch something
+new only when none is.
**4. The catalog, then Hugging Face.** `catalog --json` is sized for the host: keep entries that are
`runnable`, whose `fit.ctx` is at least 65536, and that are good at code; pull `fit.version`'s
@@ -135,6 +147,7 @@ here is the pick ("already on this computer, no download"); fetch something new
much it can hold in pages and whether it reads images; offer a shortlist of 2–3 only when the person
asks for other options. No speed figure (`fit.est_tok_s` only orders the list for "fast"), no quant
name, no token count as the whole answer.
+
When the person names a model the catalog lacks, the catalog is not a wall — `pull` takes any
`<repo>:<file>.gguf` on Hugging Face and fetches its projector too:
@@ -145,7 +158,7 @@ When the person names a model the catalog lacks, the catalog is not a wall — `
for f in json.load(sys.stdin)["siblings"]:
if f["rfilename"].endswith(".gguf"): print(f["rfilename"], f.get("size"))'
-Prefer an official org (`ggml-org`, `Qwen`, `google`, `unsloth`, `bartowski`, `lmstudio-community`)
+Prefer the model's own publisher, or a quantizer that publisher's model card or the catalog names,
over an unknown uploader, then downloads; say which in half a line. Pick the quant by size against
`usable_bytes` (or ≈0.6 bytes per parameter for Q4_K_M), leaving room for context. A model outside
the catalog has been sized by nobody but you — say the fit is your estimate.
@@ -169,36 +182,44 @@ fails, `leave`, `sync`, and join again at half — 256K → 128K → 64K. If 64K
this computer does not have the memory for this model with a 64K context; offer a smaller model. *Vision:* on by default when the projector is
on disk. When the person asks for text only, rename the projector yourself
(`mv <stem>.mmproj.gguf <stem>.mmproj.gguf.off`) before the join and say so in one line.
+Sizing (weights + context × cache per token within free memory) is `run-local-model` §3.
"$GRID_FLEET" run --machine MACHINE -- pull OWNER/REPO:EXACT_FILE.gguf
"$GRID_FLEET" run --machine MACHINE -- join GRID --serve EXACT_FILE.gguf --advertise-as MODEL_ALIAS \
--max-concurrency N --ctx-size CTX --endpoint-port PORT
+A GGUF **already on that machine** in another folder (step 0 found it) is linked, never pulled again:
+`"$GRID_FLEET" link --machine MACHINE FILE NAME.gguf [--projector MMPROJ_FILE]`, then `join … --serve
+NAME.gguf` — FILE is the full path `fleet models --machine MACHINE` (JSON, `path`) printed. `link`
+says `NOT READY` when that machine's Grid engine is missing or does not start: ask, then `run
+--machine MACHINE -- engine install llama.cpp` in the next turn.
+
`--advertise-as` is the name the person will see in their model picker. **Don't pass `--name`:**
the runner sets it on every `join` to the machine's name as Harness Machines shows it right now, and
replaces any you give — it is the label under the model in every picker, and a name made up here
-(`macbookpro-qwen3.6-35b`) or Grid's host-name default (`mac.lan`) read as a different computer.
+(a machine name joined to a model name) or Grid's host-name default read as a different computer.
Say the machine by that same name. `--max-concurrency N` is 1 unless they asked for more; don't pass
`--parallel` (grid derives the slot count from it) and don't pass `--jinja` (on by default in the
engine grid ships). Never pin `--ctx-size` under 65536 — a window that small cannot hold a coding
agent's own prompt (a 32K engine here refused an agent's first request of 59,561 tokens). Use
explicit ports when several instances share a host. An existing Ollama, vLLM, MLX or LM Studio
-engine can join with `--at URL -m MODEL` (the runner names it too); do not install a second engine needlessly.
+engine joins with `--at http://HOST:PORT/v1 -m MODEL --advertise-as ALIAS` (the runner names it too) — the
+`/v1` is required, `fleet models` prints the exact URL, and the matching `skills/engine-*/SKILL.md` has
+its start and ready checks; do not install a second engine needlessly.
-Choose a reasoning budget deliberately. Grid's GPU default can spend more tokens thinking than a
-small output limit permits, yielding no final answer. For an everyday low-latency assistant, start
-with `fleet run --thinking off -- join ... --reasoning-budget 0`. `--thinking off` sets llama.cpp's
-`enable_thinking:false` template parameter for the newly started engine; it is needed on builds
-where a zero token budget alone still produces reasoning. Use `--thinking on` to enable a supported
-model's thinking explicitly. These switches configure startup, not an already running instance.
-For a reasoning model, reserve an explicit budget smaller than `--n-predict`, leaving room for the
-answer.
+Thinking: for everyday use `fleet run --thinking off -- join … --reasoning-budget 0`. `--thinking off`
+sets `enable_thinking:false` at startup — needed, because a zero budget alone still let a model spend
+every token thinking [run]. `--thinking on` enables it; either applies only to a new start.
**6. Prove it answers — once, bounded.** A successful `join` means *starting*, not ready. First wait
for the relay to list it (a call before that answers `No providers available for this model`, which
is "not yet", not "broken"):
- until "$GRID_FLEET" run -- models GRID 2>/dev/null | grep -qx 'MODEL_ALIAS'; do sleep 10; done
+ # engine on this computer: its own checks first, then the relay
+ "$GRID_FLEET" verify --at http://127.0.0.1:PORT/v1 --model MODEL_ALIAS --kind llama.cpp --grid GRID --alias MODEL_ALIAS
+ # engine on another machine: the relay only
+ "$GRID_FLEET" verify --grid GRID --alias MODEL_ALIAS
+ # once it passes: the receipt the Models panel shows
"$GRID_FLEET" verify --grid GRID --model MODEL_ALIAS
A `verify` that fails with a compute or out-of-memory error is the context not fitting — step down
@@ -207,19 +228,15 @@ as step 5 says rather than retrying the same size. Then read the window it actua
65536, `leave` it, say in one line it could only get N pages here, and move to a smaller model or
quant. A null means Grid did not say; report the context you passed.
-⚠️ Not `chat` for this check. `chat` sets no output limit and the engine's default is tens of
-thousands of tokens, so a small model that runs away answering "ok" holds the slot for minutes — and
-with one slot everything after it waits, including a second check. `max_tokens` is what makes this
-finish in seconds whatever the model does. The timeout is long on purpose: right after a join, grid
-sends the new engine a probe of about 5K tokens to measure what it can do, and on a host without a
-GPU that alone takes 3–5 minutes at ~30 tokens/s while holding the single slot. Tell the user the
-model is warming up and the first answer can take a few minutes on that machine. Send the check
-ONCE and leave it alone — a second call, a `chat`, a log tail all queue behind the same probe. A
-reply with a `choices` entry means the whole path works. Nothing within the timeout: stop the model
-(`leave`), say in one line it started but did not answer in time, and offer through a tool a model
-one step larger from the same list (tiny models loop), or more requests at once, or stop. Verify
-`message.content` contains the requested result; reasoning text alone or a successful HTTP status is
-not acceptance. Then `"$GRID_FLEET" refresh`.
+Every step prints its elapsed time and every wait has a deadline (listing: every 10 s up to 300 s;
+the relay answer: up to 420 s with a "still waiting" line every 15 s) — never a bare `until … sleep`
+loop, which waits forever on an engine that never registers. Relay the progress lines to the person.
+
+Never `chat` for this check: it has no output limit, and a small model that runs away holds the only
+slot for minutes. Right after a join, grid probes the new engine with ~5K tokens (3–5 minutes on a
+host without a GPU): tell the person it is warming up, run `verify` once, and leave it alone. It fails
+within its deadline: `leave`, say so in one line, and offer the next model up, or stop. Then
+`"$GRID_FLEET" refresh`.
**7. Say where it is, and what it costs.** "<alias> is running on <machine> with room for about
N pages (<K>K tokens), one at a time, vision on/off; it uses about M GB of memory. Pick it from the
@@ -233,7 +250,7 @@ The person names the one thing they want different; everything else stays. Read
settings from the viewer or `stats GRID --verbose --json` (model, context, slots) so you change only
that and can say what changed. Context, concurrency and vision all mean a restart — `leave` that
instance, then the same `join` with the one flag (or the projector file) changed; check the fit
-first as in step 5. A different model is step 3 onward, keeping the context and concurrency they
+first as in step 5. A different model is step 4 onward, keeping the context and concurrency they
already have; stop the old one only when the new one is about to serve. Stop is `leave` and one
line. A restart drops the model from the picker for the seconds it takes and any agent mid-turn on
it loses that turn: say so in one line, and ask first only when `stats` shows requests in flight.
diff --git a/store/agents/autonomous-grid/skills/grid-operations/references/commands.md b/store/agents/autonomous-grid/skills/grid-operations/references/commands.md
index b46c804e6..42c962d1a 100644
--- a/store/agents/autonomous-grid/skills/grid-operations/references/commands.md
+++ b/store/agents/autonomous-grid/skills/grid-operations/references/commands.md
@@ -8,10 +8,12 @@ https://github.com/autonomous-ai/autonomous-grid/blob/main/docs/cli.md.
|---|---|
| Lifecycle and selection | `start`, `stop`, `delete`, `ls`, `info`, `mode`, `use` — `use` is the one source of truth for which grid; the viewer follows it, and `connect` sets it (SKILL.md, "Targets and access"). Change grids through `connect`, which verifies first |
| Host inventory | `device-info --json` |
-| Model discovery and weights | `~/.grid/models/*.gguf` on disk first, then `catalog --json`, then Hugging Face; `pull REPO:FILE`, `ctx FILE --json`, `rm FILE` |
+| Model discovery and weights | this computer first: `"$GRID_FLEET" models` (files from every app, engines answering, memory and swap now; not a Grid command, no `run --`); an SSH machine: `"$GRID_FLEET" models --machine M` (a Harness-linked one cannot be listed); then `catalog --json`, then Hugging Face; `pull REPO:FILE`, `ctx FILE --json`, `rm FILE` |
+| Engine configuration from official sources | `"$GRID_FLEET" recipe vllm|sglang ORG/NAME` (exit 3 = no recipe), `"$GRID_FLEET" model-facts ORG/NAME|DIR` (what the model says about itself); see `skills/run-local-model/SKILL.md` |
| Engine provisioning | `engine install llama.cpp [--from-source]`; the install check is `~/.grid/bin/llama-server --version` — `engine status`/`start`/`stop` are the media engine (ComfyUI) |
| Serving and removal | `join`, `leave --engine SELECTOR`, `engines --json`, `models --json` |
-| Inference checks | a bounded relay call with `max_tokens` (SKILL.md step 6), never `chat` for verification (no output limit); `image`, `edit`, `video`, `stt` for media |
+| Inference checks | `"$GRID_FLEET" verify` (SKILL.md step 6: engine and relay, every step bounded and narrated), never `chat` for verification (no output limit); `image`, `edit`, `video`, `stt` for media |
+| Models for an installed engine with none | `"$GRID_FLEET" candidates mlx` (Mac), recipes via `"$GRID_FLEET" recipe` (GPU), `catalog --json` (Grid's engine) |
| Remote telemetry | `stats --verbose --json`, `usage --by model|member|engine --json` |
| Remote request routing | `router --help` (inspect before changing policies or advisors) |
| Training and evaluation | `train --help`, `train doctor`, `train packs`, `train eval`, `train deploy` |
diff --git a/store/agents/autonomous-grid/skills/run-local-model/SKILL.md b/store/agents/autonomous-grid/skills/run-local-model/SKILL.md
new file mode 100644
index 000000000..50c4bbb5b
--- /dev/null
+++ b/store/agents/autonomous-grid/skills/run-local-model/SKILL.md
@@ -0,0 +1,113 @@
+---
+name: run-local-model
+description: "Details behind AGENTS.md's 'Starting a model on this computer' flow: reading `fleet models`, sizing a start, choosing file and engine, picking a model for an engine with none, what `fleet verify` checks, and where to read when unsure. Open it when a step of that flow needs more than the flow says."
+---
+
+# Run a local model — details
+
+The flow itself is in AGENTS.md. Tags: `[run]` seen on a real machine (M1 Pro 32 GB, macOS 26.6,
+2026-09-29), `[doc]` official docs (see the `engine-*` skill), `[code]` read in the tool's source.
+
+## Purpose → settings (the only thing asked)
+
+*Coding · Chat and writing · Reading images · Just something fast.* **Context is never below 64K**:
+every model here is used by an agent whose own prompt fills a small window; Ollama's docs set the same
+floor for agents [doc]. When 64K does not fit, take a smaller model, never a smaller window.
+
+| Purpose | Context | At once | Also needs |
+|---|---|---|---|
+| Coding | 128K when it fits, else 64K | 1 | tool calls |
+| Chat and writing, or fast | 64K | 1 | — |
+| Reading images | 64K | 1 | a projector beside the file |
+
+Thinking is off for everyday use.
+
+## Reading `fleet models`
+
+- `machine.accelerators[]`: a GPU counts only with `active: true` (its own tool answered). `active: false`
+ comes with an `error` (e.g. no driver): say it in one line; plan no GPU engine on it.
+- `machine.engines[]`: installed engines, running or not. `machine.canRun`: the kinds this hardware can
+ run at all — never propose one outside it.
+- `machine.memory.availableBytes`, `swapUsedBytes`: room right now. Metal and `device-info` do not see
+ other apps (Metal said 25 GiB free while macOS was 11 GB into swap [run]).
+- `machine.accelerators[].totalBytes`: the GPU ceiling, below total RAM on a Mac — read it here, never assume it; `device-info` reports a different figure
+ there [run] — use the smaller.
+- `engines[]`: answering engines and their exact `--at` URL. `openai-compatible` whose "models" are not
+ models is another app: leave it alone.
+- `models[]`: one entry per real file (`alsoAt` = other apps holding the same file), with `format`,
+ `bytes`, `projector`, and for GGUF `contextLength`, `kvBytesPerToken`, `toolCalls`, `unsupportedTensorTypes`.
+
+## Size
+
+ need = weights + context × kvBytesPerToken × slots + 0.5 GB (+ projector when vision is on)
+
+`kvBytesPerToken` matched what llama.cpp allocated exactly [run]; files of similar size needed from
+20 KiB to 160 KiB per token, so never skip this. Null (latent attention): start at 64K and read the
+engine's memory report. Fits when `need ≤ availableBytes + 3 GB` and ≤ the GPU ceiling (macOS moves
+idle pages to swap once; swap still climbing a minute after start means too big). Too big: a smaller
+file, or ask the one trade-off — "close other apps first, or a lighter model beside your work?".
+
+## Choose the file, then the engine
+
+Drop: `unsupportedTensorTypes` not empty (llama.cpp refuses them [run]); context below 64K; no tool
+calls for coding; no projector for images; anything that does not fit. Prefer newer families, more
+parameters at 4-bit over fewer at 8-bit, and mixture-of-experts models when memory allows.
+
+| The file | Mac (Apple silicon) | Linux + active NVIDIA/AMD GPU | CPU only |
+|---|---|---|---|
+| served by an engine already answering | `join --at URL/v1 -m ID --advertise-as ALIAS` | same | same |
+| GGUF from any app | Grid's engine (link into `~/.grid/models`) | same | same |
+| MLX folder | mlx-lm (`engine-mlx-lm`) | — | — |
+| Hugging Face safetensors | mlx-lm | vLLM or SGLang from the model's recipe | find a GGUF |
+
+Grid only serves from `~/.grid/models`: its launcher keeps just the file name of `--serve` and looks
+there; a projector must sit beside it [code: grid `shared/engine/launcher.py`]. A symlink costs no disk.
+Never `ollama create` to reuse a file (it copies it [run]); never vLLM or SGLang on a Mac (CPU-only /
+no macOS build [run]); never download a second copy only to switch engines. On Apple silicon, when a
+*new* download is needed and mlx-lm or LM Studio is installed, an MLX build is sound — Ollama itself
+moved its Apple engine to MLX [doc: ollama.com/blog/mlx].
+
+On Apple silicon with mlx-lm (or LM Studio) installed, **MLX first**: when the same model exists as an
+MLX folder and as a GGUF and the MLX one fits, serve the MLX one. The choice is still model-first —
+never a bigger MLX model that does not fit over a smaller GGUF that does. The report names what was
+passed over and why, one line each ("<model> (<format>): needs <N> GB, <M> GB free"), so "why not MLX?"
+is answered before anyone asks.
+
+## An engine installed, nothing to serve
+
+Offer 2–3 that fit (plain name, GB, what it is good at) plus "none of these"; the download waits for
+the go-ahead.
+- Mac with mlx-lm or LM Studio: `"$GRID_FLEET" candidates mlx [--search WORDS] [--sort downloads|trending|recent]`
+ — mlx-community models sized from their real files, cache at 64K, `fits` against the GPU ceiling.
+- Linux GPU with vLLM or SGLang: pick families from [recipes.vllm.ai/llms.txt](https://recipes.vllm.ai/llms.txt)
+ or the SGLang cookbook, then `fleet recipe` each; `vramMinimumGb` against the GPU decides.
+- Grid's engine or Ollama with nothing: Grid's catalog (`grid-operations` step 4), pulled into Grid.
+- The person names an engine outside `canRun`: say why in one line (no active GPU, not a Mac).
+
+## This computer already on that grid
+
+In remote mode a computer joins a grid as one identity, and Grid's `--serve` engine cannot share it: a
+second model is refused with "can't join a multi-engine identity. Run `grid leave`, then re-join every
+engine as external `--at <url> -m <model>`" [run]. Ask *Replace X with Y · Keep X*; never leave an
+engine the person did not agree to.
+
+## What `fleet verify` checks
+
+Engine: **ready** (`/models` within 180 s, every 3 s), **listed**, **answer** (max_tokens 16, thinking off;
+reasoning-only output fails), **tool call** (`read_file` must come back as `tool_calls`), **speed**. With
+`--grid`: **relay listed** (every 10 s up to 300 s) and **relay answer** (up to 420 s, a "still waiting"
+line every 15 s). Exit 0 only when all pass. Seen end to end: an mlx-lm engine joined a local grid with
+`--at …/v1` in 6 s and passed all seven in 10 s; a Grid-engine GGUF passed at 19 tok/s [run].
+
+## When unsure: read, never guess
+
+1. `"$GRID_FLEET" recipe vllm|sglang ORG/NAME` — the official recipe for that exact model (exit 3: none).
+2. `"$GRID_FLEET" model-facts ORG/NAME|DIR` — architecture, context, sampling defaults, the template's
+ tool-call syntax, the authors' serve commands, and whether each engine lists the architecture
+ (`listed: null` = cannot tell).
+3. Agent indexes: [docs.ollama.com/llms.txt](https://docs.ollama.com/llms.txt),
+ [lmstudio.ai/llms.txt](https://lmstudio.ai/llms.txt), [recipes.vllm.ai/llms.txt](https://recipes.vllm.ai/llms.txt),
+ [docs.sglang.io/llms.txt](https://docs.sglang.io/llms.txt); vLLM and mlx-lm docs as raw markdown on GitHub.
+4. Nothing says it: tell the person this model cannot be set up reliably here and offer the next one.
+
+`fleet models` reads this computer only; a Harness-linked machine's disk is not visible yet.
diff --git a/store/agents/autonomous-grid/test/candidates.test.mjs b/store/agents/autonomous-grid/test/candidates.test.mjs
new file mode 100644
index 000000000..b1f8c20fe
--- /dev/null
+++ b/store/agents/autonomous-grid/test/candidates.test.mjs
@@ -0,0 +1,54 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { estimateBytes, kvBytesFromConfig, mlxCandidates } from '../lib/candidates.mjs';
+import { parseNvidiaSmi, runnableEngines } from '../lib/inventory.mjs';
+
+test('the filter estimate reads the Hub\'s unpacked parameter count at the repo\'s bit width', () => {
+ // A 4B model at 4 bits: the Hub reports ~4.0e9 "U32"; its files are 2.26 GB.
+ assert.equal(estimateBytes({ total: 4_022_468_096 }, 4), Math.round(4_022_468_096 * 0.5 * 1.15));
+ assert.equal(estimateBytes(undefined, 4), 0);
+});
+
+test('cache per token counts only the layers that keep one', () => {
+ const dense = { num_hidden_layers: 28, num_attention_heads: 16, num_key_value_heads: 8, head_dim: 128 };
+ assert.equal(kvBytesFromConfig(dense), 28 * 8 * 128 * 4);
+ assert.equal(kvBytesFromConfig({ ...dense, full_attention_interval: 4 }), 7 * 8 * 128 * 4);
+ assert.equal(kvBytesFromConfig({ text_config: { ...dense, layer_types: ['linear_attention', 'full_attention', 'linear_attention', 'full_attention'] } }), 2 * 8 * 128 * 4);
+ assert.equal(kvBytesFromConfig({ num_hidden_layers: 4 }), null);
+});
+
+test('MLX candidates are chat models that fit the budget at the asked context, sized without downloading', async () => {
+ const GB = 1024 ** 3;
+ const rows = [
+ { id: 'mlx-community/Small-4bit', pipeline_tag: 'text-generation', downloads: 900, gated: false, config: { quantization_config: { bits: 4 }, architectures: ['SmallForCausalLM'], tokenizer_config: { chat_template: '{% if tools %}{% endif %}' } }, safetensors: { total: 4 * GB } },
+ { id: 'mlx-community/Huge-4bit', pipeline_tag: 'text-generation', downloads: 800, gated: false, config: { quantization_config: { bits: 4 } }, safetensors: { total: 60 * GB } },
+ { id: 'mlx-community/Speech', pipeline_tag: 'automatic-speech-recognition', downloads: 5000, gated: false, config: { quantization_config: { bits: 4 } }, safetensors: { total: GB } },
+ { id: 'mlx-community/Locked-4bit', pipeline_tag: 'text-generation', downloads: 700, gated: 'manual', config: { quantization_config: { bits: 4 } }, safetensors: { total: GB } },
+ ];
+ const fetchImpl = async url => {
+ if (url.startsWith('https://huggingface.co/api/models?')) {
+ assert.match(url, /author=mlx-community/);
+ return new Response(JSON.stringify(rows));
+ }
+ if (url === 'https://huggingface.co/api/models/mlx-community/Small-4bit?blobs=true') {
+ return new Response(JSON.stringify({ siblings: [{ rfilename: 'model.safetensors', size: 2.26 * GB }, { rfilename: 'tokenizer.json', size: 11e6 }] }));
+ }
+ if (url.endsWith('/Small-4bit/raw/main/config.json')) return new Response(JSON.stringify({ num_hidden_layers: 24, num_attention_heads: 16, num_key_value_heads: 4, head_dim: 128, max_position_embeddings: 131072 }));
+ return new Response('{}', { status: 404 });
+ };
+ const found = await mlxCandidates({ budgetBytes: 20 * GB, fetchImpl });
+ assert.deepEqual(found.candidates.map(c => c.id), ['mlx-community/Small-4bit']);
+ const [small] = found.candidates;
+ assert.equal(small.bytes, 2.26 * GB, 'the real file size, not the parameter count');
+ assert.equal(small.kvBytesPerToken, 24 * 4 * 128 * 4);
+ assert.equal(small.fits, true);
+ assert.equal(small.toolCalls, true);
+});
+
+test('a GPU counts only when its own tool answers; the engines a machine can run follow from it', () => {
+ const [gpu] = parseNvidiaSmi('0, NVIDIA RTX 4090, 24564, 1200, 3, 580.1\n');
+ assert.deepEqual([gpu.name, gpu.totalBytes, gpu.active, gpu.driver], ['NVIDIA RTX 4090', 24564 * 1024 ** 2, true, '580.1']);
+ assert.deepEqual(runnableEngines('darwin', 'arm64', []), ['llama.cpp', 'mlx-lm', 'ollama', 'lm-studio']);
+ assert.deepEqual(runnableEngines('linux', 'x64', [gpu]), ['llama.cpp', 'ollama', 'vllm', 'sglang']);
+ assert.deepEqual(runnableEngines('linux', 'x64', [{ vendor: 'nvidia', active: false }]), ['llama.cpp', 'ollama']);
+});
diff --git a/store/agents/autonomous-grid/test/inventory.test.mjs b/store/agents/autonomous-grid/test/inventory.test.mjs
new file mode 100644
index 000000000..457d018f8
--- /dev/null
+++ b/store/agents/autonomous-grid/test/inventory.test.mjs
@@ -0,0 +1,213 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { createServer } from 'node:http';
+import { mkdir, mkdtemp, symlink, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import {
+ defaultRoots, joinedGrids, modelFamily, parseListDevices, parseListeningPorts, parseMeminfo, parseNetstatListening,
+ parseProcNetTcp, parseSwapUsage, parseVmStat,
+ probeEngines, readGguf, scanModels, summarize, summarizeGguf,
+} from '../lib/inventory.mjs';
+
+const u32 = n => { const b = Buffer.alloc(4); b.writeUInt32LE(n); return b; };
+const u64 = n => { const b = Buffer.alloc(8); b.writeBigUInt64LE(BigInt(n)); return b; };
+const str = s => Buffer.concat([u64(Buffer.byteLength(s)), Buffer.from(s)]);
+const kv = {
+ string: (key, value) => Buffer.concat([str(key), u32(8), str(value)]),
+ u32: (key, value) => Buffer.concat([str(key), u32(4), u32(value)]),
+ i32s: (key, list) => Buffer.concat([str(key), u32(9), u32(5), u64(list.length), ...list.map(u32)]),
+ strings: (key, list) => Buffer.concat([str(key), u32(9), u32(8), u64(list.length), ...list.map(str)]),
+};
+const gguf = (pairs, tensorTypes = [12]) => Buffer.concat([
+ Buffer.from('GGUF'), u32(3), u64(tensorTypes.length), u64(pairs.length), ...pairs,
+ ...tensorTypes.map((type, i) => Buffer.concat([str(`blk.${i}.weight`), u32(2), u64(4), u64(4), u32(type), u64(0)])),
+]);
+const put = async (path, data) => { await mkdir(dirname(path), { recursive: true }); await writeFile(path, data); };
+const tiny = arch => gguf([kv.string('general.architecture', arch), kv.u32(`${arch}.block_count`, 2)]);
+
+test('hybrid attention caches KV only on its full-attention layers; foreign tensor types are flagged', async () => {
+ const file = join(await mkdtemp(join(tmpdir(), 'gguf-')), 'qwen.gguf');
+ await writeFile(file, gguf([
+ kv.string('general.architecture', 'qwen35'), kv.string('general.size_label', '4B'),
+ kv.strings('tokenizer.ggml.tokens', Array.from({ length: 5000 }, (_, i) => `token-${i}`)),
+ kv.u32('qwen35.block_count', 32), kv.u32('qwen35.context_length', 262144),
+ kv.u32('qwen35.attention.head_count', 16), kv.u32('qwen35.attention.head_count_kv', 4),
+ kv.u32('qwen35.attention.key_length', 256), kv.u32('qwen35.attention.value_length', 256),
+ kv.u32('qwen35.full_attention_interval', 4),
+ kv.string('tokenizer.chat_template', '{% if tools %}<tools>{% endif %}{% if enable_thinking %}<think>{% endif %}'),
+ ], [12, 143]));
+ const summary = summarizeGguf(await readGguf(file));
+ // llama.cpp measured 1024 MiB for 32768 cells on 8 layers of Qwen3.5-4B: 32 KiB per token.
+ assert.equal(summary.kvBytesPerToken, 32768);
+ assert.equal(summary.kvBasis, 'f16 cache');
+ assert.deepEqual([summary.architecture, summary.parameters, summary.contextLength], ['qwen35', '4B', 262144]);
+ assert.equal(summary.toolCalls, true);
+ assert.equal(summary.thinking, true);
+ assert.deepEqual(summary.unsupportedTensorTypes, [143]);
+});
+
+test('per-layer KV heads count only the attention layers, and non-GGUF files are ignored', async () => {
+ const dir = await mkdtemp(join(tmpdir(), 'gguf-'));
+ await writeFile(join(dir, 'lfm.gguf'), gguf([
+ kv.string('general.architecture', 'lfm2'), kv.u32('lfm2.block_count', 6), kv.u32('lfm2.embedding_length', 1024),
+ kv.u32('lfm2.attention.head_count', 16), kv.i32s('lfm2.attention.head_count_kv', [0, 0, 8, 0, 8, 0]),
+ ]));
+ assert.equal(summarizeGguf(await readGguf(join(dir, 'lfm.gguf'))).kvBytesPerToken, 16 * (64 + 64) * 2);
+ await writeFile(join(dir, 'blob'), 'not a model');
+ assert.equal(await readGguf(join(dir, 'blob')), null);
+});
+
+test('scan finds models other apps downloaded, names them, and reports one entry per real file', async () => {
+ const home = await mkdtemp(join(tmpdir(), 'home-'));
+ const ollama = join(home, '.ollama', 'models');
+ await put(join(ollama, 'blobs', 'sha256-aaa'), tiny('qwen2'));
+ await put(join(ollama, 'manifests', 'registry.ollama.ai', 'library', 'qwen2.5', '0.5b'),
+ JSON.stringify({ layers: [{ mediaType: 'application/vnd.ollama.image.model', digest: 'sha256:aaa' }] }));
+ await mkdir(join(home, '.grid', 'models'), { recursive: true });
+ await symlink(join(ollama, 'blobs', 'sha256-aaa'), join(home, '.grid', 'models', 'qwen2.5-0.5b.gguf'));
+ const studio = join(home, '.lmstudio', 'models', 'publisher', 'repo');
+ await put(join(studio, 'Model-7B-Q4_K_M.gguf'), tiny('llama'));
+ await put(join(studio, 'mmproj-Model-7B-F16.gguf'), tiny('clip'));
+ const snapshot = join(home, '.cache', 'huggingface', 'hub', 'models--mlx-community--Tiny-4bit', 'snapshots', 'abc');
+ await put(join(snapshot, 'config.json'), JSON.stringify({ architectures: ['Qwen2ForCausalLM'], quantization: { bits: 4, group_size: 64 } }));
+ await put(join(snapshot, 'model.safetensors'), 'weights');
+ await put(join(home, 'models', 'plain', 'config.json'), JSON.stringify({ architectures: ['LlamaForCausalLM'], max_position_embeddings: 8192 }));
+ await put(join(home, 'models', 'plain', 'model-00001-of-00001.safetensors'), 'weights');
+ await put(join(home, 'Downloads', 'notes.txt'), 'not a model');
+
+ const { models } = await scanModels({ roots: defaultRoots({}, home).filter(root => root.path.startsWith(home)), minBytes: 0 });
+ const byName = Object.fromEntries(models.map(model => [model.name, model]));
+ assert.equal(models.length, 4);
+ assert.equal(byName['qwen2.5-0.5b'].source, 'grid');
+ assert.deepEqual(byName['qwen2.5-0.5b'].alsoAt.map(seen => [seen.source, seen.name]), [['ollama', 'qwen2.5:0.5b']]);
+ assert.equal(byName['Model-7B-Q4_K_M'].quant, 'Q4_K_M');
+ assert.match(byName['Model-7B-Q4_K_M'].projector, /mmproj-Model-7B-F16\.gguf$/);
+ assert.equal(byName['mlx-community/Tiny-4bit'].format, 'mlx');
+ assert.equal(byName['mlx-community/Tiny-4bit'].config.quantization, '4-bit');
+ assert.equal(byName.plain.format, 'safetensors');
+ assert.ok(byName.plain.readableBy.includes('vllm'));
+ assert.ok(byName['qwen2.5-0.5b'].readableBy.includes('llama.cpp'));
+});
+
+test('the summary table lists every model of every format, with the reasons one cannot be used', () => {
+ const text = summarize({
+ machine: { chip: 'Apple M1 Pro', platform: 'darwin', arch: 'arm64', memory: { totalBytes: 34e9, availableBytes: 12e9, swapUsedBytes: 11e9 },
+ accelerators: [{ vendor: 'apple', name: 'Apple M1 Pro', active: true, totalBytes: 26.8e9 }], engines: [{ kind: 'ollama', version: '0.34.4' }],
+ canRun: ['llama.cpp', 'mlx-lm'], listeningPorts: [8082] },
+ engines: [{ kind: 'llama.cpp', url: 'http://127.0.0.1:8082/v1', models: ['lfm'] }],
+ models: [
+ { name: 'big-gguf', source: 'grid', format: 'gguf', bytes: 6.1e9, projector: null, alsoAt: [], readableBy: ['llama.cpp'],
+ gguf: { contextLength: 262144, kvBytesPerToken: 32768, toolCalls: true, unsupportedTensorTypes: [] } },
+ { name: 'odd-gguf', source: 'folder', format: 'gguf', bytes: 5.9e9, projector: null, alsoAt: [], readableBy: ['llama.cpp'],
+ gguf: { contextLength: 262144, kvBytesPerToken: 65536, toolCalls: true, unsupportedTensorTypes: [143] } },
+ { name: 'org/Model-9B-4bit', source: 'huggingface', format: 'mlx', bytes: 5.95e9, projector: null, alsoAt: [], readableBy: ['mlx-lm'],
+ gguf: null, config: { contextLength: 262144, kvBytesPerToken: 32768, toolCalls: true } },
+ { name: 'small-ctx', source: 'ollama', format: 'gguf', bytes: 0.4e9, projector: null, alsoAt: [{ source: 'lm-studio' }], readableBy: ['llama.cpp'],
+ gguf: { contextLength: 32768, kvBytesPerToken: 12288, toolCalls: true, unsupportedTensorTypes: [] } },
+ ],
+ });
+ assert.match(text, /^machine\s+Apple M1 Pro/m);
+ assert.match(text, /apple Apple M1 Pro active/);
+ assert.match(text, /mlx .*org\/Model-9B-4bit \(huggingface\)/, 'MLX models appear, never filtered out');
+ assert.match(text, /odd-gguf .*llama\.cpp cannot load \(tensor type 143\)/);
+ assert.match(text, /small-ctx \(ollama, also lm-studio\)\s+! context 32768 < 65536/);
+ assert.match(text, /answering llama\.cpp http:\/\/127\.0\.0\.1:8082\/v1/);
+ assert.match(text, /^joined\s+none$/m);
+});
+
+test('joined grids come from Grid run files with a live PID, whatever the relay says', async () => {
+ const home = await mkdtemp(join(tmpdir(), 'grid-home-'));
+ const write = async (dir, state) => { await mkdir(join(home, 'run', 'engines', dir), { recursive: true }); await writeFile(join(home, 'run', 'engines', dir, 'remote.json'), JSON.stringify(state)); };
+ await write('grid-aaa', { grid_id: 'grid-aaa', pid: 111, advertise_as: [], models: ['lfm.gguf'], engines: [{ endpoint_url: null, models: ['lfm.gguf'] }] });
+ await write('grid-bbb', { grid_id: 'grid-bbb', pid: 222, advertise_as: ['Coder'], models: ['/snap'], engines: [{ endpoint_url: 'http://127.0.0.1:8084/v1' }] });
+ await write('grid-gone', { grid_id: 'grid-gone', pid: 333, models: ['old.gguf'] });
+ await write('n1', {});
+ const rows = await joinedGrids(home, { alive: pid => pid !== 333 });
+ assert.deepEqual(rows.map(r => [r.gridId, r.models, r.at]).sort(), [
+ ['grid-aaa', ['lfm.gguf'], null], ['grid-bbb', ['Coder'], 'http://127.0.0.1:8084/v1']]);
+ assert.deepEqual(await joinedGrids(join(home, 'missing')), []);
+ const text = summarize({
+ machine: { platform: 'darwin', arch: 'arm64', accelerators: [], engines: [], canRun: [], listeningPorts: [] }, engines: [], models: [],
+ joined: [{ grid: 'my-grid', gridId: 'grid-aaa', models: ['lfm.gguf'], at: null }],
+ });
+ assert.match(text, /^joined\s+my-grid serving \[lfm\.gguf\] — a join to that grid adds to what it serves \(Grid 0\.3\.53\+\)$/m);
+});
+
+test('engines are identified by what they answer, and the --at URL always carries /v1', async () => {
+ const serve = routes => new Promise(resolve => {
+ const server = createServer((req, res) => {
+ const body = routes[req.url];
+ if (!body) { res.statusCode = 404; res.end(); return; }
+ res.setHeader('content-type', 'application/json'); res.end(JSON.stringify(body));
+ });
+ server.listen(0, '127.0.0.1', () => resolve(server));
+ });
+ const servers = await Promise.all([
+ serve({ '/v1/models': { data: [{ id: 'qwen', owned_by: 'vllm' }] } }),
+ serve({ '/v1/models': { data: [{ id: 'qwen2.5:0.5b', owned_by: 'library' }] }, '/api/version': { version: '0.34.4' } }),
+ serve({ '/v1/models': { models: [{ name: 'lfm' }], data: [{ id: 'lfm', owned_by: 'llamacpp' }] } }),
+ serve({ '/v1/models': { data: [{ id: 'mlx-community/x' }] } }),
+ serve({}),
+ ]);
+ try {
+ const ports = servers.map(server => server.address().port);
+ const found = await probeEngines(ports);
+ assert.deepEqual(found.map(engine => engine.kind), ['vllm', 'ollama', 'llama.cpp', 'openai-compatible']);
+ assert.ok(found.every(engine => engine.url === `http://127.0.0.1:${engine.port}/v1`));
+ assert.deepEqual(found[1].models, ['qwen2.5:0.5b']);
+ } finally { servers.forEach(server => server.close()); }
+});
+
+test('system readings parse the exact text macOS, Linux, lsof and llama-server print', () => {
+ assert.equal(parseVmStat('Mach Virtual Memory Statistics: (page size of 16384 bytes)\nPages free: 10.\nPages active: 500.\nPages inactive: 20.\nPages speculative: 5.\nPages purgeable: 1.\n'), 36 * 16384);
+ assert.equal(parseSwapUsage('total = 11264.00M used = 10945.44M free = 318.56M (encrypted)').used, 10945.44 * 1024 ** 2);
+ assert.deepEqual(parseMeminfo('MemTotal: 1000 kB\nMemAvailable: 600 kB\nSwapTotal: 200 kB\nSwapFree: 50 kB\n'), { total: 1000 * 1024, available: 600 * 1024, swapUsed: 150 * 1024 });
+ assert.deepEqual(parseListDevices('Available devices:\n MTL0: Apple M1 Pro (25559 MiB, 25558 MiB free)\n BLAS: Accelerate (0 MiB, 0 MiB free)\n'),
+ [{ id: 'MTL0', name: 'Apple M1 Pro', totalBytes: 25559 * 1024 ** 2, freeBytes: 25558 * 1024 ** 2 }]);
+ assert.deepEqual(parseListeningPorts('p123\nf4\nn*:8082\nf5\nn127.0.0.1:8130\np9\nf3\nn[::1]:631\n'), [631, 8082, 8130]);
+});
+
+test('listening ports are read on Linux without lsof, and on Windows from netstat', () => {
+ const proc = [
+ ' sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode',
+ ' 0: 00000000:1F90 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 12345 1 0 100 0 0 10 0',
+ ' 1: 0100007F:2328 0100007F:A1B2 01 00000000:00000000 00:00000000 00000000 0 0 12346 1 0 20 4 30 10 -1',
+ ' 0: 00000000000000000000000000000000:2CAA 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 1 1 0 100 0 0 10 0',
+ ].join('\n');
+ assert.deepEqual(parseProcNetTcp(proc), [8080, 11434], 'only LISTEN (0A) rows, IPv4 and IPv6');
+ const netstat = [
+ 'Active Connections', '', ' Proto Local Address Foreign Address State PID',
+ ' TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 1016',
+ ' TCP 127.0.0.1:11434 0.0.0.0:0 LISTENING 4812',
+ ' TCP 192.168.1.5:52100 20.1.2.3:443 ESTABLISHED 7000',
+ ' TCP [::]:1234 [::]:0 LISTENING 5000',
+ ' UDP 0.0.0.0:5353 *:* 2100',
+ ].join('\r\n');
+ assert.deepEqual(parseNetstatListening(netstat), [135, 1234, 11434]);
+});
+
+test('one model in two formats is one model, and on a Mac with mlx-lm the GGUF row points at the MLX copy', () => {
+ assert.equal(modelFamily('Qwen3.5-9B-UD-Q4_K_XL'), 'qwen3.5-9b');
+ assert.equal(modelFamily('mlx-community/Qwen3.5-9B-4bit'), 'qwen3.5-9b');
+ assert.equal(modelFamily('qwen2.5-0.5b-instruct-q8_0'), 'qwen2.5-0.5b-instruct');
+ assert.equal(modelFamily('Qwen3.5-9B-MTP-Q6_K'), 'qwen3.5-9b-mtp', 'a variant stays its own model');
+ const row = (name, format, readableBy) => ({ name, source: 'x', format, bytes: 6e9, projector: null, alsoAt: [], readableBy, gguf: format === 'gguf' ? { unsupportedTensorTypes: [] } : null, config: {} });
+ const models = [row('Qwen3.5-9B-UD-Q4_K_XL', 'gguf', ['llama.cpp']), row('mlx-community/Qwen3.5-9B-4bit', 'mlx', ['mlx-lm'])];
+ const mac = { platform: 'darwin', arch: 'arm64', accelerators: [], engines: [{ kind: 'mlx-lm' }], canRun: ['llama.cpp', 'mlx-lm'], listeningPorts: [] };
+ assert.match(summarize({ machine: mac, engines: [], models }), /Qwen3\.5-9B-UD-Q4_K_XL .*an MLX copy is on disk \(mlx-community\/Qwen3\.5-9B-4bit\)/);
+ assert.doesNotMatch(summarize({ machine: { ...mac, engines: [] }, engines: [], models }), /MLX copy/, 'no MLX engine installed, no pointer');
+});
+
+test('a model no engine on this machine can run says so, instead of looking usable', () => {
+ const text = summarize({
+ machine: { platform: 'linux', arch: 'x86_64', accelerators: [], engines: [], canRun: ['llama.cpp', 'ollama'], listeningPorts: [] },
+ engines: [],
+ models: [
+ { name: 'org/Model-4bit', source: 'huggingface', format: 'mlx', bytes: 5e9, projector: null, alsoAt: [], readableBy: ['mlx-lm', 'lm-studio'], config: {} },
+ { name: 'model-q4', source: 'grid', format: 'gguf', bytes: 5e9, projector: null, alsoAt: [], readableBy: ['llama.cpp'], gguf: { unsupportedTensorTypes: [] } },
+ ],
+ });
+ assert.match(text, /org\/Model-4bit .*! no engine this machine can run reads mlx \(Apple silicon only\)/);
+ assert.doesNotMatch(text, /model-q4 .*no engine/);
+});
diff --git a/store/agents/autonomous-grid/test/modelFacts.test.mjs b/store/agents/autonomous-grid/test/modelFacts.test.mjs
new file mode 100644
index 000000000..4023aacd2
--- /dev/null
+++ b/store/agents/autonomous-grid/test/modelFacts.test.mjs
@@ -0,0 +1,70 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { mkdtemp, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { modelFacts } from '../lib/modelFacts.mjs';
+
+const template = '{%- if tools %}<tools>{{ tools }}</tools>\nReturn <tool_call>{"name": ..., "arguments": ...}</tool_call>{%- endif %}{% if enable_thinking %}<think>{% endif %}';
+const card = [
+ '# Widget-7B', '', 'Serve it:', '', '```bash', 'vllm serve Acme/Widget-7B --enable-auto-tool-choice --tool-call-parser widget --reasoning-parser widget_r', '```',
+ '', '```python', 'print("not a serve command")', '```',
+].join('\n');
+const hub = {
+ 'https://huggingface.co/api/models/Acme/Widget-7B': { gated: false, siblings: ['config.json', 'generation_config.json', 'tokenizer_config.json', 'README.md'].map(rfilename => ({ rfilename })) },
+ 'https://huggingface.co/Acme/Widget-7B/raw/main/config.json': { architectures: ['WidgetForCausalLM'], model_type: 'widget', max_position_embeddings: 131072, torch_dtype: 'bfloat16' },
+ 'https://huggingface.co/Acme/Widget-7B/raw/main/generation_config.json': { temperature: 0.7, top_p: 0.8, eos_token_id: 2 },
+ 'https://huggingface.co/Acme/Widget-7B/raw/main/tokenizer_config.json': { chat_template: template },
+ 'https://huggingface.co/Acme/Widget-7B/raw/main/README.md': card,
+ 'https://huggingface.co/api/models/Acme/Locked-7B': { gated: 'manual', siblings: [{ rfilename: 'config.json' }] },
+ 'https://raw.githubusercontent.com/vllm-project/vllm/main/docs/models/supported_models.md': '| `WidgetForCausalLM` | Widget | `Acme/Widget-7B` | ✅︎ |\n| `OtherForCausalLM` | Other |',
+ 'https://raw.githubusercontent.com/sgl-project/sglang/main/python/sglang/srt/models/widget.py': 'class WidgetForCausalLM:\n pass\n\nEntryClass = WidgetForCausalLM\n',
+ 'https://huggingface.co/api/models/Acme/Gizmo-3B': { gated: false, siblings: [{ rfilename: 'config.json' }] },
+ 'https://huggingface.co/Acme/Gizmo-3B/raw/main/config.json': { architectures: ['GizmoForCausalLM'], model_type: 'gizmo' },
+};
+const fetchImpl = async url => {
+ const body = hub[url];
+ if (body === undefined) return new Response('', { status: url.includes('/Locked-7B/raw/') ? 401 : 404 });
+ return new Response(typeof body === 'string' ? body : JSON.stringify(body), { status: 200 });
+};
+
+test('a model with no recipe is read from its own files: architecture, context, sampling, tool syntax, card commands, support', async () => {
+ const facts = await modelFacts('Acme/Widget-7B', { fetchImpl, token: undefined });
+ assert.deepEqual([facts.found, facts.gated, facts.architecture, facts.modelType, facts.contextLength, facts.dtype],
+ [true, false, 'WidgetForCausalLM', 'widget', 131072, 'bfloat16']);
+ assert.deepEqual(facts.samplingDefaults, { temperature: 0.7, top_p: 0.8 });
+ assert.equal(facts.chatTemplate.toolCalls, true);
+ assert.equal(facts.chatTemplate.thinking, true);
+ assert.match(facts.chatTemplate.toolSyntax, /<tool_call>\{"name"/);
+ assert.equal(facts.modelCard.serveCommands.length, 1);
+ assert.deepEqual(facts.modelCard.toolCallParsers, ['widget']);
+ assert.deepEqual(facts.modelCard.reasoningParsers, ['widget_r']);
+ assert.equal(facts.support.vllm.listed, true);
+ assert.equal(facts.support.sglang.listed, true);
+});
+
+test('engine support is only claimed from the engine\'s own list; an absent registry file means unknown, not unsupported', async () => {
+ const facts = await modelFacts('Acme/Gizmo-3B', { fetchImpl, token: undefined });
+ assert.equal(facts.support.vllm.listed, false);
+ assert.equal(facts.support.sglang.listed, null);
+ assert.match(facts.support.sglang.note, /model-impl transformers/);
+});
+
+test('a gated or unknown model is reported as such, never filled in', async () => {
+ const locked = await modelFacts('Acme/Locked-7B', { fetchImpl, token: undefined });
+ assert.equal(locked.gated, true);
+ assert.match(locked.note, /accept its license/);
+ const missing = await modelFacts('Acme/Nothing-1B', { fetchImpl, token: undefined });
+ assert.equal(missing.found, false);
+ await assert.rejects(modelFacts('not an id', { fetchImpl }), /ORG\/NAME/);
+});
+
+test('a model folder already on disk is read in place', async () => {
+ const dir = await mkdtemp(join(tmpdir(), 'model-'));
+ await writeFile(join(dir, 'config.json'), JSON.stringify({ architectures: ['WidgetForCausalLM'], max_position_embeddings: 65536, quantization: { bits: 4, group_size: 64 } }));
+ await writeFile(join(dir, 'chat_template.jinja'), template);
+ const facts = await modelFacts(dir, { fetchImpl, checkSupport: false });
+ assert.deepEqual([facts.source, facts.architecture, facts.contextLength, facts.quantization], [dir, 'WidgetForCausalLM', 65536, 'mlx 4-bit']);
+ assert.equal(facts.chatTemplate.toolCalls, true);
+ assert.equal(facts.support, null);
+});
diff --git a/store/agents/autonomous-grid/test/recipes.test.mjs b/store/agents/autonomous-grid/test/recipes.test.mjs
new file mode 100644
index 000000000..439a65d21
--- /dev/null
+++ b/store/agents/autonomous-grid/test/recipes.test.mjs
@@ -0,0 +1,97 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { NoRecipe, sglangCookbook, vllmRecipe } from '../lib/recipes.mjs';
+
+const site = routes => async url => {
+ const body = routes[url];
+ if (body === undefined) return new Response('not found', { status: 404 });
+ return new Response(typeof body === 'string' ? body : JSON.stringify(body), { status: 200 });
+};
+
+const vllmSite = site({
+ 'https://recipes.vllm.ai/models.json': [
+ { hf_id: 'Acme/Widget-7B', url: '/Acme/Widget-7B', json: '/Acme/Widget-7B.json' },
+ { hf_id: 'Acme/Widget-70B', url: '/Acme/Widget-70B', json: '/Acme/Widget-70B.json' },
+ ],
+ 'https://recipes.vllm.ai/Acme/Widget-7B.json': {
+ hf_id: 'Acme/Widget-7B',
+ model: { model_id: 'Acme/Widget-7B', min_vllm_version: '0.17.0', context_length: 262144, base_args: ['--trust-remote-code'], base_env: {} },
+ features: {
+ tool_calling: { args: ['--enable-auto-tool-choice', '--tool-call-parser', 'widget'] },
+ reasoning: { args: ['--reasoning-parser', 'widget'] },
+ spec_decoding: { args: ['--speculative-config', '{"method":"mtp"}'] },
+ },
+ opt_in_features: ['spec_decoding'],
+ variants: { default: { precision: 'bf16', vram_minimum_gb: 22 }, fp8: { model_id: 'Acme/Widget-7B-FP8', precision: 'fp8', vram_minimum_gb: 11, extra_args: ['--kv-cache-dtype', 'fp8'] } },
+ recommended_command: { command: 'vllm serve Acme/Widget-7B', docker_image: 'vllm/vllm-openai:latest' },
+ guide: '## Troubleshooting\n- lower the capture size',
+ },
+});
+
+test('a vLLM recipe gives the minimum version, the flags an agent always needs, and the variants by GPU memory', async () => {
+ const found = await vllmRecipe('acme/widget-7b', { fetchImpl: vllmSite });
+ assert.equal(found.source, 'https://recipes.vllm.ai/Acme/Widget-7B');
+ assert.equal(found.minVersion, '0.17.0');
+ assert.deepEqual(found.baseArgs, ['--trust-remote-code']);
+ assert.deepEqual(found.features.tool_calling, { optIn: false, args: ['--enable-auto-tool-choice', '--tool-call-parser', 'widget'], env: {}, description: '' });
+ assert.equal(found.features.spec_decoding.optIn, true);
+ assert.deepEqual(found.variants.fp8, { modelId: 'Acme/Widget-7B-FP8', precision: 'fp8', vramMinimumGb: 11, extraArgs: ['--kv-cache-dtype', 'fp8'], description: '' });
+ assert.equal(found.variants.default.modelId, 'Acme/Widget-7B');
+ assert.match(found.guide, /Troubleshooting/);
+});
+
+test('no vLLM recipe is an explicit answer, never a sibling model standing in for it', async () => {
+ await assert.rejects(vllmRecipe('Acme/Widget-3B', { fetchImpl: vllmSite }), NoRecipe);
+ await assert.rejects(vllmRecipe('not an id', { fetchImpl: vllmSite }), /ORG\/NAME/);
+});
+
+const cookbookPage = `# Widget
+
+## 1. Model Introduction
+Text.
+
+## 2. SGLang Installation
+
+SGLang 0.5.9 or newer is required.
+
+\`\`\`bash Command theme={null}
+uv pip install sglang
+\`\`\`
+
+## 3. Model Deployment
+
+### 3.2 Configuration Tips
+* Lower \`--mem-fraction-static\` when images are large.
+
+## 4. Model Invocation
+
+\`\`\`shell Command theme={null}
+sglang serve \\
+ --model-path Acme/Widget-2.5-7B \\
+ --reasoning-parser widget \\
+ --tool-call-parser widget_xml \\
+ --host 0.0.0.0 --port 30000
+\`\`\`
+`;
+const sglangSite = site({
+ 'https://docs.sglang.io/llms.txt': [
+ '- [Widget](https://docs.sglang.io/cookbook/autoregressive/Acme/Widget.md): older family',
+ '- [Widget 2.5](https://docs.sglang.io/cookbook/autoregressive/Acme/Widget-2.5.md): current family',
+ '- [Image](https://docs.sglang.io/cookbook/diffusion/Acme/Paint.md): not a language model',
+ ].join('\n'),
+ 'https://docs.sglang.io/cookbook/autoregressive/Acme/Widget-2.5.md': cookbookPage,
+});
+
+test('the SGLang cookbook page is the most specific family match, with its commands, parsers and notes', async () => {
+ const found = await sglangCookbook('Acme/Widget-2.5-7B-Instruct', { fetchImpl: sglangSite });
+ assert.equal(found.source, 'https://docs.sglang.io/cookbook/autoregressive/Acme/Widget-2.5.md');
+ assert.deepEqual(found.toolCallParsers, ['widget_xml']);
+ assert.deepEqual(found.reasoningParsers, ['widget']);
+ assert.equal(found.serveCommands.length, 1);
+ assert.match(found.installation, /0\.5\.9 or newer/);
+ assert.match(found.configurationTips, /mem-fraction-static/);
+});
+
+test('no SGLang cookbook page is an explicit answer, never a guess', async () => {
+ await assert.rejects(sglangCookbook('Other/Gadget-1B', { fetchImpl: sglangSite }), NoRecipe);
+});
diff --git a/store/agents/autonomous-grid/test/remoteModels.test.mjs b/store/agents/autonomous-grid/test/remoteModels.test.mjs
new file mode 100644
index 000000000..7b103bdbc
--- /dev/null
+++ b/store/agents/autonomous-grid/test/remoteModels.test.mjs
@@ -0,0 +1,89 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { EventEmitter } from 'node:events';
+import { PassThrough } from 'node:stream';
+import { mkdtemp, readlink, writeFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { FINDER_FILES, linkModel, remoteModels, sshBase } from '../lib/remoteModels.mjs';
+
+/** A fake `ssh`: records each call and its stdin, and answers the run with `reply(remoteCommand)`. */
+function fakeSsh(reply) {
+ const calls = [];
+ const spawnImpl = (file, args) => {
+ const child = new EventEmitter();
+ child.stdout = new PassThrough(); child.stderr = new PassThrough(); child.stdin = new PassThrough();
+ child.kill = () => {};
+ const call = { file, args, input: '' };
+ calls.push(call);
+ child.stdin.on('data', chunk => { call.input += chunk; });
+ child.stdin.on('finish', () => {
+ const out = reply(args.at(-1));
+ child.stdout.end(out.stdout ?? ''); child.stderr.end(out.stderr ?? '');
+ setImmediate(() => child.emit('close', out.code ?? 0));
+ });
+ return child;
+ };
+ return { calls, spawnImpl };
+}
+
+const machine = { id: 'box', name: 'box', transport: 'ssh', host: 'me@box', port: 2222 };
+
+test('the finder is copied to the machine and run by its own Node', async () => {
+ const ssh = fakeSsh(remote => (remote.startsWith('if ') ? { stdout: 'machine Intel Xeon · linux/x64\n' } : {}));
+ const result = await remoteModels(machine, { spawnImpl: ssh.spawnImpl });
+ assert.equal(result.full, true);
+ assert.match(result.text, /^machine\s+Intel Xeon/);
+ assert.deepEqual(ssh.calls.slice(0, 3).map(c => c.args.at(-1).split('/').pop()), FINDER_FILES);
+ assert.match(ssh.calls[0].input, /export async function inventory/, 'the real finder source is sent');
+ assert.match(ssh.calls.at(-1).args.at(-1), /exec node \.cache\/grid-harness\/finder\/finderMain\.mjs --summary/);
+ assert.deepEqual(ssh.calls[0].args.slice(0, 9), sshBase(machine).slice(0, 9));
+ assert.ok(ssh.calls.every(c => c.file === 'ssh' && c.args.includes('BatchMode=yes') && c.args.includes('StrictHostKeyChecking=yes')));
+});
+
+test('without Node there it still lists the model files, and says what is missing', async () => {
+ const ssh = fakeSsh(remote => (remote.startsWith('if ')
+ ? { stdout: '__GRID_FINDER_NO_NODE__\nLinux x86_64\n/root/.ollama/models/manifests/registry.ollama.ai/library/qwen/0.5b\n' } : {}));
+ const result = await remoteModels(machine, { spawnImpl: ssh.spawnImpl });
+ assert.equal(result.full, false);
+ assert.match(result.text, /Linux x86_64 \(me@box\) — Node 18\+ is not installed there/);
+ assert.match(result.text, /manifests\/registry\.ollama\.ai\/library\/qwen\/0\.5b/);
+});
+
+test('a machine that cannot be reached, or is linked through Harness, fails with the reason', async () => {
+ const down = fakeSsh(() => ({ code: 255, stderr: 'ssh: connect to host box port 2222: Connection refused\n' }));
+ await assert.rejects(remoteModels(machine, { spawnImpl: down.spawnImpl }), /Could not copy the finder to me@box: .*Connection refused/);
+ await assert.rejects(remoteModels({ id: 'air', name: 'MacBook-Air', transport: 'harness', machineId: 'x' }),
+ /Harness, which runs Grid commands only.*device-info/);
+});
+
+test('a model on this machine is linked into Grid\'s folder once, and a taken name is never replaced', async () => {
+ const dir = await mkdtemp(join(tmpdir(), 'link-'));
+ const model = join(dir, 'lfm.gguf'), mmproj = join(dir, 'mmproj-lfm.gguf'), other = join(dir, 'other.gguf');
+ for (const file of [model, mmproj, other]) await writeFile(file, 'GGUF');
+ const env = { GRID_HOME: join(dir, 'grid') }, local = { id: 'local', transport: 'local' };
+
+ const first = await linkModel(local, model, 'LFM.gguf', { projector: mmproj, env });
+ assert.match(first.text, /linked .*LFM\.gguf -> .*lfm\.gguf/);
+ assert.equal(await readlink(join(dir, 'grid', 'models', 'LFM.gguf')), model);
+ assert.equal(await readlink(join(dir, 'grid', 'models', 'LFM.mmproj.gguf')), mmproj, 'the projector name --serve looks for');
+ assert.match(first.text, /NOT READY: Grid's engine \(llama\.cpp\) is not installed.*run -- engine install llama\.cpp/,
+ 'a join would die at start without the engine, so the link says so');
+ assert.match((await linkModel(local, model, 'LFM.gguf', { env })).text, /already linked/);
+ await assert.rejects(linkModel(local, other, 'LFM.gguf', { env }), /already exists and is not a link/);
+ await assert.rejects(linkModel(local, join(dir, 'missing.gguf'), 'M.gguf', { env }), /No file at/);
+ await assert.rejects(linkModel(local, 'relative.gguf', 'M.gguf', { env }), /full paths/);
+ await assert.rejects(linkModel(local, model, 'bad name.gguf', { env }), /plain file name/);
+});
+
+test('on an SSH machine the link is made there, with every path quoted', async () => {
+ const ssh = fakeSsh(() => ({ stdout: 'linked /root/.grid/models/LFM.gguf -> /root/my models/lfm.gguf\n' }));
+ const result = await linkModel(machine, "/root/my models/lfm.gguf", 'LFM.gguf', { spawnImpl: ssh.spawnImpl });
+ assert.match(result.text, /^linked /);
+ const script = ssh.calls[0].args.at(-1);
+ assert.match(script, /link '\/root\/my models\/lfm\.gguf' 'LFM\.gguf'/);
+ assert.match(script, /ln -s "\$1" "\$d\/\$2"/);
+ assert.match(script, /bin\/llama-server".*command -v llama-server.*NOT READY.*--machine box -- engine install llama\.cpp/);
+ assert.match(script, /err="\$\("\$e" --version 2>&1\)".*does not start/, 'installed is not enough: it has to run');
+ await assert.rejects(linkModel({ id: 'air', transport: 'harness', machineId: 'x' }, '/m.gguf', 'M.gguf'), /catalog/);
+});
diff --git a/store/agents/autonomous-grid/test/serve.test.mjs b/store/agents/autonomous-grid/test/serve.test.mjs
new file mode 100644
index 000000000..4890b7081
--- /dev/null
+++ b/store/agents/autonomous-grid/test/serve.test.mjs
@@ -0,0 +1,34 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { execFileSync } from 'node:child_process';
+import { mkdtemp, readFile } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { recordedPid, runFiles, serve, stop } from '../lib/serve.mjs';
+
+const alive = pid => { try { process.kill(pid, 0); return true; } catch { return false; } };
+
+test('an engine started with serve leads its own process group, logs, records its PID, and stop ends only it', async () => {
+ const workspace = await mkdtemp(join(tmpdir(), 'serve-'));
+ const started = await serve(workspace, 'probe', [process.execPath, '-e', 'console.log(process.env.PROBE); setTimeout(() => {}, 60000)'], { env: { PROBE: 'ready' } });
+ try {
+ assert.equal(await recordedPid(workspace, 'probe'), started.pid);
+ assert.ok(alive(started.pid));
+ const group = Number(execFileSync('ps', ['-o', 'pgid=', '-p', String(started.pid)], { encoding: 'utf8' }).trim());
+ assert.equal(group, started.pid, 'its own process group, so the starting shell cannot take it down');
+ await new Promise(resolve => setTimeout(resolve, 300));
+ assert.match(await readFile(runFiles(workspace, 'probe').log, 'utf8'), /ready/);
+ await assert.rejects(serve(workspace, 'probe', [process.execPath, '-e', '']), /already running/);
+ } finally {
+ const stopped = await stop(workspace, 'probe', { graceMs: 2_000 });
+ assert.equal(stopped.stopped, true);
+ }
+ assert.equal(alive(started.pid), false);
+ assert.equal(await recordedPid(workspace, 'probe'), null);
+ assert.equal((await stop(workspace, 'probe')).stopped, false, 'nothing recorded, nothing killed');
+});
+
+test('names cannot reach outside run/', () => {
+ assert.throws(() => runFiles('/tmp/x', '../escape'), /engine name/);
+ assert.throws(() => runFiles('/tmp/x', ''), /engine name/);
+});
diff --git a/store/agents/autonomous-grid/test/verify.test.mjs b/store/agents/autonomous-grid/test/verify.test.mjs
new file mode 100644
index 000000000..f2a426e2b
--- /dev/null
+++ b/store/agents/autonomous-grid/test/verify.test.mjs
@@ -0,0 +1,101 @@
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { createLog, verifyEngine, verifyRelay } from '../lib/verify.mjs';
+
+const json = (body, status = 200) => new Response(JSON.stringify(body), { status });
+const quiet = () => {};
+const instant = async () => {};
+
+function engine({ bootFailures = 1, content = 'ok', reasoning = '', toolCall = true, seen = [], thinksFirst = false } = {}) {
+ let boots = 0;
+ return async (url, init = {}) => {
+ const body = init.body ? JSON.parse(init.body) : null;
+ seen.push({ url, body });
+ if (url.endsWith('/models')) return boots++ < bootFailures ? new Response('loading', { status: 503 }) : json({ data: [{ id: 'widget' }] });
+ if (body.tools) return json({ choices: [{ message: toolCall ? { tool_calls: [{ function: { name: 'read_file', arguments: '{"path":"README.md"}' } }] } : { content: 'I cannot' } }] });
+ if (body.max_tokens === 128) return json({ usage: { completion_tokens: 128 }, timings: { predicted_per_second: 21.5 }, choices: [{ message: { content: 'one two' } }] });
+ // A model an engine keeps thinking: 16 tokens are all reasoning, a larger budget reaches the answer.
+ if (thinksFirst) return json({ choices: [{ message: body.max_tokens > 16 ? { content: 'ok', reasoning_content: 'hmm' } : { content: '', reasoning_content: 'hmm' } }] });
+ return json({ choices: [{ message: { content, reasoning_content: reasoning } }] });
+ };
+}
+
+test('an engine passes only after ready, listed, a real answer, a tool call and a speed reading, each narrated', async () => {
+ const lines = [], seen = [];
+ const result = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ seen }), log: createLog(line => lines.push(line)), sleep: instant });
+ assert.equal(result.ok, true);
+ assert.deepEqual(result.steps.map(step => step.name), ['ready', 'listed', 'answer', 'tool call', 'speed']);
+ assert.equal(result.steps.at(-1).tokensPerSecond, 21.5);
+ assert.ok(lines.some(line => /attempt 1: 503/.test(line)), 'a failed readiness attempt is shown, not hidden');
+ assert.ok(lines.every(line => /^\[\+\s*\d+s\]/.test(line)), 'every line carries elapsed time');
+ assert.deepEqual(seen.find(call => call.body?.max_tokens === 16).body.chat_template_kwargs, { enable_thinking: false });
+});
+
+test('a model that always thinks first passes once it answers with room to think, and says thinking stays on', async () => {
+ const lines = [], seen = [];
+ const result = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ bootFailures: 0, thinksFirst: true, seen }), log: createLog(line => lines.push(line)), sleep: instant });
+ const answer = result.steps.find(step => step.name === 'answer');
+ assert.equal(answer.ok, true);
+ assert.match(answer.note, /"ok" — thinking stays on here/);
+ assert.ok(lines.some(line => /only thinking came back in 16 tokens; asking once more with room for it \(1024 tokens\)/.test(line)));
+ assert.deepEqual(seen.filter(call => call.body?.messages?.[0]?.content?.startsWith('Reply')).map(call => call.body.max_tokens), [16, 1024]);
+});
+
+test('thinking-only output, a missing tool call and a URL without /v1 each fail with the reason', async () => {
+ const thinking = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ bootFailures: 0, content: '', reasoning: 'Let me think' }), log: quiet, sleep: instant });
+ assert.equal(thinking.ok, false);
+ assert.match(thinking.steps.at(-1).note, /thinking is still on/);
+ const noTool = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ bootFailures: 0, toolCall: false }), log: quiet, sleep: instant });
+ assert.equal(noTool.steps.at(-1).name, 'tool call');
+ assert.equal(noTool.ok, false);
+ const root = await verifyEngine({ url: 'http://127.0.0.1:9', model: 'widget', fetchImpl: engine(), log: quiet, sleep: instant });
+ assert.match(root.steps[0].note, /must end in \/v1/);
+});
+
+test('Ollama gets its documented thinking switch; readiness gives up at its deadline', async () => {
+ const seen = [];
+ await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', kind: 'ollama', fetchImpl: engine({ bootFailures: 0, seen }), log: quiet, sleep: instant });
+ assert.equal(seen.find(call => call.body?.max_tokens === 16).body.reasoning_effort, 'none');
+ const never = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', readyMs: 30, fetchImpl: engine({ bootFailures: Infinity }), log: quiet, sleep: instant });
+ assert.equal(never.ok, false);
+ assert.match(never.steps[0].note, /within/);
+});
+
+test('the relay must list the alias before one bounded answer through the grid counts', async () => {
+ let polls = 0;
+ const listModels = async () => ({ ok: true, models: ++polls < 3 ? ['other'] : ['other', 'widget-alias'] });
+ const fetchImpl = async (url, init) => {
+ assert.equal(init.headers.authorization, 'Bearer key');
+ return json({ choices: [{ message: { content: 'ok' } }] });
+ };
+ const lines = [];
+ const result = await verifyRelay({ alias: 'widget-alias', listModels, relay: { baseUrl: 'https://relay.test/v1', apiKey: 'key' }, fetchImpl, log: line => lines.push(line), sleep: instant });
+ assert.equal(result.ok, true);
+ assert.equal(polls, 3);
+ assert.equal(lines.filter(line => /attempt \d+: .*not yet widget-alias/.test(line)).length, 2);
+ const missing = await verifyRelay({ alias: 'gone', listModels: async () => ({ ok: true, models: [] }), relay: { baseUrl: 'https://relay.test/v1' }, listMs: 20, fetchImpl, log: quiet, sleep: instant });
+ assert.equal(missing.ok, false);
+});
+
+test('a model registered under its raw id fails at once with the fix, instead of waiting out the deadline', async () => {
+ let polls = 0;
+ const result = await verifyRelay({
+ alias: 'Coder', engineModel: '/Users/me/.cache/hub/snap', relay: { baseUrl: 'https://relay.test/v1' }, log: quiet, sleep: instant,
+ listModels: async () => { polls++; return { ok: true, models: ['/users/me/.cache/hub/snap'] }; },
+ });
+ assert.equal(result.ok, false);
+ assert.equal(polls, 1);
+ assert.match(result.steps[0].note, /listed as \/users\/me\/\.cache\/hub\/snap, not Coder: join again with --advertise-as Coder/);
+});
+
+test('an alias the relay lists in another case passes at once, and is asked by the listed name', async () => {
+ let polls = 0, asked = null;
+ const result = await verifyRelay({
+ alias: 'lfm2.5-vl-3b', relay: { baseUrl: 'https://relay.test/v1' }, log: quiet, sleep: instant,
+ listModels: async () => { polls++; return { ok: true, models: ['LFM2.5-VL-3B', 'Qwen3-0.6B'] }; },
+ fetchImpl: async (url, init) => { asked = JSON.parse(init.body).model; return json({ choices: [{ message: { content: 'ok' } }] }); },
+ });
+ assert.equal(result.ok, true);
+ assert.equal(polls, 1);
+ assert.equal(asked, 'LFM2.5-VL-3B');
+});
diff --git a/store/agents/autonomous-grid/toolchain/fleet.mjs b/store/agents/autonomous-grid/toolchain/fleet.mjs
index 2d3a34af9..cc075e932 100644
--- a/store/agents/autonomous-grid/toolchain/fleet.mjs
+++ b/store/agents/autonomous-grid/toolchain/fleet.mjs
@@ -1,9 +1,15 @@
#!/usr/bin/env node
-import { verifyModel } from '../lib/verify.mjs';
+import { createLog, verifyEngine, verifyModel, verifyRelay } from '../lib/verify.mjs';
import { join, resolve } from 'node:path';
-import { atomicJson, DEFAULT_CONFIG, execute, harnessNameFor, nameJoin, readConfig, runTracked } from '../lib/fleet.mjs';
+import { atomicJson, DEFAULT_CONFIG, execute, gridJson, harnessNameFor, nameJoin, readConfig, runTracked } from '../lib/fleet.mjs';
import { createCollector } from '../lib/telemetry.mjs';
import { discoverMachines } from '../lib/harness.mjs';
+import { inventory, machineState, summarize } from '../lib/inventory.mjs';
+import { mlxCandidates } from '../lib/candidates.mjs';
+import { recipe } from '../lib/recipes.mjs';
+import { modelFacts } from '../lib/modelFacts.mjs';
+import { linkModel, remoteModels } from '../lib/remoteModels.mjs';
+import { serve, stop } from '../lib/serve.mjs';
import { connectWorkspace, initializeWorkspace, mergeMachines, readStatus } from '../lib/connect.mjs';
const workspace = resolve(process.env.HARNESS_WORKSPACE || process.cwd());
@@ -51,7 +57,8 @@ try {
process.exitCode = snapshot.status === 'unavailable' ? 1 : 0;
} else if (command === 'config') {
console.log(JSON.stringify(await readConfig(workspace), null, 2));
- } else if (command === 'verify') {
+ } else if (command === 'verify' && !args.includes('--at') && !args.includes('--alias')) {
+ // The Models panel's receipt form: one request, published as an operation the panel reads.
const flags = new Map();
for (let i = 0; i < args.length; i += 2) {
if (!['--grid', '--model'].includes(args[i]) || !args[i + 1] || flags.has(args[i])) throw new Error('Use: fleet verify --grid NAME --model MODEL');
@@ -60,6 +67,115 @@ try {
const config = await readConfig(workspace);
const machine = config.machines.find(m => m.id === config.controller);
console.log(JSON.stringify(await verifyModel(workspace, machine, config.mode, flags.get('--grid') || config.grid, flags.get('--model'))));
+ } else if (command === 'models') {
+ const extraRoots = [], extraPorts = [];
+ let summary = false, target;
+ for (let i = 0; i < args.length; i++) {
+ if (args[i] === '--root' && args[i + 1]) extraRoots.push(resolve(args[++i]));
+ else if (args[i] === '--port' && /^\d+$/.test(args[i + 1] || '')) extraPorts.push(Number(args[++i]));
+ else if (args[i] === '--machine' && args[i + 1]) target = args[++i];
+ else if (args[i] === '--summary') summary = true;
+ else if (args[i] !== '--json') throw new Error('Use: fleet models [--machine ID] [--summary] [--root DIR]... [--port N]...');
+ }
+ const config = target ? await readConfig(workspace) : null;
+ const machine = target ? config.machines.find(m => m.id === target) : null;
+ if (target && !machine) throw new Error(`Unknown machine ${target}. Add it to grid-fleet.json first.`);
+ if (machine && machine.transport !== 'local') {
+ if (extraRoots.length || extraPorts.length) throw new Error('--root and --port apply to this machine only.');
+ console.log((await remoteModels(machine, { summary })).text);
+ } else {
+ const found = await inventory({ extraRoots, extraPorts });
+ console.log(summary ? summarize(found) : JSON.stringify(found, null, 2));
+ }
+ } else if (command === 'link') {
+ const usage = 'Use: fleet link [--machine ID] FILE [NAME.gguf] [--projector FILE]';
+ let target, projector;
+ const rest = [];
+ for (let i = 0; i < args.length; i++) {
+ if (args[i] === '--machine' && args[i + 1]) target = args[++i];
+ else if (args[i] === '--projector' && args[i + 1]) projector = args[++i];
+ else if (args[i].startsWith('-')) throw new Error(usage);
+ else rest.push(args[i]);
+ }
+ if (rest.length < 1 || rest.length > 2) throw new Error(usage);
+ const config = await readConfig(workspace);
+ const machine = config.machines.find(m => m.id === (target || config.controller));
+ if (!machine) throw new Error(`Unknown machine ${target}. Add it to grid-fleet.json first.`);
+ const local = machine.transport === 'local';
+ const file = local ? resolve(rest[0]) : rest[0];
+ const name = rest[1] || file.split('/').pop();
+ console.log((await linkModel(machine, file, name, { projector: projector && local ? resolve(projector) : projector })).text);
+ } else if (command === 'recipe') {
+ if (args.length !== 2) throw new Error('Use: fleet recipe vllm|sglang ORG/NAME');
+ try { console.log(JSON.stringify({ found: true, ...(await recipe(args[0], args[1])) }, null, 2)); }
+ catch (error) {
+ if (error.code !== 'NO_RECIPE') throw error;
+ console.log(JSON.stringify({ found: false, message: error.message,
+ next: `No official recipe. Read the model itself: fleet model-facts ${args[1]} — then follow "No recipe" in the engine skill. Never invent flags.` }, null, 2));
+ process.exitCode = 3;
+ }
+ } else if (command === 'candidates') {
+ const [engine, ...rest] = args;
+ if (engine !== 'mlx') throw new Error('Use: fleet candidates mlx [--search WORDS] [--sort downloads|trending|recent] [--context TOKENS]');
+ const options = { search: '', sort: 'downloads', contextTokens: 65536 };
+ for (let i = 0; i < rest.length; i += 2) {
+ if (rest[i] === '--search' && rest[i + 1]) options.search = rest[i + 1];
+ else if (rest[i] === '--sort' && rest[i + 1]) options.sort = rest[i + 1];
+ else if (rest[i] === '--context' && Number(rest[i + 1]) >= 65536) options.contextTokens = Number(rest[i + 1]);
+ else throw new Error('Use: fleet candidates mlx [--search WORDS] [--sort downloads|trending|recent] [--context TOKENS≥65536]');
+ }
+ const machine = await machineState();
+ const metal = machine.accelerators.find(gpu => gpu.vendor === 'apple' && gpu.totalBytes);
+ console.log(JSON.stringify({ machine: { chip: machine.chip, gpuBudgetBytes: metal?.totalBytes ?? null, availableBytes: machine.memory?.availableBytes ?? null },
+ ...(await mlxCandidates({ ...options, budgetBytes: metal?.totalBytes })) }, null, 2));
+ } else if (command === 'verify') {
+ const options = { tools: true };
+ for (let i = 0; i < args.length; i++) {
+ if (['--at', '--model', '--kind', '--grid', '--alias'].includes(args[i]) && args[i + 1]) options[args[i].slice(2)] = args[++i];
+ else if (args[i] === '--no-tools') options.tools = false;
+ else throw new Error('Use: fleet verify --at http://HOST:PORT/v1 --model ID [--kind ENGINE] [--no-tools] [--grid GRID --alias NAME], or --grid GRID --alias NAME alone');
+ }
+ const usage = 'Use: fleet verify --at http://HOST:PORT/v1 --model ID [--kind ENGINE] [--no-tools] [--grid GRID --alias NAME], or --grid GRID --alias NAME alone for an engine on another machine';
+ if (Boolean(options.grid) !== Boolean(options.alias) || (options.at ? !options.model : !options.grid)) throw new Error(usage);
+ const log = createLog();
+ const engine = options.at
+ ? await verifyEngine({ url: options.at, model: options.model, kind: options.kind, tools: options.tools, log })
+ : { ok: true, steps: [] };
+ let relay = null;
+ if (engine.ok && options.grid) {
+ const config = await readConfig(workspace), local = { id: 'local', transport: 'local' };
+ const info = await execute(local, [`--${config.mode}`, 'info', options.grid, '--env'], { timeoutMs: 30_000 });
+ const env = Object.fromEntries([...`${info.stdout}`.matchAll(/(?:export\s+)?(OPENAI_BASE_URL|OPENAI_API_KEY)=["']?([^"'\s]+)/g)].map(m => [m[1], m[2]]));
+ relay = env.OPENAI_BASE_URL
+ ? await verifyRelay({ alias: options.alias, engineModel: options.model, log, relay: { baseUrl: env.OPENAI_BASE_URL, apiKey: env.OPENAI_API_KEY },
+ listModels: async () => {
+ const listed = await gridJson(local, config.mode, ['models', options.grid], { timeoutMs: 30_000 });
+ return listed.ok ? { ok: true, models: (Array.isArray(listed.value) ? listed.value : []).map(row => row.model ?? row.id) } : listed;
+ } })
+ : { ok: false, steps: [{ name: 'relay endpoint', ok: false, note: `grid info ${options.grid} --env gave no OPENAI_BASE_URL` }] };
+ }
+ const ok = engine.ok && (relay?.ok ?? true);
+ console.log(JSON.stringify({ ok, engine: engine.steps, relay: relay?.steps ?? null }, null, 2));
+ process.exitCode = ok ? 0 : 1;
+ } else if (command === 'serve') {
+ const boundary = args.indexOf('--');
+ const usage = 'Use: fleet serve NAME [--env KEY=VALUE]... -- ENGINE COMMAND...';
+ if (boundary < 1 || boundary === args.length - 1) throw new Error(usage);
+ const env = {};
+ for (let i = 1; i < boundary; i += 2) {
+ const pair = args[i] === '--env' ? args[i + 1]?.match(/^([A-Z_][A-Z0-9_]*)=(.*)$/) : null;
+ if (!pair) throw new Error(usage);
+ env[pair[1]] = pair[2];
+ }
+ console.log(JSON.stringify(await serve(workspace, args[0], args.slice(boundary + 1), { env }), null, 2));
+ } else if (command === 'stop') {
+ if (args.length !== 1) throw new Error('Use: fleet stop NAME');
+ console.log(JSON.stringify(await stop(workspace, args[0]), null, 2));
+ } else if (command === 'model-facts') {
+ if (args.length !== 1) throw new Error('Use: fleet model-facts ORG/NAME|MODEL_FOLDER');
+ const facts = await modelFacts(args[0]);
+ console.log(JSON.stringify(facts, null, 2));
+ if (!facts.found || facts.gated && !facts.architecture) process.exitCode = 3;
} else if (command === 'run') {
const boundary = args.indexOf('--');
if (boundary < 0 || boundary === args.length - 1) throw new Error('Use: fleet run [--machine ID] -- <grid arguments>');
@@ -94,6 +210,28 @@ try {
fleet doctor Check Grid and Node
fleet discover [--add] Discover your Harness machines
fleet config Validate and print fleet inventory
+ fleet models [--machine ID] [--summary] [--root DIR] [--port N]
+ Read-only: model files other apps downloaded, engines
+ answering, memory and swap now. --machine reads an SSH
+ machine with its own Node (a file list without it)
+ fleet link [--machine ID] FILE [NAME.gguf] [--projector FILE]
+ Link a GGUF already on that machine into its
+ ~/.grid/models so join --serve can serve it (no copy)
+ fleet recipe vllm|sglang ORG/NAME The engine's official recipe for that model, read live;
+ exit 3 when there is none (never invent flags then)
+ fleet model-facts ORG/NAME|DIR What the model says about itself: architecture, context,
+ chat-template tool syntax, card commands, engine support
+ fleet candidates mlx [--search WORDS] [--sort downloads|trending|recent]
+ MLX models sized for this Mac at a 64K+ context (no download)
+ fleet verify --at URL/v1 --model ID [--kind ENGINE] [--no-tools] [--grid GRID --alias NAME]
+ Bounded, narrated acceptance: ready, listed, answer, tool call,
+ speed; with --grid, listed by the relay and answering through it
+ fleet serve NAME [--env KEY=VALUE]... -- COMMAND...
+ Start an engine other than Grid's so it outlives this shell:
+ own session, log in run/NAME.log, PID in run/NAME.pid
+ fleet stop NAME Stop only the PID that serve recorded for NAME
+ fleet verify --grid GRID --alias NAME
+ The relay half only, for an engine on another machine
fleet refresh Read Grid, update viewer and verdict
fleet run [--machine ID] -- ARGS Run any Grid command on a configured machine