The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.
TypeScript 3d-modelingagent-frameworkagent-harnessai-agentscadclaude-codecodexcoding-agentsdeveloper-toolsesp32game-developmentmcp-servermulti-agentopen-hardwareopencodeopenharnesspcbroboticsterminaltmux
Grid harness: find, start and verify models on any machine and engine
Timd97257fc7af1
diff --git a/store/agents/autonomous-grid/AGENTS.md b/store/agents/autonomous-grid/AGENTS.mdindex df79304e6..ba92c40b3 100644--- a/store/agents/autonomous-grid/AGENTS.md+++ b/store/agents/autonomous-grid/AGENTS.md@@ -26,8 +26,72 @@ paragraph is not a stop. **Talk the way Harness talks:** short, declarative, secis true and what happens next; no "I'd be happy to", no exclamation marks; a thing that is not setup is said plainly and stopped at, the way a failed build is reported.-Read `skills/grid-operations/SKILL.md` from this package (or its workspace skill link) before operating-the fleet. `$GRID_FLEET` is the workspace-aware command runner; `$GRID_CLI` is the selected Grid CLI.+**Starting a model on this computer — the whole flow.** Do it in this order; open a skill only when a+step names one. The commands below are verified: run them as written, without reading `--help` first —+each extra step costs a minute. Technical choices (file, engine, context, port, flags) are yours, from+measurements; the person is asked only what the model is for, and about trade-offs they can feel. When+the person names an engine, that engine is the plan: if it has no suitable model, ask one question —+*Use <file on disk> with Grid's engine now · Get a model for <engine> (N GB download)* — never switch+engines silently.++1. `"$GRID_FLEET" models --summary` — one short table: this machine (chip, GPUs and whether each is+ active, engines installed, what it can run, ports in use), memory and swap now, engines already+ answering with their `--at` URL, grids this computer is joined to (`joined`), and one row per model file on disk of every format (size, context,+ cache at 64K, tool calls, vision, which engines read it). Read the table; do not filter it yourself.+ Drop `--summary` for the full JSON only when a field you need is not in the table. A request that+ does not say what the model is for ("I have some models, can we run one?") gets the purpose question+ right after this table — question tool, before reading any skill: a person waited 6 minutes for+ one [run]. Everything this flow needs is on this page; open a skill only where a step names it.+2. Which grid: "on this computer only", "not online", "just here" → a **local grid** (below); otherwise+ the one the person named, else `personalGrid` (never a shared grid unasked — below). `"$GRID_FLEET" run -- models GRID --json` — already served+ there? Then no start. This computer already joined to it = the `joined` line of step 1, never the+ relay's engine list (an asleep grid lists none). Since Grid 0.3.53 a join there **adds** a model beside+ what this computer already serves (each with its own port and alias) [run]; adding or removing a+ `--serve` model restarts the others for a few seconds, so say so first. If memory will not hold both,+ ask *Replace it · Keep it* (Replace = `leave GRID --engine ALIAS`, then join).+3. Pick a file already on disk: context ≥ 64K (128K for coding when it fits); weights + context × cache+ per token + 0.5 GB within free memory (+3 GB) and the GPU ceiling; tool calls for coding. Nothing+ fits: `skills/grid-operations` step 4 (catalog). An engine installed with nothing to serve:+ `"$GRID_FLEET" candidates mlx` on a Mac, recipes on a GPU box (`skills/run-local-model`).+4. Engine by format: GGUF → Grid's own engine; an engine already answering → `join --at http://127.0.0.1:PORT/v1+ -m ID --advertise-as ALIAS` (the `/v1` is required; without the alias the picker shows the raw id);+ engines already answering that belong to a `joined` grid are not free to reuse; MLX folder → `skills/engine-mlx-lm`; safetensors on an NVIDIA/AMD box →+ `skills/engine-vllm` or `engine-sglang`; Ollama or LM Studio → their `skills/engine-*`.+5. GGUF start: `"$GRID_FLEET" link FILE NAME.gguf` (Grid serves only from `~/.grid/models`; a link, no copy), then+ `"$GRID_FLEET" run --thinking off -- join GRID --serve NAME.gguf --advertise-as ALIAS --name MACHINE-ALIAS+ --ctx-size CTX --endpoint-port PORT --reasoning-budget 0` plus `--max-concurrency 1` on a remote grid or+ `--parallel 1` on a local one (`--max-concurrency` is remote-only), PORT not in `listeningPorts`.+6. `"$GRID_FLEET" verify --at http://127.0.0.1:PORT/v1 --model ALIAS --kind llama.cpp --grid GRID --alias ALIAS`+ — `--alias` is the `--advertise-as` name; `--model` is what the engine itself lists. Run it right after+ the start, without `| tail` or `| head` — it waits by itself and narrates; pass its progress on. Every+ engine the same way (Ollama, LM Studio, mlx-lm…: `--kind` is that engine, and after its `join --at`,+ `verify --grid GRID --alias ALIAS` proves the relay half). This is the only check: never `sleep`,+ `stats`, `grid chat` or a log in `~/.grid` to find out whether it works [run: 2.5 minutes lost that way]. Report only what passed — an engine that answers but is+ not joined is "running here, not in your model picker yet", never "done" — then `fleet models` once more+ (swap still climbing a minute later = too big: stop, take a smaller file), `refresh`, and the hand-off.++**Another machine in `grid-fleet.json`** runs the same flow with `--machine M`: `"$GRID_FLEET" models+--machine M --summary` (an SSH machine, read with its own Node; drop `--summary` for file paths),+`"$GRID_FLEET" link --machine M FILE NAME.gguf` — it prints `NOT READY` when that machine lacks Grid's+engine: ask, then `"$GRID_FLEET" run --machine M -- engine install llama.cpp` before any join (without it+the join still says "starting" and the engine dies at once) —, then+`"$GRID_FLEET" run --machine M --thinking off -- join GRID --serve NAME.gguf --advertise-as ALIAS --name M+--ctx-size CTX --max-concurrency 1 --reasoning-budget 0`, where ALIAS is a name no other machine serves on+that grid (`run -- models GRID --json`) — a shared name cannot show which machine answered — and then+`"$GRID_FLEET" verify --grid GRID --alias ALIAS` (through the relay only).+Only Grid's own engine can be started there; a Harness-linked machine's files cannot be listed, so its+model comes from the catalog.++"On this computer only", "not online", "just here" mean a **local grid** — the personal grid is online+and is not local. Kept on this computer only: `"$GRID_CLI" --local start NAME --port P --host 127.0.0.1 --advertise-host+127.0.0.1`, then `"$GRID_FLEET" connect --mode local --grid NAME`, and every `join` also takes+`--advertise-host 127.0.0.1` — without it the engine registers this computer's LAN address [run]. **Never run `grid mode`** — it switches+the CLI's mode for every workspace and app on the machine — and never `start` a hosted grid unasked.+A Grid refusal names its own fix: read the whole message and follow it; never open `~/.grid`.++The skills hold the detail: `skills/grid-operations/SKILL.md` (grids and access, catalog downloads,+vision, change, stop, move), `skills/run-local-model/SKILL.md` (sizing, choosing, reading official+sources when unsure), `skills/engine-*/SKILL.md` (one per engine). `$GRID_FLEET` is the workspace-aware command runner; `$GRID_CLI` is the selected Grid CLI.Use the runner for operations so progress appears in the viewer. The terminal beside the viewer isthe conversation; do not build another chat UI or run a second background agent.@@ -36,15 +100,24 @@ viewer's published observations without network access. For questions about runnmachines, use a fresh, live snapshot and its observation time; do not start a second network poll.Downloaded files and catalog entries are not proof of serving models. The user's own grid is`personalGrid` in `grid-fleet.json`, named by Harness from the signed-in account: "my grid" means-that exact name, never asked for. Which grid to use otherwise: `grid`, the workspace's selected-fleet; never assume one named `home`. Add `--remember` when the user wants that fleet reused by future Grid workspaces.+that exact name, never asked for. **A model started on the person's own machine joins `personalGrid`+by default** — "run a model on my Mac" names no grid, and that is not consent to serve a shared one:+`grid` (the workspace's selected fleet) can be a team or company grid where other people's requests+would land on this machine [run: a model went to a company grid that way]. Use `grid`, or any other+grid, only when the person names it or asks to share with that team; never assume one named `home`.+Add `--remember` when the user wants that fleet reused by future Grid workspaces.**This harness runs without Codex's sandbox**, the way a terminal does: `join` starts an enginewith the GPU, and every Grid command reaches the network. So run fleet commands directly(`"$GRID_FLEET" run -- …`) — never request escalation, and never treat a failure as a sandboxproblem. Every fleet operation goes through the runner or `$GRID_CLI`: never `ps`, `pgrep`, `lsof`,`sysctl`, `kill`, llama logs or hand-written scripts to start, stop, inspect or diagnose an engine;-the command's own output is the evidence.+the command's own output is the evidence. Two written exceptions: `"$GRID_FLEET" models`, `recipe`+and `model-facts` read this machine and the engines' official sources for you; and an engine other+than Grid's own (Ollama, LM Studio, mlx-lm, vLLM, SGLang) is started with `"$GRID_FLEET" serve NAME -- …`+and stopped with `"$GRID_FLEET" stop NAME`, using the command in its `skills/engine-*/SKILL.md`. A plain+`nohup … &` dies when your command returns [run]; `serve` gives the engine its own session and records+its PID, and `stop` kills only that PID.**Two failed attempts at the same step is the limit.** Report what failed in one line, with thecommand's own error, and wait — never improvise a workaround. If the viewer observation is stale,diff --git a/store/agents/autonomous-grid/lib/candidates.mjs b/store/agents/autonomous-grid/lib/candidates.mjsnew file mode 100644index 000000000..4b3fbc650--- /dev/null+++ b/store/agents/autonomous-grid/lib/candidates.mjs@@ -0,0 +1,69 @@+// Models to download for an engine that is installed but has nothing to serve. MLX: the mlx-community+// organisation on Hugging Face, sized from each repo's own safetensors and config — nothing is downloaded+// here. Every candidate carries its size, its cache cost at the context asked, and whether it fits.+const HUB = 'https://huggingface.co';+const CHAT_TASKS = new Set(['text-generation', 'image-text-to-text']);++/**+ * A first estimate for filtering only: the Hub counts a quantized repo's parameters unpacked (a 4B model at+ * 4 bits reports 4.0e9 "U32"), so size ≈ parameters × bits ÷ 8, plus ~15% for scales. The real size comes+ * from the files (`repoBytes`).+ */+export function estimateBytes(safetensors, bits) {+ return Math.round((safetensors?.total ?? 0) * (bits / 8) * 1.15);+}++/** The exact size of a repo's weight files, from the Hub's file listing. */+export async function repoBytes(id, fetchImpl) {+ const info = await fetchImpl(`${HUB}/api/models/${id}?blobs=true`, { signal: AbortSignal.timeout(20_000) }).then(r => (r.ok ? r.json() : null)).catch(() => null);+ if (!info) return null;+ return (info.siblings ?? []).filter(file => file.rfilename.endsWith('.safetensors')).reduce((sum, file) => sum + (file.size ?? 0), 0) || null;+}++/** f16/bf16 KV-cache bytes per token from a Hugging Face config, counting only layers that keep a cache. */+export function kvBytesFromConfig(config) {+ const text = config.text_config ?? config;+ const layers = text.num_hidden_layers, heads = text.num_attention_heads;+ const kvHeads = text.num_key_value_heads ?? heads;+ const headDim = text.head_dim ?? (text.hidden_size && heads ? text.hidden_size / heads : undefined);+ if (!layers || !kvHeads || !headDim) return null;+ let cached = layers;+ if (Array.isArray(text.layer_types)) cached = text.layer_types.filter(type => /full|global/.test(type) || type === 'attention').length;+ else if (text.full_attention_interval > 1) cached = Math.floor(layers / text.full_attention_interval);+ return cached * kvHeads * headDim * 2 * 2;+}++/**+ * MLX models for this Mac: chat-capable, quantized, public, newest-first within popularity, each checked+ * against `budgetBytes` at `contextTokens`. `search` narrows by name (e.g. a family the person asked for).+ */+const SORTS = { downloads: 'downloads', trending: 'trendingScore', recent: 'lastModified' };++export async function mlxCandidates({ budgetBytes, contextTokens = 65536, search = '', sort = 'downloads', limit = 12, fetchImpl = globalThis.fetch } = {}) {+ if (!SORTS[sort]) throw new Error('sort is downloads, trending or recent');+ const query = new URLSearchParams({ author: 'mlx-community', sort: SORTS[sort], direction: '-1', limit: '300' });+ if (search) query.set('search', search);+ for (const field of ['safetensors', 'config', 'downloads', 'pipeline_tag', 'lastModified', 'gated']) query.append('expand[]', field);+ const response = await fetchImpl(`${HUB}/api/models?${query}`, { signal: AbortSignal.timeout(30_000) });+ if (!response.ok) throw new Error(`huggingface.co answered HTTP ${response.status}`);+ // Weights alone must fit before the config is worth reading.+ const listed = (await response.json()).filter(row => CHAT_TASKS.has(row.pipeline_tag) && !row.gated && row.config?.quantization_config?.bits+ && (!budgetBytes || estimateBytes(row.safetensors, row.config.quantization_config.bits) < budgetBytes));+ const out = [];+ for (const row of listed.slice(0, limit)) {+ const bytes = await repoBytes(row.id, fetchImpl) ?? estimateBytes(row.safetensors, row.config.quantization_config.bits);+ const config = await fetchImpl(`${HUB}/${row.id}/raw/main/config.json`, { signal: AbortSignal.timeout(20_000) })+ .then(r => (r.ok ? r.json() : null)).catch(() => null);+ const kv = config ? kvBytesFromConfig(config) : null;+ const contextMax = config ? (config.text_config ?? config).max_position_embeddings ?? null : null;+ const need = kv === null ? null : bytes + kv * contextTokens + 512 * 1024 ** 2;+ out.push({+ id: row.id, downloads: row.downloads, lastModified: row.lastModified, task: row.pipeline_tag,+ bits: row.config.quantization_config.bits, architecture: row.config.architectures?.[0] ?? null,+ bytes, kvBytesPerToken: kv, contextMax, needBytesAtContext: need,+ fits: need === null || !budgetBytes ? null : need <= budgetBytes && (contextMax ?? contextTokens) >= contextTokens,+ toolCalls: /\btools\b/.test(row.config.tokenizer_config?.chat_template ?? '') || null,+ });+ }+ return { source: `${HUB}/models?author=mlx-community`, contextTokens, budgetBytes: budgetBytes ?? null, candidates: out };+}diff --git a/store/agents/autonomous-grid/lib/finderMain.mjs b/store/agents/autonomous-grid/lib/finderMain.mjsnew file mode 100644index 000000000..a3bee380a--- /dev/null+++ b/store/agents/autonomous-grid/lib/finderMain.mjs@@ -0,0 +1,7 @@+// Entry point for `fleet models --machine ID`: copied to that machine beside inventory.mjs and+// candidates.mjs and run by its own Node, so the table describes that machine — its disk, its engines,+// its memory — not the controller's.+import { inventory, summarize } from './inventory.mjs';++const found = await inventory();+console.log(process.argv.includes('--summary') ? summarize(found) : JSON.stringify(found));diff --git a/store/agents/autonomous-grid/lib/inventory.mjs b/store/agents/autonomous-grid/lib/inventory.mjsnew file mode 100644index 000000000..e984909df--- /dev/null+++ b/store/agents/autonomous-grid/lib/inventory.mjs@@ -0,0 +1,579 @@+// Read-only inventory of this computer for local models: model files other apps already downloaded,+// engines answering on loopback, and the memory the OS reports right now. Nothing here starts, stops,+// moves or downloads anything; every probe is a file read, a GET, or a read-only system command.+import { execFile } from 'node:child_process';+import { open, readdir, readFile, realpath, stat } from 'node:fs/promises';+import { arch, cpus, freemem, homedir, platform, totalmem } from 'node:os';+import { basename, dirname, join, relative, sep } from 'node:path';+import { promisify } from 'node:util';++import { kvBytesFromConfig } from './candidates.mjs';++const exec = promisify(execFile);+const SCALAR_BYTES = { 0: 1, 1: 1, 2: 2, 3: 2, 4: 4, 5: 4, 6: 4, 7: 1, 10: 8, 11: 8, 12: 8 };+// Grid's llama.cpp refuses higher tensor types: "invalid ggml type 143. should be in [0, 43)".+export const GGML_TYPE_LIMIT = 43;+const SKIP_DIRS = new Set(['node_modules', '.git', '__pycache__', '.Trash']);+const KNOWN_PORTS = [11434, 1234, 8000, 8080, 8081, 30000];++class HeaderReader {+ constructor(handle) { this.handle = handle; this.buffer = Buffer.alloc(0); this.offset = 0; this.next = 0; }+ async need(bytes) {+ if (this.buffer.length - this.offset >= bytes) return;+ const kept = this.buffer.subarray(this.offset);+ const chunk = Buffer.alloc(Math.max(bytes - kept.length, 1 << 20));+ const { bytesRead } = await this.handle.read(chunk, 0, chunk.length, this.next);+ this.next += bytesRead;+ this.buffer = Buffer.concat([kept, chunk.subarray(0, bytesRead)]);+ this.offset = 0;+ if (this.buffer.length < bytes) throw new Error('truncated GGUF header');+ }+ skip(bytes) {+ const buffered = this.buffer.length - this.offset;+ if (bytes <= buffered) { this.offset += bytes; return; }+ this.next += bytes - buffered; this.buffer = Buffer.alloc(0); this.offset = 0;+ }+ async number(type) {+ const size = SCALAR_BYTES[type];+ await this.need(size);+ const b = this.buffer, o = this.offset;+ this.offset += size;+ switch (type) {+ case 0: return b.readUInt8(o); case 1: return b.readInt8(o);+ case 2: return b.readUInt16LE(o); case 3: return b.readInt16LE(o);+ case 4: return b.readUInt32LE(o); case 5: return b.readInt32LE(o);+ case 6: return b.readFloatLE(o); case 7: return b.readUInt8(o) !== 0;+ case 10: return Number(b.readBigUInt64LE(o)); case 11: return Number(b.readBigInt64LE(o));+ default: return b.readDoubleLE(o);+ }+ }+ async string(limit = 1 << 20) {+ const length = await this.number(10);+ if (length > limit) { this.skip(length); return null; }+ await this.need(length);+ const text = this.buffer.toString('utf8', this.offset, this.offset + length);+ this.offset += length;+ return text;+ }+ async value(type, keep) {+ if (type === 8) return this.string();+ if (type in SCALAR_BYTES) return this.number(type);+ if (type !== 9) throw new Error(`unsupported GGUF value type ${type}`);+ const itemType = await this.number(4), length = await this.number(10);+ if (itemType in SCALAR_BYTES) {+ if (!keep || length > 4096) { this.skip(length * SCALAR_BYTES[itemType]); return undefined; }+ const items = [];+ for (let i = 0; i < length; i++) items.push(await this.number(itemType));+ return items;+ }+ if (itemType !== 8) throw new Error(`unsupported GGUF array type ${itemType}`);+ for (let i = 0; i < length; i++) this.skip(await this.number(10));+ return undefined;+ }+}++/** The metadata and tensor types of a GGUF file, or null when the file is not GGUF. */+export async function readGguf(path) {+ const handle = await open(path, 'r');+ try {+ const reader = new HeaderReader(handle);+ await reader.need(4);+ if (reader.buffer.toString('latin1', 0, 4) !== 'GGUF') return null;+ reader.offset = 4;+ const version = await reader.number(4);+ const tensorCount = await reader.number(10), keyCount = await reader.number(10);+ if (version < 2 || keyCount > 100_000 || tensorCount > 1_000_000) throw new Error('unsupported GGUF header');+ const meta = {};+ for (let i = 0; i < keyCount; i++) {+ const key = await reader.string(4096), type = await reader.number(4);+ const keep = !key?.startsWith('tokenizer.') || key === 'tokenizer.chat_template';+ const value = await reader.value(type, keep);+ if (key && keep && value !== undefined) meta[key] = value;+ }+ const tensorTypes = new Set();+ for (let i = 0; i < tensorCount; i++) {+ reader.skip(await reader.number(10));+ reader.skip((await reader.number(4)) * 8);+ tensorTypes.add(await reader.number(4));+ reader.skip(8);+ }+ return { meta, tensorTypes: [...tensorTypes].sort((a, b) => a - b) };+ } finally { await handle.close(); }+}++/**+ * What an agent needs from GGUF metadata to size a start. The KV figure is f16 cache bytes per token of+ * context; llama.cpp reported exactly this (1024 MiB for 32768 cells, 8 of 32 layers) for Qwen3.5-4B.+ */+export function summarizeGguf({ meta, tensorTypes }) {+ const architecture = meta['general.architecture'];+ const get = key => meta[`${architecture}.${key}`];+ const layers = get('block_count'), heads = get('attention.head_count');+ const kvHeads = get('attention.head_count_kv') ?? heads;+ const widest = Array.isArray(heads) ? Math.max(...heads) : heads;+ const keyLength = get('attention.key_length') ?? (get('embedding_length') && widest ? get('embedding_length') / widest : undefined);+ const valueLength = get('attention.value_length') ?? keyLength;+ let kvBytesPerToken = null, kvBasis = null;+ if (get('attention.kv_lora_rank') !== undefined) kvBasis = 'latent attention: not estimated';+ else if (layers && keyLength && kvHeads !== undefined) {+ const perLayer = Array.isArray(kvHeads) ? kvHeads.slice(0, layers) : Array(layers).fill(kvHeads);+ const interval = get('full_attention_interval');+ const cached = perLayer.map((n, i) => (interval > 1 && (i + 1) % interval !== 0 ? 0 : n));+ kvBytesPerToken = cached.reduce((sum, n) => sum + n, 0) * (keyLength + valueLength) * 2;+ kvBasis = get('attention.sliding_window') ? 'upper bound (sliding-window layers hold less)' : 'f16 cache';+ }+ const template = meta['tokenizer.chat_template'] || '';+ return {+ architecture: architecture ?? null, name: meta['general.name'] ?? null, parameters: meta['general.size_label'] ?? null,+ contextLength: get('context_length') ?? null, layers: layers ?? null,+ experts: get('expert_count') ? { total: get('expert_count'), active: get('expert_used_count') ?? null } : null,+ kvBytesPerToken, kvBasis,+ toolCalls: /\btools\b/.test(template), thinking: /enable_thinking|<think>/.test(template),+ unsupportedTensorTypes: tensorTypes.filter(type => type >= GGML_TYPE_LIMIT),+ };+}++export function defaultRoots(env = process.env, home = homedir()) {+ const hub = env.HF_HUB_CACHE || (env.HF_HOME ? join(env.HF_HOME, 'hub') : join(env.XDG_CACHE_HOME || join(home, '.cache'), 'huggingface', 'hub'));+ return [+ { source: 'grid', path: join(env.GRID_HOME || join(home, '.grid'), 'models') },+ { source: 'ollama', path: env.OLLAMA_MODELS || join(home, '.ollama', 'models') },+ { source: 'ollama', path: '/usr/share/ollama/.ollama/models' },+ { source: 'lm-studio', path: join(home, '.lmstudio', 'models') },+ { source: 'lm-studio', path: join(home, '.cache', 'lm-studio', 'models') },+ { source: 'huggingface', path: hub },+ { source: 'folder', path: join(home, 'models') },+ { source: 'folder', path: join(home, 'Models') },+ { source: 'folder', path: join(home, 'Downloads'), depth: 3 },+ ];+}++const QUANT = /(?:^|[-_.])((?:UD-)?(?:I?Q\d(?:_[A-Z0-9]+)*|BF16|F16|F32|MXFP4|NVFP4))(?=[-_.]|$)/i;+const READERS = { gguf: ['llama.cpp', 'ollama', 'lm-studio'], mlx: ['mlx-lm', 'lm-studio'], safetensors: ['vllm', 'sglang', 'mlx-lm'] };++async function walk(dir, depth, visit, budget) {+ if (depth < 0 || budget.dirs-- <= 0) return;+ let entries;+ try { entries = await readdir(dir, { withFileTypes: true }); } catch { return; }+ if (entries.some(e => e.name === 'config.json') && entries.some(e => e.name.endsWith('.safetensors'))) await visit('folder', dir);+ for (const entry of entries) {+ const path = join(dir, entry.name);+ if (entry.isDirectory()) { if (!SKIP_DIRS.has(entry.name) && entry.name !== 'blobs') await walk(path, depth - 1, visit, budget); }+ else if (/\.gguf$/i.test(entry.name) && (entry.isFile() || entry.isSymbolicLink())) await visit('gguf', path);+ }+}++async function listFiles(dir, depth) {+ if (depth < 0) return [];+ let entries;+ try { entries = await readdir(dir, { withFileTypes: true }); } catch { return []; }+ const nested = await Promise.all(entries.map(e => (e.isDirectory() ? listFiles(join(dir, e.name), depth - 1) : [join(dir, e.name)])));+ return nested.flat();+}++async function ollamaEntries(root) {+ const manifests = join(root, 'manifests'), entries = [];+ for (const file of await listFiles(manifests, 4)) {+ const [host, namespace, model, tag] = relative(manifests, file).split(sep);+ if (!tag) continue;+ let layers;+ try { ({ layers = [] } = JSON.parse(await readFile(file, 'utf8'))); } catch { continue; }+ const blob = mediaType => layers.find(layer => layer.mediaType === `application/vnd.ollama.image.${mediaType}`)?.digest;+ const weights = blob('model'), projector = blob('projector');+ if (!weights) continue;+ const short = namespace === 'library' ? `${model}:${tag}` : `${namespace}/${model}:${tag}`;+ entries.push({+ kind: 'gguf', path: join(root, 'blobs', weights.replace(':', '-')),+ name: host === 'registry.ollama.ai' ? short : `${host}/${short}`,+ projector: projector ? join(root, 'blobs', projector.replace(':', '-')) : null,+ });+ }+ return entries;+}++function hubName(path) {+ const repo = path.split(sep).find(part => part.startsWith('models--'));+ return repo ? repo.slice('models--'.length).replace('--', '/') : null;+}++async function describeFolder(dir) {+ const config = JSON.parse(await readFile(join(dir, 'config.json'), 'utf8'));+ let bytes = 0;+ for (const name of await readdir(dir)) if (name.endsWith('.safetensors')) bytes += (await stat(join(dir, name))).size;+ const mlx = Boolean(config.quantization?.bits) || /mlx/i.test(dir);+ const template = await readFile(join(dir, 'chat_template.jinja'), 'utf8')+ .catch(async () => JSON.parse(await readFile(join(dir, 'tokenizer_config.json'), 'utf8')).chat_template ?? '')+ .catch(() => '');+ return {+ format: mlx ? 'mlx' : 'safetensors', bytes,+ config: {+ architecture: config.architectures?.[0] ?? config.model_type ?? null,+ contextLength: config.max_position_embeddings ?? config.text_config?.max_position_embeddings ?? null,+ quantization: config.quantization?.bits ? `${config.quantization.bits}-bit` : config.quantization_config?.quant_method ?? null,+ kvBytesPerToken: kvBytesFromConfig(config),+ toolCalls: typeof template === 'string' && template ? /\btools\b/.test(template) : null,+ },+ };+}++/** Every model file under the roots, one entry per real file; later sightings of a file become `alsoAt`. */+export async function scanModels({ roots = defaultRoots(), minBytes = 50 * 1024 ** 2 } = {}) {+ const byRealPath = new Map(), projectors = [], seenRoots = new Set(), scanned = [];+ const add = async (source, found) => {+ let realPath, info;+ try { realPath = await realpath(found.path); info = await stat(realPath); } catch { return; }+ const sighting = { source, path: found.path, name: found.name ?? null };+ if (byRealPath.has(realPath)) { byRealPath.get(realPath).alsoAt.push(sighting); return; }+ const entry = { name: found.name ?? null, source, path: found.path, realPath, format: found.kind, bytes: info.size,+ quant: null, projector: found.projector ?? null, alsoAt: [], readableBy: [], gguf: null, config: null, error: null };+ try {+ if (found.kind === 'folder') {+ Object.assign(entry, await describeFolder(realPath));+ if (entry.bytes < minBytes) return;+ } else {+ if (info.size < minBytes) return;+ const header = await readGguf(realPath);+ if (!header) return;+ entry.gguf = summarizeGguf(header);+ entry.quant = basename(found.path).match(QUANT)?.[1]?.toUpperCase() ?? null;+ }+ } catch (error) { entry.error = error.message; }+ entry.name ??= hubName(found.path) ?? basename(found.path).replace(/\.gguf$/i, '');+ entry.readableBy = READERS[entry.format] ?? [];+ byRealPath.set(realPath, entry);+ };+ for (const root of roots) {+ let real;+ try { real = await realpath(root.path); } catch { continue; }+ if (seenRoots.has(real)) continue;+ seenRoots.add(real); scanned.push(root.path);+ if (root.source === 'ollama') { for (const found of await ollamaEntries(real)) await add('ollama', found); continue; }+ await walk(root.path, root.depth ?? 6, async (kind, path) => {+ if (kind === 'gguf' && /mmproj/i.test(basename(path))) projectors.push(path);+ else await add(root.source, { kind, path });+ }, { dirs: 20_000 });+ }+ const models = [...byRealPath.values()];+ for (const path of projectors) {+ const siblings = models.filter(m => m.format === 'gguf' && dirname(m.path) === dirname(path));+ const stem = basename(path).split('.mmproj')[0];+ const owner = siblings.find(m => basename(m.path).replace(/\.gguf$/i, '') === stem) ?? (siblings.length === 1 ? siblings[0] : null);+ if (owner && !owner.projector) owner.projector = path;+ }+ return { models: models.sort((a, b) => b.bytes - a.bytes), scannedRoots: scanned };+}++export function parseListeningPorts(lsofOutput) {+ const ports = new Set();+ for (const line of lsofOutput.split('\n')) if (line.startsWith('n')) {+ const port = Number(line.slice(line.lastIndexOf(':') + 1));+ if (Number.isInteger(port) && port > 0) ports.add(port);+ }+ return [...ports].sort((a, b) => a - b);+}++/**+ * Linux listening TCP ports from `/proc/net/tcp` and `/proc/net/tcp6` (state `0A`, port in hex after the+ * last colon). A file read, so it works where `lsof` is missing — slim containers and many servers [run].+ */+export function parseProcNetTcp(text) {+ const ports = new Set();+ for (const line of text.split('\n')) {+ const cols = line.trim().split(/\s+/);+ if (cols[3] !== '0A') continue;+ const port = parseInt(cols[1]?.slice(cols[1].lastIndexOf(':') + 1) ?? '', 16);+ if (Number.isInteger(port) && port > 0) ports.add(port);+ }+ return [...ports].sort((a, b) => a - b);+}++/** Windows `netstat -ano` lines such as ` TCP [::]:11434 [::]:0 LISTENING 4812`. */+export function parseNetstatListening(text) {+ const ports = new Set();+ for (const [, port] of text.matchAll(/^\s*TCP\s+\S*:(\d+)\s+\S+\s+LISTENING\b/gim)) ports.add(Number(port));+ return [...ports].sort((a, b) => a - b);+}++async function listeningPorts(os) {+ if (os === 'linux') {+ const tables = await Promise.all(['/proc/net/tcp', '/proc/net/tcp6'].map(file => readFile(file, 'utf8').catch(() => '')));+ return parseProcNetTcp(tables.join('\n'));+ }+ if (os === 'win32') return parseNetstatListening(await output('netstat', ['-ano']));+ return parseListeningPorts(await output('lsof', ['-nP', '-iTCP', '-sTCP:LISTEN', '-Fn']));+}++function engineKind(body, ownedBy) {+ if (Array.isArray(body.models) && body.data) return 'llama.cpp';+ return { vllm: 'vllm', sglang: 'sglang', llamacpp: 'llama.cpp', organization_owner: 'lm-studio', library: 'ollama' }[ownedBy] ?? 'openai-compatible';+}++/** One GET per port. An answer with a model list is an engine; `url` is the exact `--at` form. */+export async function probeEngines(ports, { host = '127.0.0.1', timeoutMs = 800, fetchImpl = globalThis.fetch } = {}) {+ const probe = async port => {+ const root = `http://${host}:${port}`;+ try {+ const response = await fetchImpl(`${root}/v1/models`, { signal: AbortSignal.timeout(timeoutMs) });+ if (!response.ok) return null;+ const body = await response.json();+ const rows = Array.isArray(body.data) ? body.data : null;+ if (!rows) return null;+ let kind = engineKind(body, rows[0]?.owned_by);+ if (kind === 'openai-compatible' || kind === 'ollama') {+ const version = await fetchImpl(`${root}/api/version`, { signal: AbortSignal.timeout(timeoutMs) })+ .then(r => (r.ok ? r.json() : null)).catch(() => null);+ if (typeof version?.version === 'string') kind = 'ollama';+ }+ return { kind, port, url: `${root}/v1`, models: rows.map(row => row.id).filter(Boolean) };+ } catch { return null; }+ };+ return (await Promise.all([...new Set(ports)].map(probe))).filter(Boolean);+}++const pidAlive = pid => { try { process.kill(pid, 0); return true; } catch (error) { return error.code === 'EPERM'; } };++/**+ * Grids this computer is joined to, from Grid's own run files rather than the relay: an asleep grid lists+ * no engines, so an agent that read the relay once joined over a live engine [run]. Grid itself refuses a+ * second `--serve` model per grid while its run record exists; an engine whose record is gone is invisible+ * here too, and a join displaces it.+ */+export async function joinedGrids(gridHome, { alive = pidAlive } = {}) {+ const dir = join(gridHome, 'run', 'engines');+ const rows = [];+ for (const entry of await readdir(dir, { withFileTypes: true }).catch(() => [])) {+ if (!entry.isDirectory()) continue;+ const state = await readFile(join(dir, entry.name, 'remote.json'), 'utf8').then(JSON.parse).catch(() => null);+ if (!Number.isInteger(state?.pid) || !alive(state.pid)) continue;+ rows.push({+ gridId: state.grid_id ?? entry.name, grid: null, pid: state.pid, startedAt: state.started_at ?? null,+ models: state.advertise_as?.length ? state.advertise_as : state.models ?? [],+ at: state.engines?.find(engine => engine.endpoint_url)?.endpoint_url ?? state.endpoint_url ?? null,+ });+ }+ return rows;+}++export function parseVmStat(text) {+ const pageSize = Number(text.match(/page size of (\d+) bytes/)?.[1] || 4096);+ const pages = label => Number(text.match(new RegExp(`Pages ${label}:\\s+(\\d+)`))?.[1] || 0);+ return (pages('free') + pages('inactive') + pages('speculative') + pages('purgeable')) * pageSize;+}++export function parseSwapUsage(text) {+ const mb = label => Number(text.match(new RegExp(`${label} = ([\\d.]+)M`))?.[1] || 0) * 1024 ** 2;+ return { total: mb('total'), used: mb('used') };+}++export function parseMeminfo(text) {+ const kb = label => Number(text.match(new RegExp(`^${label}:\\s+(\\d+) kB`, 'm'))?.[1] || 0) * 1024;+ return { total: kb('MemTotal'), available: kb('MemAvailable'), swapUsed: kb('SwapTotal') - kb('SwapFree') };+}++/** `llama-server --list-devices`: each accelerator's memory budget as llama.cpp will use it. */+export function parseListDevices(text) {+ return [...text.matchAll(/^\s*(\w+): (.+?) \((\d+) MiB, (\d+) MiB free\)/gm)]+ .filter(([, id, , total]) => Number(total) > 0 && !/^(CPU|BLAS)/.test(id))+ .map(([, id, name, total, free]) => ({ id, name, totalBytes: Number(total) * 1024 ** 2, freeBytes: Number(free) * 1024 ** 2 }));+}++async function output(file, args, timeout = 10_000) {+ try { return (await exec(file, args, { timeout, maxBuffer: 4 * 1024 * 1024 })).stdout; } catch { return ''; }+}++/** stdout and whether the command itself answered: a present binary that errors is "not active". */+async function probe(file, args, timeout = 10_000) {+ try { const { stdout, stderr } = await exec(file, args, { timeout, maxBuffer: 4 * 1024 * 1024 }); return { ran: true, ok: true, out: `${stdout}${stderr}` }; }+ catch (error) { return { ran: error.code !== 'ENOENT', ok: false, out: `${error.stdout ?? ''}${error.stderr ?? ''}`.trim() || error.message }; }+}++export function parseNvidiaSmi(text) {+ return text.split('\n').map(line => line.split(',').map(cell => cell.trim())).filter(cells => cells.length >= 6 && /^\d+$/.test(cells[0]))+ .map(([index, name, total, used, utilization, driver]) => ({+ vendor: 'nvidia', id: `CUDA${index}`, name, totalBytes: Number(total) * 1024 ** 2, usedBytes: Number(used) * 1024 ** 2,+ utilizationPct: Number(utilization), driver, active: true,+ }));+}++/**+ * The accelerators that actually answer. Apple silicon: llama.cpp's Metal device list. NVIDIA: nvidia-smi+ * must run and list the card; a card that lspci sees while nvidia-smi fails is reported inactive, with why.+ */+async function accelerators(llamaServer, os) {+ let listed = parseListDevices((await probe(llamaServer, ['--list-devices'], 30_000)).out);+ // Seen once on a Mac under memory pressure: an empty list that the next call fills. One retry, then report what is there.+ if (!listed.length && (await stat(llamaServer).catch(() => null))) {+ await new Promise(resolve => setTimeout(resolve, 1_000));+ listed = parseListDevices((await probe(llamaServer, ['--list-devices'], 30_000)).out);+ }+ const found = listed+ .map(device => ({ vendor: /^MTL/.test(device.id) ? 'apple' : /^CUDA/.test(device.id) ? 'nvidia' : /^(ROCm|HIP)/i.test(device.id) ? 'amd' : 'other', ...device, active: true, source: 'llama-server --list-devices' }));+ const smi = await probe('nvidia-smi', ['--query-gpu=index,name,memory.total,memory.used,utilization.gpu,driver_version', '--format=csv,noheader,nounits']);+ if (smi.ok) {+ for (const gpu of parseNvidiaSmi(smi.out)) if (!found.some(d => d.vendor === 'nvidia' && d.name === gpu.name)) found.push({ ...gpu, source: 'nvidia-smi' });+ } else if (smi.ran) found.push({ vendor: 'nvidia', active: false, source: 'nvidia-smi', error: smi.out.split('\n')[0].slice(0, 200) });+ const rocm = await probe('rocm-smi', ['--showproductname', '--showmeminfo', 'vram', '--json']);+ if (rocm.ok) found.push({ vendor: 'amd', active: true, source: 'rocm-smi', detail: rocm.out.slice(0, 400) });+ else if (rocm.ran) found.push({ vendor: 'amd', active: false, source: 'rocm-smi', error: rocm.out.split('\n')[0].slice(0, 200) });+ if (os === 'linux' && !found.some(d => d.vendor === 'nvidia')) {+ const pci = (await probe('lspci', [])).out;+ if (/nvidia/i.test(pci)) found.push({ vendor: 'nvidia', active: false, source: 'lspci', error: 'card present, nvidia-smi missing: driver not installed or not loaded' });+ }+ return found;+}++const firstLine = text => text.split('\n').map(line => line.trim()).find(Boolean) ?? null;++/** Engines installed here (on PATH or in their standard install place), with the version each reports. */+async function installedEngines(llamaServer, home) {+ const engines = [];+ const add = (kind, path, version, note) => engines.push({ kind, path, version: version || null, ...(note ? { note } : {}) });+ const grid = await probe(llamaServer, ['--version']);+ if (grid.ok) add('llama.cpp', llamaServer, firstLine(grid.out), "Grid's own engine");+ // PATH first, then ~/.grid/envs/<engine>/bin, where the engine skills install Python engines.+ const envs = join(home, '.grid', 'envs');+ const windows = platform() === 'win32';+ const onPath = async name => (windows+ ? (await output('where', [name])).split(/\r?\n/)[0] // Windows has no `sh`; `where` prints each match+ : await output('sh', ['-c', `command -v ${name}`])).trim();+ const inEnv = async (name, env) => {+ if (!env) return null;+ const file = windows ? join(envs, env, 'Scripts', `${name}.exe`) : join(envs, env, 'bin', name); // a venv's layout+ return (await stat(file).catch(() => null)) ? file : null;+ };+ const exists = async file => ((await stat(file).catch(() => null)) ? file : null);+ const which = async (name, env) => (await onPath(name)) || inEnv(name, env);+ const own = await which('llama-server');+ if (own && own !== llamaServer) {+ const answer = await probe(own, ['--version']);+ add('llama.cpp', own, answer.ok ? firstLine(answer.out) : null, answer.ok ? null : 'on PATH but did not answer --version');+ }+ const ollama = (await which('ollama'))+ || ((await exists('/Applications/Ollama.app')) ? '/Applications/Ollama.app/Contents/Resources/ollama' : null)+ // Windows installs per user under %LOCALAPPDATA%\Programs\Ollama and adds it to PATH [doc: docs.ollama.com/windows].+ || (process.env.LOCALAPPDATA ? await exists(join(process.env.LOCALAPPDATA, 'Programs', 'Ollama', 'ollama.exe')) : null);+ if (ollama) add('ollama', ollama, (await probe(ollama, ['--version'])).out.match(/version is ([\w.-]+)/)?.[1]);+ const lms = (await which('lms')) || (await exists(join(home, '.lmstudio', 'bin', 'lms')))+ || (windows ? await exists(join(home, '.lmstudio', 'bin', 'lms.exe')) : null);+ const studioApp = await stat('/Applications/LM Studio.app').catch(() => null);+ if (lms || studioApp) add('lm-studio', lms ?? '/Applications/LM Studio.app', null, lms ? null : 'app installed; lms appears after its first launch');+ const mlx = await which('mlx_lm.server', 'mlx-lm');+ if (mlx) {+ const python = (await readFile(mlx, 'utf8').catch(() => '')).match(/^#!(\S+)/)?.[1];+ add('mlx-lm', mlx, python ? firstLine((await probe(python, ['-c', 'import mlx_lm; print(mlx_lm.__version__)'])).out) : null);+ }+ const vllm = await which('vllm', 'vllm');+ if (vllm) add('vllm', vllm, firstLine((await probe(vllm, ['--version'], 60_000)).out));+ const sglang = await which('sglang', 'sglang');+ if (sglang) add('sglang', sglang, null);+ return engines;+}++/** Which engines this hardware can run at all, from each engine's documented platforms. */+export function runnableEngines(os, cpu, gpus) {+ const active = vendor => gpus.some(gpu => gpu.vendor === vendor && gpu.active);+ if (os === 'darwin' && cpu === 'arm64') return ['llama.cpp', 'mlx-lm', 'ollama', 'lm-studio'];+ if (os === 'linux' && (active('nvidia') || active('amd'))) return ['llama.cpp', 'ollama', 'vllm', 'sglang'];+ return ['llama.cpp', 'ollama'];+}++export async function machineState({ env = process.env, home = homedir() } = {}) {+ const machine = { platform: platform(), arch: arch(), chip: null, memory: null, accelerators: [], engines: [], canRun: [], listeningPorts: [] };+ machine.chip = (machine.platform === 'darwin'+ ? (await output('sysctl', ['-n', 'machdep.cpu.brand_string'])).trim()+ : (await readFile('/proc/cpuinfo', 'utf8').catch(() => '')).match(/^model name\s*:\s*(.+)$/m)?.[1])+ // ARM Linux has no "model name" line and Windows no /proc; Node's own CPU list covers both — except+ // on ARM Linux, where Node itself answers the word "unknown" [run].+ || [cpus()[0]?.model?.trim()].find(model => model && model !== 'unknown') || null;+ if (machine.platform === 'darwin') {+ const total = Number((await output('sysctl', ['-n', 'hw.memsize'])).trim()) || null;+ const swap = parseSwapUsage(await output('sysctl', ['-n', 'vm.swapusage']));+ machine.memory = { totalBytes: total, availableBytes: parseVmStat(await output('vm_stat', [])), swapUsedBytes: swap.used,+ note: 'available = free + inactive + speculative + purgeable pages; Metal and Grid do not see other apps, this does' };+ } else if (machine.platform === 'linux') {+ const info = parseMeminfo(await readFile('/proc/meminfo', 'utf8').catch(() => ''));+ machine.memory = { totalBytes: info.total, availableBytes: info.available, swapUsedBytes: info.swapUsed };+ }+ if (!machine.memory?.totalBytes) {+ // Windows and anything else: the OS's own figures through Node. Swap is not reported there.+ machine.memory = { totalBytes: totalmem(), availableBytes: freemem(), swapUsedBytes: null, note: 'as the OS reports it' };+ }+ const llamaServer = join(env.GRID_HOME || join(home, '.grid'), 'bin', machine.platform === 'win32' ? 'llama-server.exe' : 'llama-server');+ [machine.accelerators, machine.engines] = await Promise.all([accelerators(llamaServer, machine.platform), installedEngines(llamaServer, home)]);+ if (machine.platform === 'darwin' && machine.arch === 'arm64' && !machine.accelerators.some(gpu => gpu.vendor === 'apple')) {+ machine.accelerators.push({ vendor: 'apple', name: machine.chip, active: true, source: 'Apple silicon GPU (built in; budget unknown until llama.cpp is installed)' });+ }+ machine.canRun = runnableEngines(machine.platform, machine.arch, machine.accelerators);+ machine.listeningPorts = await listeningPorts(machine.platform);+ return machine;+}++const gb = bytes => (bytes === null || bytes === undefined ? '—' : `${(bytes / 1e9).toFixed(1)} GB`);++/**+ * One model's identity across formats and publishers: `Qwen3.5-9B-UD-Q4_K_XL` (GGUF) and+ * `mlx-community/Qwen3.5-9B-4bit` (MLX) are both `qwen3.5-9b`. Only the packaging words go —+ * quantization, bit width, the format — so a variant such as `-MTP` or `-Coder` stays its own model.+ */+export function modelFamily(name) {+ return String(name).split('/').pop().toLowerCase()+ .replace(/\.gguf$/, '')+ .split(/[-_.](?=(?:ud|i?q\d[a-z0-9_]*|\d+bit|bf16|f16|f32|fp16|mxfp4|nvfp4|gguf|mlx|awq|gptq)(?:[-_.]|$))/)[0];+}++/**+ * The inventory as a short table an agent reads directly, so it never writes its own filter over the+ * JSON (an agent's hand-written `jq` once dropped every MLX model and reported none on disk).+ */+export function summarize({ machine, engines, models, joined = [] }, contextTokens = 65536) {+ const mem = machine.memory ?? {};+ const gpus = machine.accelerators.map(g => `${g.vendor}${g.name ? ` ${g.name}` : ''} ${g.active ? 'active' : `INACTIVE (${g.error ?? 'no answer'})`}${g.totalBytes ? ` ${gb(g.totalBytes)}` : ''}`);+ const lines = [+ `machine ${machine.chip ?? '?'} · ${machine.platform}/${machine.arch} · memory ${gb(mem.totalBytes)}, available ${gb(mem.availableBytes)}, swap used ${gb(mem.swapUsedBytes)}`,+ `gpu ${gpus.join(' · ') || 'none'}`,+ `installed ${machine.engines.map(e => `${e.kind}${e.version ? ` ${e.version}` : ''}${e.note ? ` (${e.note})` : ''}`).join(' · ') || 'none'}`,+ `can run ${machine.canRun.join(', ')}`,+ `answering ${engines.map(e => `${e.kind} ${e.url} [${e.models.slice(0, 3).join(', ')}]`).join(' · ') || 'none'}`,+ `joined ${joined.map(j => `${j.grid ?? j.gridId} serving [${j.models.join(', ')}]${j.at ? ` at ${j.at}` : ''}`).join(' · ') || 'none'}${joined.length ? ' — a join to that grid adds to what it serves (Grid 0.3.53+)' : ''}`,+ `ports in use ${machine.listeningPorts.join(' ')}`,+ '',+ `FORMAT SIZE CONTEXT CACHE@${contextTokens / 1024}K TOOLS VISION READ BY MODEL (where)`,+ ];+ // On Apple silicon with an MLX engine installed, a GGUF that also exists as an MLX folder says so on+ // its own row: an agent handed the rule "prefer the MLX copy" still served the GGUF, because nothing+ // told it the two names were one model [run].+ const mlxHere = machine.canRun.includes('mlx-lm') && machine.engines.some(e => e.kind === 'mlx-lm' || e.kind === 'lm-studio');+ const mlxCopies = new Map();+ if (mlxHere) for (const m of models) if (m.format === 'mlx') mlxCopies.set(modelFamily(m.name), m.name);+ for (const m of models) {+ const facts = m.gguf ?? m.config ?? {};+ const kv = facts.kvBytesPerToken;+ const runnable = m.readableBy.some(reader => machine.canRun.includes(reader));+ const mlxCopy = m.format === 'gguf' ? mlxCopies.get(modelFamily(m.name)) : null;+ const note = [runnable ? '' : `no engine this machine can run reads ${m.format}${m.format === 'mlx' ? ' (Apple silicon only)' : ''}`,+ mlxCopy ? `an MLX copy is on disk (${mlxCopy}): serve that one on this Mac unless vision is needed` : '',+ m.gguf?.unsupportedTensorTypes?.length ? `llama.cpp cannot load (tensor type ${m.gguf.unsupportedTensorTypes.join(', ')})` : '',+ facts.contextLength && facts.contextLength < contextTokens ? `context ${facts.contextLength} < ${contextTokens}` : '', m.error ?? ''].filter(Boolean).join('; ');+ lines.push([+ m.format.padEnd(11), gb(m.bytes).padStart(8), String(facts.contextLength ?? '—').padStart(8),+ (kv ? gb(kv * contextTokens) : '—').padStart(10), String(facts.toolCalls ?? '?').padEnd(5), (m.projector ? 'yes' : 'no').padEnd(6),+ m.readableBy.join(',').padEnd(27), `${m.name} (${m.source}${m.alsoAt.length ? `, also ${m.alsoAt.map(a => a.source).join('/')}` : ''})${note ? ` ! ${note}` : ''}`,+ ].join(' '));+ }+ return lines.join('\n');+}++/** The whole picture for one start decision: memory now, engines answering, and models on disk. */+export async function inventory({ extraRoots = [], extraPorts = [], env = process.env, home = homedir() } = {}) {+ const machine = await machineState({ env, home });+ const roots = [...defaultRoots(env, home), ...extraRoots.map(path => ({ source: 'folder', path }))];+ const ports = [...KNOWN_PORTS, ...machine.listeningPorts.filter(port => port >= 1024), ...extraPorts];+ const [engines, disk, joined] = await Promise.all([probeEngines(ports), scanModels({ roots }), joinedGrids(env.GRID_HOME || join(home, '.grid'))]);+ if (joined.length) {+ // Run files carry the grid id; the name the agent uses comes from the CLI's own list.+ let grids = [];+ try { grids = JSON.parse(await output(env.GRID_CLI || 'grid', ['ls', '--json'], 8_000)); } catch {}+ for (const row of joined) row.grid = (Array.isArray(grids) ? grids : []).find(g => g.id === row.gridId)?.grid ?? null;+ }+ return { machine, engines, joined, ...disk };+}diff --git a/store/agents/autonomous-grid/lib/modelFacts.mjs b/store/agents/autonomous-grid/lib/modelFacts.mjsnew file mode 100644index 000000000..11e0e472b--- /dev/null+++ b/store/agents/autonomous-grid/lib/modelFacts.mjs@@ -0,0 +1,98 @@+// What a model says about itself, for the case no engine recipe covers it: the files every Hugging Face+// model ships (config, generation defaults, chat template, model card), read from the Hub or a local+// folder, plus whether each engine's official supported-model list names its architecture. Facts only.+import { readFile, stat } from 'node:fs/promises';+import { join } from 'node:path';++const HUB = 'https://huggingface.co';+const FILES = ['config.json', 'generation_config.json', 'tokenizer_config.json', 'chat_template.jinja', 'chat_template.json', 'README.md'];+const VLLM_SUPPORTED = 'https://raw.githubusercontent.com/vllm-project/vllm/main/docs/models/supported_models.md';+// SGLang's docs name families, not architectures; its model registry is one file per model_type with an EntryClass.+const SGLANG_MODELS = 'https://raw.githubusercontent.com/sgl-project/sglang/main/python/sglang/srt/models';+const SERVE = /\b(vllm serve|sglang serve|sglang\.launch_server|llama-server|mlx_lm\.server|ollama (run|create|pull)|lms (get|load))\b/;+const SAMPLING = ['temperature', 'top_p', 'top_k', 'min_p', 'repetition_penalty', 'presence_penalty'];++const json = text => { try { return JSON.parse(text); } catch { return null; } };++async function hubReader(hfId, { fetchImpl, token }) {+ const headers = token ? { authorization: `Bearer ${token}` } : {};+ const response = await fetchImpl(`${HUB}/api/models/${hfId}`, { headers, signal: AbortSignal.timeout(20_000) });+ if (response.status === 401 || response.status === 404) return { missing: true };+ if (!response.ok) throw new Error(`huggingface.co answered HTTP ${response.status} for ${hfId}`);+ const info = await response.json();+ const present = new Set((info.siblings ?? []).map(file => file.rfilename));+ return {+ gated: Boolean(info.gated), present,+ read: async name => {+ if (!present.has(name)) return null;+ const file = await fetchImpl(`${HUB}/${hfId}/raw/main/${name}`, { headers, signal: AbortSignal.timeout(20_000) });+ return file.ok ? file.text() : null;+ },+ };+}++function localReader(dir) {+ return { gated: false, read: name => readFile(join(dir, name), 'utf8').catch(() => null) };+}++/** A few lines of the chat template around its tool-call markup: compare them with the engine's parser docs. */+function toolSyntax(template) {+ const at = template.search(/<tool_call>|\[TOOL_CALLS\]|<\|python_tag\|>|tool_calls|<function|functools|tool▁call/);+ if (at < 0) return null;+ return template.slice(Math.max(0, at - 160), at + 320).trim();+}++function codeBlocks(markdown) {+ return [...markdown.matchAll(/```[^\n]*\n([\s\S]*?)```/g)].map(match => match[1].trim());+}++/** `listed`: true = named by the engine's own list, false = its full list omits it, null = this check cannot tell. */+async function support(architecture, modelType, fetchImpl) {+ const text = url => fetchImpl(url, { signal: AbortSignal.timeout(20_000) }).then(r => (r.ok ? r.text() : null)).catch(() => null);+ const vllmList = architecture ? await text(VLLM_SUPPORTED) : null;+ const vllm = vllmList === null+ ? { listed: null, source: VLLM_SUPPORTED, note: architecture ? 'list unreachable' : 'config.json names no architecture' }+ : { listed: vllmList.split('\n').some(line => line.includes(`\`${architecture}\``)), source: VLLM_SUPPORTED,+ lines: vllmList.split('\n').filter(line => line.includes(`\`${architecture}\``)).map(line => line.trim().slice(0, 240)).slice(0, 3) };+ const registryFile = modelType && /^[\w.-]+$/.test(modelType) ? `${SGLANG_MODELS}/${modelType}.py` : null;+ const registry = registryFile && architecture ? await text(registryFile) : null;+ const sglang = registry?.includes(architecture)+ ? { listed: true, source: registryFile }+ : { listed: null, source: registryFile ?? SGLANG_MODELS,+ note: 'not found in the registry file for this model_type; it may live in another file, or run through --model-impl transformers' };+ return { vllm, sglang };+}++export async function modelFacts(target, { fetchImpl = globalThis.fetch, token = process.env.HF_TOKEN, checkSupport = true } = {}) {+ const isLocal = await stat(target).then(info => info.isDirectory()).catch(() => false);+ if (!isLocal && !/^[\w.-]+\/[\w.-]+$/.test(target || '')) throw new Error('Pass a Hugging Face id (ORG/NAME) or a model folder');+ const reader = isLocal ? localReader(target) : await hubReader(target, { fetchImpl, token });+ const source = isLocal ? target : `${HUB}/${target}`;+ if (reader.missing) return { source, found: false, note: 'Hugging Face has no such public model, or it is private: check the id; a private model needs HF_TOKEN.' };+ const files = Object.fromEntries(await Promise.all(FILES.map(async name => [name, await reader.read(name)])));+ if (reader.gated && !files['config.json']) {+ return { source, found: true, gated: true, note: 'Gated model: the person must accept its license on Hugging Face and provide HF_TOKEN. Ask; do not work around it.' };+ }+ const config = json(files['config.json']) ?? {}, text = config.text_config ?? {};+ const generation = json(files['generation_config.json']) ?? {};+ const tokenizer = json(files['tokenizer_config.json']) ?? {};+ const template = files['chat_template.jinja'] ?? json(files['chat_template.json'])?.chat_template+ ?? (typeof tokenizer.chat_template === 'string' ? tokenizer.chat_template : tokenizer.chat_template?.map?.(t => t.template).join('\n')) ?? '';+ const card = files['README.md'] ?? '';+ const architecture = config.architectures?.[0] ?? null, modelType = config.model_type ?? null;+ const flagValues = flag => [...new Set([...card.matchAll(new RegExp(`--${flag}[ =]+([A-Za-z0-9_.-]+)`, 'g'))].map(match => match[1]))];+ return {+ source, found: true, gated: reader.gated,+ architecture, modelType,+ contextLength: config.max_position_embeddings ?? text.max_position_embeddings ?? null,+ dtype: config.torch_dtype ?? config.dtype ?? text.torch_dtype ?? null,+ quantization: config.quantization_config?.quant_method ?? (config.quantization?.bits ? `mlx ${config.quantization.bits}-bit` : null),+ samplingDefaults: Object.fromEntries(SAMPLING.filter(key => generation[key] !== undefined).map(key => [key, generation[key]])),+ chatTemplate: template ? { toolCalls: /\btools\b/.test(template), toolSyntax: toolSyntax(template), thinking: /<think>|enable_thinking/.test(template) } : null,+ modelCard: {+ serveCommands: codeBlocks(card).filter(block => SERVE.test(block)).slice(0, 8),+ toolCallParsers: flagValues('tool-call-parser'), reasoningParsers: flagValues('reasoning-parser'),+ },+ support: checkSupport ? await support(architecture, modelType, fetchImpl) : null,+ };+}diff --git a/store/agents/autonomous-grid/lib/recipes.mjs b/store/agents/autonomous-grid/lib/recipes.mjsnew file mode 100644index 000000000..cb43eb39e--- /dev/null+++ b/store/agents/autonomous-grid/lib/recipes.mjs@@ -0,0 +1,95 @@+// Official serving recipes, read live from the engines' own sites. When a site has no entry for the+// model this throws NoRecipe: the caller says so and never invents flags.+const VLLM_SITE = 'https://recipes.vllm.ai';+const SGLANG_INDEX = 'https://docs.sglang.io/llms.txt';++export class NoRecipe extends Error {+ constructor(message) { super(message); this.code = 'NO_RECIPE'; }+}++const squash = text => text.toLowerCase().replace(/[^a-z0-9]/g, '');++async function get(fetchImpl, url, as) {+ const response = await fetchImpl(url, { signal: AbortSignal.timeout(20_000) });+ if (response.status === 404) return null;+ if (!response.ok) throw new Error(`${new URL(url).host} answered HTTP ${response.status} for ${url}`);+ return as === 'json' ? response.json() : response.text();+}++function checkId(hfId) {+ if (!/^[\w.-]+\/[\w.-]+$/.test(hfId || '')) throw new Error('Pass the Hugging Face id: ORG/NAME');+}++/** vLLM's per-model recipe: minimum version, always-on args, features (tool calls, reasoning, opt-ins), variants. */+export async function vllmRecipe(hfId, { fetchImpl = globalThis.fetch } = {}) {+ checkId(hfId);+ const catalogue = await get(fetchImpl, `${VLLM_SITE}/models.json`, 'json');+ if (!Array.isArray(catalogue)) throw new Error('recipes.vllm.ai/models.json did not return a list');+ const entry = catalogue.find(row => row.hf_id?.toLowerCase() === hfId.toLowerCase());+ if (!entry) throw new NoRecipe(`recipes.vllm.ai has no recipe for ${hfId}.`);+ const recipe = await get(fetchImpl, `${VLLM_SITE}${entry.json}`, 'json');+ if (!recipe) throw new NoRecipe(`recipes.vllm.ai lists ${entry.hf_id} but its recipe file is missing.`);+ const model = recipe.model ?? {}, optIn = new Set(recipe.opt_in_features ?? []);+ const recommended = recipe.recommended_command ?? {};+ return {+ engine: 'vllm', source: `${VLLM_SITE}${entry.url}`, hfId: entry.hf_id,+ minVersion: model.min_vllm_version ?? null, contextLength: Number(model.context_length) || null,+ architecture: model.architecture ?? null, parameters: model.parameter_count ?? null, activeParameters: model.active_parameters ?? null,+ baseArgs: model.base_args ?? [], baseEnv: model.base_env ?? {},+ features: Object.fromEntries(Object.entries(recipe.features ?? {}).map(([name, feature]) =>+ [name, { optIn: optIn.has(name), args: feature.args ?? [], env: feature.env ?? {}, description: feature.description ?? '' }])),+ variants: Object.fromEntries(Object.entries(recipe.variants ?? {}).map(([name, variant]) => [name, {+ modelId: variant.model_id ?? model.model_id ?? entry.hf_id, precision: variant.precision ?? null,+ vramMinimumGb: variant.vram_minimum_gb ?? null, extraArgs: variant.extra_args ?? [], description: variant.description ?? '',+ }])),+ install: model.install ?? null,+ recommended: { hardware: recommended.hardware ?? null, variant: recommended.variant ?? null, command: recommended.command ?? null, dockerImage: recommended.docker_image ?? null },+ guide: recipe.guide ?? '',+ };+}++/** A heading's text up to the next heading of the same or a higher level; `#` lines inside code fences are not headings. */+function section(markdown, heading) {+ const lines = markdown.split('\n'), wanted = new RegExp(`^(#{2,3}) .*${heading}`, 'i');+ let fenced = false, level = 0;+ const kept = [];+ for (const line of lines) {+ if (/^```/.test(line)) fenced = !fenced;+ const depth = !fenced && !/^```/.test(line) ? line.match(/^(#{1,6}) /)?.[1].length : undefined;+ if (!level) { if (depth && wanted.test(line)) { level = depth; kept.push(line); } continue; }+ if (depth && depth <= level) break;+ kept.push(line);+ }+ return kept.join('\n').trim();+}++/** SGLang's cookbook page for the model's family: its serve commands, parsers, install and tuning notes. */+export async function sglangCookbook(hfId, { fetchImpl = globalThis.fetch } = {}) {+ checkId(hfId);+ const index = await get(fetchImpl, SGLANG_INDEX, 'text');+ if (!index) throw new Error('docs.sglang.io/llms.txt is unavailable');+ const pages = [...index.matchAll(/\]\((https:\/\/docs\.sglang\.io\/cookbook\/autoregressive\/([^/)]+)\/([^/)]+)\.md)\)/g)]+ .map(([, url, org, family]) => ({ url, org, family }));+ const name = squash(hfId.split('/')[1]);+ const fits = pages.filter(page => squash(page.family) && name.startsWith(squash(page.family)));+ const longest = Math.max(0, ...fits.map(page => squash(page.family).length));+ const best = fits.filter(page => squash(page.family).length === longest);+ if (!best.length) throw new NoRecipe(`The SGLang cookbook has no page for ${hfId}.`);+ const page = best[0];+ const markdown = await get(fetchImpl, page.url, 'text');+ if (!markdown) throw new NoRecipe(`${page.url} is listed but missing.`);+ const blocks = [...markdown.matchAll(/```[^\n]*\n([\s\S]*?)```/g)].map(match => match[1].trim());+ const values = flag => [...new Set([...markdown.matchAll(new RegExp(`--${flag}[ =]+([A-Za-z0-9_.-]+)`, 'g'))].map(match => match[1]))];+ return {+ engine: 'sglang', source: page.url, family: page.family, alsoMatching: best.slice(1).map(other => other.url),+ toolCallParsers: values('tool-call-parser'), reasoningParsers: values('reasoning-parser'),+ serveCommands: blocks.filter(block => /sglang serve|sglang\.launch_server/.test(block)).slice(0, 6),+ installation: section(markdown, 'Installation'), configurationTips: section(markdown, 'Configuration Tips'),+ };+}++export async function recipe(engine, hfId, options) {+ if (engine === 'vllm') return vllmRecipe(hfId, options);+ if (engine === 'sglang') return sglangCookbook(hfId, options);+ throw new Error('Use: fleet recipe vllm|sglang ORG/NAME');+}diff --git a/store/agents/autonomous-grid/lib/remoteModels.mjs b/store/agents/autonomous-grid/lib/remoteModels.mjsnew file mode 100644index 000000000..b168d4faf--- /dev/null+++ b/store/agents/autonomous-grid/lib/remoteModels.mjs@@ -0,0 +1,130 @@+// `fleet models` for a machine other than this one. The finder reads a machine's own disk and asks its+// own loopback ports, so it has to run there: over SSH it is copied to the machine and run by that+// machine's Node. Without Node 18+ there, a plain `find` still lists the model files, and says what is+// missing. A Harness link runs Grid commands only, so it cannot run the finder at all.+import { spawn } from 'node:child_process';+import { mkdir, readFile, readlink, stat, symlink } from 'node:fs/promises';+import { homedir } from 'node:os';+import { dirname, isAbsolute, join } from 'node:path';+import { fileURLToPath } from 'node:url';++import { shellQuote } from './fleet.mjs';++const HERE = dirname(fileURLToPath(import.meta.url));+export const FINDER_FILES = ['inventory.mjs', 'candidates.mjs', 'finderMain.mjs'];+const DIR = '.cache/grid-harness/finder'; // relative to the remote home, where an SSH command starts+const NO_NODE = '__GRID_FINDER_NO_NODE__';++// The same roots the finder scans (inventory.mjs `defaultRoots`), for the listing without Node. Sizes+// in kilobytes: BusyBox `find` (Alpine) has no `M` suffix [run].+const FALLBACK = [+ 'for d in ~/.grid/models ~/.ollama/models /usr/share/ollama/.ollama/models ~/.lmstudio/models',+ '~/.cache/lm-studio/models "${HF_HUB_CACHE:-${HF_HOME:-$HOME/.cache/huggingface}/hub}" ~/models ~/Models ~/Downloads;',+ 'do [ -d "$d" ] && find -L "$d" -maxdepth 8 \\( -iname "*.gguf" -size +51200k -o -name config.json',+ '-o -path "*/manifests/*" -type f \\) 2>/dev/null; done | head -300',+].join(' ');++/** SSH options every call shares: the user's existing trust, never a prompt (as `invocation` in fleet.mjs). */+export function sshBase(machine) {+ return ['-o', 'BatchMode=yes', '-o', 'StrictHostKeyChecking=yes', '-o', 'ConnectTimeout=8',+ ...(machine.port ? ['-p', String(machine.port)] : []), machine.host];+}++function sshRun(machine, remote, { input = null, timeoutMs = 90_000, spawnImpl = spawn } = {}) {+ return new Promise(resolve => {+ let stdout = '', stderr = '', done = false;+ const child = spawnImpl('ssh', [...sshBase(machine), remote], { stdio: ['pipe', 'pipe', 'pipe'] });+ const finish = result => { if (!done) { done = true; clearTimeout(timer); resolve({ stdout, stderr, ...result }); } };+ const timer = setTimeout(() => { child.kill('SIGTERM'); finish({ code: 124, error: `no answer from ${machine.host} within ${timeoutMs / 1000}s` }); }, timeoutMs);+ child.stdout.setEncoding('utf8').on('data', chunk => { stdout += chunk; });+ child.stderr.setEncoding('utf8').on('data', chunk => { stderr += chunk; });+ child.once('error', error => finish({ code: 127, error: error.code === 'ENOENT' ? 'ssh is not installed here' : error.message }));+ child.once('close', code => finish({ code: code ?? 1, error: code === 0 ? null : stderr.trim().split('\n').pop() || `ssh exited ${code}` }));+ child.stdin.end(input ?? '');+ });+}++/**+ * The finder's output for `machine`: `{ text, full }`, where `full` says whether it is the finder's own+ * table (Node there) or only a file list. Throws with the reason when the machine cannot be read.+ */+export async function remoteModels(machine, { summary = true, spawnImpl = spawn } = {}) {+ if (machine.transport === 'harness') {+ throw new Error(`${machine.name || machine.id} is linked through Harness, which runs Grid commands only, so its model `+ + `files cannot be listed from here. Its hardware: fleet run --machine ${machine.id} -- device-info --json`);+ }+ if (machine.transport !== 'ssh') throw new Error(`fleet models --machine needs an SSH machine; ${machine.id} is ${machine.transport}.`);+ const call = (remote, options = {}) => sshRun(machine, remote, { spawnImpl, ...options });+ for (const file of FINDER_FILES) {+ const sent = await call(`mkdir -p ${DIR} && cat > ${DIR}/${file}`, { input: await readFile(join(HERE, file), 'utf8'), timeoutMs: 30_000 });+ if (sent.code !== 0) throw new Error(`Could not copy the finder to ${machine.host}: ${sent.error}`);+ }+ // A number, not a boolean: newer Node throws on `process.exit(true)`, which would read as "no Node".+ const node = 'command -v node >/dev/null 2>&1 && node -e "process.exit(Number(+process.versions.node.split(\'.\')[0] < 18))"';+ const run = await call(`if ${node}; then exec node ${DIR}/finderMain.mjs${summary ? ' --summary' : ''}; `+ + `else echo ${NO_NODE}; uname -sm; ${FALLBACK}; fi`);+ if (run.code !== 0) throw new Error(`The finder failed on ${machine.host}: ${run.error}`);+ if (!run.stdout.startsWith(NO_NODE)) return { text: run.stdout.trimEnd(), full: true };+ const [, system, ...files] = run.stdout.trimEnd().split('\n');+ return {+ full: false,+ text: [`machine ${system} (${machine.host}) — Node 18+ is not installed there, so this is a file list only:`,+ 'no context, memory fit, tool calls, engines or ports. Install Node there for the full table.', '',+ ...(files.length ? files : ['(no model files found)'])].join('\n'),+ };+}++const GGUF_NAME = /^[A-Za-z0-9][A-Za-z0-9._-]{0,200}\.gguf$/;++/**+ * Make a GGUF already on a machine servable by Grid there, without copying it: `join --serve` reads only+ * `~/.grid/models` and keeps just the file name [run], so the file is linked in. A projector is linked as+ * `<name>.mmproj.gguf`, the name `--serve` finds it by [run]. An existing name is never replaced.+ */+export async function linkModel(machine, file, name, { projector, env = process.env, spawnImpl = spawn } = {}) {+ if (!GGUF_NAME.test(name)) throw new Error('NAME is a plain file name ending in .gguf.');+ const pairs = [[file, name], ...(projector ? [[projector, name.replace(/\.gguf$/, '.mmproj.gguf')]] : [])];+ if (pairs.some(([source]) => !isAbsolute(source))) throw new Error('Give FILE (and --projector) as full paths on that machine, as `fleet models` prints them.');+ if (machine.transport === 'harness') throw new Error(`${machine.name || machine.id} is linked through Harness, which runs Grid commands only; a model for it comes from the catalog (pull).`);+ if (machine.transport === 'local') {+ const dir = join(env.GRID_HOME || join(homedir(), '.grid'), 'models');+ await mkdir(dir, { recursive: true });+ const lines = [];+ for (const [source, target] of pairs) {+ if (!(await stat(source).catch(() => null))?.isFile()) throw new Error(`No file at ${source}.`);+ const at = join(dir, target);+ const current = await readlink(at).catch(error => (error.code === 'ENOENT' ? null : '(not a link)'));+ if (current === source) { lines.push(`already linked ${at}`); continue; }+ if (current !== null) throw new Error(`${at} already exists and is not a link to ${source}; choose another NAME.`);+ await symlink(source, at);+ lines.push(`linked ${at} -> ${source}`);+ }+ const engine = join(env.GRID_HOME || join(homedir(), '.grid'), 'bin', process.platform === 'win32' ? 'llama-server.exe' : 'llama-server');+ if (!(await stat(engine).catch(() => null))) lines.push(noEngine(machine.id).replace(` --machine ${machine.id}`, ''));+ return { text: lines.join('\n') };+ }+ if (machine.transport !== 'ssh') throw new Error(`Unknown transport for ${machine.id}.`);+ const script = [+ ...(machine.gridHome ? [`GRID_HOME=${shellQuote(machine.gridHome)}; export GRID_HOME;`] : []),+ 'd="${GRID_HOME:-$HOME/.grid}/models"; mkdir -p "$d" || exit 5;',+ 'link() { if [ ! -f "$1" ]; then echo "No file at $1." >&2; exit 3; fi;',+ 'if [ -L "$d/$2" ] && [ "$(readlink "$d/$2")" = "$1" ]; then echo "already linked $d/$2";',+ 'elif [ -e "$d/$2" ] || [ -L "$d/$2" ]; then echo "$d/$2 already exists and is not a link to $1; choose another NAME." >&2; exit 4;',+ 'else ln -s "$1" "$d/$2" && echo "linked $d/$2 -> $1"; fi; };',+ ...pairs.map(([source, target]) => `link ${shellQuote(source)} ${shellQuote(target)} || exit $?;`),+ // The join after this needs Grid's own engine there, and needs it to RUN: without it the join still+ // says "starting" and the engine dies at once in a log on that machine — "llama-server not found", or+ // "libgomp.so.1: cannot open shared object file" on a minimal Ubuntu [run].+ 'e="${GRID_HOME:-$HOME/.grid}/bin/llama-server"; [ -x "$e" ] || e="$(command -v llama-server 2>/dev/null)";',+ `if [ -z "$e" ]; then echo ${shellQuote(noEngine(machine.id))};`,+ 'elif ! err="$("$e" --version 2>&1)"; then',+ `echo "NOT READY: Grid's engine is installed on this machine but does not start: $(printf %s "$err" | head -1)";`,+ `echo " A missing lib*.so is a system package there (libgomp.so.1 = libgomp1); tell the person which one — installing it is theirs to do."; fi;`,+ ].join(' ');+ const result = await sshRun(machine, script, { spawnImpl, timeoutMs: 30_000 });+ if (result.code !== 0) throw new Error(result.stderr.trim() || `Could not link on ${machine.host}: ${result.error}`);+ return { text: result.stdout.trimEnd() };+}++const noEngine = id => `NOT READY: Grid's engine (llama.cpp) is not installed on this machine, so a join here would die at start. `+ + `Ask first (a download), then: "$GRID_FLEET" run --machine ${id} -- engine install llama.cpp`;diff --git a/store/agents/autonomous-grid/lib/serve.mjs b/store/agents/autonomous-grid/lib/serve.mjsnew file mode 100644index 000000000..54c3b26ed--- /dev/null+++ b/store/agents/autonomous-grid/lib/serve.mjs@@ -0,0 +1,56 @@+// Engines other than Grid's own, started so they outlive the command that started them. An agent's+// shell tears down what it spawned when the command returns — a `nohup … &` mlx-lm died that way with an+// empty log — so the engine runs as the leader of its own session, logging to run/NAME.log, with its PID+// in run/NAME.pid. Stop kills only a PID this file recorded.+import { spawn } from 'node:child_process';+import { mkdir, open, readFile, rm, writeFile } from 'node:fs/promises';+import { join } from 'node:path';++const NAME = /^[a-z0-9][a-z0-9._-]{0,62}$/;++export function runFiles(workspace, name) {+ if (!NAME.test(name || '')) throw new Error('The engine name is lowercase letters, digits, dot, dash or underscore.');+ const dir = join(workspace, 'run');+ return { dir, log: join(dir, `${name}.log`), pid: join(dir, `${name}.pid`) };+}++const alive = pid => { try { process.kill(pid, 0); return true; } catch (error) { return error.code === 'EPERM'; } };++export async function recordedPid(workspace, name) {+ const pid = Number((await readFile(runFiles(workspace, name).pid, 'utf8').catch(() => '')).trim());+ return Number.isInteger(pid) && pid > 1 ? pid : null;+}++/** Start `argv` detached: own session and process group, stdio to the log, PID recorded. */+export async function serve(workspace, name, argv, { env = {}, spawnImpl = spawn } = {}) {+ if (!Array.isArray(argv) || !argv.length) throw new Error('Give the engine command after --.');+ const files = runFiles(workspace, name);+ const running = await recordedPid(workspace, name);+ if (running && alive(running)) throw new Error(`${name} is already running as PID ${running}; stop it first.`);+ await mkdir(files.dir, { recursive: true });+ const log = await open(files.log, 'a');+ try {+ await log.write(`\n=== ${new Date().toISOString()} ${argv.join(' ')}\n`);+ const child = spawnImpl(argv[0], argv.slice(1), {+ cwd: workspace, detached: true, stdio: ['ignore', log.fd, log.fd], env: { ...process.env, ...env },+ });+ await new Promise((resolve, reject) => { child.once('spawn', resolve); child.once('error', reject); });+ child.unref();+ await writeFile(files.pid, `${child.pid}\n`);+ return { name, pid: child.pid, log: files.log };+ } finally { await log.close(); }+}++/** Stop the PID recorded for NAME: TERM, then KILL after `graceMs`; never any other process. */+export async function stop(workspace, name, { graceMs = 10_000 } = {}) {+ const files = runFiles(workspace, name);+ const pid = await recordedPid(workspace, name);+ if (!pid) return { name, stopped: false, note: `no PID recorded for ${name}` };+ if (!alive(pid)) { await rm(files.pid, { force: true }); return { name, pid, stopped: true, note: 'already gone' }; }+ process.kill(-pid, 'SIGTERM');+ const deadline = Date.now() + graceMs;+ while (alive(pid) && Date.now() < deadline) await new Promise(resolve => setTimeout(resolve, 250));+ if (alive(pid)) process.kill(-pid, 'SIGKILL');+ await rm(files.pid, { force: true });+ return { name, pid, stopped: true };+}diff --git a/store/agents/autonomous-grid/lib/verify.mjs b/store/agents/autonomous-grid/lib/verify.mjsindex d6ab8b40c..248308eac 100644--- a/store/agents/autonomous-grid/lib/verify.mjs+++ b/store/agents/autonomous-grid/lib/verify.mjs@@ -47,3 +47,133 @@ export async function verifyModel(workspace, machine, mode, grid, model, { run =/^(Could not read|The model|The model endpoint)/.test(error.message) ? error.message : 'The model check was interrupted. Ask Model Manager to retry.');} finally { clearInterval(heartbeat); }}++// Acceptance for a served model, step by step, every step bounded and every wait narrated: the engine+// (ready, listed, answers, calls a tool, speed) and then, with a grid, the relay (listed, answers).+// Nothing here waits without a deadline, and nothing is reported as working that was not seen working.+const TOOL = { type: 'function', function: { name: 'read_file', description: 'Read a text file from the project', parameters: { type: 'object', properties: { path: { type: 'string' } }, required: ['path'] } } };+// chat_template_kwargs is read by llama.cpp, mlx-lm, vLLM and SGLang; Ollama documents reasoning_effort "none".+const noThinking = kind => (kind === 'ollama' ? { reasoning_effort: 'none' } : { chat_template_kwargs: { enable_thinking: false } });++export function createLog(write = line => process.stderr.write(`${line}\n`), clock = Date.now) {+ const start = clock();+ return message => write(`[+${String(Math.round((clock() - start) / 1000)).padStart(4)}s] ${message}`);+}++async function call(fetchImpl, url, { body, headers = {}, timeoutMs, log, label }) {+ const started = Date.now();+ const heartbeat = setInterval(() => log(`${label}: still waiting (${Math.round((Date.now() - started) / 1000)}s of ${Math.round(timeoutMs / 1000)}s)`), 15_000);+ try {+ const response = await fetchImpl(url, {+ method: body ? 'POST' : 'GET', headers: { 'content-type': 'application/json', ...headers },+ body: body ? JSON.stringify(body) : undefined, signal: AbortSignal.timeout(timeoutMs),+ });+ const text = await response.text();+ let json = null;+ try { json = JSON.parse(text); } catch {}+ return { status: response.status, json, text: text.slice(0, 300), seconds: (Date.now() - started) / 1000 };+ } catch (error) {+ return { status: 0, json: null, text: error.name === 'TimeoutError' ? `no answer within ${Math.round(timeoutMs / 1000)}s` : error.message, seconds: (Date.now() - started) / 1000 };+ } finally { clearInterval(heartbeat); }+}++const message = json => json?.choices?.[0]?.message ?? {};+const reasoningOf = json => message(json).reasoning_content || message(json).reasoning || '';+const THINKING_ROOM = 1024;++/**+ * The one bounded "does it answer?" request. Some engines keep a model thinking whatever the request+ * says (LM Studio with a reasoning MLX model ignored every switch [run]), so 16 tokens come back as+ * reasoning only. That model still works — it answers after thinking — so ask once more with room for+ * the thinking, and say that it stays on, instead of failing a working engine.+ */+async function answerOnce(fetchImpl, url, { body, log, label, timeoutMs, headers }) {+ const first = await call(fetchImpl, url, { timeoutMs, log, label, headers, body });+ const content = (message(first.json).content ?? '').trim();+ if (content || !reasoningOf(first.json)) return { answer: first, content, thinkingStaysOn: false };+ log(` ${label}: only thinking came back in ${body.max_tokens} tokens; asking once more with room for it (${THINKING_ROOM} tokens)`);+ const second = await call(fetchImpl, url, { timeoutMs, log, label, headers, body: { ...body, max_tokens: THINKING_ROOM } });+ return { answer: second, content: (message(second.json).content ?? '').trim(), thinkingStaysOn: true };+}++/** The engine itself, at its own `/v1` URL. */+const wait = ms => new Promise(resolve => setTimeout(resolve, ms));++export async function verifyEngine({ url, model, kind, readyMs = 180_000, answerMs = 300_000, tools = true, fetchImpl = globalThis.fetch, log = createLog(), sleep = wait }) {+ const base = url.replace(/\/+$/, '');+ const steps = [];+ const record = (name, ok, detail) => { steps.push({ name, ok, ...detail }); log(`${ok ? 'PASS' : 'FAIL'} ${name}${detail.note ? ` — ${detail.note}` : ''}`); return ok; };+ if (!/\/v1$/.test(base)) return { ok: false, steps: [{ name: 'url', ok: false, note: 'the URL must end in /v1' }] };++ log(`step 1/4 ready: GET ${base}/models (up to ${readyMs / 1000}s, every 3s)`);+ const deadline = Date.now() + readyMs;+ let listed = null, attempt = 0;+ while (Date.now() < deadline) {+ attempt++;+ const models = await call(fetchImpl, `${base}/models`, { timeoutMs: 5_000, log, label: 'models' });+ const ids = (models.json?.data ?? []).map(row => row.id);+ if (models.status === 200) { listed = ids; break; }+ if (attempt === 1 || attempt % 5 === 0) log(` attempt ${attempt}: ${models.status || models.text}`);+ await sleep(3_000);+ }+ if (!record('ready', listed !== null, { note: listed === null ? `no 200 from /models within ${readyMs / 1000}s` : `${listed.length} model(s) listed` })) return { ok: false, steps };+ if (!record('listed', listed.includes(model), { note: listed.includes(model) ? model : `${model} not in ${JSON.stringify(listed.slice(0, 8))}` })) return { ok: false, steps };++ log(`step 2/4 answer: one request, max_tokens 16, thinking off (up to ${answerMs / 1000}s)`);+ const { answer, content, thinkingStaysOn } = await answerOnce(fetchImpl, `${base}/chat/completions`, { timeoutMs: answerMs, log, label: 'answer', body: {+ model, max_tokens: 16, temperature: 0, messages: [{ role: 'user', content: 'Reply with the single word: ok' }], ...noThinking(kind) } });+ const thinkingOnly = !content && reasoningOf(answer.json);+ if (!record('answer', Boolean(content), { seconds: answer.seconds, note: content+ ? `${JSON.stringify(content.slice(0, 40))}${thinkingStaysOn ? ` — thinking stays on here: it answers after thinking, slower and with more tokens` : ''}`+ : thinkingOnly ? `only reasoning text came back, even with ${THINKING_ROOM} tokens: thinking is still on` : `HTTP ${answer.status}: ${answer.text}` })) return { ok: false, steps };++ if (tools) {+ log('step 3/4 tool call: a read_file tool, "Open the file README.md."');+ const tool = await call(fetchImpl, `${base}/chat/completions`, { timeoutMs: answerMs, log, label: 'tool call', body: {+ model, max_tokens: 256, temperature: 0, tools: [TOOL], messages: [{ role: 'user', content: 'Open the file README.md.' }], ...noThinking(kind) } });+ const called = message(tool.json).tool_calls?.[0]?.function;+ if (!record('tool call', called?.name === 'read_file', { seconds: tool.seconds, note: called ? `${called.name}(${called.arguments})` : `no tool_calls; HTTP ${tool.status}` })) return { ok: false, steps };+ } else log('step 3/4 tool call: skipped');++ log('step 4/4 speed: 128 tokens');+ const speed = await call(fetchImpl, `${base}/chat/completions`, { timeoutMs: answerMs, log, label: 'speed', body: {+ model, max_tokens: 128, temperature: 0, messages: [{ role: 'user', content: 'Count from 1 to 60 in words.' }], ...noThinking(kind) } });+ const reported = speed.json?.timings?.predicted_per_second;+ const tokens = speed.json?.usage?.completion_tokens;+ const perSecond = reported ?? (tokens ? tokens / speed.seconds : null);+ record('speed', perSecond !== null, { seconds: speed.seconds, tokensPerSecond: perSecond && Math.round(perSecond * 10) / 10,+ note: perSecond ? `${perSecond.toFixed(1)} tokens/s${reported ? ' (engine timing)' : ' (wall clock, includes reading the prompt)'}` : `HTTP ${speed.status}` });+ return { ok: steps.every(step => step.ok), steps };+}++/** After a join: the relay must list the alias, then answer through the grid's own endpoint. */+export async function verifyRelay({ alias, engineModel, listModels, relay, listMs = 300_000, answerMs = 420_000, fetchImpl = globalThis.fetch, log = createLog(), sleep = wait }) {+ const steps = [];+ const record = (name, ok, detail) => { steps.push({ name, ok, ...detail }); log(`${ok ? 'PASS' : 'FAIL'} ${name}${detail.note ? ` — ${detail.note}` : ''}`); return ok; };+ log(`relay 1/2 listed: grid models, every 10s, up to ${listMs / 1000}s`);+ const deadline = Date.now() + listMs;+ let seen = false, attempt = 0, last = '';+ // The relay changes a name's case (an alias given in lower case came back upper case, and the other+ // way round), so names match whatever their case [run: 5 minutes lost waiting on a case difference].+ const same = (a, b) => typeof a === 'string' && typeof b === 'string' && a.toLowerCase() === b.toLowerCase();+ while (Date.now() < deadline) {+ attempt++;+ const listed = await listModels();+ const found = listed.ok && listed.models.find(name => same(name, alias));+ if (found) { seen = true; alias = found; break; }+ // Registered under the engine's own id: waiting cannot fix a missing --advertise-as [run: 5 minutes lost].+ const raw = listed.ok && engineModel && listed.models.find(name => same(name, engineModel));+ if (raw) return (record('relay listed', false, { note: `listed as ${raw}, not ${alias}: join again with --advertise-as ${alias}` }), { ok: false, steps });+ last = listed.ok ? `${listed.models.length} model(s), not yet ${alias}: ${listed.models.slice(0, 5).map(name => String(name).slice(0, 60)).join(', ') || 'none'}` : listed.error;+ log(` attempt ${attempt}: ${last}`);+ await sleep(10_000);+ }+ if (!record('relay listed', seen, { note: seen ? alias : `not listed within ${listMs / 1000}s (${last})` })) return { ok: false, steps };+ log(`relay 2/2 answer through the grid (up to ${answerMs / 1000}s: a new engine is probed first and may hold its slot for minutes)`);+ const { answer, content, thinkingStaysOn } = await answerOnce(fetchImpl, `${relay.baseUrl.replace(/\/+$/, '')}/chat/completions`, {+ timeoutMs: answerMs, log, label: 'relay answer', headers: relay.apiKey ? { authorization: `Bearer ${relay.apiKey}` } : {},+ body: { model: alias, max_tokens: 16, messages: [{ role: 'user', content: 'Reply with the single word: ok' }], ...noThinking() } });+ record('relay answer', Boolean(content), { seconds: answer.seconds, note: content+ ? `${JSON.stringify(content.slice(0, 40))}${thinkingStaysOn ? ' — thinking stays on' : ''}` : `HTTP ${answer.status}: ${answer.text}` });+ return { ok: steps.every(step => step.ok), steps };+}diff --git a/store/agents/autonomous-grid/skills/engine-lm-studio/SKILL.md b/store/agents/autonomous-grid/skills/engine-lm-studio/SKILL.mdnew file mode 100644index 000000000..66581a59e--- /dev/null+++ b/store/agents/autonomous-grid/skills/engine-lm-studio/SKILL.md@@ -0,0 +1,105 @@+---+name: engine-lm-studio+description: "Reuse what LM Studio already has on this computer through its `lms` CLI: adopt its running server into the person's fleet, serve its MLX models, or serve its GGUF files without it. Load before touching LM Studio, its models or its server."+---++# LM Studio (`lms` CLI, no clicks)++Official docs, read 2026-09-29 (source files in github.com/lmstudio-ai/docs):+[App, llmster and lms](https://lmstudio.ai/docs/app/basics/lmstudio-vs-llmster-vs-lms) ·+[Headless](https://lmstudio.ai/docs/developer/core/headless) · [lms server start](https://lmstudio.ai/docs/cli/serve/server-start) ·+[lms server stop](https://lmstudio.ai/docs/cli/serve/server-stop) · [lms load](https://lmstudio.ai/docs/cli/local-models/load) ·+[lms import](https://lmstudio.ai/docs/cli/local-models/import) · [lms daemon up/down](https://lmstudio.ai/docs/cli/daemon/daemon-up) ·+[Idle TTL](https://lmstudio.ai/docs/developer/core/ttl-and-auto-evict) · [OpenAI compatibility](https://lmstudio.ai/docs/developer/openai-compat) ·+[Tools](https://lmstudio.ai/docs/developer/openai-compat/tools) · [Structured output](https://lmstudio.ai/docs/developer/openai-compat/structured-output) ·+[Import models](https://lmstudio.ai/docs/app/advanced/import-model) · [Server settings](https://lmstudio.ai/docs/developer/core/server/settings)+Tested: LM Studio 0.4.25 (`brew install --cask lm-studio`), MacBook Pro M1 Pro 32 GB, macOS 26.6, 2026-09-29.+Tags: `[doc]` official page above, `[run]` seen on the tested machine, `[?]` unverified.++## When to use it++- **Its server is answering** (`fleet models` → `kind: lm-studio`): adopt it, but load the model+ yourself with a 64K+ context (below) — a model it loads on demand gets 8192 [run], too small for an agent.+- **An MLX model sits in LM Studio's folder and mlx-lm is not installed**: LM Studio runs MLX on Apple+ silicon (its structured output uses Outlines for MLX [doc]), so use it instead of installing anything.+- **A GGUF in LM Studio's folder**: Grid's own engine serves it in place; LM Studio need not run.+- Never install LM Studio just to serve a model: Grid's engine covers GGUF, mlx-lm covers MLX.++## Where its models live / how to list them++- `~/.lmstudio/models/<publisher>/<model>/<file>` — it keeps Hugging Face's layout [doc]. The folder can be+ moved in the app's My Models tab [doc]; no documented setting file or env var [?]. Older versions used+ `~/.cache/lm-studio/models` [?]. `fleet models` scans both.+- `lms ls --json` → `modelKey`, `path`, `format` (gguf/mlx), `sizeBytes` [run]. Keys prefix-match: a short+ key matched two downloaded models and loaded "the first one" [run] — always pass the exact key.+- A fresh install holds only an embedding model, no chat model [run].++## Installed? Running?++- App: `/Applications/LM Studio.app`. `lms` lives in `~/.lmstudio/bin` and appears only after the app ran+ once [run]. Headless servers use **llmster** instead of the app: `curl -fsSL https://lmstudio.ai/install.sh | bash` [doc].+- `lms server status` → "The server is not running." or its port [run]. Launching the app does not+ start the server [run]. `fleet models` lists a running one as `kind: lm-studio` (its `owned_by` is+ `organization_owner` [run]).++## Start an already-downloaded model++Port P from outside `machine.listeningPorts`. Always `--port`: without it the last used port is reused [doc].++ export PATH="$HOME/.lmstudio/bin:$PATH"+ lms server start --port P --bind 127.0.0.1 # starts LM Studio in the background if needed [doc][run]+ lms load <exact key> --estimate-only --context-length 65536 --gpu max -y # fits? [doc][run]+ lms load <exact key> --context-length 65536 --gpu max --identifier <id> -y++- With the app closed, `server start` printed "Waking up LM Studio service..." and was up in 4 s; it runs+ the app as `LM Studio --run-as-service`, with no window [run]. It returns at once (not a foreground process).+- `--estimate-only` prints GPU and total memory for that context and whether guardrails allow it [doc][run].+- A model loaded with `lms load` has no idle timer and stays until unloaded [doc]; on-demand loads unload+ after 60 minutes idle [doc]. Nothing downloads on load [run].++## Ready means++Run `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <id> --kind lm-studio` — it performs the+request checks with deadlines and prints each one. Also:++1. `lms server status` shows port P [run]. 2. `GET http://127.0.0.1:P/v1/models` lists `<id>` [doc][run].+3. `lms ps` shows `<id>` with CONTEXT ≥ 65536 [run]. 4. One bounded `/v1/chat/completions` request with+`model: <id>`, `max_tokens` 16 → non-empty `content` [run].+A 401 means the person turned on "Require Authentication" [doc]: do not change their setting; say so and+serve the file with Grid's engine instead.++## Join Harness Compute++ "$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <id> --advertise-as ALIAS --name NAME++`/v1` is required: with the bare root, Grid's capability probe records structured output as unsupported [run].+Grid's own detector finds LM Studio only on 1234 [run].++## Tool calls, JSON output, thinking++- Tools: `tools` on `/v1/chat/completions` and `/v1/responses`, OpenAI format [doc]. Grid's probe: tools yes [run].+- Structured output: `response_format` with `json_schema` [doc]; GGUF uses llama.cpp grammars, MLX uses+ Outlines [doc]. Grid's probe: json_schema yes, json_object no [run].+- Thinking control through the OpenAI endpoint: not documented [?]. Check `content` in the ready request.++## Memory and speed knobs++`--context-length` (≥ 65536 here), `--gpu max|off|0–1` (automatic when omitted) [doc], `--ttl <seconds>` [doc],+`--parallel <n>` (default 4 on this build [run]), `--speculative-draft-mtp` for models with MTP layers [run: in help].++## Stop++- Your model: `lms unload <id>` [doc]. Your server: `lms server stop` [doc] — the background service keeps running [run].+- The service you woke: `lms daemon down` stops llmster only, not the app [doc]; the app service ends with+ `osascript -e 'quit app "LM Studio"'` [run].+- If LM Studio was running before you came, unload only your model and leave the rest as it was.++## Known failures → what to do++| Sign | Do |+|---|---|+| "2 models match the provided model key" | use the exact `modelKey` from `lms ls --json` [run] |+| CONTEXT 8192 in `lms ps` | the model was loaded on demand; unload it and `lms load` with `--context-length 65536` [run] |+| `lms get … -y` shows a spinner forever | it downloads in the background [run]; never start a download without the go-ahead |+| a foreign GGUF must appear in LM Studio | `lms import -l -y --user-repo <user>/<repo> FILE.gguf`: **without `-l` it moves the file** [doc]; the file name must end in `.gguf` and the link must point at the real file, not a temp copy [run] |+| 401 from `/v1/models` | authentication is on in the person's settings [doc]; do not turn it off |diff --git a/store/agents/autonomous-grid/skills/engine-mlx-lm/SKILL.md b/store/agents/autonomous-grid/skills/engine-mlx-lm/SKILL.mdnew file mode 100644index 000000000..b497a721e--- /dev/null+++ b/store/agents/autonomous-grid/skills/engine-mlx-lm/SKILL.md@@ -0,0 +1,121 @@+---+name: engine-mlx-lm+description: "Serve an MLX or Hugging Face safetensors model already on this Mac with mlx-lm's server and join it to the person's fleet. Load before installing, starting or stopping mlx-lm, or when `fleet models` lists an `mlx` or `safetensors` model on Apple silicon."+---++# mlx-lm server (Apple silicon)++Official docs, read 2026-09-29:+[SERVER.md](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/SERVER.md) ·+[server.py (flags, routes)](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/server.py) ·+[README](https://github.com/ml-explore/mlx-lm/blob/main/README.md) ·+[Hugging Face cache variables](https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables) ·+[Hugging Face cache layout](https://huggingface.co/docs/huggingface_hub/guides/manage-cache)+Tested: mlx-lm 0.31.3 (the latest on PyPI that day), uv venv with Python 3.12, MacBook Pro M1 Pro 32 GB,+macOS 26.6, 2026-09-29. Tags: `[doc]` official source above, `[run]` seen on the tested machine, `[?]` unverified.++## When to use it++- `fleet models` lists a model with `format: mlx` (an MLX-converted folder: `config.json` with a `quantization` block) on+ a Mac: this is its engine. Grid's own engine reads only GGUF.+- `format: safetensors` (a plain Hugging Face model) on a Mac: mlx-lm served one directly [run]; vLLM and+ SGLang are for NVIDIA/AMD servers.+- If LM Studio is installed and already holds the MLX model, LM Studio can serve it with nothing new+ installed (`engine-lm-studio`). If a GGUF of the same model exists, prefer Grid's engine.+- The docs call the server "not recommended for production" (basic security only) [doc]: bind 127.0.0.1.+- A GGUF of the same model already on disk is served by Grid's engine; do not download an MLX copy of it.+- mlx-lm installed and no MLX model yet: `"$GRID_FLEET" candidates mlx` (see `run-local-model`).+- Verified end to end [run]: started offline on a local snapshot, `fleet verify` passed ready, answer,+ tool call and speed; `grid join … --at http://127.0.0.1:P/v1 -m <snapshot path> --advertise-as NAME`+ joined in 6 s and the relay answered through the grid.++## Where its models live / how to list them++- Hugging Face cache: `$HF_HUB_CACHE`, else `$HF_HOME/hub`, else `~/.cache/huggingface/hub` [doc].+ Layout `models--<org>--<name>/snapshots/<commit>/` with files linked into `blobs/` [doc].+- An MLX folder has `config.json` with a `quantization` block (`bits`, `group_size`) or sits in an+ `mlx-community` repo; a plain model has `config.json` and `*.safetensors` without it. `fleet models`+ reports both, including folders outside the cache (`~/models`, LM Studio's folder).+- Server side: `GET /v1/models` lists every cached repo as `org/name` plus the served path [doc][run].++## Installed? Running?++- Installed: `fleet models` lists `mlx-lm` under installed engines — on `PATH` or in `~/.grid/envs/mlx-lm`.+ Install there and nowhere else, never into the system Python (a slow step: ask for the go-ahead first):+ `uv venv --python 3.12 ~/.grid/envs/mlx-lm && uv pip install --python ~/.grid/envs/mlx-lm/bin/python mlx-lm`+ [run] (`pip install mlx-lm` or `conda install -c conda-forge mlx-lm` [doc]). ENV below is that folder.+- Running: `fleet models` → an engine on 8080 labelled `openai-compatible` (it has no `owned_by`) whose+ models are Hugging Face ids [run].++## Start an already-downloaded model (never downloads)++Port P from outside `machine.listeningPorts` (8090 was taken by another app during testing [run]):++ "$GRID_FLEET" serve mlx-P --env HF_HUB_OFFLINE=1 -- ENV/bin/mlx_lm.server \+ --model <snapshot dir or cached org/name> --host 127.0.0.1 --port P --max-tokens 32768 \+ --chat-template-args '{"enable_thinking":false}'++- `fleet serve` starts it in its own session, so it outlives your shell; log in `run/mlx-P.log`, PID in+ `run/mlx-P.pid`. Under the agent's shell a plain `nohup … &` died with an empty log when the command+ returned, while a `fleet serve` process was still alive in a later command [run]. Never `launchd`.++- `HF_HUB_OFFLINE=1`: no HTTP calls, cached files only, an error if missing [doc]. Without it an uncached+ model is downloaded from Hugging Face [doc].+- `--max-tokens` defaults to 512 [doc]: a request without its own limit would stop mid-answer.+- Sampling defaults are greedy (`--temp 0.0`, `--top-p 1.0`) [doc]; set `--temp`/`--top-p`/`--top-k` from+ the model card's recommended settings.+- Ready log: `Starting httpd at 127.0.0.1 on port P...` [run].+- There is no context size to set: the cache grows with the conversation, up to the model's+ `max_position_embeddings`. Check that it is ≥ 65536 and that weights + 64K of cache fit before starting.++## Ready means++Run `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <path or id> --kind mlx-lm` right after+`serve` — it waits for loading itself; no `sleep`, `curl` or log reading first. What it checks:++1. The PID is alive. 2. `GET /health` → `{"status": "ok"}` [run] (in the source, not in SERVER.md [doc]).+3. `GET /v1/models` → 200 [doc]. 4. One bounded `/v1/chat/completions` request, `max_tokens` 16, `model`+ = the path or id you started with → non-empty `content` [run].++## Join Harness Compute++ "$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <path or id you started with> --advertise-as ALIAS --name NAME++`/v1` is required: `/models` without it answers 404 (only chat accepts both) [doc][run]. Without+`--advertise-as` the picker shows the whole snapshot path, and `verify --alias` waits five minutes for a+name that never appears [run]. Grid's detector labels anything on 8080 as `mlx`, whatever it is [run].++## Tool calls, JSON output, thinking++- Tools: passed to the chat template; when the tokenizer has no tool-calling support the server logs+ "Received tools but model does not support tool calling" and answers without calls [doc]. Grid's probe+ reported no tool calls for a 0.5B 4-bit model whose template does mention tools [run]: run the tool+ check before offering a model for coding.+- Thinking: `--chat-template-args '{"enable_thinking":false}'` at start, or `chat_template_kwargs` per+ request [doc]. Reasoning text comes back in `message.reasoning` [doc].+- JSON: Grid's probe reported JSON object and schema modes as supported [run].++## Memory and speed knobs++| Flag | Default | Use |+|---|---|---|+| `--decode-concurrency` | 32 | requests decoded together [doc] |+| `--prompt-concurrency` | 8 | prompts prefilled together [doc] |+| `--prefill-step-size` | 2048 | lower it if prefill spikes memory [doc] |+| `--prompt-cache-size` / `--prompt-cache-bytes` | 10 caches / unlimited | cap memory kept for reuse [doc] |+| `--kv-bits 4\|8` | off | smaller cache for long context, but one request at a time [doc]; newer than 0.31.3 [run: absent from its help] |+| `--draft-model`, `--num-draft-tokens` | none, 3 | speculative decoding [doc] |++## Stop++`"$GRID_FLEET" stop mlx-P`, then confirm P is gone from `fleet models --summary` (ports in use).++## Known failures → what to do++| Sign | Do |+|---|---|+| `OfflineModeIsEnabled` or file-not-found at start | the model is not fully cached; offer the download as a slow step [doc] |+| answers stop around 512 tokens | start with `--max-tokens` [doc] |+| empty `content`, text in `reasoning` | disable thinking in the chat template [doc] |+| "Received tools but model does not support tool calling" | not a coding model; pick another [doc] |+| address already in use | pick another port; never stop the other process [run] |diff --git a/store/agents/autonomous-grid/skills/engine-ollama/SKILL.md b/store/agents/autonomous-grid/skills/engine-ollama/SKILL.mdnew file mode 100644index 000000000..32a7fbfcb--- /dev/null+++ b/store/agents/autonomous-grid/skills/engine-ollama/SKILL.md@@ -0,0 +1,131 @@+---+name: engine-ollama+description: "Reuse what Ollama already has on this computer: adopt a running Ollama server into the person's fleet, or serve an Ollama-downloaded model without Ollama. Load before touching Ollama, its models or its settings."+---++# Ollama++Official docs, read 2026-09-29 (source files in github.com/ollama/ollama/tree/main/docs):+[FAQ](https://docs.ollama.com/faq) · [Context length](https://docs.ollama.com/context-length) ·+[OpenAI compatibility](https://docs.ollama.com/api/openai-compatibility) ·+[Thinking](https://docs.ollama.com/capabilities/thinking) · [CLI](https://docs.ollama.com/cli) ·+[macOS](https://docs.ollama.com/macos) · [Linux](https://docs.ollama.com/linux) · [Import](https://docs.ollama.com/import) ·+[API reference](https://github.com/ollama/ollama/blob/main/docs/openapi.yaml)+Tested: Ollama 0.34.4 (`brew install ollama`), MacBook Pro M1 Pro 32 GB, macOS 26.6, 2026-09-29.+Tags: `[doc]` official page above, `[run]` seen on the tested machine, `[?]` unverified.++## When to use it++- **Ollama is already answering** (`fleet models` lists `kind: ollama`): adopt it as it is; it is the+ person's app, so never restart or reconfigure it without asking.+- **Ollama is installed but stopped, or its settings do not fit the job**: do not start it. Its models+ are GGUF files that Grid's engine serves in place — link the blob into `~/.grid/models` and+ `join --serve` [run]. That also gives the context you choose instead of Ollama's default.+- Start Ollama yourself only when Grid's engine refuses a blob Ollama runs (Ollama ships its own engine+ for some new architectures [?]); then use the start below.+- Never `ollama create` to reuse a file from another app: it copies the whole file into Ollama's blobs+ (+624 MB for a 640 MB GGUF) [run]. Never `ollama pull` without the person's go-ahead: it downloads.++## MLX inside Ollama (Apple silicon)++Since 0.19 Ollama has its own MLX engine on Apple silicon, announced as a preview on 2026-03-30+([blog](https://ollama.com/blog/mlx)) [doc]. It runs on MLX only the architectures registered in its+source — read the list live, never from memory:+`https://raw.githubusercontent.com/ollama/ollama/main/mlxrunner/model/architectures/architectures.go`+(one import per architecture folder under `mlxrunner/model/`) [doc]. Compare with `model_type` from+`fleet model-facts`. Everything else keeps running on Ollama's GGML engine. The announced models are+Ollama-library tags in their own quantization, and the post asks for more than 32 GB of unified memory [doc];+importing other MLX models is not yet documented [?].++## Where its models live / how to list them++- Default dir: macOS `~/.ollama/models`, Linux service `/usr/share/ollama/.ollama/models`, Windows+ `C:\Users\%username%\.ollama\models`; `OLLAMA_MODELS` moves it [doc].+- Layout: `manifests/registry.ollama.ai/library/<model>/<tag>` (a JSON file) and `blobs/sha256-<hex>`;+ the manifest layer `application/vnd.ollama.image.model` names the weights blob, a GGUF [run]. Model id is+ `<model>:<tag>`; other namespaces are `<user>/<model>:<tag>` [?].+- `fleet models` already reads the manifests: entries with `source: ollama`, the blob path, and+ `alsoAt` when another app links the same file [run].+- From a running server: `GET /api/tags` (downloaded), `GET /api/ps` (loaded, with `context_length` and+ `size_vram`) [doc], `GET /v1/models` (`owned_by` is the Ollama user, `library` by default) [doc][run].+ `POST /api/show {"model":ID}` lists `capabilities` (completion, tools, thinking, vision) [doc].++## Installed? Running?++- Installed: `command -v ollama` or `/Applications/Ollama.app`. `ollama --version` with no server prints+ "Warning: could not connect to a running Ollama instance" and the client version [run].+- The macOS app starts the server at login (a login item) [doc]; `brew install ollama` starts nothing [run];+ the Linux installer creates a systemd service `ollama` [doc].+- Running: `fleet models` → `engines[]` with `kind: ollama`. Default bind is 127.0.0.1:11434 [doc].++## Start an already-downloaded model (only when Grid's engine refused it)++Port P from outside `machine.listeningPorts`; the port is set only through `OLLAMA_HOST` (no `--port`) [doc]:++ "$GRID_FLEET" serve ollama-P --env OLLAMA_HOST=127.0.0.1:P --env OLLAMA_CONTEXT_LENGTH=65536 -- ollama serve++(`fleet serve` keeps it alive after your shell returns; log `run/ollama-P.log`, PID `run/ollama-P.pid`.)++- Foreground process; the log says `Listening on 127.0.0.1:P (version …)` when ready [run].+- Every other `ollama` command must carry the same `OLLAMA_HOST`, or it talks to 11434 [run].+- Nothing downloads unless you pull; the model loads on its first request [run].+- Context: the default depends on GPU memory — 4K below 24 GiB, 32K at 24–48 GiB, 256K at 48 GiB or+ more [doc] (the FAQ still says 4096 [doc]). Agents need at least 64000 [doc], and every model here is+ used by an agent: always set `OLLAMA_CONTEXT_LENGTH` to 65536 or more. The OpenAI API cannot set+ context per request [doc].++## Ready means++Run `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <model>:<tag> --kind ollama` — it performs+these checks with deadlines and prints each one. What it checks:++1. The PID is alive. 2. `GET /` answers "Ollama is running" [run]. 3. `GET /api/version` → 200 [run].+4. One bounded request through `/v1/chat/completions` with the model id, `max_tokens` 16 and+ `"reasoning_effort":"none"` for a thinking model → non-empty `content` [doc][run].+Then `GET /api/ps` must show the model with `context_length` ≥ 65536 [doc]; less is not started.++## Join Harness Compute++ "$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <model>:<tag> --advertise-as ALIAS --name NAME++`/v1` is required: without it `/models` and `/chat/completions` answer 404, and Grid's capability probe+records JSON output as unsupported without any error [run]. Grid's own detector finds Ollama only on+11434 [run].++## Tool calls, JSON output, thinking++- `/v1/chat/completions` supports `tools`, `response_format` and `reasoning_effort` [doc].+ `reasoning_effort: "none"` asks for no thinking; the native API uses `"think": false` [doc].+- A model's thinking values and default: `/api/show` → `thinking.values`, `thinking.default` [doc].+- Tool support is per model: check `capabilities` contains `tools` before offering it for coding [doc].++## Memory and speed knobs (server environment, apply to all models)++| Variable | Default | Effect |+|---|---|---|+| `OLLAMA_CONTEXT_LENGTH` | by GPU memory (above) | context for every model [doc] |+| `OLLAMA_NUM_PARALLEL` | 1 | requests at once per model; memory scales with parallel × context [doc] |+| `OLLAMA_KV_CACHE_TYPE` | `f16` | `q8_0` ≈ half the KV memory, `q4_0` ≈ a quarter; needs flash attention [doc] |+| `OLLAMA_FLASH_ATTENTION` | automatic | `1` forces on, `0` off [doc] |+| `OLLAMA_KEEP_ALIVE` | 5m | how long an idle model stays loaded [doc] |+| `OLLAMA_MAX_LOADED_MODELS` | 3 × GPUs (3 on CPU) | models loaded at once [doc] |++`brew services start ollama` sets `OLLAMA_FLASH_ATTENTION=1` and `OLLAMA_KV_CACHE_TYPE=q8_0` [run: brew caveat].+`ollama ps` → `PROCESSOR` must read `100% GPU`; a CPU/GPU split is slow [doc].++## Stop++- A server you started: `"$GRID_FLEET" stop ollama-P`.+- Unload one model but keep the server: `OLLAMA_HOST=… ollama stop <model>` [doc].+- The person's app or service: ask first. macOS app: quit from the menu bar; Linux: `sudo systemctl stop ollama` [doc].++## Known failures → what to do++| Sign | Do |+|---|---|+| 404 on `/models` or `/chat/completions` | the URL lacks `/v1` [run] |+| `content` empty, `thinking` full | add `reasoning_effort: "none"` [doc] |+| `/api/ps` context below 65536 | the person's Ollama runs its default (4K or 32K); serve the blob with Grid's engine instead of changing their app |+| 503 "server is overloaded" | queue full (`OLLAMA_MAX_QUEUE`, default 512) [doc]; wait, do not retry in a loop |+| `PROCESSOR` shows CPU share | model plus context does not fit the GPU; smaller context or model [doc] |+| model not found | not downloaded; offer the download as a slow step, never pull silently |diff --git a/store/agents/autonomous-grid/skills/engine-sglang/SKILL.md b/store/agents/autonomous-grid/skills/engine-sglang/SKILL.mdnew file mode 100644index 000000000..e8f42ff19--- /dev/null+++ b/store/agents/autonomous-grid/skills/engine-sglang/SKILL.md@@ -0,0 +1,95 @@+---+name: engine-sglang+description: "Serve a Hugging Face model with SGLang on a Linux machine with an NVIDIA or AMD GPU, configured from the model's SGLang cookbook page — or, when it has none, from the model's own files — and join it to the person's fleet. Load before installing, configuring, starting or stopping SGLang."+---++# SGLang (Linux + NVIDIA CUDA or AMD ROCm)++Official sources, read 2026-09-29. Agent index: [docs.sglang.io/llms.txt](https://docs.sglang.io/llms.txt)+(every page is available as `.md`). [Cookbook](https://docs.sglang.io/cookbook/intro) ·+[Models](https://www.sglang.io/models) · [Quickstart](https://docs.sglang.io/docs/get-started/quickstart.md) ·+[Server arguments](https://docs.sglang.io/docs/advanced_features/server_arguments.md) ·+[Tool parser](https://docs.sglang.io/docs/advanced_features/tool_parser.md) ·+[Separate reasoning](https://docs.sglang.io/docs/advanced_features/separate_reasoning.md) ·+[Native API](https://docs.sglang.io/docs/basic_usage/native_api.md) ·+[Transformers fallback](https://docs.sglang.io/docs/supported-models/transformers_fallback.md) ·+source: [github.com/sgl-project/sglang](https://github.com/sgl-project/sglang)+Tested: not runnable on macOS (below); no GPU run yet. Tags: `[doc]` official source, `[run]` seen on a machine, `[?]` unverified.++## When to use it++- Linux with an NVIDIA GPU of compute capability 8.0 or newer (A10, A100, L4, L40S, H100), Python ≥ 3.10 [doc];+ AMD GPUs, Intel Xeon CPUs and TPUs have their own guides [doc]. Hugging Face models, many requests at once,+ several GPUs.+- **Never on macOS.** The current release ships Linux wheels only and requires CUDA builds of its kernels;+ on a Mac the installer falls back to an old release whose server does not import [run].+- A GGUF file for one person: Grid's own engine serves it with nothing to install.++## 1. The cookbook page — `"$GRID_FLEET" recipe sglang ORG/NAME`++Finds the model family's page through `llms.txt` (the most specific family name the model name starts+with) and prints: `source` (the page), `installation` (its version requirement — some families need a+release or the main branch — and install commands), `serveCommands` (the page's own launch commands per+GPU type), `toolCallParsers`, `reasoningParsers`, and `configurationTips`. Read the page itself for+anything else. Exit code 3 means no page: go to step 2.++## 2. No cookbook page — read the model: `"$GRID_FLEET" model-facts ORG/NAME`++1. `support.sglang.listed: true` means SGLang's model registry has the architecture; `null` means this+ check cannot tell. SGLang can run most decoder models through `--model-impl transformers` [doc] — a test,+ not a promise.+2. `modelCard.serveCommands` and parsers: the model authors' own SGLang command. Use it when present.+3. Otherwise `--tool-call-parser auto` and `--reasoning-parser auto`: SGLang detects both from the model's+ chat template [doc]. Then the tool-call acceptance check decides; if it fails, say so and stop — do not+ cycle through parser names.+4. `contextLength` must be ≥ 65536.++## Install and version++- Slow step, ask first. `uv pip install --prerelease=allow sglang` [doc]; a family whose cookbook page asks+ for the main branch: `uv pip install --prerelease=allow 'git+https://github.com/sgl-project/sglang.git#subdirectory=python'` [doc].+ `OSError: CUDA_HOME environment variable is not set` → `export CUDA_HOME=/usr/local/cuda-<version>` [doc].+- Docker: `lmsysorg/sglang:latest` (NVIDIA), `lmsysorg/sglang-rocm:<tag>` (AMD, tag per GPU generation on+ the cookbook page) [doc], run with `--gpus all --shm-size 32g --ipc=host -p 127.0.0.1:P:30000+ -v ~/.cache/huggingface:/root/.cache/huggingface` [doc].++## Common configs (the cookbook's flags win)++Each adds the parsers from step 1 or 2, `--host 127.0.0.1 --port P` and `--context-length` of at least 65536+(the default is the model's own maximum [doc]).++| Case | Add |+|---|---|+| One GPU, one agent | `--max-running-requests 4` |+| One GPU, many people | leave `--max-running-requests` to SGLang |+| Several GPUs, one machine | `--tp-size <GPU count>` [doc] |+| Out of memory | lower `--mem-fraction-static` (weights plus KV pool share) [doc]; `--kv-cache-dtype fp8_e4m3` [doc] |++## Start, ready, join++ "$GRID_FLEET" serve sglang-P --env HF_HUB_OFFLINE=1 -- ~/.grid/envs/sglang/bin/sglang serve \+ --model-path <model id or snapshot dir> --served-model-name <id> --host 127.0.0.1 --port P <flags>++(`python3 -m sglang.launch_server` takes the same arguments [doc].) Defaults are 127.0.0.1 and port 30000 [doc].+Install into `~/.grid/envs/sglang` so `fleet models` finds it; `fleet serve` keeps it alive after your shell+returns (log `run/sglang-P.log`).++- Verify: `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <id> --kind sglang` (bounded, narrated).+- Ready: the log says `The server is fired up and ready to roll!` [doc]; `GET /health` → 200;+ `GET /health_generate` generates one token [doc]; `GET /v1/models` lists `<id>`; one bounded answer; one tool call.+- Join: `"$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <id> --advertise-as ALIAS --name NAME`. Grid's detector has+ no SGLang probe and would mislabel it on 8000 or 8080 [run], so always `--at`, always with `/v1`.+- Thinking off per request: `"chat_template_kwargs": {"enable_thinking": false}` where the cookbook page shows it [doc].++## Stop++`"$GRID_FLEET" stop sglang-P` (or `docker stop` your container); confirm the port is free.++## Known failures → what to do++| Sign | Do |+|---|---|+| out of memory while serving | lower `--mem-fraction-static` [doc] |+| no `tool_calls` | the cookbook's `--tool-call-parser`, else `auto` [doc]; still none → report |+| a CUDA crash, a multi-GPU hang, a production incident | read SGLang's own maintainer skills: `.agents/skills/debug-cuda-crash`, `debug-distributed-hang`, `sglang-prod-incident-triage` (`SKILL.md` in each) at `https://raw.githubusercontent.com/sgl-project/sglang/main/` |+| 404 on `/models` | the `--at` URL lacks `/v1` |diff --git a/store/agents/autonomous-grid/skills/engine-vllm/SKILL.md b/store/agents/autonomous-grid/skills/engine-vllm/SKILL.mdnew file mode 100644index 000000000..7968bdb49--- /dev/null+++ b/store/agents/autonomous-grid/skills/engine-vllm/SKILL.md@@ -0,0 +1,117 @@+---+name: engine-vllm+description: "Serve a Hugging Face model with vLLM on a Linux machine with an NVIDIA or AMD GPU, configured from the model's official vLLM recipe — or, when it has none, from the model's own files — and join it to the person's fleet. Load before installing, configuring, starting or stopping vLLM."+---++# vLLM (Linux + NVIDIA CUDA or AMD ROCm)++Official sources, read 2026-09-29. Agent index: [recipes.vllm.ai/llms.txt](https://recipes.vllm.ai/llms.txt).+Reference docs as raw markdown under `https://raw.githubusercontent.com/vllm-project/vllm/main/docs/`:+[OpenAI-compatible server](https://docs.vllm.ai/en/latest/serving/online_serving/openai_compatible_server/) (`serving/online_serving/openai_compatible_server.md`) ·+[Tool calling](https://docs.vllm.ai/en/latest/features/tool_calling/) (`features/tool_calling.md`) ·+[Reasoning outputs](https://docs.vllm.ai/en/latest/features/reasoning_outputs/) (`features/reasoning_outputs.md`) ·+[Supported models](https://docs.vllm.ai/en/latest/models/supported_models/) (`models/supported_models.md`) ·+[GPU install](https://docs.vllm.ai/en/latest/getting_started/installation/gpu/) · [Docker](https://docs.vllm.ai/en/latest/deployment/docker/) ·+[Optimization](https://docs.vllm.ai/en/latest/configuration/optimization/) · [Conserving memory](https://docs.vllm.ai/en/latest/configuration/conserving_memory/)+Tested: only vLLM 0.11.0 on an Apple-silicon Mac (CPU backend), 2026-09-29 — no GPU run yet.+Tags: `[doc]` official source above, `[run]` seen on the tested machine, `[?]` unverified.++## When to use it++- Linux with an NVIDIA GPU of compute capability 7.5 or newer [doc], or an AMD GPU with ROCm [doc], and a+ Hugging Face model (safetensors, FP8, AWQ…). vLLM is built for many requests at once and for models split+ across several GPUs.+- A GGUF file for one person on one GPU: Grid's own engine serves it with nothing to install.+- **Not on a Mac.** There it runs on the CPU only, slowly (26 tokens in 2.3 s for a 0.5B model), and needed+ pinned `transformers<5`, `--dtype float32` for JSON output and a batched-token limit to start [run].++## 1. The recipe — `"$GRID_FLEET" recipe vllm ORG/NAME`++Reads [recipes.vllm.ai](https://recipes.vllm.ai/) live (`models.json`, then `<ORG>/<NAME>.json`) and prints:++| Field | Use |+|---|---|+| `minVersion` | the installed `vllm --version` must be at least this |+| `baseArgs`, `baseEnv` | always |+| `features.tool_calling.args`, `features.reasoning.args` | always for an agent |+| `features.*` with `optIn: true` | only on purpose (faster decoding, text only to free memory, longer context) |+| `variants.<name>` | pick by `vramMinimumGb` against the GPU; a variant can carry its own `modelId` and `extraArgs` |+| `recommended` | the site's reference command and Docker image for its reference hardware |+| `install` | the recipe's own install steps, including any pinned extras |+| `guide` | prose; its **Troubleshooting** holds model-specific fixes |++The recipe beats the generic docs: a family's generic parser entry and a model's recipe can name different+parsers [doc]. Exit code 3 means there is no recipe: go to step 2.++## 2. No recipe — read the model: `"$GRID_FLEET" model-facts ORG/NAME`++1. `support.vllm.listed` must be `true` (its architecture is in vLLM's supported-models list). `false`:+ the Transformers backend (`--model-impl transformers`) may still run it [doc] — a test, not a promise.+2. `modelCard.serveCommands` / `toolCallParsers` / `reasoningParsers`: the model authors' own vLLM command.+ Use it when present.+3. Otherwise match `chatTemplate.toolSyntax` against the parser descriptions in `features/tool_calling.md`+ (each parser documents the output format it reads) and, when `chatTemplate.thinking`, pick the reasoning+ parser from the table in `features/reasoning_outputs.md` [doc]. Quote the doc line you matched.+4. `contextLength` must be ≥ 65536. Sampling defaults come from the model's `generation_config.json`+ automatically [doc]; leave them.+5. No match in the docs: say so and stop — do not cycle through parsers until one passes.++## Install and version++- Slow step, ask first. Venv: `uv venv && uv pip install -U vllm --torch-backend=auto` [doc]; pin with+ `"vllm==X.Y.Z"` at or above the recipe's `minVersion`. Wheels target CUDA 12.9 by default, builds for 12.8+ and 13.0 exist, Blackwell needs CUDA ≥ 12.8 [doc]. AMD: `--extra-index-url https://wheels.vllm.ai/rocm`+ (Python 3.12, ROCm 7.0, glibc ≥ 2.35) [doc]. Check the GPU and driver with `nvidia-smi`.+- Docker: `vllm/vllm-openai` (NVIDIA), `vllm/vllm-openai-rocm` (AMD) [doc], with+ `--gpus all --ipc=host -p 127.0.0.1:P:8000 -v ~/.cache/huggingface:/root/.cache/huggingface` [doc].++## Common configs (the recipe's flags win)++Each adds the recipe's (or step 2's) tool and reasoning flags, `--host 127.0.0.1 --port P` and a+`--max-model-len` of at least 65536.++| Case | Add |+|---|---|+| One GPU, one agent | `--max-model-len 131072 --max-num-seqs 4 --enable-prefix-caching` |+| One GPU, many people | `--max-model-len 65536 --enable-prefix-caching` (`max-num-seqs` left to vLLM) |+| Several GPUs, one machine | `--tensor-parallel-size <GPU count>` [doc] |+| Memory is tight | the recipe's FP8 variant, `--kv-cache-dtype fp8`, fewer `--max-num-seqs` [doc] |++## Start, ready, join++ "$GRID_FLEET" serve vllm-P --env HF_HUB_OFFLINE=1 -- ~/.grid/envs/vllm/bin/vllm serve <model id or snapshot dir> \+ --served-model-name <id> <flags>++(Install into `~/.grid/envs/vllm` so `fleet models` finds it; `fleet serve` keeps it alive after your shell+returns; log `run/vllm-P.log`.)++- Verify: `"$GRID_FLEET" verify --at http://127.0.0.1:P/v1 --model <id> --kind vllm` (bounded, narrated).+- Ready: PID alive; log shows `Application startup complete.` [run] (minutes for a big model); `GET /health`+ → 200 and `GET /v1/models` lists `<id>` [run]; one bounded answer; one tool call.+- Join: `"$GRID_FLEET" run -- join GRID --at http://127.0.0.1:P/v1 -m <id> --advertise-as ALIAS --name NAME`. `/v1` is required —+ without it `/models` and `/chat/completions` answer 404 [run]. `--api-key` guards only `/v1` routes [doc].+- Thinking off for everyday use: the recipe's guide names the flag when there is one+ (`--default-chat-template-kwargs '{"enable_thinking": false}'` in the recipes that document it [doc]).++## Knobs++`--gpu-memory-utilization` (share of GPU memory pre-allocated for weights plus cache), `--max-model-len`,+`--max-num-seqs`, `--max-num-batched-tokens`, `--kv-cache-dtype fp8`, `--enforce-eager` (fastest start,+slower serving), `--tensor-parallel-size` [doc].++## Stop++`"$GRID_FLEET" stop vllm-P` (or `docker stop` the container you started); confirm the port is free.++## Known failures → what to do++| Sign | Do |+|---|---|+| "preempted … not enough KV cache space" | raise `--gpu-memory-utilization`, lower `--max-num-seqs`, or more tensor parallel [doc] |+| CUDA out of memory at start | FP8 variant, `--kv-cache-dtype fp8`, fewer sequences — context stays ≥ 65536 |+| "max_num_batched_tokens … smaller than max_model_len" | raise `--max-num-batched-tokens` to at least the context [doc][run] |+| an error the recipe's guide names | apply the guide's fix exactly [doc] |+| tokenizer `AttributeError` after install | transformers newer than that vLLM supports (0.11.0 needed `transformers<5` [run]); use a vLLM at or above the recipe minimum |+| no `tool_calls` | `--enable-auto-tool-choice` plus the recipe's parser [doc][run] |+| 404 on `/models` | the `--at` URL lacks `/v1` [run] |+| "CUDA error: an illegal memory access" | read vLLM's own maintainer skill `.agents/skills/debug-ima/SKILL.md` at `https://raw.githubusercontent.com/vllm-project/vllm/main/` |diff --git a/store/agents/autonomous-grid/skills/grid-operations/SKILL.md b/store/agents/autonomous-grid/skills/grid-operations/SKILL.mdindex 7a45c7e40..cdb86692f 100644--- a/store/agents/autonomous-grid/skills/grid-operations/SKILL.md+++ b/store/agents/autonomous-grid/skills/grid-operations/SKILL.md@@ -40,7 +40,8 @@ company grid") goes to that row's exact `grid` name; a request that names none g(also `$HARNESS_PRIVATE_GRID`). Harness put it there from the account that is signed in, so it isthe answer, not a guess: pass that exact name to `join`, `leave`, `engines` and `models`, and do notask the person which grid is theirs. It can differ from `grid` (the workspace's selected fleet) —-a request that says "my grid" goes to `personalGrid`, whatever is selected. The `type` column of+a request that says "my grid" goes to `personalGrid`, whatever is selected, and so does a request that+names no grid at all ("run a model on my Mac"). The `type` column of`ls --json` is what kind of grid a row is, not a permission to ask about: `permissioned-public` isa person's own grid, `private-domain` a company's, `domain-restricted` a team's, `os-community` apublic one. Only when `personalGrid` is null: say in one line that Harness has not named this@@ -88,9 +89,22 @@ success, say **"Your model is running. Select it from the model picker in a sessSlow steps are a real stop: a download or an engine build is asked about through the question tooland runs in the **next** turn, never in the message that asks. Every model you offer comes from-something you looked up — the host's disk, the catalog, or Hugging Face — never from memory. The-pick is made once; that is the model through the download and the start, unless it fails (won't-fit, won't pull, won't answer) or the person asks for a different one.+something you looked up — the grid, a machine's disk (step 0), the catalog, or Hugging Face — never+from memory. The pick is made once; that is the model through the download and the start, unless it+fails (won't fit, won't pull, won't answer) or the person asks for a different one.++**0. Scan first — what the grid serves, and what every machine already holds.**++- *The grid:* `"$GRID_FLEET" run -- models GRID --json` for the grid it will go on. A model already+ served there needs no start at all — say "use it from the model picker now".+- *This computer:* `skills/run-local-model/SKILL.md` — it takes stock (`fleet models`), sizes, picks+ file and engine, starts and verifies. This skill adds only the grid parts below.+- *An SSH machine in `grid-fleet.json`:* `"$GRID_FLEET" models --machine M --summary` runs the same+ finder on that machine with its own Node — its disk, engines, ports and memory [run]. Without Node+ 18+ there it prints a file list only and says so.+- *A Harness-linked machine:* its disk cannot be listed from here yet. Read its hardware+ (`run --machine M -- device-info --json`) and say in one line that its downloaded files are not+ visible, so a model for it comes from the catalog.**1. What they need — decided for them, not asked.** Harness is a coding tool: the model is for acoding agent unless the person says otherwise. Do **not** ask them to choose an engine,@@ -116,18 +130,16 @@ before the start say "the model takes W GB, and its context uses the rest of thecan give models" (`usable_bytes` from `device-info`); after the start, give Grid's measured figurewhen `engines --json` reports one. Never invent a number.-**2. What the host has.** `device-info --json` on the intended host first: `usable_bytes` is the-real ceiling for weights plus context. Per-machine, never summed across hosts. The engine check is-the binary, not a status command: `~/.grid/bin/llama-server --version` printing a version line means-llama.cpp is installed — go on without a word. ⚠️ `engine status` reports the **media** engine-(ComfyUI) and says `Installed: no` on a host whose llama.cpp is fine; it sent an agent asking to-install what was already built. Only when the binary is missing, ask (build from source or prebuilt)-and run `engine install llama.cpp [--from-source]` in the next turn.+**2. Another machine's hardware.** `run --machine M -- device-info --json`: `usable_bytes` is its+ceiling, per machine, never summed. Its llama.cpp is installed when `~/.grid/bin/llama-server --version`+prints a version. ⚠️ `engine status` reports the **media** engine (ComfyUI) and says `Installed: no`+on a host whose llama.cpp is fine. Only when the binary is missing, ask and run+`engine install llama.cpp [--from-source]` in the next turn.-**3. What is already on the host's disk comes first.** `ls ~/.grid/models/*.gguf` minus the-`.mmproj.gguf` sidecars; `ctx FILE --json` says the most each was trained to hold — a file under-64K is skipped; a `<stem>.mmproj.gguf` beside a file means it reads images. A suitable file already-here is the pick ("already on this computer, no download"); fetch something new only when none is.+**3. What is already on disk comes first.** Step 0's table lists every model file on that machine,+from every app, with its context and whether it reads images — a file under 64K is skipped. A+suitable file already there is the pick ("already on this computer, no download"); fetch something+new only when none is.**4. The catalog, then Hugging Face.** `catalog --json` is sized for the host: keep entries that are`runnable`, whose `fit.ctx` is at least 65536, and that are good at code; pull `fit.version`'s@@ -135,6 +147,7 @@ here is the pick ("already on this computer, no download"); fetch something newmuch it can hold in pages and whether it reads images; offer a shortlist of 2–3 only when the personasks for other options. No speed figure (`fit.est_tok_s` only orders the list for "fast"), no quantname, no token count as the whole answer.+When the person names a model the catalog lacks, the catalog is not a wall — `pull` takes any`<repo>:<file>.gguf` on Hugging Face and fetches its projector too:@@ -145,7 +158,7 @@ When the person names a model the catalog lacks, the catalog is not a wall — `for f in json.load(sys.stdin)["siblings"]:if f["rfilename"].endswith(".gguf"): print(f["rfilename"], f.get("size"))'-Prefer an official org (`ggml-org`, `Qwen`, `google`, `unsloth`, `bartowski`, `lmstudio-community`)+Prefer the model's own publisher, or a quantizer that publisher's model card or the catalog names,over an unknown uploader, then downloads; say which in half a line. Pick the quant by size against`usable_bytes` (or ≈0.6 bytes per parameter for Q4_K_M), leaving room for context. A model outsidethe catalog has been sized by nobody but you — say the fit is your estimate.@@ -169,36 +182,44 @@ fails, `leave`, `sync`, and join again at half — 256K → 128K → 64K. If 64Kthis computer does not have the memory for this model with a 64K context; offer a smaller model. *Vision:* on by default when the projector ison disk. When the person asks for text only, rename the projector yourself(`mv <stem>.mmproj.gguf <stem>.mmproj.gguf.off`) before the join and say so in one line.+Sizing (weights + context × cache per token within free memory) is `run-local-model` §3."$GRID_FLEET" run --machine MACHINE -- pull OWNER/REPO:EXACT_FILE.gguf"$GRID_FLEET" run --machine MACHINE -- join GRID --serve EXACT_FILE.gguf --advertise-as MODEL_ALIAS \--max-concurrency N --ctx-size CTX --endpoint-port PORT+A GGUF **already on that machine** in another folder (step 0 found it) is linked, never pulled again:+`"$GRID_FLEET" link --machine MACHINE FILE NAME.gguf [--projector MMPROJ_FILE]`, then `join … --serve+NAME.gguf` — FILE is the full path `fleet models --machine MACHINE` (JSON, `path`) printed. `link`+says `NOT READY` when that machine's Grid engine is missing or does not start: ask, then `run+--machine MACHINE -- engine install llama.cpp` in the next turn.+`--advertise-as` is the name the person will see in their model picker. **Don't pass `--name`:**the runner sets it on every `join` to the machine's name as Harness Machines shows it right now, andreplaces any you give — it is the label under the model in every picker, and a name made up here-(`macbookpro-qwen3.6-35b`) or Grid's host-name default (`mac.lan`) read as a different computer.+(a machine name joined to a model name) or Grid's host-name default read as a different computer.Say the machine by that same name. `--max-concurrency N` is 1 unless they asked for more; don't pass`--parallel` (grid derives the slot count from it) and don't pass `--jinja` (on by default in theengine grid ships). Never pin `--ctx-size` under 65536 — a window that small cannot hold a codingagent's own prompt (a 32K engine here refused an agent's first request of 59,561 tokens). Useexplicit ports when several instances share a host. An existing Ollama, vLLM, MLX or LM Studio-engine can join with `--at URL -m MODEL` (the runner names it too); do not install a second engine needlessly.+engine joins with `--at http://HOST:PORT/v1 -m MODEL --advertise-as ALIAS` (the runner names it too) — the+`/v1` is required, `fleet models` prints the exact URL, and the matching `skills/engine-*/SKILL.md` has+its start and ready checks; do not install a second engine needlessly.-Choose a reasoning budget deliberately. Grid's GPU default can spend more tokens thinking than a-small output limit permits, yielding no final answer. For an everyday low-latency assistant, start-with `fleet run --thinking off -- join ... --reasoning-budget 0`. `--thinking off` sets llama.cpp's-`enable_thinking:false` template parameter for the newly started engine; it is needed on builds-where a zero token budget alone still produces reasoning. Use `--thinking on` to enable a supported-model's thinking explicitly. These switches configure startup, not an already running instance.-For a reasoning model, reserve an explicit budget smaller than `--n-predict`, leaving room for the-answer.+Thinking: for everyday use `fleet run --thinking off -- join … --reasoning-budget 0`. `--thinking off`+sets `enable_thinking:false` at startup — needed, because a zero budget alone still let a model spend+every token thinking [run]. `--thinking on` enables it; either applies only to a new start.**6. Prove it answers — once, bounded.** A successful `join` means *starting*, not ready. First waitfor the relay to list it (a call before that answers `No providers available for this model`, whichis "not yet", not "broken"):- until "$GRID_FLEET" run -- models GRID 2>/dev/null | grep -qx 'MODEL_ALIAS'; do sleep 10; done+ # engine on this computer: its own checks first, then the relay+ "$GRID_FLEET" verify --at http://127.0.0.1:PORT/v1 --model MODEL_ALIAS --kind llama.cpp --grid GRID --alias MODEL_ALIAS+ # engine on another machine: the relay only+ "$GRID_FLEET" verify --grid GRID --alias MODEL_ALIAS+ # once it passes: the receipt the Models panel shows"$GRID_FLEET" verify --grid GRID --model MODEL_ALIASA `verify` that fails with a compute or out-of-memory error is the context not fitting — step down@@ -207,19 +228,15 @@ as step 5 says rather than retrying the same size. Then read the window it actua65536, `leave` it, say in one line it could only get N pages here, and move to a smaller model orquant. A null means Grid did not say; report the context you passed.-⚠️ Not `chat` for this check. `chat` sets no output limit and the engine's default is tens of-thousands of tokens, so a small model that runs away answering "ok" holds the slot for minutes — and-with one slot everything after it waits, including a second check. `max_tokens` is what makes this-finish in seconds whatever the model does. The timeout is long on purpose: right after a join, grid-sends the new engine a probe of about 5K tokens to measure what it can do, and on a host without a-GPU that alone takes 3–5 minutes at ~30 tokens/s while holding the single slot. Tell the user the-model is warming up and the first answer can take a few minutes on that machine. Send the check-ONCE and leave it alone — a second call, a `chat`, a log tail all queue behind the same probe. A-reply with a `choices` entry means the whole path works. Nothing within the timeout: stop the model-(`leave`), say in one line it started but did not answer in time, and offer through a tool a model-one step larger from the same list (tiny models loop), or more requests at once, or stop. Verify-`message.content` contains the requested result; reasoning text alone or a successful HTTP status is-not acceptance. Then `"$GRID_FLEET" refresh`.+Every step prints its elapsed time and every wait has a deadline (listing: every 10 s up to 300 s;+the relay answer: up to 420 s with a "still waiting" line every 15 s) — never a bare `until … sleep`+loop, which waits forever on an engine that never registers. Relay the progress lines to the person.++Never `chat` for this check: it has no output limit, and a small model that runs away holds the only+slot for minutes. Right after a join, grid probes the new engine with ~5K tokens (3–5 minutes on a+host without a GPU): tell the person it is warming up, run `verify` once, and leave it alone. It fails+within its deadline: `leave`, say so in one line, and offer the next model up, or stop. Then+`"$GRID_FLEET" refresh`.**7. Say where it is, and what it costs.** "<alias> is running on <machine> with room for aboutN pages (<K>K tokens), one at a time, vision on/off; it uses about M GB of memory. Pick it from the@@ -233,7 +250,7 @@ The person names the one thing they want different; everything else stays. Readsettings from the viewer or `stats GRID --verbose --json` (model, context, slots) so you change onlythat and can say what changed. Context, concurrency and vision all mean a restart — `leave` thatinstance, then the same `join` with the one flag (or the projector file) changed; check the fit-first as in step 5. A different model is step 3 onward, keeping the context and concurrency they+first as in step 5. A different model is step 4 onward, keeping the context and concurrency theyalready have; stop the old one only when the new one is about to serve. Stop is `leave` and oneline. A restart drops the model from the picker for the seconds it takes and any agent mid-turn onit loses that turn: say so in one line, and ask first only when `stats` shows requests in flight.diff --git a/store/agents/autonomous-grid/skills/grid-operations/references/commands.md b/store/agents/autonomous-grid/skills/grid-operations/references/commands.mdindex b46c804e6..42c962d1a 100644--- a/store/agents/autonomous-grid/skills/grid-operations/references/commands.md+++ b/store/agents/autonomous-grid/skills/grid-operations/references/commands.md@@ -8,10 +8,12 @@ https://github.com/autonomous-ai/autonomous-grid/blob/main/docs/cli.md.|---|---|| Lifecycle and selection | `start`, `stop`, `delete`, `ls`, `info`, `mode`, `use` — `use` is the one source of truth for which grid; the viewer follows it, and `connect` sets it (SKILL.md, "Targets and access"). Change grids through `connect`, which verifies first || Host inventory | `device-info --json` |-| Model discovery and weights | `~/.grid/models/*.gguf` on disk first, then `catalog --json`, then Hugging Face; `pull REPO:FILE`, `ctx FILE --json`, `rm FILE` |+| Model discovery and weights | this computer first: `"$GRID_FLEET" models` (files from every app, engines answering, memory and swap now; not a Grid command, no `run --`); an SSH machine: `"$GRID_FLEET" models --machine M` (a Harness-linked one cannot be listed); then `catalog --json`, then Hugging Face; `pull REPO:FILE`, `ctx FILE --json`, `rm FILE` |+| Engine configuration from official sources | `"$GRID_FLEET" recipe vllm|sglang ORG/NAME` (exit 3 = no recipe), `"$GRID_FLEET" model-facts ORG/NAME|DIR` (what the model says about itself); see `skills/run-local-model/SKILL.md` || Engine provisioning | `engine install llama.cpp [--from-source]`; the install check is `~/.grid/bin/llama-server --version` — `engine status`/`start`/`stop` are the media engine (ComfyUI) || Serving and removal | `join`, `leave --engine SELECTOR`, `engines --json`, `models --json` |-| Inference checks | a bounded relay call with `max_tokens` (SKILL.md step 6), never `chat` for verification (no output limit); `image`, `edit`, `video`, `stt` for media |+| Inference checks | `"$GRID_FLEET" verify` (SKILL.md step 6: engine and relay, every step bounded and narrated), never `chat` for verification (no output limit); `image`, `edit`, `video`, `stt` for media |+| Models for an installed engine with none | `"$GRID_FLEET" candidates mlx` (Mac), recipes via `"$GRID_FLEET" recipe` (GPU), `catalog --json` (Grid's engine) || Remote telemetry | `stats --verbose --json`, `usage --by model|member|engine --json` || Remote request routing | `router --help` (inspect before changing policies or advisors) || Training and evaluation | `train --help`, `train doctor`, `train packs`, `train eval`, `train deploy` |diff --git a/store/agents/autonomous-grid/skills/run-local-model/SKILL.md b/store/agents/autonomous-grid/skills/run-local-model/SKILL.mdnew file mode 100644index 000000000..50c4bbb5b--- /dev/null+++ b/store/agents/autonomous-grid/skills/run-local-model/SKILL.md@@ -0,0 +1,113 @@+---+name: run-local-model+description: "Details behind AGENTS.md's 'Starting a model on this computer' flow: reading `fleet models`, sizing a start, choosing file and engine, picking a model for an engine with none, what `fleet verify` checks, and where to read when unsure. Open it when a step of that flow needs more than the flow says."+---++# Run a local model — details++The flow itself is in AGENTS.md. Tags: `[run]` seen on a real machine (M1 Pro 32 GB, macOS 26.6,+2026-09-29), `[doc]` official docs (see the `engine-*` skill), `[code]` read in the tool's source.++## Purpose → settings (the only thing asked)++*Coding · Chat and writing · Reading images · Just something fast.* **Context is never below 64K**:+every model here is used by an agent whose own prompt fills a small window; Ollama's docs set the same+floor for agents [doc]. When 64K does not fit, take a smaller model, never a smaller window.++| Purpose | Context | At once | Also needs |+|---|---|---|---|+| Coding | 128K when it fits, else 64K | 1 | tool calls |+| Chat and writing, or fast | 64K | 1 | — |+| Reading images | 64K | 1 | a projector beside the file |++Thinking is off for everyday use.++## Reading `fleet models`++- `machine.accelerators[]`: a GPU counts only with `active: true` (its own tool answered). `active: false`+ comes with an `error` (e.g. no driver): say it in one line; plan no GPU engine on it.+- `machine.engines[]`: installed engines, running or not. `machine.canRun`: the kinds this hardware can+ run at all — never propose one outside it.+- `machine.memory.availableBytes`, `swapUsedBytes`: room right now. Metal and `device-info` do not see+ other apps (Metal said 25 GiB free while macOS was 11 GB into swap [run]).+- `machine.accelerators[].totalBytes`: the GPU ceiling, below total RAM on a Mac — read it here, never assume it; `device-info` reports a different figure+ there [run] — use the smaller.+- `engines[]`: answering engines and their exact `--at` URL. `openai-compatible` whose "models" are not+ models is another app: leave it alone.+- `models[]`: one entry per real file (`alsoAt` = other apps holding the same file), with `format`,+ `bytes`, `projector`, and for GGUF `contextLength`, `kvBytesPerToken`, `toolCalls`, `unsupportedTensorTypes`.++## Size++ need = weights + context × kvBytesPerToken × slots + 0.5 GB (+ projector when vision is on)++`kvBytesPerToken` matched what llama.cpp allocated exactly [run]; files of similar size needed from+20 KiB to 160 KiB per token, so never skip this. Null (latent attention): start at 64K and read the+engine's memory report. Fits when `need ≤ availableBytes + 3 GB` and ≤ the GPU ceiling (macOS moves+idle pages to swap once; swap still climbing a minute after start means too big). Too big: a smaller+file, or ask the one trade-off — "close other apps first, or a lighter model beside your work?".++## Choose the file, then the engine++Drop: `unsupportedTensorTypes` not empty (llama.cpp refuses them [run]); context below 64K; no tool+calls for coding; no projector for images; anything that does not fit. Prefer newer families, more+parameters at 4-bit over fewer at 8-bit, and mixture-of-experts models when memory allows.++| The file | Mac (Apple silicon) | Linux + active NVIDIA/AMD GPU | CPU only |+|---|---|---|---|+| served by an engine already answering | `join --at URL/v1 -m ID --advertise-as ALIAS` | same | same |+| GGUF from any app | Grid's engine (link into `~/.grid/models`) | same | same |+| MLX folder | mlx-lm (`engine-mlx-lm`) | — | — |+| Hugging Face safetensors | mlx-lm | vLLM or SGLang from the model's recipe | find a GGUF |++Grid only serves from `~/.grid/models`: its launcher keeps just the file name of `--serve` and looks+there; a projector must sit beside it [code: grid `shared/engine/launcher.py`]. A symlink costs no disk.+Never `ollama create` to reuse a file (it copies it [run]); never vLLM or SGLang on a Mac (CPU-only /+no macOS build [run]); never download a second copy only to switch engines. On Apple silicon, when a+*new* download is needed and mlx-lm or LM Studio is installed, an MLX build is sound — Ollama itself+moved its Apple engine to MLX [doc: ollama.com/blog/mlx].++On Apple silicon with mlx-lm (or LM Studio) installed, **MLX first**: when the same model exists as an+MLX folder and as a GGUF and the MLX one fits, serve the MLX one. The choice is still model-first —+never a bigger MLX model that does not fit over a smaller GGUF that does. The report names what was+passed over and why, one line each ("<model> (<format>): needs <N> GB, <M> GB free"), so "why not MLX?"+is answered before anyone asks.++## An engine installed, nothing to serve++Offer 2–3 that fit (plain name, GB, what it is good at) plus "none of these"; the download waits for+the go-ahead.+- Mac with mlx-lm or LM Studio: `"$GRID_FLEET" candidates mlx [--search WORDS] [--sort downloads|trending|recent]`+ — mlx-community models sized from their real files, cache at 64K, `fits` against the GPU ceiling.+- Linux GPU with vLLM or SGLang: pick families from [recipes.vllm.ai/llms.txt](https://recipes.vllm.ai/llms.txt)+ or the SGLang cookbook, then `fleet recipe` each; `vramMinimumGb` against the GPU decides.+- Grid's engine or Ollama with nothing: Grid's catalog (`grid-operations` step 4), pulled into Grid.+- The person names an engine outside `canRun`: say why in one line (no active GPU, not a Mac).++## This computer already on that grid++In remote mode a computer joins a grid as one identity, and Grid's `--serve` engine cannot share it: a+second model is refused with "can't join a multi-engine identity. Run `grid leave`, then re-join every+engine as external `--at <url> -m <model>`" [run]. Ask *Replace X with Y · Keep X*; never leave an+engine the person did not agree to.++## What `fleet verify` checks++Engine: **ready** (`/models` within 180 s, every 3 s), **listed**, **answer** (max_tokens 16, thinking off;+reasoning-only output fails), **tool call** (`read_file` must come back as `tool_calls`), **speed**. With+`--grid`: **relay listed** (every 10 s up to 300 s) and **relay answer** (up to 420 s, a "still waiting"+line every 15 s). Exit 0 only when all pass. Seen end to end: an mlx-lm engine joined a local grid with+`--at …/v1` in 6 s and passed all seven in 10 s; a Grid-engine GGUF passed at 19 tok/s [run].++## When unsure: read, never guess++1. `"$GRID_FLEET" recipe vllm|sglang ORG/NAME` — the official recipe for that exact model (exit 3: none).+2. `"$GRID_FLEET" model-facts ORG/NAME|DIR` — architecture, context, sampling defaults, the template's+ tool-call syntax, the authors' serve commands, and whether each engine lists the architecture+ (`listed: null` = cannot tell).+3. Agent indexes: [docs.ollama.com/llms.txt](https://docs.ollama.com/llms.txt),+ [lmstudio.ai/llms.txt](https://lmstudio.ai/llms.txt), [recipes.vllm.ai/llms.txt](https://recipes.vllm.ai/llms.txt),+ [docs.sglang.io/llms.txt](https://docs.sglang.io/llms.txt); vLLM and mlx-lm docs as raw markdown on GitHub.+4. Nothing says it: tell the person this model cannot be set up reliably here and offer the next one.++`fleet models` reads this computer only; a Harness-linked machine's disk is not visible yet.diff --git a/store/agents/autonomous-grid/test/candidates.test.mjs b/store/agents/autonomous-grid/test/candidates.test.mjsnew file mode 100644index 000000000..b1f8c20fe--- /dev/null+++ b/store/agents/autonomous-grid/test/candidates.test.mjs@@ -0,0 +1,54 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { estimateBytes, kvBytesFromConfig, mlxCandidates } from '../lib/candidates.mjs';+import { parseNvidiaSmi, runnableEngines } from '../lib/inventory.mjs';++test('the filter estimate reads the Hub\'s unpacked parameter count at the repo\'s bit width', () => {+ // A 4B model at 4 bits: the Hub reports ~4.0e9 "U32"; its files are 2.26 GB.+ assert.equal(estimateBytes({ total: 4_022_468_096 }, 4), Math.round(4_022_468_096 * 0.5 * 1.15));+ assert.equal(estimateBytes(undefined, 4), 0);+});++test('cache per token counts only the layers that keep one', () => {+ const dense = { num_hidden_layers: 28, num_attention_heads: 16, num_key_value_heads: 8, head_dim: 128 };+ assert.equal(kvBytesFromConfig(dense), 28 * 8 * 128 * 4);+ assert.equal(kvBytesFromConfig({ ...dense, full_attention_interval: 4 }), 7 * 8 * 128 * 4);+ assert.equal(kvBytesFromConfig({ text_config: { ...dense, layer_types: ['linear_attention', 'full_attention', 'linear_attention', 'full_attention'] } }), 2 * 8 * 128 * 4);+ assert.equal(kvBytesFromConfig({ num_hidden_layers: 4 }), null);+});++test('MLX candidates are chat models that fit the budget at the asked context, sized without downloading', async () => {+ const GB = 1024 ** 3;+ const rows = [+ { id: 'mlx-community/Small-4bit', pipeline_tag: 'text-generation', downloads: 900, gated: false, config: { quantization_config: { bits: 4 }, architectures: ['SmallForCausalLM'], tokenizer_config: { chat_template: '{% if tools %}{% endif %}' } }, safetensors: { total: 4 * GB } },+ { id: 'mlx-community/Huge-4bit', pipeline_tag: 'text-generation', downloads: 800, gated: false, config: { quantization_config: { bits: 4 } }, safetensors: { total: 60 * GB } },+ { id: 'mlx-community/Speech', pipeline_tag: 'automatic-speech-recognition', downloads: 5000, gated: false, config: { quantization_config: { bits: 4 } }, safetensors: { total: GB } },+ { id: 'mlx-community/Locked-4bit', pipeline_tag: 'text-generation', downloads: 700, gated: 'manual', config: { quantization_config: { bits: 4 } }, safetensors: { total: GB } },+ ];+ const fetchImpl = async url => {+ if (url.startsWith('https://huggingface.co/api/models?')) {+ assert.match(url, /author=mlx-community/);+ return new Response(JSON.stringify(rows));+ }+ if (url === 'https://huggingface.co/api/models/mlx-community/Small-4bit?blobs=true') {+ return new Response(JSON.stringify({ siblings: [{ rfilename: 'model.safetensors', size: 2.26 * GB }, { rfilename: 'tokenizer.json', size: 11e6 }] }));+ }+ if (url.endsWith('/Small-4bit/raw/main/config.json')) return new Response(JSON.stringify({ num_hidden_layers: 24, num_attention_heads: 16, num_key_value_heads: 4, head_dim: 128, max_position_embeddings: 131072 }));+ return new Response('{}', { status: 404 });+ };+ const found = await mlxCandidates({ budgetBytes: 20 * GB, fetchImpl });+ assert.deepEqual(found.candidates.map(c => c.id), ['mlx-community/Small-4bit']);+ const [small] = found.candidates;+ assert.equal(small.bytes, 2.26 * GB, 'the real file size, not the parameter count');+ assert.equal(small.kvBytesPerToken, 24 * 4 * 128 * 4);+ assert.equal(small.fits, true);+ assert.equal(small.toolCalls, true);+});++test('a GPU counts only when its own tool answers; the engines a machine can run follow from it', () => {+ const [gpu] = parseNvidiaSmi('0, NVIDIA RTX 4090, 24564, 1200, 3, 580.1\n');+ assert.deepEqual([gpu.name, gpu.totalBytes, gpu.active, gpu.driver], ['NVIDIA RTX 4090', 24564 * 1024 ** 2, true, '580.1']);+ assert.deepEqual(runnableEngines('darwin', 'arm64', []), ['llama.cpp', 'mlx-lm', 'ollama', 'lm-studio']);+ assert.deepEqual(runnableEngines('linux', 'x64', [gpu]), ['llama.cpp', 'ollama', 'vllm', 'sglang']);+ assert.deepEqual(runnableEngines('linux', 'x64', [{ vendor: 'nvidia', active: false }]), ['llama.cpp', 'ollama']);+});diff --git a/store/agents/autonomous-grid/test/inventory.test.mjs b/store/agents/autonomous-grid/test/inventory.test.mjsnew file mode 100644index 000000000..457d018f8--- /dev/null+++ b/store/agents/autonomous-grid/test/inventory.test.mjs@@ -0,0 +1,213 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { createServer } from 'node:http';+import { mkdir, mkdtemp, symlink, writeFile } from 'node:fs/promises';+import { tmpdir } from 'node:os';+import { dirname, join } from 'node:path';+import {+ defaultRoots, joinedGrids, modelFamily, parseListDevices, parseListeningPorts, parseMeminfo, parseNetstatListening,+ parseProcNetTcp, parseSwapUsage, parseVmStat,+ probeEngines, readGguf, scanModels, summarize, summarizeGguf,+} from '../lib/inventory.mjs';++const u32 = n => { const b = Buffer.alloc(4); b.writeUInt32LE(n); return b; };+const u64 = n => { const b = Buffer.alloc(8); b.writeBigUInt64LE(BigInt(n)); return b; };+const str = s => Buffer.concat([u64(Buffer.byteLength(s)), Buffer.from(s)]);+const kv = {+ string: (key, value) => Buffer.concat([str(key), u32(8), str(value)]),+ u32: (key, value) => Buffer.concat([str(key), u32(4), u32(value)]),+ i32s: (key, list) => Buffer.concat([str(key), u32(9), u32(5), u64(list.length), ...list.map(u32)]),+ strings: (key, list) => Buffer.concat([str(key), u32(9), u32(8), u64(list.length), ...list.map(str)]),+};+const gguf = (pairs, tensorTypes = [12]) => Buffer.concat([+ Buffer.from('GGUF'), u32(3), u64(tensorTypes.length), u64(pairs.length), ...pairs,+ ...tensorTypes.map((type, i) => Buffer.concat([str(`blk.${i}.weight`), u32(2), u64(4), u64(4), u32(type), u64(0)])),+]);+const put = async (path, data) => { await mkdir(dirname(path), { recursive: true }); await writeFile(path, data); };+const tiny = arch => gguf([kv.string('general.architecture', arch), kv.u32(`${arch}.block_count`, 2)]);++test('hybrid attention caches KV only on its full-attention layers; foreign tensor types are flagged', async () => {+ const file = join(await mkdtemp(join(tmpdir(), 'gguf-')), 'qwen.gguf');+ await writeFile(file, gguf([+ kv.string('general.architecture', 'qwen35'), kv.string('general.size_label', '4B'),+ kv.strings('tokenizer.ggml.tokens', Array.from({ length: 5000 }, (_, i) => `token-${i}`)),+ kv.u32('qwen35.block_count', 32), kv.u32('qwen35.context_length', 262144),+ kv.u32('qwen35.attention.head_count', 16), kv.u32('qwen35.attention.head_count_kv', 4),+ kv.u32('qwen35.attention.key_length', 256), kv.u32('qwen35.attention.value_length', 256),+ kv.u32('qwen35.full_attention_interval', 4),+ kv.string('tokenizer.chat_template', '{% if tools %}<tools>{% endif %}{% if enable_thinking %}<think>{% endif %}'),+ ], [12, 143]));+ const summary = summarizeGguf(await readGguf(file));+ // llama.cpp measured 1024 MiB for 32768 cells on 8 layers of Qwen3.5-4B: 32 KiB per token.+ assert.equal(summary.kvBytesPerToken, 32768);+ assert.equal(summary.kvBasis, 'f16 cache');+ assert.deepEqual([summary.architecture, summary.parameters, summary.contextLength], ['qwen35', '4B', 262144]);+ assert.equal(summary.toolCalls, true);+ assert.equal(summary.thinking, true);+ assert.deepEqual(summary.unsupportedTensorTypes, [143]);+});++test('per-layer KV heads count only the attention layers, and non-GGUF files are ignored', async () => {+ const dir = await mkdtemp(join(tmpdir(), 'gguf-'));+ await writeFile(join(dir, 'lfm.gguf'), gguf([+ kv.string('general.architecture', 'lfm2'), kv.u32('lfm2.block_count', 6), kv.u32('lfm2.embedding_length', 1024),+ kv.u32('lfm2.attention.head_count', 16), kv.i32s('lfm2.attention.head_count_kv', [0, 0, 8, 0, 8, 0]),+ ]));+ assert.equal(summarizeGguf(await readGguf(join(dir, 'lfm.gguf'))).kvBytesPerToken, 16 * (64 + 64) * 2);+ await writeFile(join(dir, 'blob'), 'not a model');+ assert.equal(await readGguf(join(dir, 'blob')), null);+});++test('scan finds models other apps downloaded, names them, and reports one entry per real file', async () => {+ const home = await mkdtemp(join(tmpdir(), 'home-'));+ const ollama = join(home, '.ollama', 'models');+ await put(join(ollama, 'blobs', 'sha256-aaa'), tiny('qwen2'));+ await put(join(ollama, 'manifests', 'registry.ollama.ai', 'library', 'qwen2.5', '0.5b'),+ JSON.stringify({ layers: [{ mediaType: 'application/vnd.ollama.image.model', digest: 'sha256:aaa' }] }));+ await mkdir(join(home, '.grid', 'models'), { recursive: true });+ await symlink(join(ollama, 'blobs', 'sha256-aaa'), join(home, '.grid', 'models', 'qwen2.5-0.5b.gguf'));+ const studio = join(home, '.lmstudio', 'models', 'publisher', 'repo');+ await put(join(studio, 'Model-7B-Q4_K_M.gguf'), tiny('llama'));+ await put(join(studio, 'mmproj-Model-7B-F16.gguf'), tiny('clip'));+ const snapshot = join(home, '.cache', 'huggingface', 'hub', 'models--mlx-community--Tiny-4bit', 'snapshots', 'abc');+ await put(join(snapshot, 'config.json'), JSON.stringify({ architectures: ['Qwen2ForCausalLM'], quantization: { bits: 4, group_size: 64 } }));+ await put(join(snapshot, 'model.safetensors'), 'weights');+ await put(join(home, 'models', 'plain', 'config.json'), JSON.stringify({ architectures: ['LlamaForCausalLM'], max_position_embeddings: 8192 }));+ await put(join(home, 'models', 'plain', 'model-00001-of-00001.safetensors'), 'weights');+ await put(join(home, 'Downloads', 'notes.txt'), 'not a model');++ const { models } = await scanModels({ roots: defaultRoots({}, home).filter(root => root.path.startsWith(home)), minBytes: 0 });+ const byName = Object.fromEntries(models.map(model => [model.name, model]));+ assert.equal(models.length, 4);+ assert.equal(byName['qwen2.5-0.5b'].source, 'grid');+ assert.deepEqual(byName['qwen2.5-0.5b'].alsoAt.map(seen => [seen.source, seen.name]), [['ollama', 'qwen2.5:0.5b']]);+ assert.equal(byName['Model-7B-Q4_K_M'].quant, 'Q4_K_M');+ assert.match(byName['Model-7B-Q4_K_M'].projector, /mmproj-Model-7B-F16\.gguf$/);+ assert.equal(byName['mlx-community/Tiny-4bit'].format, 'mlx');+ assert.equal(byName['mlx-community/Tiny-4bit'].config.quantization, '4-bit');+ assert.equal(byName.plain.format, 'safetensors');+ assert.ok(byName.plain.readableBy.includes('vllm'));+ assert.ok(byName['qwen2.5-0.5b'].readableBy.includes('llama.cpp'));+});++test('the summary table lists every model of every format, with the reasons one cannot be used', () => {+ const text = summarize({+ machine: { chip: 'Apple M1 Pro', platform: 'darwin', arch: 'arm64', memory: { totalBytes: 34e9, availableBytes: 12e9, swapUsedBytes: 11e9 },+ accelerators: [{ vendor: 'apple', name: 'Apple M1 Pro', active: true, totalBytes: 26.8e9 }], engines: [{ kind: 'ollama', version: '0.34.4' }],+ canRun: ['llama.cpp', 'mlx-lm'], listeningPorts: [8082] },+ engines: [{ kind: 'llama.cpp', url: 'http://127.0.0.1:8082/v1', models: ['lfm'] }],+ models: [+ { name: 'big-gguf', source: 'grid', format: 'gguf', bytes: 6.1e9, projector: null, alsoAt: [], readableBy: ['llama.cpp'],+ gguf: { contextLength: 262144, kvBytesPerToken: 32768, toolCalls: true, unsupportedTensorTypes: [] } },+ { name: 'odd-gguf', source: 'folder', format: 'gguf', bytes: 5.9e9, projector: null, alsoAt: [], readableBy: ['llama.cpp'],+ gguf: { contextLength: 262144, kvBytesPerToken: 65536, toolCalls: true, unsupportedTensorTypes: [143] } },+ { name: 'org/Model-9B-4bit', source: 'huggingface', format: 'mlx', bytes: 5.95e9, projector: null, alsoAt: [], readableBy: ['mlx-lm'],+ gguf: null, config: { contextLength: 262144, kvBytesPerToken: 32768, toolCalls: true } },+ { name: 'small-ctx', source: 'ollama', format: 'gguf', bytes: 0.4e9, projector: null, alsoAt: [{ source: 'lm-studio' }], readableBy: ['llama.cpp'],+ gguf: { contextLength: 32768, kvBytesPerToken: 12288, toolCalls: true, unsupportedTensorTypes: [] } },+ ],+ });+ assert.match(text, /^machine\s+Apple M1 Pro/m);+ assert.match(text, /apple Apple M1 Pro active/);+ assert.match(text, /mlx .*org\/Model-9B-4bit \(huggingface\)/, 'MLX models appear, never filtered out');+ assert.match(text, /odd-gguf .*llama\.cpp cannot load \(tensor type 143\)/);+ assert.match(text, /small-ctx \(ollama, also lm-studio\)\s+! context 32768 < 65536/);+ assert.match(text, /answering llama\.cpp http:\/\/127\.0\.0\.1:8082\/v1/);+ assert.match(text, /^joined\s+none$/m);+});++test('joined grids come from Grid run files with a live PID, whatever the relay says', async () => {+ const home = await mkdtemp(join(tmpdir(), 'grid-home-'));+ const write = async (dir, state) => { await mkdir(join(home, 'run', 'engines', dir), { recursive: true }); await writeFile(join(home, 'run', 'engines', dir, 'remote.json'), JSON.stringify(state)); };+ await write('grid-aaa', { grid_id: 'grid-aaa', pid: 111, advertise_as: [], models: ['lfm.gguf'], engines: [{ endpoint_url: null, models: ['lfm.gguf'] }] });+ await write('grid-bbb', { grid_id: 'grid-bbb', pid: 222, advertise_as: ['Coder'], models: ['/snap'], engines: [{ endpoint_url: 'http://127.0.0.1:8084/v1' }] });+ await write('grid-gone', { grid_id: 'grid-gone', pid: 333, models: ['old.gguf'] });+ await write('n1', {});+ const rows = await joinedGrids(home, { alive: pid => pid !== 333 });+ assert.deepEqual(rows.map(r => [r.gridId, r.models, r.at]).sort(), [+ ['grid-aaa', ['lfm.gguf'], null], ['grid-bbb', ['Coder'], 'http://127.0.0.1:8084/v1']]);+ assert.deepEqual(await joinedGrids(join(home, 'missing')), []);+ const text = summarize({+ machine: { platform: 'darwin', arch: 'arm64', accelerators: [], engines: [], canRun: [], listeningPorts: [] }, engines: [], models: [],+ joined: [{ grid: 'my-grid', gridId: 'grid-aaa', models: ['lfm.gguf'], at: null }],+ });+ assert.match(text, /^joined\s+my-grid serving \[lfm\.gguf\] — a join to that grid adds to what it serves \(Grid 0\.3\.53\+\)$/m);+});++test('engines are identified by what they answer, and the --at URL always carries /v1', async () => {+ const serve = routes => new Promise(resolve => {+ const server = createServer((req, res) => {+ const body = routes[req.url];+ if (!body) { res.statusCode = 404; res.end(); return; }+ res.setHeader('content-type', 'application/json'); res.end(JSON.stringify(body));+ });+ server.listen(0, '127.0.0.1', () => resolve(server));+ });+ const servers = await Promise.all([+ serve({ '/v1/models': { data: [{ id: 'qwen', owned_by: 'vllm' }] } }),+ serve({ '/v1/models': { data: [{ id: 'qwen2.5:0.5b', owned_by: 'library' }] }, '/api/version': { version: '0.34.4' } }),+ serve({ '/v1/models': { models: [{ name: 'lfm' }], data: [{ id: 'lfm', owned_by: 'llamacpp' }] } }),+ serve({ '/v1/models': { data: [{ id: 'mlx-community/x' }] } }),+ serve({}),+ ]);+ try {+ const ports = servers.map(server => server.address().port);+ const found = await probeEngines(ports);+ assert.deepEqual(found.map(engine => engine.kind), ['vllm', 'ollama', 'llama.cpp', 'openai-compatible']);+ assert.ok(found.every(engine => engine.url === `http://127.0.0.1:${engine.port}/v1`));+ assert.deepEqual(found[1].models, ['qwen2.5:0.5b']);+ } finally { servers.forEach(server => server.close()); }+});++test('system readings parse the exact text macOS, Linux, lsof and llama-server print', () => {+ assert.equal(parseVmStat('Mach Virtual Memory Statistics: (page size of 16384 bytes)\nPages free: 10.\nPages active: 500.\nPages inactive: 20.\nPages speculative: 5.\nPages purgeable: 1.\n'), 36 * 16384);+ assert.equal(parseSwapUsage('total = 11264.00M used = 10945.44M free = 318.56M (encrypted)').used, 10945.44 * 1024 ** 2);+ assert.deepEqual(parseMeminfo('MemTotal: 1000 kB\nMemAvailable: 600 kB\nSwapTotal: 200 kB\nSwapFree: 50 kB\n'), { total: 1000 * 1024, available: 600 * 1024, swapUsed: 150 * 1024 });+ assert.deepEqual(parseListDevices('Available devices:\n MTL0: Apple M1 Pro (25559 MiB, 25558 MiB free)\n BLAS: Accelerate (0 MiB, 0 MiB free)\n'),+ [{ id: 'MTL0', name: 'Apple M1 Pro', totalBytes: 25559 * 1024 ** 2, freeBytes: 25558 * 1024 ** 2 }]);+ assert.deepEqual(parseListeningPorts('p123\nf4\nn*:8082\nf5\nn127.0.0.1:8130\np9\nf3\nn[::1]:631\n'), [631, 8082, 8130]);+});++test('listening ports are read on Linux without lsof, and on Windows from netstat', () => {+ const proc = [+ ' sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode',+ ' 0: 00000000:1F90 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 12345 1 0 100 0 0 10 0',+ ' 1: 0100007F:2328 0100007F:A1B2 01 00000000:00000000 00:00000000 00000000 0 0 12346 1 0 20 4 30 10 -1',+ ' 0: 00000000000000000000000000000000:2CAA 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 1 1 0 100 0 0 10 0',+ ].join('\n');+ assert.deepEqual(parseProcNetTcp(proc), [8080, 11434], 'only LISTEN (0A) rows, IPv4 and IPv6');+ const netstat = [+ 'Active Connections', '', ' Proto Local Address Foreign Address State PID',+ ' TCP 0.0.0.0:135 0.0.0.0:0 LISTENING 1016',+ ' TCP 127.0.0.1:11434 0.0.0.0:0 LISTENING 4812',+ ' TCP 192.168.1.5:52100 20.1.2.3:443 ESTABLISHED 7000',+ ' TCP [::]:1234 [::]:0 LISTENING 5000',+ ' UDP 0.0.0.0:5353 *:* 2100',+ ].join('\r\n');+ assert.deepEqual(parseNetstatListening(netstat), [135, 1234, 11434]);+});++test('one model in two formats is one model, and on a Mac with mlx-lm the GGUF row points at the MLX copy', () => {+ assert.equal(modelFamily('Qwen3.5-9B-UD-Q4_K_XL'), 'qwen3.5-9b');+ assert.equal(modelFamily('mlx-community/Qwen3.5-9B-4bit'), 'qwen3.5-9b');+ assert.equal(modelFamily('qwen2.5-0.5b-instruct-q8_0'), 'qwen2.5-0.5b-instruct');+ assert.equal(modelFamily('Qwen3.5-9B-MTP-Q6_K'), 'qwen3.5-9b-mtp', 'a variant stays its own model');+ const row = (name, format, readableBy) => ({ name, source: 'x', format, bytes: 6e9, projector: null, alsoAt: [], readableBy, gguf: format === 'gguf' ? { unsupportedTensorTypes: [] } : null, config: {} });+ const models = [row('Qwen3.5-9B-UD-Q4_K_XL', 'gguf', ['llama.cpp']), row('mlx-community/Qwen3.5-9B-4bit', 'mlx', ['mlx-lm'])];+ const mac = { platform: 'darwin', arch: 'arm64', accelerators: [], engines: [{ kind: 'mlx-lm' }], canRun: ['llama.cpp', 'mlx-lm'], listeningPorts: [] };+ assert.match(summarize({ machine: mac, engines: [], models }), /Qwen3\.5-9B-UD-Q4_K_XL .*an MLX copy is on disk \(mlx-community\/Qwen3\.5-9B-4bit\)/);+ assert.doesNotMatch(summarize({ machine: { ...mac, engines: [] }, engines: [], models }), /MLX copy/, 'no MLX engine installed, no pointer');+});++test('a model no engine on this machine can run says so, instead of looking usable', () => {+ const text = summarize({+ machine: { platform: 'linux', arch: 'x86_64', accelerators: [], engines: [], canRun: ['llama.cpp', 'ollama'], listeningPorts: [] },+ engines: [],+ models: [+ { name: 'org/Model-4bit', source: 'huggingface', format: 'mlx', bytes: 5e9, projector: null, alsoAt: [], readableBy: ['mlx-lm', 'lm-studio'], config: {} },+ { name: 'model-q4', source: 'grid', format: 'gguf', bytes: 5e9, projector: null, alsoAt: [], readableBy: ['llama.cpp'], gguf: { unsupportedTensorTypes: [] } },+ ],+ });+ assert.match(text, /org\/Model-4bit .*! no engine this machine can run reads mlx \(Apple silicon only\)/);+ assert.doesNotMatch(text, /model-q4 .*no engine/);+});diff --git a/store/agents/autonomous-grid/test/modelFacts.test.mjs b/store/agents/autonomous-grid/test/modelFacts.test.mjsnew file mode 100644index 000000000..4023aacd2--- /dev/null+++ b/store/agents/autonomous-grid/test/modelFacts.test.mjs@@ -0,0 +1,70 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { mkdtemp, writeFile } from 'node:fs/promises';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { modelFacts } from '../lib/modelFacts.mjs';++const template = '{%- if tools %}<tools>{{ tools }}</tools>\nReturn <tool_call>{"name": ..., "arguments": ...}</tool_call>{%- endif %}{% if enable_thinking %}<think>{% endif %}';+const card = [+ '# Widget-7B', '', 'Serve it:', '', '```bash', 'vllm serve Acme/Widget-7B --enable-auto-tool-choice --tool-call-parser widget --reasoning-parser widget_r', '```',+ '', '```python', 'print("not a serve command")', '```',+].join('\n');+const hub = {+ 'https://huggingface.co/api/models/Acme/Widget-7B': { gated: false, siblings: ['config.json', 'generation_config.json', 'tokenizer_config.json', 'README.md'].map(rfilename => ({ rfilename })) },+ 'https://huggingface.co/Acme/Widget-7B/raw/main/config.json': { architectures: ['WidgetForCausalLM'], model_type: 'widget', max_position_embeddings: 131072, torch_dtype: 'bfloat16' },+ 'https://huggingface.co/Acme/Widget-7B/raw/main/generation_config.json': { temperature: 0.7, top_p: 0.8, eos_token_id: 2 },+ 'https://huggingface.co/Acme/Widget-7B/raw/main/tokenizer_config.json': { chat_template: template },+ 'https://huggingface.co/Acme/Widget-7B/raw/main/README.md': card,+ 'https://huggingface.co/api/models/Acme/Locked-7B': { gated: 'manual', siblings: [{ rfilename: 'config.json' }] },+ 'https://raw.githubusercontent.com/vllm-project/vllm/main/docs/models/supported_models.md': '| `WidgetForCausalLM` | Widget | `Acme/Widget-7B` | ✅︎ |\n| `OtherForCausalLM` | Other |',+ 'https://raw.githubusercontent.com/sgl-project/sglang/main/python/sglang/srt/models/widget.py': 'class WidgetForCausalLM:\n pass\n\nEntryClass = WidgetForCausalLM\n',+ 'https://huggingface.co/api/models/Acme/Gizmo-3B': { gated: false, siblings: [{ rfilename: 'config.json' }] },+ 'https://huggingface.co/Acme/Gizmo-3B/raw/main/config.json': { architectures: ['GizmoForCausalLM'], model_type: 'gizmo' },+};+const fetchImpl = async url => {+ const body = hub[url];+ if (body === undefined) return new Response('', { status: url.includes('/Locked-7B/raw/') ? 401 : 404 });+ return new Response(typeof body === 'string' ? body : JSON.stringify(body), { status: 200 });+};++test('a model with no recipe is read from its own files: architecture, context, sampling, tool syntax, card commands, support', async () => {+ const facts = await modelFacts('Acme/Widget-7B', { fetchImpl, token: undefined });+ assert.deepEqual([facts.found, facts.gated, facts.architecture, facts.modelType, facts.contextLength, facts.dtype],+ [true, false, 'WidgetForCausalLM', 'widget', 131072, 'bfloat16']);+ assert.deepEqual(facts.samplingDefaults, { temperature: 0.7, top_p: 0.8 });+ assert.equal(facts.chatTemplate.toolCalls, true);+ assert.equal(facts.chatTemplate.thinking, true);+ assert.match(facts.chatTemplate.toolSyntax, /<tool_call>\{"name"/);+ assert.equal(facts.modelCard.serveCommands.length, 1);+ assert.deepEqual(facts.modelCard.toolCallParsers, ['widget']);+ assert.deepEqual(facts.modelCard.reasoningParsers, ['widget_r']);+ assert.equal(facts.support.vllm.listed, true);+ assert.equal(facts.support.sglang.listed, true);+});++test('engine support is only claimed from the engine\'s own list; an absent registry file means unknown, not unsupported', async () => {+ const facts = await modelFacts('Acme/Gizmo-3B', { fetchImpl, token: undefined });+ assert.equal(facts.support.vllm.listed, false);+ assert.equal(facts.support.sglang.listed, null);+ assert.match(facts.support.sglang.note, /model-impl transformers/);+});++test('a gated or unknown model is reported as such, never filled in', async () => {+ const locked = await modelFacts('Acme/Locked-7B', { fetchImpl, token: undefined });+ assert.equal(locked.gated, true);+ assert.match(locked.note, /accept its license/);+ const missing = await modelFacts('Acme/Nothing-1B', { fetchImpl, token: undefined });+ assert.equal(missing.found, false);+ await assert.rejects(modelFacts('not an id', { fetchImpl }), /ORG\/NAME/);+});++test('a model folder already on disk is read in place', async () => {+ const dir = await mkdtemp(join(tmpdir(), 'model-'));+ await writeFile(join(dir, 'config.json'), JSON.stringify({ architectures: ['WidgetForCausalLM'], max_position_embeddings: 65536, quantization: { bits: 4, group_size: 64 } }));+ await writeFile(join(dir, 'chat_template.jinja'), template);+ const facts = await modelFacts(dir, { fetchImpl, checkSupport: false });+ assert.deepEqual([facts.source, facts.architecture, facts.contextLength, facts.quantization], [dir, 'WidgetForCausalLM', 65536, 'mlx 4-bit']);+ assert.equal(facts.chatTemplate.toolCalls, true);+ assert.equal(facts.support, null);+});diff --git a/store/agents/autonomous-grid/test/recipes.test.mjs b/store/agents/autonomous-grid/test/recipes.test.mjsnew file mode 100644index 000000000..439a65d21--- /dev/null+++ b/store/agents/autonomous-grid/test/recipes.test.mjs@@ -0,0 +1,97 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { NoRecipe, sglangCookbook, vllmRecipe } from '../lib/recipes.mjs';++const site = routes => async url => {+ const body = routes[url];+ if (body === undefined) return new Response('not found', { status: 404 });+ return new Response(typeof body === 'string' ? body : JSON.stringify(body), { status: 200 });+};++const vllmSite = site({+ 'https://recipes.vllm.ai/models.json': [+ { hf_id: 'Acme/Widget-7B', url: '/Acme/Widget-7B', json: '/Acme/Widget-7B.json' },+ { hf_id: 'Acme/Widget-70B', url: '/Acme/Widget-70B', json: '/Acme/Widget-70B.json' },+ ],+ 'https://recipes.vllm.ai/Acme/Widget-7B.json': {+ hf_id: 'Acme/Widget-7B',+ model: { model_id: 'Acme/Widget-7B', min_vllm_version: '0.17.0', context_length: 262144, base_args: ['--trust-remote-code'], base_env: {} },+ features: {+ tool_calling: { args: ['--enable-auto-tool-choice', '--tool-call-parser', 'widget'] },+ reasoning: { args: ['--reasoning-parser', 'widget'] },+ spec_decoding: { args: ['--speculative-config', '{"method":"mtp"}'] },+ },+ opt_in_features: ['spec_decoding'],+ variants: { default: { precision: 'bf16', vram_minimum_gb: 22 }, fp8: { model_id: 'Acme/Widget-7B-FP8', precision: 'fp8', vram_minimum_gb: 11, extra_args: ['--kv-cache-dtype', 'fp8'] } },+ recommended_command: { command: 'vllm serve Acme/Widget-7B', docker_image: 'vllm/vllm-openai:latest' },+ guide: '## Troubleshooting\n- lower the capture size',+ },+});++test('a vLLM recipe gives the minimum version, the flags an agent always needs, and the variants by GPU memory', async () => {+ const found = await vllmRecipe('acme/widget-7b', { fetchImpl: vllmSite });+ assert.equal(found.source, 'https://recipes.vllm.ai/Acme/Widget-7B');+ assert.equal(found.minVersion, '0.17.0');+ assert.deepEqual(found.baseArgs, ['--trust-remote-code']);+ assert.deepEqual(found.features.tool_calling, { optIn: false, args: ['--enable-auto-tool-choice', '--tool-call-parser', 'widget'], env: {}, description: '' });+ assert.equal(found.features.spec_decoding.optIn, true);+ assert.deepEqual(found.variants.fp8, { modelId: 'Acme/Widget-7B-FP8', precision: 'fp8', vramMinimumGb: 11, extraArgs: ['--kv-cache-dtype', 'fp8'], description: '' });+ assert.equal(found.variants.default.modelId, 'Acme/Widget-7B');+ assert.match(found.guide, /Troubleshooting/);+});++test('no vLLM recipe is an explicit answer, never a sibling model standing in for it', async () => {+ await assert.rejects(vllmRecipe('Acme/Widget-3B', { fetchImpl: vllmSite }), NoRecipe);+ await assert.rejects(vllmRecipe('not an id', { fetchImpl: vllmSite }), /ORG\/NAME/);+});++const cookbookPage = `# Widget++## 1. Model Introduction+Text.++## 2. SGLang Installation++SGLang 0.5.9 or newer is required.++\`\`\`bash Command theme={null}+uv pip install sglang+\`\`\`++## 3. Model Deployment++### 3.2 Configuration Tips+* Lower \`--mem-fraction-static\` when images are large.++## 4. Model Invocation++\`\`\`shell Command theme={null}+sglang serve \\+ --model-path Acme/Widget-2.5-7B \\+ --reasoning-parser widget \\+ --tool-call-parser widget_xml \\+ --host 0.0.0.0 --port 30000+\`\`\`+`;+const sglangSite = site({+ 'https://docs.sglang.io/llms.txt': [+ '- [Widget](https://docs.sglang.io/cookbook/autoregressive/Acme/Widget.md): older family',+ '- [Widget 2.5](https://docs.sglang.io/cookbook/autoregressive/Acme/Widget-2.5.md): current family',+ '- [Image](https://docs.sglang.io/cookbook/diffusion/Acme/Paint.md): not a language model',+ ].join('\n'),+ 'https://docs.sglang.io/cookbook/autoregressive/Acme/Widget-2.5.md': cookbookPage,+});++test('the SGLang cookbook page is the most specific family match, with its commands, parsers and notes', async () => {+ const found = await sglangCookbook('Acme/Widget-2.5-7B-Instruct', { fetchImpl: sglangSite });+ assert.equal(found.source, 'https://docs.sglang.io/cookbook/autoregressive/Acme/Widget-2.5.md');+ assert.deepEqual(found.toolCallParsers, ['widget_xml']);+ assert.deepEqual(found.reasoningParsers, ['widget']);+ assert.equal(found.serveCommands.length, 1);+ assert.match(found.installation, /0\.5\.9 or newer/);+ assert.match(found.configurationTips, /mem-fraction-static/);+});++test('no SGLang cookbook page is an explicit answer, never a guess', async () => {+ await assert.rejects(sglangCookbook('Other/Gadget-1B', { fetchImpl: sglangSite }), NoRecipe);+});diff --git a/store/agents/autonomous-grid/test/remoteModels.test.mjs b/store/agents/autonomous-grid/test/remoteModels.test.mjsnew file mode 100644index 000000000..7b103bdbc--- /dev/null+++ b/store/agents/autonomous-grid/test/remoteModels.test.mjs@@ -0,0 +1,89 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { EventEmitter } from 'node:events';+import { PassThrough } from 'node:stream';+import { mkdtemp, readlink, writeFile } from 'node:fs/promises';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { FINDER_FILES, linkModel, remoteModels, sshBase } from '../lib/remoteModels.mjs';++/** A fake `ssh`: records each call and its stdin, and answers the run with `reply(remoteCommand)`. */+function fakeSsh(reply) {+ const calls = [];+ const spawnImpl = (file, args) => {+ const child = new EventEmitter();+ child.stdout = new PassThrough(); child.stderr = new PassThrough(); child.stdin = new PassThrough();+ child.kill = () => {};+ const call = { file, args, input: '' };+ calls.push(call);+ child.stdin.on('data', chunk => { call.input += chunk; });+ child.stdin.on('finish', () => {+ const out = reply(args.at(-1));+ child.stdout.end(out.stdout ?? ''); child.stderr.end(out.stderr ?? '');+ setImmediate(() => child.emit('close', out.code ?? 0));+ });+ return child;+ };+ return { calls, spawnImpl };+}++const machine = { id: 'box', name: 'box', transport: 'ssh', host: 'me@box', port: 2222 };++test('the finder is copied to the machine and run by its own Node', async () => {+ const ssh = fakeSsh(remote => (remote.startsWith('if ') ? { stdout: 'machine Intel Xeon · linux/x64\n' } : {}));+ const result = await remoteModels(machine, { spawnImpl: ssh.spawnImpl });+ assert.equal(result.full, true);+ assert.match(result.text, /^machine\s+Intel Xeon/);+ assert.deepEqual(ssh.calls.slice(0, 3).map(c => c.args.at(-1).split('/').pop()), FINDER_FILES);+ assert.match(ssh.calls[0].input, /export async function inventory/, 'the real finder source is sent');+ assert.match(ssh.calls.at(-1).args.at(-1), /exec node \.cache\/grid-harness\/finder\/finderMain\.mjs --summary/);+ assert.deepEqual(ssh.calls[0].args.slice(0, 9), sshBase(machine).slice(0, 9));+ assert.ok(ssh.calls.every(c => c.file === 'ssh' && c.args.includes('BatchMode=yes') && c.args.includes('StrictHostKeyChecking=yes')));+});++test('without Node there it still lists the model files, and says what is missing', async () => {+ const ssh = fakeSsh(remote => (remote.startsWith('if ')+ ? { stdout: '__GRID_FINDER_NO_NODE__\nLinux x86_64\n/root/.ollama/models/manifests/registry.ollama.ai/library/qwen/0.5b\n' } : {}));+ const result = await remoteModels(machine, { spawnImpl: ssh.spawnImpl });+ assert.equal(result.full, false);+ assert.match(result.text, /Linux x86_64 \(me@box\) — Node 18\+ is not installed there/);+ assert.match(result.text, /manifests\/registry\.ollama\.ai\/library\/qwen\/0\.5b/);+});++test('a machine that cannot be reached, or is linked through Harness, fails with the reason', async () => {+ const down = fakeSsh(() => ({ code: 255, stderr: 'ssh: connect to host box port 2222: Connection refused\n' }));+ await assert.rejects(remoteModels(machine, { spawnImpl: down.spawnImpl }), /Could not copy the finder to me@box: .*Connection refused/);+ await assert.rejects(remoteModels({ id: 'air', name: 'MacBook-Air', transport: 'harness', machineId: 'x' }),+ /Harness, which runs Grid commands only.*device-info/);+});++test('a model on this machine is linked into Grid\'s folder once, and a taken name is never replaced', async () => {+ const dir = await mkdtemp(join(tmpdir(), 'link-'));+ const model = join(dir, 'lfm.gguf'), mmproj = join(dir, 'mmproj-lfm.gguf'), other = join(dir, 'other.gguf');+ for (const file of [model, mmproj, other]) await writeFile(file, 'GGUF');+ const env = { GRID_HOME: join(dir, 'grid') }, local = { id: 'local', transport: 'local' };++ const first = await linkModel(local, model, 'LFM.gguf', { projector: mmproj, env });+ assert.match(first.text, /linked .*LFM\.gguf -> .*lfm\.gguf/);+ assert.equal(await readlink(join(dir, 'grid', 'models', 'LFM.gguf')), model);+ assert.equal(await readlink(join(dir, 'grid', 'models', 'LFM.mmproj.gguf')), mmproj, 'the projector name --serve looks for');+ assert.match(first.text, /NOT READY: Grid's engine \(llama\.cpp\) is not installed.*run -- engine install llama\.cpp/,+ 'a join would die at start without the engine, so the link says so');+ assert.match((await linkModel(local, model, 'LFM.gguf', { env })).text, /already linked/);+ await assert.rejects(linkModel(local, other, 'LFM.gguf', { env }), /already exists and is not a link/);+ await assert.rejects(linkModel(local, join(dir, 'missing.gguf'), 'M.gguf', { env }), /No file at/);+ await assert.rejects(linkModel(local, 'relative.gguf', 'M.gguf', { env }), /full paths/);+ await assert.rejects(linkModel(local, model, 'bad name.gguf', { env }), /plain file name/);+});++test('on an SSH machine the link is made there, with every path quoted', async () => {+ const ssh = fakeSsh(() => ({ stdout: 'linked /root/.grid/models/LFM.gguf -> /root/my models/lfm.gguf\n' }));+ const result = await linkModel(machine, "/root/my models/lfm.gguf", 'LFM.gguf', { spawnImpl: ssh.spawnImpl });+ assert.match(result.text, /^linked /);+ const script = ssh.calls[0].args.at(-1);+ assert.match(script, /link '\/root\/my models\/lfm\.gguf' 'LFM\.gguf'/);+ assert.match(script, /ln -s "\$1" "\$d\/\$2"/);+ assert.match(script, /bin\/llama-server".*command -v llama-server.*NOT READY.*--machine box -- engine install llama\.cpp/);+ assert.match(script, /err="\$\("\$e" --version 2>&1\)".*does not start/, 'installed is not enough: it has to run');+ await assert.rejects(linkModel({ id: 'air', transport: 'harness', machineId: 'x' }, '/m.gguf', 'M.gguf'), /catalog/);+});diff --git a/store/agents/autonomous-grid/test/serve.test.mjs b/store/agents/autonomous-grid/test/serve.test.mjsnew file mode 100644index 000000000..4890b7081--- /dev/null+++ b/store/agents/autonomous-grid/test/serve.test.mjs@@ -0,0 +1,34 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { execFileSync } from 'node:child_process';+import { mkdtemp, readFile } from 'node:fs/promises';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { recordedPid, runFiles, serve, stop } from '../lib/serve.mjs';++const alive = pid => { try { process.kill(pid, 0); return true; } catch { return false; } };++test('an engine started with serve leads its own process group, logs, records its PID, and stop ends only it', async () => {+ const workspace = await mkdtemp(join(tmpdir(), 'serve-'));+ const started = await serve(workspace, 'probe', [process.execPath, '-e', 'console.log(process.env.PROBE); setTimeout(() => {}, 60000)'], { env: { PROBE: 'ready' } });+ try {+ assert.equal(await recordedPid(workspace, 'probe'), started.pid);+ assert.ok(alive(started.pid));+ const group = Number(execFileSync('ps', ['-o', 'pgid=', '-p', String(started.pid)], { encoding: 'utf8' }).trim());+ assert.equal(group, started.pid, 'its own process group, so the starting shell cannot take it down');+ await new Promise(resolve => setTimeout(resolve, 300));+ assert.match(await readFile(runFiles(workspace, 'probe').log, 'utf8'), /ready/);+ await assert.rejects(serve(workspace, 'probe', [process.execPath, '-e', '']), /already running/);+ } finally {+ const stopped = await stop(workspace, 'probe', { graceMs: 2_000 });+ assert.equal(stopped.stopped, true);+ }+ assert.equal(alive(started.pid), false);+ assert.equal(await recordedPid(workspace, 'probe'), null);+ assert.equal((await stop(workspace, 'probe')).stopped, false, 'nothing recorded, nothing killed');+});++test('names cannot reach outside run/', () => {+ assert.throws(() => runFiles('/tmp/x', '../escape'), /engine name/);+ assert.throws(() => runFiles('/tmp/x', ''), /engine name/);+});diff --git a/store/agents/autonomous-grid/test/verify.test.mjs b/store/agents/autonomous-grid/test/verify.test.mjsnew file mode 100644index 000000000..f2a426e2b--- /dev/null+++ b/store/agents/autonomous-grid/test/verify.test.mjs@@ -0,0 +1,101 @@+import test from 'node:test';+import assert from 'node:assert/strict';+import { createLog, verifyEngine, verifyRelay } from '../lib/verify.mjs';++const json = (body, status = 200) => new Response(JSON.stringify(body), { status });+const quiet = () => {};+const instant = async () => {};++function engine({ bootFailures = 1, content = 'ok', reasoning = '', toolCall = true, seen = [], thinksFirst = false } = {}) {+ let boots = 0;+ return async (url, init = {}) => {+ const body = init.body ? JSON.parse(init.body) : null;+ seen.push({ url, body });+ if (url.endsWith('/models')) return boots++ < bootFailures ? new Response('loading', { status: 503 }) : json({ data: [{ id: 'widget' }] });+ if (body.tools) return json({ choices: [{ message: toolCall ? { tool_calls: [{ function: { name: 'read_file', arguments: '{"path":"README.md"}' } }] } : { content: 'I cannot' } }] });+ if (body.max_tokens === 128) return json({ usage: { completion_tokens: 128 }, timings: { predicted_per_second: 21.5 }, choices: [{ message: { content: 'one two' } }] });+ // A model an engine keeps thinking: 16 tokens are all reasoning, a larger budget reaches the answer.+ if (thinksFirst) return json({ choices: [{ message: body.max_tokens > 16 ? { content: 'ok', reasoning_content: 'hmm' } : { content: '', reasoning_content: 'hmm' } }] });+ return json({ choices: [{ message: { content, reasoning_content: reasoning } }] });+ };+}++test('an engine passes only after ready, listed, a real answer, a tool call and a speed reading, each narrated', async () => {+ const lines = [], seen = [];+ const result = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ seen }), log: createLog(line => lines.push(line)), sleep: instant });+ assert.equal(result.ok, true);+ assert.deepEqual(result.steps.map(step => step.name), ['ready', 'listed', 'answer', 'tool call', 'speed']);+ assert.equal(result.steps.at(-1).tokensPerSecond, 21.5);+ assert.ok(lines.some(line => /attempt 1: 503/.test(line)), 'a failed readiness attempt is shown, not hidden');+ assert.ok(lines.every(line => /^\[\+\s*\d+s\]/.test(line)), 'every line carries elapsed time');+ assert.deepEqual(seen.find(call => call.body?.max_tokens === 16).body.chat_template_kwargs, { enable_thinking: false });+});++test('a model that always thinks first passes once it answers with room to think, and says thinking stays on', async () => {+ const lines = [], seen = [];+ const result = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ bootFailures: 0, thinksFirst: true, seen }), log: createLog(line => lines.push(line)), sleep: instant });+ const answer = result.steps.find(step => step.name === 'answer');+ assert.equal(answer.ok, true);+ assert.match(answer.note, /"ok" — thinking stays on here/);+ assert.ok(lines.some(line => /only thinking came back in 16 tokens; asking once more with room for it \(1024 tokens\)/.test(line)));+ assert.deepEqual(seen.filter(call => call.body?.messages?.[0]?.content?.startsWith('Reply')).map(call => call.body.max_tokens), [16, 1024]);+});++test('thinking-only output, a missing tool call and a URL without /v1 each fail with the reason', async () => {+ const thinking = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ bootFailures: 0, content: '', reasoning: 'Let me think' }), log: quiet, sleep: instant });+ assert.equal(thinking.ok, false);+ assert.match(thinking.steps.at(-1).note, /thinking is still on/);+ const noTool = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', fetchImpl: engine({ bootFailures: 0, toolCall: false }), log: quiet, sleep: instant });+ assert.equal(noTool.steps.at(-1).name, 'tool call');+ assert.equal(noTool.ok, false);+ const root = await verifyEngine({ url: 'http://127.0.0.1:9', model: 'widget', fetchImpl: engine(), log: quiet, sleep: instant });+ assert.match(root.steps[0].note, /must end in \/v1/);+});++test('Ollama gets its documented thinking switch; readiness gives up at its deadline', async () => {+ const seen = [];+ await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', kind: 'ollama', fetchImpl: engine({ bootFailures: 0, seen }), log: quiet, sleep: instant });+ assert.equal(seen.find(call => call.body?.max_tokens === 16).body.reasoning_effort, 'none');+ const never = await verifyEngine({ url: 'http://127.0.0.1:9/v1', model: 'widget', readyMs: 30, fetchImpl: engine({ bootFailures: Infinity }), log: quiet, sleep: instant });+ assert.equal(never.ok, false);+ assert.match(never.steps[0].note, /within/);+});++test('the relay must list the alias before one bounded answer through the grid counts', async () => {+ let polls = 0;+ const listModels = async () => ({ ok: true, models: ++polls < 3 ? ['other'] : ['other', 'widget-alias'] });+ const fetchImpl = async (url, init) => {+ assert.equal(init.headers.authorization, 'Bearer key');+ return json({ choices: [{ message: { content: 'ok' } }] });+ };+ const lines = [];+ const result = await verifyRelay({ alias: 'widget-alias', listModels, relay: { baseUrl: 'https://relay.test/v1', apiKey: 'key' }, fetchImpl, log: line => lines.push(line), sleep: instant });+ assert.equal(result.ok, true);+ assert.equal(polls, 3);+ assert.equal(lines.filter(line => /attempt \d+: .*not yet widget-alias/.test(line)).length, 2);+ const missing = await verifyRelay({ alias: 'gone', listModels: async () => ({ ok: true, models: [] }), relay: { baseUrl: 'https://relay.test/v1' }, listMs: 20, fetchImpl, log: quiet, sleep: instant });+ assert.equal(missing.ok, false);+});++test('a model registered under its raw id fails at once with the fix, instead of waiting out the deadline', async () => {+ let polls = 0;+ const result = await verifyRelay({+ alias: 'Coder', engineModel: '/Users/me/.cache/hub/snap', relay: { baseUrl: 'https://relay.test/v1' }, log: quiet, sleep: instant,+ listModels: async () => { polls++; return { ok: true, models: ['/users/me/.cache/hub/snap'] }; },+ });+ assert.equal(result.ok, false);+ assert.equal(polls, 1);+ assert.match(result.steps[0].note, /listed as \/users\/me\/\.cache\/hub\/snap, not Coder: join again with --advertise-as Coder/);+});++test('an alias the relay lists in another case passes at once, and is asked by the listed name', async () => {+ let polls = 0, asked = null;+ const result = await verifyRelay({+ alias: 'lfm2.5-vl-3b', relay: { baseUrl: 'https://relay.test/v1' }, log: quiet, sleep: instant,+ listModels: async () => { polls++; return { ok: true, models: ['LFM2.5-VL-3B', 'Qwen3-0.6B'] }; },+ fetchImpl: async (url, init) => { asked = JSON.parse(init.body).model; return json({ choices: [{ message: { content: 'ok' } }] }); },+ });+ assert.equal(result.ok, true);+ assert.equal(polls, 1);+ assert.equal(asked, 'LFM2.5-VL-3B');+});diff --git a/store/agents/autonomous-grid/toolchain/fleet.mjs b/store/agents/autonomous-grid/toolchain/fleet.mjsindex 2d3a34af9..cc075e932 100644--- a/store/agents/autonomous-grid/toolchain/fleet.mjs+++ b/store/agents/autonomous-grid/toolchain/fleet.mjs@@ -1,9 +1,15 @@#!/usr/bin/env node-import { verifyModel } from '../lib/verify.mjs';+import { createLog, verifyEngine, verifyModel, verifyRelay } from '../lib/verify.mjs';import { join, resolve } from 'node:path';-import { atomicJson, DEFAULT_CONFIG, execute, harnessNameFor, nameJoin, readConfig, runTracked } from '../lib/fleet.mjs';+import { atomicJson, DEFAULT_CONFIG, execute, gridJson, harnessNameFor, nameJoin, readConfig, runTracked } from '../lib/fleet.mjs';import { createCollector } from '../lib/telemetry.mjs';import { discoverMachines } from '../lib/harness.mjs';+import { inventory, machineState, summarize } from '../lib/inventory.mjs';+import { mlxCandidates } from '../lib/candidates.mjs';+import { recipe } from '../lib/recipes.mjs';+import { modelFacts } from '../lib/modelFacts.mjs';+import { linkModel, remoteModels } from '../lib/remoteModels.mjs';+import { serve, stop } from '../lib/serve.mjs';import { connectWorkspace, initializeWorkspace, mergeMachines, readStatus } from '../lib/connect.mjs';const workspace = resolve(process.env.HARNESS_WORKSPACE || process.cwd());@@ -51,7 +57,8 @@ try {process.exitCode = snapshot.status === 'unavailable' ? 1 : 0;} else if (command === 'config') {console.log(JSON.stringify(await readConfig(workspace), null, 2));- } else if (command === 'verify') {+ } else if (command === 'verify' && !args.includes('--at') && !args.includes('--alias')) {+ // The Models panel's receipt form: one request, published as an operation the panel reads.const flags = new Map();for (let i = 0; i < args.length; i += 2) {if (!['--grid', '--model'].includes(args[i]) || !args[i + 1] || flags.has(args[i])) throw new Error('Use: fleet verify --grid NAME --model MODEL');@@ -60,6 +67,115 @@ try {const config = await readConfig(workspace);const machine = config.machines.find(m => m.id === config.controller);console.log(JSON.stringify(await verifyModel(workspace, machine, config.mode, flags.get('--grid') || config.grid, flags.get('--model'))));+ } else if (command === 'models') {+ const extraRoots = [], extraPorts = [];+ let summary = false, target;+ for (let i = 0; i < args.length; i++) {+ if (args[i] === '--root' && args[i + 1]) extraRoots.push(resolve(args[++i]));+ else if (args[i] === '--port' && /^\d+$/.test(args[i + 1] || '')) extraPorts.push(Number(args[++i]));+ else if (args[i] === '--machine' && args[i + 1]) target = args[++i];+ else if (args[i] === '--summary') summary = true;+ else if (args[i] !== '--json') throw new Error('Use: fleet models [--machine ID] [--summary] [--root DIR]... [--port N]...');+ }+ const config = target ? await readConfig(workspace) : null;+ const machine = target ? config.machines.find(m => m.id === target) : null;+ if (target && !machine) throw new Error(`Unknown machine ${target}. Add it to grid-fleet.json first.`);+ if (machine && machine.transport !== 'local') {+ if (extraRoots.length || extraPorts.length) throw new Error('--root and --port apply to this machine only.');+ console.log((await remoteModels(machine, { summary })).text);+ } else {+ const found = await inventory({ extraRoots, extraPorts });+ console.log(summary ? summarize(found) : JSON.stringify(found, null, 2));+ }+ } else if (command === 'link') {+ const usage = 'Use: fleet link [--machine ID] FILE [NAME.gguf] [--projector FILE]';+ let target, projector;+ const rest = [];+ for (let i = 0; i < args.length; i++) {+ if (args[i] === '--machine' && args[i + 1]) target = args[++i];+ else if (args[i] === '--projector' && args[i + 1]) projector = args[++i];+ else if (args[i].startsWith('-')) throw new Error(usage);+ else rest.push(args[i]);+ }+ if (rest.length < 1 || rest.length > 2) throw new Error(usage);+ const config = await readConfig(workspace);+ const machine = config.machines.find(m => m.id === (target || config.controller));+ if (!machine) throw new Error(`Unknown machine ${target}. Add it to grid-fleet.json first.`);+ const local = machine.transport === 'local';+ const file = local ? resolve(rest[0]) : rest[0];+ const name = rest[1] || file.split('/').pop();+ console.log((await linkModel(machine, file, name, { projector: projector && local ? resolve(projector) : projector })).text);+ } else if (command === 'recipe') {+ if (args.length !== 2) throw new Error('Use: fleet recipe vllm|sglang ORG/NAME');+ try { console.log(JSON.stringify({ found: true, ...(await recipe(args[0], args[1])) }, null, 2)); }+ catch (error) {+ if (error.code !== 'NO_RECIPE') throw error;+ console.log(JSON.stringify({ found: false, message: error.message,+ next: `No official recipe. Read the model itself: fleet model-facts ${args[1]} — then follow "No recipe" in the engine skill. Never invent flags.` }, null, 2));+ process.exitCode = 3;+ }+ } else if (command === 'candidates') {+ const [engine, ...rest] = args;+ if (engine !== 'mlx') throw new Error('Use: fleet candidates mlx [--search WORDS] [--sort downloads|trending|recent] [--context TOKENS]');+ const options = { search: '', sort: 'downloads', contextTokens: 65536 };+ for (let i = 0; i < rest.length; i += 2) {+ if (rest[i] === '--search' && rest[i + 1]) options.search = rest[i + 1];+ else if (rest[i] === '--sort' && rest[i + 1]) options.sort = rest[i + 1];+ else if (rest[i] === '--context' && Number(rest[i + 1]) >= 65536) options.contextTokens = Number(rest[i + 1]);+ else throw new Error('Use: fleet candidates mlx [--search WORDS] [--sort downloads|trending|recent] [--context TOKENS≥65536]');+ }+ const machine = await machineState();+ const metal = machine.accelerators.find(gpu => gpu.vendor === 'apple' && gpu.totalBytes);+ console.log(JSON.stringify({ machine: { chip: machine.chip, gpuBudgetBytes: metal?.totalBytes ?? null, availableBytes: machine.memory?.availableBytes ?? null },+ ...(await mlxCandidates({ ...options, budgetBytes: metal?.totalBytes })) }, null, 2));+ } else if (command === 'verify') {+ const options = { tools: true };+ for (let i = 0; i < args.length; i++) {+ if (['--at', '--model', '--kind', '--grid', '--alias'].includes(args[i]) && args[i + 1]) options[args[i].slice(2)] = args[++i];+ else if (args[i] === '--no-tools') options.tools = false;+ else throw new Error('Use: fleet verify --at http://HOST:PORT/v1 --model ID [--kind ENGINE] [--no-tools] [--grid GRID --alias NAME], or --grid GRID --alias NAME alone');+ }+ const usage = 'Use: fleet verify --at http://HOST:PORT/v1 --model ID [--kind ENGINE] [--no-tools] [--grid GRID --alias NAME], or --grid GRID --alias NAME alone for an engine on another machine';+ if (Boolean(options.grid) !== Boolean(options.alias) || (options.at ? !options.model : !options.grid)) throw new Error(usage);+ const log = createLog();+ const engine = options.at+ ? await verifyEngine({ url: options.at, model: options.model, kind: options.kind, tools: options.tools, log })+ : { ok: true, steps: [] };+ let relay = null;+ if (engine.ok && options.grid) {+ const config = await readConfig(workspace), local = { id: 'local', transport: 'local' };+ const info = await execute(local, [`--${config.mode}`, 'info', options.grid, '--env'], { timeoutMs: 30_000 });+ const env = Object.fromEntries([...`${info.stdout}`.matchAll(/(?:export\s+)?(OPENAI_BASE_URL|OPENAI_API_KEY)=["']?([^"'\s]+)/g)].map(m => [m[1], m[2]]));+ relay = env.OPENAI_BASE_URL+ ? await verifyRelay({ alias: options.alias, engineModel: options.model, log, relay: { baseUrl: env.OPENAI_BASE_URL, apiKey: env.OPENAI_API_KEY },+ listModels: async () => {+ const listed = await gridJson(local, config.mode, ['models', options.grid], { timeoutMs: 30_000 });+ return listed.ok ? { ok: true, models: (Array.isArray(listed.value) ? listed.value : []).map(row => row.model ?? row.id) } : listed;+ } })+ : { ok: false, steps: [{ name: 'relay endpoint', ok: false, note: `grid info ${options.grid} --env gave no OPENAI_BASE_URL` }] };+ }+ const ok = engine.ok && (relay?.ok ?? true);+ console.log(JSON.stringify({ ok, engine: engine.steps, relay: relay?.steps ?? null }, null, 2));+ process.exitCode = ok ? 0 : 1;+ } else if (command === 'serve') {+ const boundary = args.indexOf('--');+ const usage = 'Use: fleet serve NAME [--env KEY=VALUE]... -- ENGINE COMMAND...';+ if (boundary < 1 || boundary === args.length - 1) throw new Error(usage);+ const env = {};+ for (let i = 1; i < boundary; i += 2) {+ const pair = args[i] === '--env' ? args[i + 1]?.match(/^([A-Z_][A-Z0-9_]*)=(.*)$/) : null;+ if (!pair) throw new Error(usage);+ env[pair[1]] = pair[2];+ }+ console.log(JSON.stringify(await serve(workspace, args[0], args.slice(boundary + 1), { env }), null, 2));+ } else if (command === 'stop') {+ if (args.length !== 1) throw new Error('Use: fleet stop NAME');+ console.log(JSON.stringify(await stop(workspace, args[0]), null, 2));+ } else if (command === 'model-facts') {+ if (args.length !== 1) throw new Error('Use: fleet model-facts ORG/NAME|MODEL_FOLDER');+ const facts = await modelFacts(args[0]);+ console.log(JSON.stringify(facts, null, 2));+ if (!facts.found || facts.gated && !facts.architecture) process.exitCode = 3;} else if (command === 'run') {const boundary = args.indexOf('--');if (boundary < 0 || boundary === args.length - 1) throw new Error('Use: fleet run [--machine ID] -- <grid arguments>');@@ -94,6 +210,28 @@ try {fleet doctor Check Grid and Nodefleet discover [--add] Discover your Harness machinesfleet config Validate and print fleet inventory+ fleet models [--machine ID] [--summary] [--root DIR] [--port N]+ Read-only: model files other apps downloaded, engines+ answering, memory and swap now. --machine reads an SSH+ machine with its own Node (a file list without it)+ fleet link [--machine ID] FILE [NAME.gguf] [--projector FILE]+ Link a GGUF already on that machine into its+ ~/.grid/models so join --serve can serve it (no copy)+ fleet recipe vllm|sglang ORG/NAME The engine's official recipe for that model, read live;+ exit 3 when there is none (never invent flags then)+ fleet model-facts ORG/NAME|DIR What the model says about itself: architecture, context,+ chat-template tool syntax, card commands, engine support+ fleet candidates mlx [--search WORDS] [--sort downloads|trending|recent]+ MLX models sized for this Mac at a 64K+ context (no download)+ fleet verify --at URL/v1 --model ID [--kind ENGINE] [--no-tools] [--grid GRID --alias NAME]+ Bounded, narrated acceptance: ready, listed, answer, tool call,+ speed; with --grid, listed by the relay and answering through it+ fleet serve NAME [--env KEY=VALUE]... -- COMMAND...+ Start an engine other than Grid's so it outlives this shell:+ own session, log in run/NAME.log, PID in run/NAME.pid+ fleet stop NAME Stop only the PID that serve recorded for NAME+ fleet verify --grid GRID --alias NAME+ The relay half only, for an engine on another machinefleet refresh Read Grid, update viewer and verdictfleet run [--machine ID] -- ARGS Run any Grid command on a configured machine