The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.
docs: formalize Itô inference serving contract (#2708)
Affaan Mustafaa73deb211e78
diff --git a/docs/design/ecc-ito-compute-integration.md b/docs/design/ecc-ito-compute-integration.mdindex a21c9bdfd..c346428a9 100644--- a/docs/design/ecc-ito-compute-integration.md+++ b/docs/design/ecc-ito-compute-integration.md@@ -110,6 +110,34 @@ adapter; the ECC bridge does not expose its paper fixture mode.Managed inference remains unavailable. ECC does not claim that Itô created amodel endpoint, deployed a workload, reserved capacity, or moved funds.+### Inference-serving contract++`skills/ito-inference` is the only canonical serving skill; `ito-serve` is+trigger language, not a second installed skill. The current ECC bridge has no+`serve` verb and rejects it before resolving or spawning the canonical client.+The canonical runtime documents `inference` only as an unsupported compatibility+probe, and MCP remains limited to auth, find, and status. Serving requests+therefore stop before login.++A future `serve` operation is not releasable until it verifies a completed+booking and fresh serving eligibility, accepts an immutable reviewed manifest,+requires a short-lived single-use confirmation bound to account, action,+manifest digest, and maximum cost, and atomically reserves a caller-provided+idempotency key. CLI arguments carry only an opaque non-authorizing confirmation+reference; bearer confirmation is resolved and consumed server-side.++Manifest handling must canonicalize the path, reject symlinks, open a regular+file without following links, validate ownership/permissions and bounded size,+and hash bytes from the opened descriptor. The digest must match the value bound+into confirmation before mutation, preventing path-swap and digest-mismatch+attacks. Authentication alone is never workload authority.++The same canonical client must expose structured, tenant-scoped status, logs,+metrics, cancel, and cleanup with bounded timeouts and revocation-aware errors.+After an ambiguous transport failure, callers reconcile by idempotency key+before retrying. ECC must never replace that control plane with root SSH, local+serving scripts, browser automation, or an unreviewed purchase endpoint.+## Skill and install shape`skills/ito-compute/SKILL.md` is an opt-in workflow installed through:diff --git a/skills/ito-inference/SKILL.md b/skills/ito-inference/SKILL.mdindex f2448256d..4a95b6c36 100644--- a/skills/ito-inference/SKILL.md+++ b/skills/ito-inference/SKILL.md@@ -1,59 +1,119 @@---name: ito-inference-description: Serve a model on a completed Itô compute booking through the canonical Itô backend. Use after ito-compute has booked GPU nodes and the user wants an OpenAI-compatible endpoint on that metal. Chains off a booking record; ECC implements no serving stack of its own.+description: Inspect the availability of model serving on a completed Itô compute booking and, when the canonical backend becomes available, hand off an explicitly confirmed serving manifest. Use after ito-compute has booked GPU nodes and the user asks for an OpenAI-compatible endpoint, ito-serve, hosted Kimi, or self-hosted open-weights inference. ECC implements no serving stack of its own.metadata:origin: ECC+ status: scaffold+ aliases: ito-serve, hosted-open-weights---# Itô Inference-Serve a model on rented Itô metal by delegating to the canonical Itô compute-backend (Layer 0.2). ECC does not implement a parallel serving stack, launch-adapter, or inference server, and does no browser automation. This skill chains-off a **completed booking** produced by `ito-compute`; it never books, reserves,-or spends.+`ito-inference` is the sole canonical ECC skill for inference serving on Itô+compute. Requests naming `ito-serve` route here; do not create or install a+second `ito-serve` skill. ECC never SSHes to nodes, downloads weights, launches+an engine, or exposes an endpoint; it never books, reserves, or spends.-## Prerequisite+## Current production boundary-A completed booking from the `ito-compute` skill: booking id, node IPs, SSH-access, GPU SKU, node count, and fabric, already recorded in harness memory.-Without a booking record, stop — this skill does not provision.+Managed serving is unavailable today. The ECC bridge exposes only `login`,+`auth`, `find`, `status`, and explicitly gated `evals`. It has no `serve` verb.+The canonical runtime documents `inference` only as an unsupported compatibility+probe; ECC does not invoke or depend on it. The MCP surface exposes only auth,+find, and status. The locally enforceable guarantee is that ECC rejects `serve`+before resolving or spawning the credential-bearing canonical client.-## Delegation+Therefore stop before authentication or any command invocation. Report the+missing capability and return to the originating agent. Never substitute a+local runner, SSH helper, browser workflow, purchase endpoint, or any untracked+local `ito-serve` draft.-ECC calls the canonical backend through the `ecc ito` bridge; it never-re-implements serving. Authenticate once with `ecc ito login` (device-authorization; no key in arguments, files, logs, or chat), exactly as-`ito-compute` documents.+## Required entitlement++When serving is implemented, its first gate is a server-verified completed+booking. Harness memory, an RFQ, a quote, node IPs, or SSH access are not proof+of entitlement. The backend must return fresh serving eligibility bound to the+authenticated account, booking, GPU topology, region, fabric, term, and model+policy. Expired, revoked, mismatched, incomplete, or already-released bookings+fail closed before confirmation.++## Future CLI and API contract++The intended command name is `serve`; `inference` may remain only as an+explicitly deprecated compatibility alias after the production contract lands.+The future handoff must be equivalent to:```shecc ito serve \- --booking <booking-id> \- --model <hf-model-id> \- [--quantization <scheme>] \- [--ttft-ms <target>] [--tpot-ms <target>]+ --booking <server-verified-booking-id> \+ --manifest <absolute-reviewed-json-file> \+ --confirmation-ref <opaque-non-authorizing-reference> \+ --idempotency-key <stable-retry-key> \+ --json```-The `--ttft-ms` / `--tpot-ms` SLO is optional; supplying it turns on-disaggregated prefill/decode, which is off by default.+The reviewed manifest must identify the model revision, engine and version,+quantization, tensor/pipeline topology, endpoint exposure policy, artifact+checksums, storage ceiling, runtime limits, optional TTFT/TPOT objectives, and+maximum incremental cost. No raw API key, SSH key, node password, or bearer+token belongs in arguments, manifests, logs, MCP results, or chat.++The client must canonicalize the manifest path, reject symlinks, open a regular+file without following links, require appropriate ownership and restrictive+permissions, enforce a bounded size, and hash bytes from the opened descriptor.+That digest must exactly equal the digest bound into confirmation before any+workload mutation. A path swap, digest mismatch, oversized file, or mutable+unsafe file fails closed.++The canonical API—not ECC—must own workload creation and return structured JSON+with `ok`, `live_api_contacted`, `notice`, and either `data` or `error`. Serving+data must include stable booking, workload, manifest, and idempotency IDs plus a+state enum; it must not claim an endpoint is live until health and model checks+pass. Errors must include a stable code and safe message without secrets.++## Confirmation and execution gates++Before workload creation, require all of the following:++1. Fresh entitlement and serving eligibility from the canonical backend.+2. A reviewable immutable manifest and deterministic digest.+3. A separate single-use confirmation bound to account, action, manifest, and+ cost, with a short expiry and replay protection. CLI arguments carry only an+ opaque, non-authorizing confirmation reference; the server resolves and+ consumes the bearer capability out of band.+4. A caller-supplied idempotency key reserved atomically with the workload.+5. Server-side fabric, capacity, model-policy, storage, and cost validation.++Authentication is identity, not workload authority. A login, API key, quote,+or completed booking never substitutes for the serving confirmation. Inspection+and plan generation must not create a workload. Cancel and cleanup are separate+mutations with their own scoped confirmation and idempotency boundaries.++## Lifecycle and recovery++The production surface is incomplete until the same canonical client exposes+tenant-scoped status, logs, metrics, cancel, and cleanup operations. Every+operation needs bounded connect and overall timeouts, revocation-aware errors,+and structured output. After an ambiguous transport failure, query status by+the idempotency key before retrying; never create a second workload merely+because the first response was lost. A revoked credential stops polling and+returns control to the originating agent without starting login automatically.-## What the backend does (Layer 0.2)+Only report `ready` after endpoint health, model identity, and canary inference+all pass. Report intermediate and terminal failure states honestly. Cleanup must+be observable and must not release or modify the underlying booking unless that+separate economic action was explicitly authorized.-The desk backend, not ECC, runs the stages, and this skill only reports them:+## Proposed backend stages-1. Fabric gate — never launch on unverified metal. Blocks below 80% of- fabric-expected bus bandwidth; advisory between 80% and 92%; fails loud on- silent NCCL socket fallback.-2. Weights download and shard to the serving layout (desk-side sharded cache- keyed by model, quantization, TP degree).-3. Topology plan (AIConfigurator): TP inside the NVLink domain, PP across nodes;- engine flags emitted as a reviewable file before launch.-4. Launch (vLLM, Dynamo when disaggregating) under systemd, warmup, SLO canary,- and registration of the endpoint URL and config to Graphiti memory.+These stages describe the future backend, not code that exists in ECC:-## Unavailable today+1. Verify entitlement, topology, fabric, and cost gates.+2. Fetch checksum-pinned weights into backend-managed storage.+3. Emit and validate a reviewable topology/engine plan.+4. Launch through the provider control plane, never direct root SSH from ECC.+5. Warm up, test health and model identity, run an SLO canary, then register the+ endpoint and redacted configuration.-The serving operation is not yet wired: the canonical CLI's `inference` verb and-the desk `serve-on-booking` backend are scaffolds. Until they land, this skill-reports the missing capability and stops. Never substitute a local runner or a-purchase endpoint.+Until every gate and lifecycle operation above exists in the canonical runtime,+this skill remains a fail-closed availability check and documentation handoff.diff --git a/tests/ci/ito-inference-skill.test.js b/tests/ci/ito-inference-skill.test.jsnew file mode 100644index 000000000..bbc02e920--- /dev/null+++ b/tests/ci/ito-inference-skill.test.js@@ -0,0 +1,130 @@+/**+ * Contract tests for the installable, fail-closed Itô inference handoff.+ */++const assert = require("assert");+const fs = require("fs");+const os = require("os");+const path = require("path");+const { spawnSync } = require("child_process");++const REPO_ROOT = path.join(__dirname, "..", "..");++function read(relativePath) {+ return fs.readFileSync(path.join(REPO_ROOT, relativePath), "utf8");+}++function readJson(relativePath) {+ return JSON.parse(read(relativePath));+}++function test(name, fn) {+ try {+ fn();+ console.log(` ✓ ${name}`);+ return true;+ } catch (error) {+ console.log(` ✗ ${name}`);+ console.error(` ${error.message}`);+ return false;+ }+}++console.log("\n=== Testing Itô inference skill lifecycle ===\n");++const results = [+ test("uses the canonical serving trigger and fails closed while unavailable", () => {+ const skill = read("skills/ito-inference/SKILL.md");+ assert.match(skill, /^name: ito-inference$/m);+ assert.match(skill, /self-host|serve a model|OpenAI-compatible endpoint/i);+ assert.match(skill, /requests naming .*ito-serve/i);+ assert.match(skill, /completed booking/i);+ assert.match(skill, /never books, reserves,\s+or spends/i);+ assert.match(skill, /serving is unavailable today/i);+ assert.match(skill, /report the\s+missing capability and return/i);+ assert.match(skill, /stop before authentication/i);+ assert.match(skill, /no `serve` verb/i);+ assert.match(skill, /`inference`.*unsupported compatibility\s+probe/i);+ assert.match(skill, /never substitute a\s+local runner, SSH helper, browser workflow, purchase endpoint/i);+ assert.doesNotMatch(skill, /ssh\s+root@|serve-status\.sh/i);+ for (const gate of [+ /server-verified completed\s+booking/i,+ /fresh serving eligibility/i,+ /single-use confirmation/i,+ /account, action, manifest, and\s+cost/i,+ /idempotency/i,+ /status, logs, metrics, cancel, and cleanup/i,+ /structured JSON/i,+ /ambiguous transport/i,+ /reject symlinks/i,+ /without following links/i,+ /hash bytes from the opened descriptor/i,+ /digest must exactly equal/i,+ ]) assert.match(skill, gate);+ assert.match(skill, /--confirmation-ref <opaque-non-authorizing-reference>/i);+ assert.doesNotMatch(skill, /--confirmation-token|--api-key|--access-token/i);+ }),+ test("keeps unsupported serving outside the executable bridge", () => {+ const bridge = read("scripts/ito.js");+ assert.match(bridge, /SUPPORTED_COMMANDS[^\n]+login[^\n]+auth[^\n]+find[^\n]+status[^\n]+evals/);+ assert.doesNotMatch(bridge, /SUPPORTED_COMMANDS[^\n]+serve/);+ assert.match(bridge, /Unsupported Itô command/);++ const fixtureRoot = fs.mkdtempSync(path.join(os.tmpdir(), "ecc-ito-serve-reject-"));+ try {+ const canonicalDir = path.join(fixtureRoot, "cli", "ito-compute-cli", "dist", "bin");+ fs.mkdirSync(canonicalDir, { recursive: true });+ const marker = path.join(fixtureRoot, "spawned");+ const executable = path.join(canonicalDir, "ito.js");+ fs.writeFileSync(executable, `require("fs").writeFileSync(${JSON.stringify(marker)}, "spawned");\n`);+ const result = spawnSync(process.execPath, [+ path.join(REPO_ROOT, "scripts", "ecc.js"), "ito", "serve",+ "--booking", "booking_test", "--model", "model_test",+ ], {+ encoding: "utf8",+ env: { ...process.env, ECC_ITO_CLI_EXECUTABLE: executable },+ });+ assert.notStrictEqual(result.status, 0);+ assert.match(result.stderr, /Unsupported Itô command "serve"/);+ assert.ok(!fs.existsSync(marker), "unsupported serve spawned the canonical child");+ } finally {+ fs.rmSync(fixtureRoot, { recursive: true, force: true });+ }+ }),+ test("ships canonical inference through the existing opt-in compute module", () => {+ const modules = readJson("manifests/install-modules.json").modules;+ const module = modules.find((candidate) => candidate.id === "ito-compute");+ assert.ok(module, "ito-compute install module is missing");+ assert.deepStrictEqual(module.paths, [+ "skills/ito-compute",+ "skills/ito-inference",+ "skills/ito-training",+ ]);+ assert.deepStrictEqual(module.dependencies, ["platform-configs"]);+ assert.strictEqual(module.defaultInstall, false);+ assert.strictEqual(module.stability, "beta");++ const components = readJson("manifests/install-components.json").components;+ assert.deepStrictEqual(+ components.find((candidate) => candidate.id === "capability:ito-compute"),+ {+ id: "capability:ito-compute",+ family: "capability",+ description: "Authenticated Itô GPU inventory, RFQ, status, and explicitly gated node-qualification workflows through the separately installed canonical CLI.",+ modules: ["ito-compute"],+ }+ );++ const profiles = readJson("manifests/install-profiles.json").profiles;+ assert.ok(profiles.full.modules.includes("ito-compute"));++ const packageFiles = readJson("package.json").files;+ assert.ok(packageFiles.includes("skills/ito-inference/"));+ assert.ok(packageFiles.includes("skills/ito-training/"));+ }),+];++const failed = results.filter((passed) => !passed).length;+console.log(`\nPassed: ${results.length - failed}`);+console.log(`Failed: ${failed}`);+process.exit(failed > 0 ? 1 : 0);