Skip to content

armgit / ECC

Public Imported from GitHub
0

The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

feat(ito): expose guarded live node qualification

Affaan Mustafa33c7dbb7d68d
diff --git a/README.md b/README.md
index f986b6da3..839c2bcf0 100644
--- a/README.md
+++ b/README.md
@@ -1215,8 +1215,9 @@ Run or self-host any open-source model behind that gateway using separate comput
`ecc ito` delegates to the separately installed canonical Itô client; ECC does
not maintain a second API client or browser handoff. The available operations
-are `ecc ito auth`, `ecc ito find`, and `ecc ito status`. The matching MCP tools
-are `ito_auth`, `ito_find`, and `ito_status`.
+are `ecc ito auth`, `ecc ito find`, `ecc ito status`, and the separately gated
+`ecc ito evals`. The matching MCP tools remain `ito_auth`, `ito_find`, and
+`ito_status`; node qualification is CLI-only.
The `ito-compute-cli` package is currently unpublished. Build it locally from the Itô runtime repo (private while the desk hardens;
design partners get access) under `cli/ito-compute-cli`, run `npm ci` and `npm run check`, then set
@@ -1226,9 +1227,13 @@ discover this credential-bearing client through `PATH`. See the [`ito-compute`
skill](skills/ito-compute/SKILL.md) for the full RFQ authority and MCP setup
contract.
-`find` submits a live authenticated RFQ. It does not reserve capacity. ECC
-exposes no quote lock, purchase, workload, node-evaluation, or inference path,
-and it never replaces a missing client or failed live call with a local result.
+`find` submits a live authenticated RFQ. It does not reserve capacity.
+`evals` requires both `ITO_ENABLE_SIXTYTWO_LIVE=1` and `--live-sixtytwo`, a
+separately installed `sixtytwo-cli==0.3.33`, an explicit node list, and an
+existing absolute configuration directory. It cannot rent, launch, recover,
+repair, or purchase.
+ECC exposes no quote lock, purchase, workload, or inference path, and it never
+replaces a missing client or failed live call with a local result.
Official references:
- [Claude Code LLM gateway docs](https://docs.anthropic.com/en/docs/claude-code/llm-gateway)
diff --git a/docs/design/ecc-ito-compute-integration.md b/docs/design/ecc-ito-compute-integration.md
index b9ed5c418..a659afb46 100644
--- a/docs/design/ecc-ito-compute-integration.md
+++ b/docs/design/ecc-ito-compute-integration.md
@@ -24,12 +24,13 @@ ECC delegates to the canonical Itô package in
`Ito-Markets/ito-cloud-runtime/cli/ito-compute-cli`. ECC does not maintain a
second API client or response schema.
-The wrapper exposes only the canonical CLI's `auth`, `find`, and `status`
+The wrapper exposes only the canonical CLI's `auth`, `find`, `status`, and `evals`
operations:
ecc ito auth
ecc ito find <all required RFQ constraints>
ecc ito status
+ ecc ito evals --cluster <id> --live-sixtytwo --nodes <list> --config-dir <dir>
The canonical MCP server exposes only `ito_auth`, `ito_find`, and `ito_status`.
ECC includes an opt-in configuration template pointing to the local built MCP
@@ -69,6 +70,12 @@ environment. It does not inspect or log the key.
or agent must gather every hard topology/economic constraint and obtain
explicit buyer authority before invoking it.
- `status` reads current RFQ and procurement status.
+- `evals` requires both `ITO_ENABLE_SIXTYTWO_LIVE=1` and
+ `--live-sixtytwo`, then runs only the canonical CLI's pinned
+ `sixtytwo-cli==0.3.33` qualification adapter against an explicit node list
+ and existing absolute configuration directory. It receives no `ITO_API_KEY`
+ or unrelated cloud/model credentials and cannot rent, launch, recover,
+ repair, reset, purchase, or order resources.
- ECC returns the canonical process's stdout, stderr, and exit code unchanged.
- An inventory row or RFQ is not a capacity reservation.
- Only a non-null canonical firm quote is firm.
@@ -81,9 +88,9 @@ Itô platform. ECC adds no shadow store.
## Unsupported in this slice
-ECC exposes no quote lock, purchase, workload execution, node qualification,
-or inference command. The canonical package contains a separately gated node
-qualification adapter, but this ECC bridge intentionally does not expose it.
+ECC exposes no quote lock, purchase, workload execution, or inference command.
+Node qualification is live-only through the separately gated canonical
+adapter; the ECC bridge does not expose its paper fixture mode.
Managed inference remains unavailable. ECC does not claim that Itô created a
model endpoint, deployed a workload, reserved capacity, or moved funds.
@@ -114,9 +121,10 @@ after review.
The local contract suite proves:
-- only the three supported operations spawn;
+- only the four supported operations spawn;
- RFQ arguments are forwarded without economic reinterpretation;
-- only approved Itô runtime variables cross the process boundary;
+- only approved Itô runtime or isolated node-qualification variables cross the
+ process boundary;
- unsupported and dry-run paths fail before spawn;
- a missing or relative executable fails closed with local-install guidance;
- canonical output and exit status pass through unchanged;
diff --git a/manifests/install-components.json b/manifests/install-components.json
index 8c457fb5c..1e674fa6d 100644
--- a/manifests/install-components.json
+++ b/manifests/install-components.json
@@ -197,7 +197,7 @@
{
"id": "capability:ito-compute",
"family": "capability",
- "description": "Authenticated Itô GPU inventory, RFQ, and status workflows through the separately installed canonical CLI.",
+ "description": "Authenticated Itô GPU inventory, RFQ, status, and explicitly gated node-qualification workflows through the separately installed canonical CLI.",
"modules": [
"ito-compute"
]
diff --git a/manifests/install-modules.json b/manifests/install-modules.json
index 97f6419e2..c55632714 100644
--- a/manifests/install-modules.json
+++ b/manifests/install-modules.json
@@ -574,7 +574,7 @@
{
"id": "ito-compute",
"kind": "skills",
- "description": "Authenticated Itô GPU inventory, RFQ, and status workflows through the separately installed canonical CLI.",
+ "description": "Authenticated Itô GPU inventory, RFQ, status, and explicitly gated node-qualification workflows through the separately installed canonical CLI.",
"paths": [
"skills/ito-compute"
],
diff --git a/scripts/ecc.js b/scripts/ecc.js
index 00928c2a1..48831bf31 100755
--- a/scripts/ecc.js
+++ b/scripts/ecc.js
@@ -4,7 +4,7 @@ const { spawnSync } = require('child_process');
const path = require('path');
const { listAvailableLanguages } = require('./lib/install-executor');
const { getComputeSponsorCopy } = require('./lib/compute-sponsor');
-const { createSafeItoEnvironment } = require('./lib/ito-environment');
+const { createSafeItoInvocationEnvironment } = require('./lib/ito-environment');
const COMMANDS = {
install: {
@@ -107,7 +107,7 @@ const PRIMARY_COMMANDS = [
];
function showHelp(exitCode = 0) {
- console.log(`
+ process.stdout.write(`
ECC selective-install CLI
Usage:
@@ -141,6 +141,7 @@ Examples:
ecc ito auth
ecc ito find --gpu h200 --count 8 --nodes 1 --gpus-per-node 8 --days 30 --storage-tb 1 --start-window 2099-08-15 --max-rate 3.00 --form-factor bare_metal --contract-type reservation --fabric infiniband --region us-east-1
ecc ito status --json
+ ecc ito evals --cluster clu_prod_example --live-sixtytwo --nodes gpu-01,gpu-02 --config-dir /absolute/path/to/qualification-config
ecc list-installed --json
ecc doctor --target cursor
ecc repair --dry-run
@@ -226,7 +227,6 @@ function runCommand(commandName, args) {
if (!command) {
throw new Error(`Unknown command: ${commandName}`);
}
-
const result = spawnSync(
process.execPath,
[path.join(__dirname, command.script), ...args],
@@ -234,9 +234,8 @@ function runCommand(commandName, args) {
cwd: process.cwd(),
env: commandName === 'ito'
? {
- ...createSafeItoEnvironment(process.env, {
+ ...createSafeItoInvocationEnvironment(process.env, args, {
includeControls: true,
- includeItoRuntime: true,
}),
}
: process.env,
diff --git a/scripts/ito.js b/scripts/ito.js
index 85f541b03..507bc18e0 100755
--- a/scripts/ito.js
+++ b/scripts/ito.js
@@ -5,9 +5,12 @@
const fs = require("fs");
const path = require("path");
const { spawnSync } = require("child_process");
-const { createSafeItoEnvironment } = require("./lib/ito-environment");
+const {
+ createSafeItoInvocationEnvironment,
+ getInvocationCommand,
+} = require("./lib/ito-environment");
-const SUPPORTED_COMMANDS = Object.freeze(["auth", "find", "status"]);
+const SUPPORTED_COMMANDS = Object.freeze(["auth", "find", "status", "evals"]);
const CANONICAL_REPOSITORY = "https://github.com/Ito-Markets/ito-cloud-runtime.git";
const CANONICAL_PACKAGE_PATH = "cli/ito-compute-cli";
const CANONICAL_ENTRY_SEGMENTS = Object.freeze([
@@ -18,25 +21,30 @@ const CANONICAL_ENTRY_SEGMENTS = Object.freeze([
]);
const EXECUTABLE_OVERRIDE = "ECC_ITO_CLI_EXECUTABLE";
const MAX_OUTPUT_BYTES = 10 * 1024 * 1024;
+const NODE_QUALIFICATION_TIMEOUT_MS = 31 * 60 * 1000;
function showHelp() {
- console.log(`
+ process.stdout.write(`
ECC × Itô local CLI bridge
Usage:
ecc ito auth
ecc ito find <all required RFQ options>
ecc ito status
- ecc ito <auth|find|status> --json
+ ecc ito evals --cluster <id> --live-sixtytwo --nodes <list> --config-dir <dir>
+ ecc ito <auth|find|status|evals> --json
The bridge invokes the separately installed canonical Itô CLI and returns its
real stdout, stderr, and exit code unchanged. It performs no browser navigation
-and adds no lock, workload, inference, evaluation, or purchase path.
+and adds no lock, workload, inference, or purchase path.
Important:
- "find" reads live inventory and submits an authenticated RFQ.
- Obtain explicit buyer authority and every hard constraint before invoking it.
- "status" reads live RFQ and procurement status.
+ - "evals" invokes only the canonical CLI's double-opt-in, pinned
+ sixtytwo-cli node-qualification adapter against explicit nodes.
+ - Node qualification cannot rent, launch, recover, repair, or purchase.
- Inventory and RFQs are not reservations; only a returned firm quote is firm.
The canonical package is currently unpublished. Install it locally:
@@ -59,11 +67,67 @@ The same package's MCP server exposes only:
Configure the MCP command as "node" with this absolute argument:
/absolute/path/to/ito-cloud-runtime/${CANONICAL_PACKAGE_PATH}/dist/bin/ito-mcp.js
-Inject ITO_API_KEY into the child process from 1Password or the launching
-environment. Never put the key in arguments, tracked files, or chat.
+For auth, find, and status, inject ITO_API_KEY into the child process from
+1Password or the launching environment. Never put the key in arguments,
+tracked files, or chat.
+
+Live node qualification requires ITO_ENABLE_SIXTYTWO_LIVE=1,
+--live-sixtytwo, an explicit node list, and an existing absolute config
+directory. It forwards only named SIXTYTWO_API_TOKEN/SIXTYTWO_TOKEN and SSH
+agent state; ITO_API_KEY is intentionally excluded. The canonical CLI requires
+sixtytwo-cli==0.3.33 and fails closed.
`);
}
+function requiredOptionValue(args, option) {
+ const indexes = args
+ .map((value, index) => (value === option ? index : -1))
+ .filter((index) => index >= 0);
+ if (indexes.length !== 1) {
+ throw new Error(`${option} is required exactly once for live node qualification.`);
+ }
+ const value = args[indexes[0] + 1];
+ if (!value?.trim() || value.startsWith("--")) {
+ throw new Error(`${option} requires a non-empty value for live node qualification.`);
+ }
+ return value;
+}
+
+function validateNodeQualificationArgs(args, environment) {
+ if (environment.ITO_ENABLE_SIXTYTWO_LIVE !== "1") {
+ throw new Error(
+ "Live node qualification requires ITO_ENABLE_SIXTYTWO_LIVE=1 before any process is started."
+ );
+ }
+ if (args.filter((value) => value === "--live-sixtytwo").length !== 1) {
+ throw new Error(
+ "Live node qualification requires --live-sixtytwo exactly once before any process is started."
+ );
+ }
+ requiredOptionValue(args, "--cluster");
+ const nodes = requiredOptionValue(args, "--nodes");
+ if (!nodes.split(",").every((node) => node.trim().length > 0)) {
+ throw new Error("--nodes must explicitly list one or more non-empty nodes.");
+ }
+ const configDirectory = requiredOptionValue(args, "--config-dir");
+ if (!path.isAbsolute(configDirectory)) {
+ throw new Error("--config-dir must be an existing absolute directory.");
+ }
+ try {
+ const resolved = fs.realpathSync.native(configDirectory);
+ if (
+ !fs.statSync(resolved).isDirectory()
+ || !fs.statSync(path.join(resolved, "sixtytwo.yaml")).isFile()
+ ) {
+ throw new Error("invalid qualification configuration");
+ }
+ } catch {
+ throw new Error(
+ "--config-dir must exist and contain a regular sixtytwo.yaml before any process is started."
+ );
+ }
+}
+
function parseArgs(argv, environment = process.env) {
const args = [...argv];
if (
@@ -90,9 +154,12 @@ function parseArgs(argv, environment = process.env) {
const command = withoutJson.shift();
if (!SUPPORTED_COMMANDS.includes(command)) {
throw new Error(
- `Unsupported Itô command "${command || "(missing)"}"; ECC permits only auth, find, and status.`
+ `Unsupported Itô command "${command || "(missing)"}"; ECC permits only auth, find, status, and evals.`
);
}
+ if (command === "evals") {
+ validateNodeQualificationArgs(withoutJson, environment);
+ }
return Object.freeze({
help: false,
@@ -186,13 +253,16 @@ function buildInvocation(executable, args) {
function invokeIto(executable, args, environment = process.env) {
const invocation = buildInvocation(executable, args);
+ const command = getInvocationCommand(args);
+ const isNodeQualification = command === "evals";
const result = spawnSync(invocation.executable, invocation.args, {
cwd: process.cwd(),
encoding: "utf8",
- env: {
- ...createSafeItoEnvironment(environment, { includeItoRuntime: true }),
- },
+ // Keep policy helpers immutable for callers, but give child-process
+ // instrumentation its own mutable copy (for example NODE_V8_COVERAGE).
+ env: { ...createSafeItoInvocationEnvironment(environment, args) },
maxBuffer: MAX_OUTPUT_BYTES,
+ timeout: isNodeQualification ? NODE_QUALIFICATION_TIMEOUT_MS : undefined,
shell: false,
windowsHide: true,
});
@@ -232,6 +302,7 @@ module.exports = Object.freeze({
CANONICAL_PACKAGE_PATH,
CANONICAL_REPOSITORY,
EXECUTABLE_OVERRIDE,
+ NODE_QUALIFICATION_TIMEOUT_MS,
SUPPORTED_COMMANDS,
buildInvocation,
invokeIto,
diff --git a/scripts/lib/ito-environment.js b/scripts/lib/ito-environment.js
index d0afa8653..03a944119 100644
--- a/scripts/lib/ito-environment.js
+++ b/scripts/lib/ito-environment.js
@@ -29,11 +29,20 @@ const ITO_RUNTIME_ENVIRONMENT_KEYS = Object.freeze([
"ITO_INVENTORY_URL",
]);
+const ITO_EVAL_ENVIRONMENT_KEYS = Object.freeze([
+ "ITO_ENABLE_SIXTYTWO_LIVE",
+ "SIXTYTWO_API_TOKEN",
+ "SIXTYTWO_TOKEN",
+ "SSH_AUTH_SOCK",
+ "SSH_AGENT_PID",
+]);
+
const ECC_ITO_CONTROL_KEYS = Object.freeze([
"ECC_DRY_RUN",
"ECC_ITO_CLI_EXECUTABLE",
"NODE_ENV",
]);
+const ITO_RUNTIME_COMMANDS = new Set(["auth", "find", "status"]);
function copyDefined(source, target, key) {
if (typeof source[key] === "string") {
@@ -56,6 +65,12 @@ function createSafeItoEnvironment(source = process.env, options = {}) {
}
}
+ if (options.includeItoEvals) {
+ for (const key of ITO_EVAL_ENVIRONMENT_KEYS) {
+ copyDefined(source, safe, key);
+ }
+ }
+
if (options.includeControls) {
for (const key of ECC_ITO_CONTROL_KEYS) {
copyDefined(source, safe, key);
@@ -65,9 +80,29 @@ function createSafeItoEnvironment(source = process.env, options = {}) {
return Object.freeze(safe);
}
+function getInvocationCommand(args = []) {
+ return args.filter((value) => value !== "--json")[0];
+}
+
+function createSafeItoInvocationEnvironment(
+ source = process.env,
+ args = [],
+ options = {},
+) {
+ const command = getInvocationCommand(args);
+ return createSafeItoEnvironment(source, {
+ includeControls: options.includeControls === true,
+ includeItoRuntime: ITO_RUNTIME_COMMANDS.has(command),
+ includeItoEvals: command === "evals",
+ });
+}
+
module.exports = Object.freeze({
ECC_ITO_CONTROL_KEYS,
+ ITO_EVAL_ENVIRONMENT_KEYS,
ITO_RUNTIME_ENVIRONMENT_KEYS,
SYSTEM_ENVIRONMENT_KEYS,
createSafeItoEnvironment,
+ createSafeItoInvocationEnvironment,
+ getInvocationCommand,
});
diff --git a/skills/ito-compute/SKILL.md b/skills/ito-compute/SKILL.md
index 11181ad14..657f85f4e 100644
--- a/skills/ito-compute/SKILL.md
+++ b/skills/ito-compute/SKILL.md
@@ -1,6 +1,6 @@
---
name: ito-compute
-description: Query live GPU inventory, submit an authenticated Itô fixed-rate RFQ, and inspect RFQ or procurement status through the separately installed canonical CLI or its exact MCP tools. Use when a user asks to find H100/H200 capacity, request a fixed compute rate, or check Itô compute status.
+description: Query live GPU inventory, submit an authenticated Itô fixed-rate RFQ, inspect RFQ or procurement status, and run explicitly gated node qualification through the separately installed canonical CLI. Use when a user asks to find H100/H200 capacity, request a fixed compute rate, check Itô compute status, or validate GPU nodes.
metadata:
origin: ECC
---
@@ -67,6 +67,32 @@ Never put it in arguments, tracked files, MCP results, logs, or chat.
Inventory prices are indicative. An RFQ is not reserved capacity. Treat a rate
as fixed only when the canonical result contains a non-null firm quote.
+## Live node qualification
+
+`ecc ito evals` exposes the canonical CLI's narrow live adapter to a separately
+installed `sixtytwo-cli==0.3.33`. It does not expose local fixture execution
+through ECC.
+Require all of the following before invoking it:
+
+- operator authorization to contact the named nodes;
+- `ITO_ENABLE_SIXTYTWO_LIVE=1`;
+- `--live-sixtytwo`;
+- an explicit node list; and
+- an existing absolute config directory containing `sixtytwo.yaml`.
+
+```sh
+ecc ito evals \
+ --cluster clu_prod_example \
+ --live-sixtytwo \
+ --nodes gpu-01,gpu-02 \
+ --config-dir /absolute/path/to/qualification-config
+```
+
+The canonical adapter can run only the pinned version check and
+`sixtytwo test --full` against the explicit nodes. It cannot rent, launch,
+recover, repair, reset, purchase, or order resources. ECC does not forward
+`ITO_API_KEY` or model/cloud credentials into node qualification.
+
## MCP workflow
Build the canonical package, then configure the stdio server with an absolute
@@ -97,7 +123,7 @@ Use `ito_auth`, gather explicit buyer authority and every hard constraint, call
## Unsupported operations
The supported client surface cannot lock quotes, reserve capacity, execute
-workloads, qualify nodes through ECC, or serve inference. Do not invent
-additional tools or a purchase path. Do not substitute a browser or fixture
-when the local CLI is missing or a live operation fails. Report the missing
-capability and stop.
+workloads, or serve inference. The MCP server does not expose qualification;
+use the explicit CLI command above. Do not invent additional tools or a
+purchase path. Do not substitute a browser or fixture when the local CLI is
+missing or a live operation fails. Report the missing capability and stop.
diff --git a/tests/ci/ito-compute-skill.test.js b/tests/ci/ito-compute-skill.test.js
index 4d4d283a9..10c33a117 100644
--- a/tests/ci/ito-compute-skill.test.js
+++ b/tests/ci/ito-compute-skill.test.js
@@ -34,10 +34,15 @@ function main() {
const tests = [
["documents only the real CLI commands and MCP tools", () => {
const skill = read("skills/ito-compute/SKILL.md");
- for (const command of ["ecc ito auth", "ecc ito find", "ecc ito status"]) {
+ for (const command of [
+ "ecc ito auth",
+ "ecc ito find",
+ "ecc ito status",
+ "ecc ito evals",
+ ]) {
assert.match(skill, new RegExp(command.replace(" ", "\\s+")));
}
- assert.doesNotMatch(skill, /^\s*ito (?:auth|find|status)\b/m);
+ assert.doesNotMatch(skill, /^\s*ito (?:auth|find|status|evals)\b/m);
for (const tool of ["ito_auth", "ito_find", "ito_status"]) {
assert.match(skill, new RegExp(`\\b${tool}\\b`));
}
@@ -52,6 +57,10 @@ function main() {
assert.match(skill, /ECC_ITO_CLI_EXECUTABLE/);
assert.match(skill, /explicit absolute built entry/);
assert.match(skill, /never discovers[^\n]*through `PATH`/);
+ assert.match(skill, /ITO_ENABLE_SIXTYTWO_LIVE/);
+ assert.match(skill, /sixtytwo-cli==0\.3\.33/);
+ assert.match(skill, /explicit node/i);
+ assert.match(skill, /cannot (?:rent|launch|recover|repair)/i);
assert.doesNotMatch(skill, /npm link/);
}],
["registers one opt-in install module and capability", () => {
@@ -72,7 +81,7 @@ function main() {
{
id: "capability:ito-compute",
family: "capability",
- description: "Authenticated Itô GPU inventory, RFQ, and status workflows through the separately installed canonical CLI.",
+ description: "Authenticated Itô GPU inventory, RFQ, status, and explicitly gated node-qualification workflows through the separately installed canonical CLI.",
modules: ["ito-compute"],
}
);
diff --git a/tests/scripts/ito-cli-bridge.test.js b/tests/scripts/ito-cli-bridge.test.js
index ef896290e..37e2ce1a0 100644
--- a/tests/scripts/ito-cli-bridge.test.js
+++ b/tests/scripts/ito-cli-bridge.test.js
@@ -13,7 +13,15 @@ const { spawnSync } = require("child_process");
const REPO_ROOT = path.join(__dirname, "..", "..");
const ECC_SCRIPT = path.join(REPO_ROOT, "scripts", "ecc.js");
+const ITO_SCRIPT = path.join(REPO_ROOT, "scripts", "ito.js");
const CANONICAL_PACKAGE = "Ito-Markets/ito-cloud-runtime/cli/ito-compute-cli";
+const {
+ NODE_QUALIFICATION_TIMEOUT_MS,
+} = require("../../scripts/ito");
+const {
+ createSafeItoInvocationEnvironment,
+ getInvocationCommand,
+} = require("../../scripts/lib/ito-environment");
function runCli(args, environment = {}) {
return spawnSync(process.execPath, [ECC_SCRIPT, ...args], {
@@ -80,7 +88,7 @@ function main() {
console.log("\n=== Testing ECC × Itô real CLI bridge ===\n");
const tests = [
- ["forwards only auth, find, and status to an explicit local executable", () => {
+ ["forwards only the reviewed RFQ CLI surface to an explicit local executable", () => {
for (const command of ["auth", "find", "status"]) {
const probe = makeItoProbe();
try {
@@ -152,15 +160,192 @@ function main() {
fs.rmSync(probe.directory, { recursive: true, force: true });
}
}],
- ["rejects unsupported, browser, simulated, and node operations before spawning", () => {
- for (const command of ["rent", "lock", "run", "inference", "evals", "mcp"]) {
+ ["isolates live node qualification from Itô and unrelated credentials", () => {
+ const probe = makeItoProbe();
+ try {
+ const configDirectory = path.join(probe.directory, "qualification");
+ fs.mkdirSync(configDirectory);
+ fs.writeFileSync(path.join(configDirectory, "sixtytwo.yaml"), "suite: full\n");
+ const result = runCli([
+ "ito",
+ "evals",
+ "--cluster", "clu_prod_example",
+ "--live-sixtytwo",
+ "--nodes", "gpu-01,gpu-02",
+ "--config-dir", configDirectory,
+ ], {
+ ECC_ITO_CLI_EXECUTABLE: probe.executable,
+ ITO_API_KEY: "must-not-cross-into-node-qualification",
+ ITO_API_URL: "https://compute.example.test",
+ ITO_INVENTORY_URL: "https://edge.example.test",
+ ITO_ENABLE_SIXTYTWO_LIVE: "1",
+ SIXTYTWO_API_TOKEN: "sixtytwo-test-token",
+ SIXTYTWO_TOKEN: "sixtytwo-legacy-test-token",
+ SSH_AUTH_SOCK: "/tmp/ecc-test-agent.sock",
+ ITO_CLI_DEMO: "1",
+ ITO_CLI_STATE_DIR: "/tmp/forbidden-paper-state",
+ AWS_SECRET_ACCESS_KEY: "must-not-cross",
+ OPENAI_API_KEY: "must-not-cross",
+ });
+ assert.strictEqual(result.status, 0, result.stderr);
+ const invocation = readInvocation(probe);
+ assert.deepStrictEqual(invocation.argv, [
+ "evals",
+ "--cluster", "clu_prod_example",
+ "--live-sixtytwo",
+ "--nodes", "gpu-01,gpu-02",
+ "--config-dir", configDirectory,
+ ]);
+ assert.strictEqual(invocation.env.ITO_ENABLE_SIXTYTWO_LIVE, "1");
+ assert.strictEqual(invocation.env.SIXTYTWO_API_TOKEN, "sixtytwo-test-token");
+ assert.strictEqual(invocation.env.SIXTYTWO_TOKEN, "sixtytwo-legacy-test-token");
+ assert.strictEqual(invocation.env.SSH_AUTH_SOCK, "/tmp/ecc-test-agent.sock");
+ assert.strictEqual(invocation.env.ITO_API_KEY, undefined);
+ assert.strictEqual(invocation.env.ITO_API_URL, undefined);
+ assert.strictEqual(invocation.env.ITO_INVENTORY_URL, undefined);
+ assert.strictEqual(invocation.env.ITO_CLI_DEMO, undefined);
+ assert.strictEqual(invocation.env.ITO_CLI_STATE_DIR, undefined);
+ assert.strictEqual(invocation.env.AWS_SECRET_ACCESS_KEY, undefined);
+ assert.strictEqual(invocation.env.OPENAI_API_KEY, undefined);
+ } finally {
+ fs.rmSync(probe.directory, { recursive: true, force: true });
+ }
+ }],
+ ["rejects every incomplete live qualification before spawning", () => {
+ const validArgs = [
+ "ito",
+ "evals",
+ "--cluster", "clu_prod_example",
+ "--live-sixtytwo",
+ "--nodes", "gpu-01,gpu-02",
+ ];
+ const cases = [
+ {
+ label: "missing environment opt-in",
+ args: [...validArgs, "--config-dir", "__CONFIG__"],
+ env: {},
+ error: /ITO_ENABLE_SIXTYTWO_LIVE=1/,
+ },
+ {
+ label: "missing live flag",
+ args: validArgs.filter((value) => value !== "--live-sixtytwo")
+ .concat("--config-dir", "__CONFIG__"),
+ env: { ITO_ENABLE_SIXTYTWO_LIVE: "1" },
+ error: /--live-sixtytwo/,
+ },
+ {
+ label: "missing nodes",
+ args: [
+ "ito", "evals",
+ "--cluster", "clu_prod_example",
+ "--live-sixtytwo",
+ "--config-dir", "__CONFIG__",
+ ],
+ env: { ITO_ENABLE_SIXTYTWO_LIVE: "1" },
+ error: /--nodes/,
+ },
+ {
+ label: "empty node list",
+ args: [
+ "ito", "evals",
+ "--cluster", "clu_prod_example",
+ "--live-sixtytwo",
+ "--nodes", ",",
+ "--config-dir", "__CONFIG__",
+ ],
+ env: { ITO_ENABLE_SIXTYTWO_LIVE: "1" },
+ error: /--nodes/,
+ },
+ {
+ label: "missing cluster",
+ args: [
+ "ito", "evals",
+ "--live-sixtytwo",
+ "--nodes", "gpu-01",
+ "--config-dir", "__CONFIG__",
+ ],
+ env: { ITO_ENABLE_SIXTYTWO_LIVE: "1" },
+ error: /--cluster/,
+ },
+ {
+ label: "relative config directory",
+ args: [...validArgs, "--config-dir", "relative/config"],
+ env: { ITO_ENABLE_SIXTYTWO_LIVE: "1" },
+ error: /absolute/,
+ },
+ {
+ label: "missing config directory",
+ args: [...validArgs, "--config-dir", "__MISSING_CONFIG__"],
+ env: { ITO_ENABLE_SIXTYTWO_LIVE: "1" },
+ error: /sixtytwo\.yaml/,
+ },
+ ];
+
+ for (const testCase of cases) {
+ const probe = makeItoProbe();
+ try {
+ const configDirectory = path.join(probe.directory, "qualification");
+ fs.mkdirSync(configDirectory);
+ fs.writeFileSync(path.join(configDirectory, "sixtytwo.yaml"), "suite: full\n");
+ const args = testCase.args.map((value) => (
+ value === "__CONFIG__"
+ ? configDirectory
+ : value === "__MISSING_CONFIG__"
+ ? path.join(probe.directory, "missing")
+ : value
+ ));
+ const result = runCli(args, {
+ ECC_ITO_CLI_EXECUTABLE: probe.executable,
+ ...testCase.env,
+ });
+ assert.notStrictEqual(result.status, 0, testCase.label);
+ assert.match(result.stderr, testCase.error, testCase.label);
+ assert.ok(
+ !fs.existsSync(probe.log),
+ `${testCase.label} must not spawn the canonical Itô CLI`,
+ );
+ } finally {
+ fs.rmSync(probe.directory, { recursive: true, force: true });
+ }
+ }
+ }],
+ ["classifies Itō child environments once and fails closed on unknown prefixes", () => {
+ const safe = createSafeItoInvocationEnvironment(
+ {
+ PATH: process.env.PATH,
+ ECC_ITO_CLI_EXECUTABLE: "/operator/canonical/ito.js",
+ ITO_API_KEY: "must-not-cross",
+ SIXTYTWO_TOKEN: "must-not-cross",
+ },
+ ["--future-ecc-flag", "evals"],
+ { includeControls: true },
+ );
+ assert.strictEqual(safe.ECC_ITO_CLI_EXECUTABLE, "/operator/canonical/ito.js");
+ assert.strictEqual(safe.ITO_API_KEY, undefined);
+ assert.strictEqual(safe.SIXTYTWO_TOKEN, undefined);
+ }],
+ ["detects the Itō command consistently with or without the global JSON flag", () => {
+ assert.strictEqual(getInvocationCommand(["auth"]), "auth");
+ assert.strictEqual(getInvocationCommand(["--json", "evals"]), "evals");
+ assert.strictEqual(getInvocationCommand([]), undefined);
+ }],
+ ["bounds the outer node-qualification process beyond the canonical timeout", () => {
+ assert.strictEqual(NODE_QUALIFICATION_TIMEOUT_MS, 31 * 60 * 1000);
+ const source = fs.readFileSync(ITO_SCRIPT, "utf8");
+ assert.match(
+ source,
+ /timeout: isNodeQualification \? NODE_QUALIFICATION_TIMEOUT_MS : undefined/,
+ );
+ }],
+ ["rejects unsupported browser, paper, and execution operations before spawning", () => {
+ for (const command of ["rent", "lock", "run", "inference", "mcp"]) {
const probe = makeItoProbe();
try {
const result = runCli(["ito", command], {
ECC_ITO_CLI_EXECUTABLE: probe.executable,
});
assert.notStrictEqual(result.status, 0, command);
- assert.match(result.stderr, /only auth, find, and status/i);
+ assert.match(result.stderr, /only auth, find, status, and evals/i);
assert.ok(!fs.existsSync(probe.log), `${command} must not spawn the Itô CLI`);
} finally {
fs.rmSync(probe.directory, { recursive: true, force: true });
@@ -335,6 +520,8 @@ function main() {
assert.match(result.stdout, /ecc ito auth/);
assert.match(result.stdout, /ecc ito find/);
assert.match(result.stdout, /ecc ito status/);
+ assert.match(result.stdout, /ecc ito evals/);
+ assert.match(result.stdout, /sixtytwo/i);
assert.match(result.stdout, /ito_auth/);
assert.match(result.stdout, /ito_find/);
assert.match(result.stdout, /ito_status/);
diff --git a/tests/scripts/ito-compute-sponsor.test.js b/tests/scripts/ito-compute-sponsor.test.js
index ecdd3967b..34e41accf 100644
--- a/tests/scripts/ito-compute-sponsor.test.js
+++ b/tests/scripts/ito-compute-sponsor.test.js
@@ -142,8 +142,10 @@ function main() {
assert.match(record, /-> any open-source model/);
assert.doesNotMatch(record, /public Kimi|Moonshot|video and sponsorship/i);
assert.match(record, /Status: \*\*Implemented local CLI bridge/i);
- assert.match(record, /auth`, `find`, and `status/);
+ assert.match(record, /auth`, `find`, `status`, and `evals/);
assert.match(record, /ito_auth`, `ito_find`, and `ito_status/);
+ assert.match(record, /sixtytwo-cli==0\.3\.33/);
+ assert.match(record, /explicit node/i);
assert.match(record, /unpublished/i);
assert.match(record, /managed inference remains unavailable/i);
assert.match(record, /version bump[\s\S]*intentionally deferred/i);