HarnessRouter Community Edition: the self-hosted, Apache-2.0 edition of the unified interface for agent harnesses. Run Codex, Claude Code, Hermes, PI, DSH, and more through one API, with sessions, streaming, files, cancellation, and failure handling. Implements the Unified Harness Protocol (UHP), an open standard. Your keys, your infrastructure.
Python agent-apiagent-frameworkagent-harnessai-agentsai-infrastructurecoding-agentconformance-testingdeepseek-harnessharness-engineeringhermesllmopen-protocolopen-sourceopen-standardpi-coding-agentself-hosteduhpunified-apiunified-harness-protocolunified-interface
Hermes: a tool's own error answer is not a strike against its MCP server (#410)
github-actions[bot]ee4a9dd25d6e
diff --git a/docker/entrypoint.sh b/docker/entrypoint.shindex 2d9be85a..18723470 100644--- a/docker/entrypoint.sh+++ b/docker/entrypoint.sh@@ -1135,7 +1135,7 @@ install_backends() {# Where the runner finds the MCP extension to mount (-e) on pi turns with MCP servers.[ -d "$TOOLS/lib/node_modules/pi-mcp-adapter" ] && export HR_PI_MCP_EXT="$TOOLS/lib/node_modules/pi-mcp-adapter"[ -x "$TOOLS/dsh-venv/bin/python" ] && export HR_DSH_PYTHON="$TOOLS/dsh-venv/bin/python"- if wanted hermes; then verify_hermes_mcp; fi+ if wanted hermes; then verify_hermes_mcp; verify_hermes_breaker; fi}# hermes 0.19.0 gates HTTP MCP on importing `streamablehttp_client`, the name the mcp SDK@@ -1164,6 +1164,30 @@ verify_hermes_bedrock() {fi}+# hermes 0.19.0 counts a tool's own error answer ("no such document") as its MCP server failing:+# three in a row and every tool of that server is refused for 60 s as "unreachable", so an agent+# that guessed three ids wrong reports the service down (runner/patches/hermes_mcp_breaker.py).+# Hermes is installed here, on first start, not in the image, so the patch is applied here too, on+# every start: a fresh install gets it, and so does a volume installed before it existed. It is+# idempotent and writes nothing unless the result compiles; a hermes whose source is not the one it+# was written for is left as it is, with one line saying so. Always returns 0: as the last command+# of a function under `set -e`, anything else would abort the entrypoint (see the note above).+verify_hermes_breaker() {+ [ -x "$TOOLS/venv/bin/python" ] || return 0+ _hmt=$("$TOOLS/venv/bin/python" -c "import importlib.metadata as m; d = m.distribution('hermes-agent'); print(next(str(d.locate_file(f)) for f in d.files if str(f).endswith('tools/mcp_tool.py')))" 2>/dev/null) || _hmt=""+ if [ -z "$_hmt" ] || [ ! -f "$_hmt" ]; then+ echo "[harnessrouter] WARNING: Hermes's MCP module was not found; a tool's own errors may be counted as its server failing"+ return 0+ fi+ _out=$("$TOOLS/venv/bin/python" /app/runner/patches/hermes_mcp_breaker.py "$_hmt" 2>&1) || true+ case "$_out" in+ *"already applied"*) : ;;+ *": applied"*) echo "[harnessrouter] Hermes: a tool's own error answer no longer counts against its MCP server" ;;+ *) echo "[harnessrouter] WARNING: $_out" ;;+ esac+ return 0+}+verify_hermes_mcp() {[ -x "$TOOLS/venv/bin/python" ] || return 0"$TOOLS/venv/bin/python" - <<'PYEOF' && return 0diff --git a/docs/support-matrix-notes.md b/docs/support-matrix-notes.mdindex 7ed9c4c1..1afab950 100644--- a/docs/support-matrix-notes.md+++ b/docs/support-matrix-notes.md@@ -3093,3 +3093,31 @@ Candidate `0.31.3-rc.1` in a side container on the test VM, on a copy of an inst| No media tools, no image credential (pi) | present | ran the skill and answered "CANNOT image generation not configured" |The image credential was removed by switching the instance's image models off on the Integrations document. Not reproduced here: the agent giving up in front of a working media tool. That was seen on the hosted service with a pi agent; Claude Code on this harness chose the media tool either way. What is shown here is that the skill is no longer offered in that state. Claude Code tasks complete, and all plugin checks pass on Claude Code, Codex and pi (the skill checks among them).+++## Hermes: a refusal from a tool is not its server failing (2026-10-06, 0.31.4)++`hermes-agent` 0.19.0 keeps a circuit breaker per MCP server and bumps it on every error answer,+including one a live server gave on purpose: a tool result marked `isError`, "no such document".+Three in a row open the breaker, and for 60 s every tool of that server is refused with "MCP server+... is unreachable after 3 consecutive failures". On the hosted service an agent that had read three+wrong ids could then not use any tool of that server and told its person the service was down.++The patch (`runner/patches/hermes_mcp_breaker.py`, the hosted one) marks the error answers that came+from the server's own tool result and resets the breaker for those. Hermes is installed on an+instance's first start, not in the image, so the entrypoint applies it to the installed copy on every+start, which also repairs a volume installed before it existed.++A small tool server was run inside the side container, with one tool that refuses on purpose (`get_document` of an id that does not exist is an error answer) and one that always answers (`list_documents`). One Hermes task with gpt-5.4: four refused `get_document` calls, then `list_documents`.++| The instance | What `list_documents` returned |+|---|---|+| Published 0.31.3 | "MCP server 'company' is unreachable after 3 consecutive failures. Auto-retry available in ~50s." The server was up and answering. |+| Candidate `0.31.4-rc.1`, started on the volume 0.31.3 had installed Hermes into | `["doc-1"]` |+| Candidate, on a fresh volume (Hermes installed on that first start) | `["doc-1"]` |++On both candidate starts the log has one line, "Hermes: a tool's own error answer no longer counts against its MCP server", and Hermes's module holds the two marks. A restart applied nothing again and logged nothing. A plain Hermes task completes. An instance started without Hermes starts, logs nothing about it, and passes the usual checks (pi and Claude Code tasks, all plugin checks on Claude Code, Codex and pi).++The patch was run against the real `hermes-agent` 0.19.0 file: it changes two places and the result compiles (the test for this needs the file at hand and is skipped in CI, since Hermes is not in the image).++Not shown live: a server that cannot be reached still opening the breaker. When the tool server was stopped mid-task Hermes dropped its tools ("Unknown tool") before the breaker came into it. That half is covered by the test of the patched check, where an error that did not come from a tool's own answer still counts.diff --git a/runner/patches/hermes_mcp_breaker.py b/runner/patches/hermes_mcp_breaker.pynew file mode 100644index 00000000..ce029620--- /dev/null+++ b/runner/patches/hermes_mcp_breaker.py@@ -0,0 +1,100 @@+"""Hermes counts a tool's own error answer as its MCP server failing.++hermes-agent 0.19.0 (tools/mcp_tool.py) bumps the per-server circuit breaker whenever a call's+answer is an error, including an answer a live server gave on purpose (a tool result marked+isError: "no such item"). Three in a row open the breaker, and for 60 s every tool of that server+is refused with "MCP server ... is unreachable": an agent that had read three wrong document ids+could then not comment on a task, and told its person the service was down (seen on the hosted+service, 2026-10-06, and fixed there the same way).++This marks the error answers that came back from the server's own tool result, and lets the+breaker count only failures to reach the server (no session, a transport error, a timeout), which+it already counts where they happen. The text the model reads is unchanged.++Hermes is not in this image (see the Dockerfile's licence note): it is installed on first start,+so the entrypoint applies this to the installed copy on every start, which also repairs a volume+installed before this existed. A source that is not the bytes this was written against is left+alone and said so; nothing is written unless the result compiles.++Usage: python hermes_mcp_breaker.py <path of tools/mcp_tool.py>+Exit: 0 applied or already applied, 3 the source is not the one this was written for, 4 the result+does not compile under the Python it was run with. Nothing is written on 3 or 4.+"""+import sys++MARK = "_hr_tool_error"++TOOL_ERROR = ''' return json.dumps({+ "error": _sanitize_error(+ error_text or "MCP tool returned an error"+ )+ }, ensure_ascii=False)+'''+TOOL_ERROR_NEW = ''' return json.dumps({+ "error": _sanitize_error(+ error_text or "MCP tool returned an error"+ ),+ "''' + MARK + '''": True,+ }, ensure_ascii=False)+'''++AFTER_CALL = ''' # Check if the MCP tool itself returned an error+ try:+ parsed = json.loads(result)+ if "error" in parsed:+ _bump_server_error(server_name)+ else:+ _reset_server_error(server_name) # success — reset+ except (json.JSONDecodeError, TypeError):+ _reset_server_error(server_name) # non-JSON = success+ return result+'''+AFTER_CALL_NEW = ''' # Check if the MCP tool itself returned an error+ try:+ parsed = json.loads(result)+ if isinstance(parsed, dict) and parsed.pop("''' + MARK + '''", False):+ # the server answered: its tool refused this call, the server is not failing+ _reset_server_error(server_name)+ return json.dumps(parsed, ensure_ascii=False)+ if "error" in parsed:+ _bump_server_error(server_name)+ else:+ _reset_server_error(server_name) # success — reset+ except (json.JSONDecodeError, TypeError):+ _reset_server_error(server_name) # non-JSON = success+ return result+'''+++class Moved(Exception):+ """The installed source is not the one this patch was written for."""+++def patch(src: str) -> str:+ for old in (TOOL_ERROR, AFTER_CALL):+ if src.count(old) != 1:+ raise Moved(f"{src.count(old)} matches of:\n{old}")+ return src.replace(TOOL_ERROR, TOOL_ERROR_NEW).replace(AFTER_CALL, AFTER_CALL_NEW)+++def apply(path: str) -> str:+ """'applied', 'already applied', or raises Moved. Nothing is written unless the result compiles."""+ src = open(path, encoding="utf-8").read()+ if MARK in src:+ return "already applied"+ out = patch(src)+ compile(out, path, "exec")+ open(path, "w", encoding="utf-8").write(out)+ return "applied"+++if __name__ == "__main__":+ try:+ print("hermes mcp breaker patch:", apply(sys.argv[1]))+ except Moved as e:+ print(f"hermes mcp breaker patch: NOT applied, the installed source is not the one it was written for ({str(e).splitlines()[0]})")+ sys.exit(3)+ except SyntaxError as e:+ # run with the interpreter Hermes itself runs on: its source needs Python 3.10 or later+ print(f"hermes mcp breaker patch: NOT applied, the result does not compile under this Python (line {e.lineno}: {e.msg})")+ sys.exit(4)diff --git a/runner/tests/test_hermes_mcp_breaker.py b/runner/tests/test_hermes_mcp_breaker.pynew file mode 100644index 00000000..c1db4993--- /dev/null+++ b/runner/tests/test_hermes_mcp_breaker.py@@ -0,0 +1,91 @@+"""The Hermes patch: a tool's own error answer is not a strike against its MCP server.++Exercised on the exact hermes-agent 0.19.0 source when it is at hand (HR_HERMES_MCP_TOOL names the+file). Hermes is installed on an instance's first start, so the entrypoint applies the patch there.+"""+import json+import os+import sys+from pathlib import Path++import pytest++ROOT = Path(__file__).resolve().parents[2]+sys.path.insert(0, str(ROOT / "runner" / "patches"))+import hermes_mcp_breaker as P # noqa: E402++SRC = "header\n" + P.TOOL_ERROR + "middle\n" + P.AFTER_CALL + "footer = 1\n"+++def test_the_patch_marks_a_tool_refusal_and_the_breaker_lets_it_pass():+ out = P.patch(SRC)+ assert P.TOOL_ERROR_NEW in out and P.AFTER_CALL_NEW in out+ # run the patched post-call check on the two kinds of error answer+ ns = {"json": json}+ strikes = {"n": 0}+ ns["_bump_server_error"] = lambda s: strikes.__setitem__("n", strikes["n"] + 1)+ ns["_reset_server_error"] = lambda s: strikes.__setitem__("n", 0)+ body = "def after(result, server_name):\n" + "\n".join(line[8:] for line in P.AFTER_CALL_NEW.splitlines())+ exec(body, ns)+ refused = json.dumps({"error": "no such item: x", P.MARK: True})+ for _ in range(5):+ shown = ns["after"](refused, "company")+ assert strikes["n"] == 0 and json.loads(shown) == {"error": "no such item: x"} # the model reads the same words+ for n in (1, 2, 3):+ ns["after"](json.dumps({"error": "MCP server 'company' is not connected"}), "company")+ assert strikes["n"] == n # a failure to reach it still counts+ ns["after"](json.dumps({"result": "ok"}), "company")+ assert strikes["n"] == 0+++def test_a_source_that_moved_is_refused_and_nothing_is_written(tmp_path):+ with pytest.raises(P.Moved):+ P.patch("nothing like hermes")+ f = tmp_path / "mcp_tool.py"+ f.write_text("nothing like hermes\n")+ with pytest.raises(P.Moved):+ P.apply(str(f))+ assert f.read_text() == "nothing like hermes\n"+++def test_applying_twice_changes_nothing_the_second_time(tmp_path):+ # the stand-in has to compile, as the real file does: the two blocks sit inside a function+ f = tmp_path / "mcp_tool.py"+ f.write_text("import json\n\n\ndef call(result, server_name, error_text, _sanitize_error, _bump_server_error, _reset_server_error):\n"+ " if True:\n if True:\n if error_text:\n" + P.TOOL_ERROR + P.AFTER_CALL)+ assert P.apply(str(f)) == "applied"+ once = f.read_text()+ assert once.count(P.MARK) == 2+ assert P.apply(str(f)) == "already applied" and f.read_text() == once+++def test_a_result_that_does_not_compile_is_not_written(tmp_path):+ f = tmp_path / "mcp_tool.py"+ broken = "def call(:\n" + P.TOOL_ERROR + P.AFTER_CALL+ f.write_text(broken)+ with pytest.raises(SyntaxError):+ P.apply(str(f))+ assert f.read_text() == broken+ import subprocess+ r = subprocess.run([sys.executable, str(ROOT / "runner" / "patches" / "hermes_mcp_breaker.py"), str(f)], capture_output=True, text=True)+ assert r.returncode == 4 and "NOT applied" in r.stdout and "Traceback" not in r.stderr and f.read_text() == broken+ f.write_text("x = 1\n")+ r = subprocess.run([sys.executable, str(ROOT / "runner" / "patches" / "hermes_mcp_breaker.py"), str(f)], capture_output=True, text=True)+ assert r.returncode == 3 and "NOT applied" in r.stdout and f.read_text() == "x = 1\n"+++def test_the_entrypoint_applies_it_on_every_start_and_can_never_stop_the_start():+ sh = (ROOT / "docker" / "entrypoint.sh").read_text()+ assert "if wanted hermes; then verify_hermes_mcp; verify_hermes_breaker; fi" in sh+ body = sh[sh.index("verify_hermes_breaker() {"):]+ body = body[:body.index("\n}\n")]+ assert "/app/runner/patches/hermes_mcp_breaker.py" in body and body.rstrip().endswith("return 0")+ assert "|| true" in body # the patch's own exit status is read from its words, not trusted to `set -e`++[email protected](not os.environ.get("HR_HERMES_MCP_TOOL"), reason="the hermes source is not at hand")+def test_the_real_file_takes_the_patch(tmp_path):+ src = Path(os.environ["HR_HERMES_MCP_TOOL"]).read_text()+ out = P.patch(src)+ assert out.count(P.MARK) == 2+ compile(out, str(tmp_path / "mcp_tool.py"), "exec")