Skip to content

HarnessRouter Community Edition: the self-hosted, Apache-2.0 edition of the unified interface for agent harnesses. Run Codex, Claude Code, Hermes, PI, DSH, and more through one API, with sessions, streaming, files, cancellation, and failure handling. Implements the Unified Harness Protocol (UHP), an open standard. Your keys, your infrastructure.

Session workspace: delete, mkdir and move beside the GET/PUT file routes (#478)

Arun Prabhakaran80793b801d46
diff --git a/docs/self-hosting-guide.md b/docs/self-hosting-guide.md
index 19e6e0ab..a4222524 100644
--- a/docs/self-hosting-guide.md
+++ b/docs/self-hosting-guide.md
@@ -481,7 +481,8 @@ the instance, and are then removed.
| What is deleted | What happens |
|---|---|
-| A task (a session) | Removed at once, for good: its conversation and trace, its responses, the files it produced and their previews, its list of changed files, its media, the record of every plugin call it made, its checkpoint and its working folder. A marker that holds its id and the time of the delete, and nothing of its content, stays for a day so that a turn still finishing cannot write it back, then goes too. |
+| A task (a session) | Removed at once, for good: its conversation and trace, its responses, the files it produced and their previews, its list of changed files, its media, the record of every plugin call it made, its checkpoint, the record of the files an app removed from its workspace, and its working folder. A marker that holds its id and the time of the delete, and nothing of its content, stays for a day so that a turn still finishing cannot write it back, then goes too. |
+| A file or folder in a task's workspace (`DELETE /v1/sessions/{session_id}/files/{path}`) | Removed at once from the workspace and from its checkpoint, so no later turn sees it, together with the copy an app saved there. A copy that a turn handed back as one of its outputs stays readable from that turn's response until the task is deleted. |
| An API key | Removed at once. It stops working immediately and leaves the list of keys. |
| A credential | Removed at once: a plugin's stored secrets when the plugin is removed, a database connection when it is removed from a harness or its harness is deleted, a stored tool token deleted with `DELETE /v1/mcp-secrets/{ref}`. |
| A harness | Kept 30 days, then removed with its skill bundles and plugin packages. A restored harness comes back without its credentials, since those went at once. The tasks and responses that used it are kept. |
@@ -1185,6 +1186,7 @@ All paths below are relative to the API base above and use the same Bearer key.
| Stream progress | Set `"stream": true` on the response request for server-sent events |
| Read session history | `GET /v1/sessions/{session_id}/turns` |
| Upload inputs or retrieve outputs | `POST /v1/files`; `GET /v1/sessions/{session_id}/files` |
+| Read or change one workspace file between turns | `GET`, `PUT` or `DELETE /v1/sessions/{session_id}/files/{path}` (`PUT` takes `{"content": "..."}` or `{"content_b64": "..."}`; `DELETE` removes a folder with everything in it); `POST /v1/sessions/{session_id}/files/{path}:mkdir` makes a folder and its parents (`created: false` when it was already there); `POST /v1/sessions/{session_id}/files/{path}:move` with `{"to": "<path>"}` renames a file or folder and never overwrites (409 `file_exists`). The next turn sees each change. All but `GET` answer 409 `session_busy` while a turn runs, and every one refuses a path that leaves the workspace; `DELETE`, `:mkdir` and `:move` also refuse the agent's own files (`.git`, `.harness`, a base's skills and memory) |
| Cancel work | `POST /v1/responses/{response_id}/cancel` |
| Delete | `DELETE /v1/sessions/{session_id}`, `DELETE /v1/responses/{response_id}`, `DELETE /v1/harnesses/{harness_id}`, `DELETE /v1/files/{file_id}`, `DELETE /v1/mcp-secrets/{ref}`; what each removes and when is in [What a delete removes, and when](#what-a-delete-removes-and-when) |
| Describe the API | `GET /v1/openapi.json`, the surface above as an OpenAPI document, readable without a login |
diff --git a/gateway/app.py b/gateway/app.py
index dc0c7672..f35caf67 100644
--- a/gateway/app.py
+++ b/gateway/app.py
@@ -22,6 +22,7 @@ import urllib.parse
import collections
import time
import uuid
+import weakref
import zipfile
import base64
@@ -5270,9 +5271,11 @@ async def _session_purge(sid: str, v: dict | None, index: dict | None = None) ->
await BACKING.graph.delete(rid, "HarnessResponse")
# 3) everything stored under its id: the checkpoint and the media scene (sessions/<sid>/ in
# the session store), the change list (sessions/<sid>/ in the response store), the files its
- # tasks produced (containers/<sid>/) and their rendered previews (previews/<sid>/)
+ # tasks produced (containers/<sid>/), their rendered previews (previews/<sid>/), and the record of
+ # what an app removed from its workspace (ws-removed/<sid>/)
for kb, prefix in ((BLOB_KB, f"sessions/{sid}/"), (RESP_BLOB_KB, f"sessions/{sid}/"),
- (RESP_BLOB_KB, f"containers/{sid}/"), (RESP_BLOB_KB, f"previews/{sid}/")):
+ (RESP_BLOB_KB, f"containers/{sid}/"), (RESP_BLOB_KB, f"previews/{sid}/"),
+ (RESP_BLOB_KB, f"ws-removed/{sid}/")):
with contextlib.suppress(Exception):
for it in await _blob_list_all(prefix, kb=kb):
await _blob_delete(str(it.get("file_id") or ""), kb=kb)
@@ -8261,7 +8264,12 @@ async def _resp_execute(translator: _RespTranslator, *, org: str, member: str, s
await _vertex_upsert(sid, {"status": "running", "turn_status": "starting",
"heartbeat": str(time.time()), "runner_turn_id": "",
**({"trace_blob": tr["prefix"]} if tr.get("prefix") else {})})
- await _hydrate(sid, rec)
+ # Under the session's change lock: an app's PUT, delete, mkdir or move that passed its busy check
+ # before this turn was stamped running finishes before the restore reads the workspace, and one
+ # that comes after waits, reads the session again under the lock, and is refused (_ws_busy_now).
+ # In this process only, like the lock itself.
+ async with _ws_change_lock(sid):
+ await _hydrate(sid, rec)
if rec.get("hydrate_failed_with_checkpoint"):
# The task has a checkpoint and it could not be restored: the turn does not run. Running it
# on the wiped workspace started the conversation over without a word (2026-09-06); an
@@ -9155,7 +9163,8 @@ async def recycle_session_sandbox(sid: str) -> dict:
if {"running", "starting"} & {str(v.get("turn_status") or ""), str(v.get("status") or "")}:
raise HTTPException(409, "a turn is running; recycle after it settles")
rec: dict = {}
- await _hydrate(sid, rec, force=True)
+ async with _ws_change_lock(sid): # as a turn's restore is (_resp_execute)
+ await _hydrate(sid, rec, force=True)
if not rec.get("hydrated"):
raise HTTPException(502, f"workspace restore failed: {_hydrate_reason(rec)}")
return {"session_id": sid, "checkpoint_sha": str(v.get("ws_sha") or ""),
@@ -10573,7 +10582,7 @@ def _ws_visible(path: str) -> bool:
# than keeping a second list in the gateway: `.harness/agent-subdir` names the agent's directory
# and `.git/info/exclude` carries the base's own machinery patterns (git status honours the same
# file, which is what keeps the produced-files list clean).
-_WS_SUBDIR_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
+_WS_SUBDIR_RE = backing.WS_SUBDIR_RE # one rule with the file routes (backing.tar_change)
def _ws_rules(tf) -> tuple[str, tuple[str, ...]]:
@@ -10750,13 +10759,40 @@ async def _newest_captures(sid: str) -> dict[str, tuple[str, bool]]:
NEWEST WINS. Two captures can name the same file — the agent's, taken when a turn ended, and
the app's, taken when somebody saved between turns — and serving the older one hands back work
the person already replaced. Ties keep the previous last-one-wins order, so a pair of legacy
- metas with no stamp resolves exactly as it did before."""
+ metas with no stamp resolves exactly as it did before.
+
+ A DELETE (or the source of a move) leaves a tombstone (_put_tombstone). It hides every capture of
+ that path, and of anything under it when it was a directory, that is not newer than it. Without it the by-path read fell back to the agent's old
+ capture of a file the app had just removed, and the next restore put the app's own copy back."""
+ return (await _capture_state(sid))[0]
+
+
+async def _capture_state(sid: str) -> tuple[dict[str, tuple[str, bool]], dict[str, float]]:
+ """(_newest_captures, the tombstones: path -> when it was removed).
+
+ The captures are the window they always were (a first page of 200 metas). The tombstones are
+ kept under their own prefix and read IN FULL: inside that window a tombstone could fall out of
+ it at random, and the file it hid would come back. There are few of them (one per removed path,
+ and a removed directory drops those under it), so reading all of them is cheap."""
out: dict[str, tuple[str, bool]] = {}
newest: dict[str, float] = {}
+ gate = asyncio.Semaphore(16)
+
+ async def _get(fid: str) -> bytes | None:
+ async with gate:
+ return await _blob_get(fid, kb=RESP_BLOB_KB)
listing = await _blob_list(f"containers/{sid}/", limit=200, kb=RESP_BLOB_KB)
- metas = [i for i in (listing.get("items") or []) if str(i.get("file_id", "")).endswith(".meta")]
- for it in metas[:200]:
- meta_b = await _blob_get(it["file_id"], kb=RESP_BLOB_KB)
+ metas = [i for i in (listing.get("items") or []) if str(i.get("file_id", "")).endswith(".meta")][:200]
+ tomb_ids = [i["file_id"] for i in await _blob_list_all(_tomb_prefix(sid), kb=RESP_BLOB_KB)]
+ bodies = await asyncio.gather(*(_get(it["file_id"]) for it in metas), *(_get(t) for t in tomb_ids))
+ tombs: dict[str, float] = {}
+ for meta_b in bodies[len(metas):]:
+ try:
+ t = json.loads(meta_b or b"")
+ tombs[str(t["filename"])] = max(float(t["removed_at"]), tombs.get(str(t["filename"]), 0.0))
+ except Exception: # noqa: BLE001
+ continue
+ for it, meta_b in zip(metas, bodies[: len(metas)]):
if not meta_b:
continue
try:
@@ -10772,7 +10808,54 @@ async def _newest_captures(sid: str) -> dict[str, tuple[str, bool]]:
newest[str(fname)] = ts
out[str(fname)] = (it["file_id"].rsplit("/", 1)[-1][: -len(".meta")],
meta.get("source") == "app")
- return out
+ if tombs:
+ out = {n: v for n, v in out.items() if not _tombstoned(n, newest[n], tombs)}
+ return out, tombs
+
+
+def _tomb_prefix(sid: str) -> str:
+ return f"ws-removed/{sid}/"
+
+
+def _tomb_key(sid: str, path: str) -> str:
+ return _tomb_prefix(sid) + hashlib.sha256(path.encode()).hexdigest()[:32]
+
+
+def _tombstoned(name: str, ts: float, tombs: dict[str, float]) -> bool:
+ """Was `name` (as of `ts`) removed by one of these tombstones: its own, or a directory's above it?"""
+ return any(ts <= t and (name == p or name.startswith(p + "/")) for p, t in tombs.items())
+
+
+async def _put_tombstone(sid: str, path: str,
+ state: tuple[dict[str, tuple[str, bool]], dict[str, float]] | None = None) -> bool:
+ """Record that an app removed `path` (see _newest_captures) → whether it was stored.
+
+ The app's own copy of the path is dropped only AFTER the store accepted the record: without the
+ record, the agent's older captures of the path would be unhidden and the by-path read would serve
+ the file the app just removed.
+
+ Then it compacts: the app's copies under a removed directory, and the records of paths under it,
+ are hidden by this one and are dropped. The agent's captures stay (a turn's record names them)."""
+ body = json.dumps({"filename": path, "removed_at": time.time()}).encode()
+ try:
+ if not await _blob_put(_tomb_key(sid, path), body, kb=RESP_BLOB_KB):
+ print(f"[workspace] tombstone rejected sid={sid} path={path}", flush=True)
+ return False
+ except Exception as e: # noqa: BLE001
+ print(f"[workspace] tombstone failed sid={sid} path={path}: {e}", flush=True)
+ return False
+ caps, tombs = state or ({}, {})
+ keys = [f"containers/{sid}/{backing.app_capture_id(path)}{x}" for x in ("", ".meta")]
+ for name, (_cid, app_) in caps.items():
+ if app_ and name.startswith(path + "/"):
+ keys += [f"containers/{sid}/{backing.app_capture_id(name)}{x}" for x in ("", ".meta")]
+ keys += [_tomb_key(sid, t) for t in tombs if t.startswith(path + "/")]
+ for key in keys:
+ try:
+ await _blob_delete(key, kb=RESP_BLOB_KB)
+ except Exception: # noqa: BLE001 — housekeeping; the record already hides it
+ pass
+ return True
# Backends whose CLI cannot resume a conversation headless. cline 3.0.60: `--json --id` refuses
@@ -11110,12 +11193,24 @@ async def read_session_file(sid: str, path: str, request: Request) -> Response:
got = await _container_file_bytes(sid, path)
else:
try:
- byname = await _cfile_by_name(sid)
+ caps, tombs = await _capture_state(sid)
except Exception: # noqa: BLE001 — blob store unavailable: keep the honest 404
- byname = {}
- cid = byname.get(path) or byname.get(path.rsplit("/", 1)[-1])
+ caps, tombs = {}, {}
+ byname = {name: c for name, (c, _app) in caps.items()}
+ cid = byname.get(path)
+ if not cid and "/" in path and not _tombstoned(path, float("-inf"), tombs):
+ # the bare-filename match is for captures named before paths were kept; a path an
+ # app removed must not come back through it as some other file of the same name
+ cid = byname.get(path.rsplit("/", 1)[-1])
if cid:
got = await _container_file_bytes(sid, cid)
+ if got is None:
+ # The checkpoint, last. A file the agent moved has no capture under its new name, and
+ # once the live directory is reaped (self-hosted) the checkpoint holds the only copy
+ # there until the next turn restores it. A path an app removed is out of the
+ # checkpoint as well (the delete changed it), so this brings nothing back.
+ with contextlib.suppress(Exception): # blob store unavailable: the honest 404
+ got = await _container_file_bytes(sid, _wf_id(path))
if got is not None:
data, media, _fname = got
return Response(content=data, media_type=media,
@@ -11148,12 +11243,7 @@ async def write_session_file(sid: str, path: str, body: WorkspaceWrite, request:
# this shipped broken: it compared v["org"], but a session vertex carries the owner in
# `tenant`, so every write 404'd on a session the caller could plainly read.
_org, v = await _owned_session(request, sid)
-
- live = {"running", "starting"} & {str(v.get("turn_status") or ""), str(v.get("status") or "")}
- if live:
- raise uhp_error(409, "session_busy",
- "The agent is working on this session. Wait for the turn to finish "
- "before writing to its workspace.")
+ _refuse_if_busy(v)
if body.content_b64 is not None:
try:
@@ -11169,7 +11259,11 @@ async def write_session_file(sid: str, path: str, body: WorkspaceWrite, request:
raise uhp_error(413, "payload_too_large",
f"A workspace file written this way is limited to {_WS_WRITE_MAX} bytes.")
- ok = await BACKING.workspace.write(sid, path, data)
+ async with _ws_change_lock(sid):
+ await _ws_busy_now(sid)
+ ok = await BACKING.workspace.write(sid, path, data)
+ if ok:
+ _ws_changed(sid)
if not ok:
# The only ways to get here are a path that tried to leave the workspace and a storage
# failure. Both are the caller's problem to see, not something to swallow.
@@ -11178,6 +11272,202 @@ async def write_session_file(sid: str, path: str, body: WorkspaceWrite, request:
return {"session_id": sid, "path": path, "bytes": len(data), "written": True}
+def _refuse_if_busy(v: dict) -> None:
+ """A workspace change while a turn runs is refused, for every route that changes one (the PUT
+ above says why)."""
+ live = {"running", "starting"} & {str(v.get("turn_status") or ""), str(v.get("status") or "")}
+ if live:
+ raise uhp_error(409, "session_busy",
+ "The agent is working on this session. Wait for the turn to finish "
+ "before writing to its workspace.")
+
+
+async def _ws_busy_now(sid: str) -> None:
+ """_refuse_if_busy again, on the session as it is now, under the change lock. The route's first
+ check read the session before it took the lock. A turn stamped running since then holds that
+ lock while it restores the workspace (_resp_execute), so a change that waited behind it reads it
+ running here and stops, instead of landing under a running turn."""
+ _refuse_if_busy(await _vertex_get(sid) or {})
+
+
+def _changeable(path: str, param: str = "path") -> str:
+ """The workspace path a delete, mkdir or move may touch → itself, normalised; else 400.
+
+ Two rules on top of PUT's. The path must stay inside the workspace, as for PUT. And it must be
+ a path the session SHOWS (_ws_visible, the listing's rule): removing or renaming `.git` or
+ `.harness` would take the session's checkpoint history and the harness's own state with it,
+ which a write cannot do but a recursive delete can."""
+ member = backing._safe_member(path)
+ if not member or not _ws_visible(member):
+ raise uhp_error(400, "invalid_input",
+ "That path must be inside the session workspace, and not one of its "
+ "internal files.", param)
+ return member
+
+
+def _ws_outcome(outcome: str, path: str, to: str = "") -> None:
+ """A backing's WS_* answer → the failure it means, or nothing when it worked. For a move, a clash
+ is at the destination, so `to` names it."""
+ if outcome in (backing.WS_OK, backing.WS_UNCHANGED):
+ return
+ if outcome == backing.WS_NOT_FOUND:
+ raise uhp_error(404, "file_not_found", f"No file or directory '{path}' in this session's workspace.", "path")
+ if outcome == backing.WS_EXISTS:
+ at = to or path
+ raise uhp_error(409, "file_exists", f"Something is already at '{at}', or a file is where one of "
+ "its folders would go.", "to" if to else "path")
+ if outcome == backing.WS_BAD_PATH:
+ if to:
+ # the backing refused the source or the destination, and does not say which
+ raise uhp_error(400, "invalid_input", "Neither the path nor `to` may be one of the agent's "
+ "own files, and a path cannot move inside itself.")
+ raise uhp_error(400, "invalid_input", "That path must be inside the session workspace, and not "
+ "one of the agent's own files.", "path")
+ raise uhp_error(502, "internal_error", "The session workspace could not be changed. Try again.")
+
+
+async def _checkpoint_rewritten(sid: str, sha: str) -> None:
+ """The hosted backing rewrote the session's checkpoint: record its sha as the session's, so a
+ sandbox kept warm with the old workspace is restored rather than trusted (CheckpointWorkspaceFiles
+ ._store says why). It must RAISE when the store refuses: the backing then leaves the archive as it
+ was rather than rewriting it under a sha that still matches the warm sandbox."""
+ await BACKING.graph.upsert("HarnessSession", sid, {"ws_sha": sha}, raise_on_fail=True)
+
+
+if hasattr(BACKING.workspace, "on_rewrite"):
+ BACKING.workspace.on_rewrite = _checkpoint_rewritten
+
+
+def _ws_unrecorded(op: str) -> None:
+ """The tombstone was not stored (_put_tombstone logged why): the change stands, and the caller is
+ told it may not last rather than handed a clean 200. A delete can be repeated (its route records
+ the removal even when the path is already gone); a move's source is gone, so it cannot."""
+ again = " Repeat the delete to record it." if op == "delete" else ""
+ raise uhp_error(502, "internal_error", f"The {op} was made, but could not be recorded, so a later "
+ f"restore of this session could undo it.{again}")
+
+
+def _ws_changed(sid: str) -> None:
+ """What _checkpoint does after the workspace changes: drop this session's cached tarball and
+ file map here and on every replica, so the listing reads the change."""
+ _ws_tar_evict(sid)
+ _WS_FILES_CACHE.pop(sid, None)
+ _bus_publish("_ctrl", "_ctrl", "", sid, "", {"type": "hr.ctrl.ws_invalidate"})
+
+
+class WorkspaceMove(BaseModel):
+ to: str
+
+
+# The action is a suffix on the path (`…/files/notes/old.md:move`), so a path with slashes is still
+# one path parameter: Starlette's {path:path} is `.*` followed by the literal, and backtracks to it.
[email protected]("/v1/sessions/{sid}/files/{path:path}:mkdir")
+async def mkdir_session_file(sid: str, path: str, request: Request) -> dict:
+ """Create a directory in a session's workspace, with any missing parents. One that already
+ exists is success (`created: false`); a FILE at that path, or where one of its parents would
+ be, is 409 `file_exists`."""
+ _org, v = await _owned_session(request, sid)
+ _refuse_if_busy(v)
+ member = _changeable(path)
+ async with _ws_change_lock(sid):
+ await _ws_busy_now(sid)
+ outcome = await BACKING.workspace.mkdir(sid, member)
+ _ws_outcome(outcome, member)
+ _ws_changed(sid)
+ return {"session_id": sid, "path": member, "dir": True, "created": outcome == backing.WS_OK}
+
+
[email protected]("/v1/sessions/{sid}/files/{path:path}:move")
+async def move_session_file(sid: str, path: str, body: WorkspaceMove, request: Request) -> dict:
+ """Rename a file or directory in a session's workspace to `to`. The destination must not exist
+ (409 `file_exists`, never an overwrite) and must not be inside the source."""
+ _org, v = await _owned_session(request, sid)
+ _refuse_if_busy(v)
+ member = _changeable(path)
+ dest = _changeable(body.to, "to")
+ if dest == member or dest.startswith(member + "/"):
+ raise uhp_error(400, "invalid_input", "A path cannot move inside itself.", "to")
+ async with _ws_change_lock(sid):
+ await _ws_busy_now(sid)
+ _ws_outcome(await BACKING.workspace.move(sid, member, dest), member, dest)
+ _ws_changed(sid)
+ state = await _capture_state_or_none(sid)
+ await _carry_captures(sid, member, dest, state)
+ if not await _put_tombstone(sid, member, state):
+ _ws_unrecorded("move")
+ return {"session_id": sid, "path": member, "to": dest, "moved": True}
+
+
[email protected]("/v1/sessions/{sid}/files/{path:path}")
+async def delete_session_file(sid: str, path: str, request: Request) -> dict:
+ """Remove a file, or a directory and everything in it, from a session's workspace."""
+ _org, v = await _owned_session(request, sid)
+ _refuse_if_busy(v)
+ member = _changeable(path)
+ async with _ws_change_lock(sid):
+ await _ws_busy_now(sid)
+ outcome = await BACKING.workspace.delete(sid, member)
+ if outcome == backing.WS_NOT_FOUND:
+ # Nothing to remove, but a durable copy of the path may still answer the by-path read
+ # (a delete whose record the store refused, repeated): record the removal when there is
+ # one to hide, and only then, so a run of deletes of paths that never existed stores
+ # nothing. Recorded, it IS the delete the caller asked to repeat, and it answers so.
+ state = await _capture_state_or_none(sid)
+ if state and any(n == member or n.startswith(member + "/") for n in state[0]):
+ if not await _put_tombstone(sid, member, state):
+ _ws_unrecorded("delete")
+ _ws_changed(sid)
+ return {"session_id": sid, "path": member, "deleted": True}
+ _ws_outcome(outcome, member)
+ _ws_changed(sid)
+ if not await _put_tombstone(sid, member, await _capture_state_or_none(sid)):
+ _ws_unrecorded("delete")
+ return {"session_id": sid, "path": member, "deleted": True}
+
+
+_WS_CHANGE_LOCKS: "weakref.WeakValueDictionary[str, asyncio.Lock]" = weakref.WeakValueDictionary()
+
+
+def _ws_change_lock(sid: str) -> asyncio.Lock:
+ """One PUT, delete, mkdir or move per session at a time on this replica: the change, the copies
+ it carries and the tombstone it writes are read-then-write, and two interleaved would each start
+ from the same state — a PUT racing a delete of its path would have its fresh copy dropped by the
+ delete's tombstone. (The backing serialises the archive rewrite itself the same way.)"""
+ return backing.keyed_lock(_WS_CHANGE_LOCKS, sid)
+
+
+async def _capture_state_or_none(sid: str):
+ try:
+ return await _capture_state(sid)
+ except Exception: # noqa: BLE001 — blob store unavailable: nothing to carry or compact
+ return None
+
+
+async def _carry_captures(sid: str, src: str, dst: str, state=None) -> None:
+ """An app's durable copy of a moved file goes to the new name.
+
+ Self-hosted, that copy can be NEWER than the checkpoint (an app saved between turns), and a
+ restore re-applies it from the capture rather than from the archive. Left under the old name it
+ would be hidden by the move's tombstone, and the restore would bring back the checkpoint's older
+ copy at the new name, losing the app's last save. Hosted, app writes go into the archive and
+ leave no capture, so there is nothing to carry.
+
+ Only the APP's: an agent's capture carried as the app's would be re-applied on every restore,
+ over whatever the agent later did with the file. Once the live directory is reaped, the by-path
+ read finds a moved agent file at its new name in the checkpoint (read_session_file's last
+ fallback)."""
+ caps = (state or ({}, {}))[0]
+ for name, (cid, from_app) in caps.items():
+ if not from_app or not (name == src or name.startswith(src + "/")):
+ continue
+ try:
+ got = await _container_file_bytes(sid, cid)
+ if got:
+ await backing.put_app_capture(BACKING.blob, RESP_BLOB_KB, sid, dst + name[len(src):], got[0])
+ except Exception as e: # noqa: BLE001 — the move is done; the source's tombstone must still land
+ print(f"[workspace] capture not carried sid={sid} {name} -> {dst}: {e}", flush=True)
+
+
@app.get("/v1/containers/{container_id}/files/{file_id}/content")
async def container_file_content(container_id: str, file_id: str, request: Request):
await _owned_session(request, container_id) # V1C02-004: container_id IS the session id
diff --git a/gateway/backing.py b/gateway/backing.py
index 885010b4..96f3e52e 100644
--- a/gateway/backing.py
+++ b/gateway/backing.py
@@ -39,6 +39,7 @@ import sqlite3
import tarfile
import time
import uuid
+import weakref
from pathlib import Path
from typing import Protocol
@@ -100,6 +101,53 @@ class WorkspaceFiles(Protocol):
"""
async def read(self, sid: str, path: str) -> bytes | None: ...
async def write(self, sid: str, path: str, data: bytes) -> bool: ...
+ # The three below answer one of the WS_* outcomes rather than a bool: a route has to tell "nothing
+ # there" (404) from "already there" (409) from a path it must refuse (400) from storage failing.
+ # mkdir answers WS_UNCHANGED for a directory that was already there.
+ async def delete(self, sid: str, path: str) -> str: ...
+ async def mkdir(self, sid: str, path: str) -> str: ...
+ async def move(self, sid: str, path: str, to: str) -> str: ...
+
+
+WS_OK, WS_NOT_FOUND, WS_EXISTS, WS_BAD_PATH, WS_FAILED = "ok", "not_found", "exists", "bad_path", "failed"
+WS_UNCHANGED = "unchanged"
+
+# The name a nesting base gives its agent's directory in `.harness/agent-subdir` (nanobot's
+# `workspace`): ONE plain directory name, or no subdir at all. The gateway's listing reads the same
+# marker with this same rule (app.py, _ws_rules), so the files a change can reach are the files the
+# listing shows.
+WS_SUBDIR_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,63}")
+
+
+def ws_protected(rel: str, pats: tuple[str, ...]) -> bool:
+ """Is `rel` (relative to the session directory) a base's own machinery, as the runner lists it
+ in `.git/info/exclude` — or a directory with some inside it? nanobot's skills and memory,
+ opensquilla's SOUL.md: the listing never shows them, and a delete or a move must not reach them
+ by naming them, or by naming a folder that holds them."""
+ for q in pats:
+ d = q.rstrip("/")
+ if rel == d or (q.endswith("/") and rel.startswith(q)) or d.startswith(rel + "/"):
+ return True
+ return False
+
+
+def keyed_lock(registry: "weakref.WeakValueDictionary[str, asyncio.Lock]", key: str) -> asyncio.Lock:
+ """The asyncio.Lock for `key` in `registry`, made on first use and dropped once nobody holds it.
+ IN THIS PROCESS only: replicas do not share it."""
+ lock = registry.get(key)
+ if lock is None:
+ lock = registry[key] = asyncio.Lock()
+ return lock
+
+
+# One rewrite of a session's checkpoint at a time. Every change reads the whole archive and writes a
+# new one, so two at once would each start from the same archive and the second write would drop the
+# first change while both answered 200.
+_CKPT_LOCKS: "weakref.WeakValueDictionary[str, asyncio.Lock]" = weakref.WeakValueDictionary()
+
+
+def _ckpt_lock(sid: str) -> asyncio.Lock:
+ return keyed_lock(_CKPT_LOCKS, sid)
class Backing:
@@ -463,11 +511,178 @@ def _safe_member(path: str) -> str | None:
Rejected rather than sanitised: a path that needed cleaning was not the path the caller meant,
and silently writing somewhere else is worse than refusing."""
p = (path or "").strip().lstrip("/")
- if not p or p != posixpath.normpath(p) or p.startswith("../") or "\\" in p:
+ # normpath leaves "." and ".." as they are, so they are named here: written into a checkpoint,
+ # a member called ".." is one that tar refuses to unpack, and every later restore would fail
+ if not p or p in (".", "..") or p != posixpath.normpath(p) or p.startswith("../") or "\\" in p:
return None
return p
+def app_capture_id(member: str) -> str:
+ """The durable id of an APP's copy of one workspace path. Derived from the path, so saving the
+ same file again replaces its one blob, and a delete (the gateway's _put_tombstone) can find and
+ drop the app's copy of a path by its name alone."""
+ return "cfile_" + hashlib.sha256(f"app:{member}".encode()).hexdigest()[:32]
+
+
+def _tar_name(m: tarfile.TarInfo) -> str:
+ return m.name[2:] if m.name.startswith("./") else m.name.rstrip("/")
+
+
+def _under(name: str, member: str) -> bool:
+ return name == member or name.startswith(member + "/")
+
+
+def _tar_subdir(tf: tarfile.TarFile) -> str:
+ """The nesting base's agent directory, from the archive's `.harness/agent-subdir`, or ""."""
+ for name in (".harness/agent-subdir", "./.harness/agent-subdir"):
+ try:
+ m = tf.getmember(name)
+ except KeyError:
+ continue
+ f = tf.extractfile(m) if m.isreg() and m.size <= 64 * 1024 else None
+ sub = f.read().decode(errors="replace").strip() if f else ""
+ return sub if WS_SUBDIR_RE.fullmatch(sub) else ""
+ return ""
+
+
+def tar_change(raw: bytes | None, op: str, member: str, to: str = "") -> tuple[str, bytes | None]:
+ """Apply delete, mkdir or move to a checkpoint tarball → (WS_* outcome, the new archive or None
+ when nothing changed).
+
+ A path names a file or a directory, and a directory is everything under it: a tarball made by
+ `tar -C ws .` lists the directory's own entry, but one rewritten by `write` may hold only the
+ files, so both shapes count as the directory existing.
+
+ WHICH paths a change may reach is the listing's rule, read from the same two files the listing
+ reads (`.harness/agent-subdir`, `.git/info/exclude`). The base that nests its workspace (nanobot)
+ keeps the user's files one level down, so a path is looked for there first. The session root is
+ second, for a file an app wrote between turns (it lands there until the next turn adopts it),
+ but never the agent's directory itself: `workspace` at the root IS nanobot's whole working
+ directory, its skills, memory and instruction file with the user's files. A move stays on the
+ side its source was found. Nothing the base excludes as its own machinery is touched.
+
+ Streamed, as `write` is: the member list is read first and each member's bytes are copied
+ straight across on the second pass, so memory is the two compressed archives and one member,
+ never the whole unpacked workspace."""
+ src = tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") if raw else None
+ try:
+ members = src.getmembers() if src else []
+ names = [_tar_name(m) for m in members]
+
+ def _small(name: str) -> str:
+ for m, n in zip(members, names):
+ if n == name and m.isreg() and m.size <= 64 * 1024:
+ f = src.extractfile(m)
+ return f.read().decode(errors="replace") if f else ""
+ return ""
+
+ sub = _small(".harness/agent-subdir").strip()
+ if not WS_SUBDIR_RE.fullmatch(sub):
+ sub = ""
+ pats = tuple(ln.strip() for ln in _small(".git/info/exclude").splitlines()
+ if ln.strip() and not ln.startswith("#"))
+ files = {n for m, n in zip(members, names) if not m.isdir()}
+
+ def _protected(p: str) -> bool:
+ # a path at the root of a nesting base is ADOPTED into the agent's directory next turn,
+ # so it is also checked as it will be there
+ return ws_protected(p, pats) or bool(sub and not _under(p, sub) and ws_protected(f"{sub}/{p}", pats))
+
+ def _exists(p: str) -> bool:
+ return any(_under(n, p) for n in names)
+
+ def _file_above(p: str) -> bool:
+ """A FILE where one of p's directories would be: an archive holding both `notes.txt` and
+ `notes.txt/a.md` cannot be unpacked, and every later restore of the session would fail."""
+ parts = p.split("/")
+ return any("/".join(parts[:k]) in files for k in range(1, len(parts)))
+
+ cands = ([f"{sub}/{member}"] if sub else []) + \
+ ([member] if not sub or member.split("/", 1)[0] != sub else [])
+ present = [c for c in cands if _exists(c)]
+ found = present[0] if present else ""
+ drop, rename, add = (), ("", ""), None
+ if op == "delete":
+ if not found:
+ return WS_NOT_FOUND, None
+ if any(_protected(c) for c in present):
+ return WS_BAD_PATH, None
+ # every copy: one left at the root of a nesting base would be adopted back next turn
+ drop = tuple(present)
+ elif op == "mkdir":
+ # a new folder goes where the runner makes it: in the agent's directory on a nesting base
+ target = found or (f"{sub}/{member}" if sub and member.split("/", 1)[0] != sub else member)
+ if _protected(target):
+ return WS_BAD_PATH, None
+ if found:
+ return (WS_EXISTS, None) if found in files else (WS_UNCHANGED, None)
+ if _file_above(target):
+ return WS_EXISTS, None
+ add = tarfile.TarInfo(target)
+ add.type, add.mode, add.mtime = tarfile.DIRTYPE, 0o755, int(time.time())
+ elif op == "move":
+ if not found:
+ return WS_NOT_FOUND, None
+ dst = f"{sub}/{to}" if sub and found != member else to
+ if _protected(found) or _protected(dst) or _under(dst, found):
+ return WS_BAD_PATH, None
+ # On a nesting base the name can be taken on the OTHER side too: a file an app put at the
+ # root (it waits there for adoption) or the agent's own. Moved in beside it, one of the two
+ # shadows the other for good (adoption never moves over a name the agent's directory has),
+ # which is an overwrite by another name.
+ taken = ([f"{sub}/{to}"] if sub else []) + ([to] if not sub or to.split("/", 1)[0] != sub else [])
+ if _exists(dst) or _file_above(dst) or any(_exists(c) for c in taken):
+ return WS_EXISTS, None
+ if any(_protected(c) for c in present[1:]):
+ return WS_BAD_PATH, None
+ rename = (found, dst)
+ # a second copy (at the root of a nesting base) would be adopted back under the old name
+ drop = tuple(present[1:])
+ else:
+ raise ValueError(op)
+ regular = {n: m for m, n in zip(members, names) if m.isreg()}
+ carried: dict[str, str] = {} # a removed file -> the link that now holds its bytes
+
+ def _renamed(n: str) -> str:
+ return rename[1] + n[len(rename[0]):] if rename[0] and _under(n, rename[0]) else n
+
+ out = io.BytesIO()
+ with tarfile.open(fileobj=out, mode="w:gz") as dst_tar:
+ for m, n in zip(members, names):
+ if any(_under(n, d) for d in drop):
+ continue
+ dot = "./" if m.name.startswith("./") else ""
+ new = _renamed(n)
+ m.name = dot + new
+ data = None
+ if m.islnk():
+ # A hard link is stored as a pointer to an EARLIER member's name. One left pointing
+ # at a removed or renamed member makes an archive no restore can unpack.
+ ldot = "./" if m.linkname.startswith("./") else ""
+ target = m.linkname[2:] if ldot else m.linkname
+ if any(_under(target, d) for d in drop):
+ if target in carried:
+ m.linkname = ldot + carried[target]
+ elif target in regular:
+ data = src.extractfile(m) # resolves to the target's bytes
+ m.type, m.linkname, m.size = tarfile.REGTYPE, "", regular[target].size
+ carried[target] = new
+ else:
+ continue
+ else:
+ m.linkname = ldot + _renamed(target)
+ elif m.isfile():
+ data = src.extractfile(m)
+ dst_tar.addfile(m, data)
+ if add is not None:
+ dst_tar.addfile(add)
+ return WS_OK, out.getvalue()
+ finally:
+ if src is not None:
+ src.close()
+
+
class RunnerWorkspaceFiles:
"""Self-hosted: a session's workspace is a live directory on the data volume, and the RUNNER is
the only principal that touches it. Behind the write-wall each session directory belongs to that
@@ -479,10 +694,11 @@ class RunnerWorkspaceFiles:
Live, unlike the checkpoint the listing route reads: an app sees a file the moment a tool call
creates it, rather than when the turn ends."""
- def __init__(self, endpoint: str, key: str, blob=None, kb: str = ""):
+ def __init__(self, endpoint: str, key: str, blob=None, kb: str = "", ckpt_kb: str = "", key_for=None):
self.endpoint = (endpoint or "").rstrip("/")
self.key = key
self.blob, self.kb = blob, kb
+ self.ckpt_kb, self.key_for = ckpt_kb, key_for
self._client: httpx.AsyncClient | None = None
def _c(self) -> httpx.AsyncClient:
@@ -519,6 +735,101 @@ class RunnerWorkspaceFiles:
await self._capture(sid, member, data)
return True
+ async def delete(self, sid: str, path: str) -> str:
+ return await self._change(sid, "delete", path)
+
+ async def mkdir(self, sid: str, path: str) -> str:
+ return await self._change(sid, "mkdir", path)
+
+ async def move(self, sid: str, path: str, to: str) -> str:
+ return await self._change(sid, "move", path, to)
+
+ async def _change(self, sid: str, op: str, path: str, to: str = "") -> str:
+ """The live directory first, through the runner (which acts as the session), then the
+ checkpoint tarball.
+
+ The checkpoint is the second half because the live directory is not the only copy: it is
+ reaped after HR_WORKSPACE_TTL_HOURS, and a sandbox that does not hold the last checkpoint
+ is wiped and restored from that tarball. A delete made only in the live directory would
+ come back on that restore, and a move would come back as two copies. `write` closes the
+ same hole with a per-file capture; a delete, a directory or a rename has no file to
+ capture, so the archive itself is changed, as the hosted backing does."""
+ member = _safe_member(path)
+ dest = _safe_member(to) if op == "move" else ""
+ if not member or (op == "move" and not dest) or not self.endpoint:
+ return WS_BAD_PATH
+ params = {"identifier": sid, "path": member, **({"to": dest} if op == "move" else {})}
+ url = f"{self.endpoint}/file" + ("" if op == "delete" else f"/{op}")
+ try:
+ r = await self._c().request("DELETE" if op == "delete" else "POST", url, params=params,
+ headers=self._headers())
+ except Exception: # noqa: BLE001
+ return WS_FAILED
+ outcome = {200: WS_OK, 404: WS_NOT_FOUND, 409: WS_EXISTS, 400: WS_BAD_PATH}.get(r.status_code, WS_FAILED)
+ if outcome == WS_NOT_FOUND:
+ # Not in the live directory. That directory is reaped after HR_WORKSPACE_TTL_HOURS while
+ # the checkpoint keeps the file, and the next turn restores from the checkpoint, so the
+ # change is the checkpoint's to make. A path in neither is a 404.
+ return await self._change_checkpoint(sid, op, member, dest, live=False)
+ if outcome != WS_OK:
+ return outcome
+ try:
+ got = r.json()
+ except ValueError:
+ got = {}
+ # The checkpoint is changed by the caller's path, not the one the runner answers: tar_change
+ # resolves it the way the runner did (the agent's directory first on a nesting base), and
+ # the runner's session-relative spelling would be looked for one level down a second time.
+ ckpt = await self._change_checkpoint(sid, op, member, dest)
+ if ckpt in (WS_EXISTS, WS_BAD_PATH):
+ return ckpt
+ return WS_UNCHANGED if op == "mkdir" and got.get("created") is False else WS_OK
+
+ async def _change_checkpoint(self, sid: str, op: str, member: str, to: str, live: bool = True) -> str:
+ """Apply the change to the checkpoint → its outcome. After a live change (`live`) the change
+ already stands, so an archive that cannot be stored is logged rather than failed; without
+ one, the archive is the only copy and its outcome is the answer.
+
+ Two answers hold even after a live change:
+ - A folder made while the session has no checkpoint (its turn ended without one) has no
+ archive to go into, and the next turn's restore empties the live directory (the captures
+ put files back, and a folder has none). So the archive is started with it, as the hosted
+ backing starts one.
+ - An archive that REFUSES what the live directory allowed (a file where the folder or the
+ destination would go) means the live directory is not the session's workspace: it is a
+ reaped one the runner made anew, which the next turn wipes and restores from this archive.
+ The refusal is the answer then, not a 200 that the next turn undoes."""
+ if not self.blob or not self.ckpt_kb or not self.key_for:
+ return WS_OK if live else WS_NOT_FOUND
+ key = self.key_for(sid)
+ try:
+ async with _ckpt_lock(sid):
+ raw = await self.blob.get(self.ckpt_kb, key)
+ if not raw and op != "mkdir":
+ # no checkpoint yet: the live directory is the only copy
+ return WS_OK if live else WS_NOT_FOUND
+ outcome, tar = await asyncio.to_thread(tar_change, raw, op, member, to)
+ if outcome in (WS_NOT_FOUND, WS_UNCHANGED):
+ # not in the checkpoint (made since it was taken), or already so
+ return WS_OK if live else outcome
+ if outcome in (WS_EXISTS, WS_BAD_PATH):
+ if live:
+ print(f"[workspace] checkpoint refused a {op} the live directory allowed "
+ f"sid={sid} path={member}: {outcome}", flush=True)
+ return outcome
+ if await self.blob.put(self.ckpt_kb, key, tar):
+ return WS_OK
+ if not live:
+ return WS_FAILED
+ why = "store refused"
+ except Exception as e: # noqa: BLE001
+ if not live:
+ return WS_FAILED
+ why = f"{type(e).__name__}: {e}"
+ # The live change stands, but a restore from this checkpoint would undo it. Say so.
+ print(f"[workspace] checkpoint {op} not applied sid={sid} path={member}: {why}", flush=True)
+ return WS_OK
+
async def _capture(self, sid: str, member: str, data: bytes) -> None:
"""Persist the app's copy of this file so it outlives the live directory.
@@ -535,21 +846,28 @@ class RunnerWorkspaceFiles:
rather than whichever the blob listing happened to return last."""
if not self.blob or not self.kb:
return
- cfile = "cfile_" + hashlib.sha256(f"app:{member}".encode()).hexdigest()[:32]
- meta = json.dumps({"filename": member,
- "media_type": mimetypes.guess_type(member)[0] or "application/octet-stream",
- "written_at": time.time(),
- "source": "app"}).encode()
- try:
- if await self.blob.put(self.kb, f"containers/{sid}/{cfile}", data):
- await self.blob.put(self.kb, f"containers/{sid}/{cfile}.meta", meta)
- return
- except Exception as e: # noqa: BLE001
- print(f"[workspace] durable capture failed sid={sid} path={member}: {e}", flush=True)
- return
- # A live write that was not captured is the old silent-loss shape. Say so rather than
- # letting it look like a clean save.
- print(f"[workspace] durable capture rejected sid={sid} path={member}", flush=True)
+ await put_app_capture(self.blob, self.kb, sid, member, data)
+
+
+async def put_app_capture(blob, kb: str, sid: str, member: str, data: bytes) -> bool:
+ """Store an app's durable copy of one workspace file (RunnerWorkspaceFiles._capture says why).
+ Also how a move carries an app's copy to the file's new name."""
+ cfile = app_capture_id(member)
+ meta = json.dumps({"filename": member,
+ "media_type": mimetypes.guess_type(member)[0] or "application/octet-stream",
+ "written_at": time.time(),
+ "source": "app"}).encode()
+ try:
+ if await blob.put(kb, f"containers/{sid}/{cfile}", data):
+ await blob.put(kb, f"containers/{sid}/{cfile}.meta", meta)
+ return True
+ except Exception as e: # noqa: BLE001
+ print(f"[workspace] durable capture failed sid={sid} path={member}: {e}", flush=True)
+ return False
+ # A live write that was not captured is the old silent-loss shape. Say so rather than
+ # letting it look like a clean save.
+ print(f"[workspace] durable capture rejected sid={sid} path={member}", flush=True)
+ return False
class CheckpointWorkspaceFiles:
@@ -560,8 +878,30 @@ class CheckpointWorkspaceFiles:
LAST one, so appending would work until something read the archive differently, and the bug
would be a stale file appearing at random."""
- def __init__(self, blob, kb: str, key_for):
+ def __init__(self, blob, kb: str, key_for, on_rewrite=None):
self.blob, self.kb, self.key_for = blob, kb, key_for
+ # async (sid, sha256 of the new archive) -> None, called after every rewrite. The gateway
+ # sets it to record the archive's sha as the session's checkpoint sha (see _store).
+ self.on_rewrite = on_rewrite
+
+ async def _store(self, sid: str, key: str, tar: bytes) -> bool:
+ """Put a rewritten archive. A sandbox kept warm from the last turn still holds the OLD
+ workspace, and hydrate skips the restore when the sandbox's sha matches the session's. Left as
+ it was, the next turn would run on the old files and its checkpoint would write them back
+ over this change: a deleted file restored, a moved one at both names.
+
+ So the new archive's sha is recorded FIRST, and the archive is stored only once it is. A sha
+ that cannot be recorded stops the change before anything moved (the caller sees a failure,
+ and a retry starts clean); an archive that cannot be stored after it leaves a sha that
+ matches no sandbox, which costs the next turn a full restore of the old archive and nothing
+ else."""
+ if self.on_rewrite is not None:
+ try:
+ await self.on_rewrite(sid, hashlib.sha256(tar).hexdigest())
+ except Exception as e: # noqa: BLE001
+ print(f"[workspace] checkpoint sha not recorded sid={sid}: {e}; not rewritten", flush=True)
+ return False
+ return bool(await self.blob.put(self.kb, key, tar))
async def read(self, sid: str, path: str) -> bytes | None:
member = _safe_member(path)
@@ -573,7 +913,10 @@ class CheckpointWorkspaceFiles:
def _extract() -> bytes | None:
with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as tf:
- for name in (member, f"./{member}"):
+ # the agent's directory first on a nesting base, the session root second: the
+ # order the runner's GET /file reads in
+ sub = _tar_subdir(tf)
+ for name in ([f"{sub}/{member}", f"./{sub}/{member}"] if sub else []) + [member, f"./{member}"]:
try:
f = tf.extractfile(name)
except KeyError:
@@ -591,21 +934,37 @@ class CheckpointWorkspaceFiles:
if not member:
return False
key = self.key_for(sid)
+ async with _ckpt_lock(sid):
+ return await self._write_locked(sid, key, member, data)
+
+ async def _write_locked(self, sid: str, key: str, member: str, data: bytes) -> bool:
raw = await self.blob.get(self.kb, key)
def _rewrite() -> bytes:
out = io.BytesIO()
wrote = False
with tarfile.open(fileobj=out, mode="w:gz") as dst:
+ target = member
if raw:
with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as src:
- for m in src.getmembers():
- name = m.name[2:] if m.name.startswith("./") else m.name
- if name == member:
+ members = src.getmembers()
+ names = [m.name[2:] if m.name.startswith("./") else m.name for m in members]
+ # a file the agent already has, on a nesting base, is replaced where it is
+ # (in its directory), not shadowed by a second copy at the root
+ sub = _tar_subdir(src)
+ top = member.split("/", 1)[0]
+ if sub and (f"{sub}/{member}" in names or (
+ "/" in member and member not in names
+ and any(_under(n, f"{sub}/{top}") for n in names))):
+ # where it is, or — a new file — in a folder the agent already has, which
+ # adoption would not move a root copy into
+ target = f"{sub}/{member}"
+ for m, name in zip(members, names):
+ if name == target:
continue # replaced below
f = src.extractfile(m) if m.isfile() else None
dst.addfile(m, f)
- info = tarfile.TarInfo(member)
+ info = tarfile.TarInfo(target)
info.size = len(data)
info.mtime = int(time.time())
info.mode = 0o644
@@ -617,7 +976,34 @@ class CheckpointWorkspaceFiles:
tar = await asyncio.to_thread(_rewrite)
except Exception: # noqa: BLE001
return False
- return bool(tar) and await self.blob.put(self.kb, key, tar)
+ return bool(tar) and await self._store(sid, key, tar)
+
+ async def delete(self, sid: str, path: str) -> str:
+ return await self._change(sid, "delete", path)
+
+ async def mkdir(self, sid: str, path: str) -> str:
+ return await self._change(sid, "mkdir", path)
+
+ async def move(self, sid: str, path: str, to: str) -> str:
+ return await self._change(sid, "move", path, to)
+
+ async def _change(self, sid: str, op: str, path: str, to: str = "") -> str:
+ """Rewrite the archive with the change applied, as `write` does: between turns the archive
+ IS the workspace."""
+ member = _safe_member(path)
+ dest = _safe_member(to) if op == "move" else ""
+ if not member or (op == "move" and not dest):
+ return WS_BAD_PATH
+ key = self.key_for(sid)
+ async with _ckpt_lock(sid):
+ try:
+ raw = await self.blob.get(self.kb, key)
+ outcome, tar = await asyncio.to_thread(tar_change, raw, op, member, dest)
+ except Exception: # noqa: BLE001
+ return WS_FAILED
+ if outcome != WS_OK or tar is None:
+ return outcome
+ return WS_OK if await self._store(sid, key, tar) else WS_FAILED
def make_backing(*, client_getter, vg_url: str, vg_key: str, vg_tenant: str,
@@ -648,4 +1034,5 @@ def make_backing(*, client_getter, vg_url: str, vg_key: str, vg_tenant: str,
workspace=RunnerWorkspaceFiles(os.environ.get("POOL_MGMT_ENDPOINT", "http://127.0.0.1:8081"),
os.environ.get("HARNESS_INTERNAL_KEY", ""),
local_blob,
- os.environ.get("HARNESS_RESP_BLOB_KB", "harness-responses")))
+ os.environ.get("HARNESS_RESP_BLOB_KB", "harness-responses"),
+ os.environ.get("HARNESS_BLOB_KB", "harness-sessions"), ws_key))
diff --git a/gateway/tests/test_session_file_changes.py b/gateway/tests/test_session_file_changes.py
new file mode 100644
index 00000000..e3bfb61c
--- /dev/null
+++ b/gateway/tests/test_session_file_changes.py
@@ -0,0 +1,943 @@
+"""Delete, mkdir and move in a session workspace, on both backings, and what keeps them from coming back.
+
+ * the hosted backing (CheckpointWorkspaceFiles): between turns the checkpoint tarball IS the
+ workspace, so each change rewrites it, as `write` does
+ * the self-hosted backing (RunnerWorkspaceFiles): the live directory through the runner, and then
+ the checkpoint, because a sandbox that does not hold the last checkpoint is restored from it and
+ a change made only live would be undone there
+ * the durable copies: a delete leaves a tombstone, so the by-path read stops falling back to the
+ agent's old capture and the restore stops re-applying the app's; a move carries the app's copy
+ to the new name
+ * the routes: the same ownership and busy rule as PUT, `..` and absolute paths refused, internal
+ paths (.git, .harness) refused, and each backing's answer mapped to its failure
+"""
+import io
+import json
+import pathlib
+import sys
+import tarfile
+import time
+
+import pytest
+
+sys.path.insert(0, str(pathlib.Path(__file__).resolve().parents[1]))
+import app as A # noqa: E402
+import backing as B # noqa: E402
+
+
+def _tar(entries: dict) -> bytes:
+ """name -> bytes (a file) or None (a directory entry)."""
+ out = io.BytesIO()
+ with tarfile.open(fileobj=out, mode="w:gz") as tf:
+ for name, data in entries.items():
+ info = tarfile.TarInfo(name)
+ if data is None:
+ info.type = tarfile.DIRTYPE
+ tf.addfile(info)
+ else:
+ info.size = len(data)
+ tf.addfile(info, io.BytesIO(data))
+ return out.getvalue()
+
+
+def _files(raw: bytes) -> dict:
+ with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as tf:
+ return {m.name: (tf.extractfile(m).read() if m.isfile() else None) for m in tf.getmembers()}
+
+
+# ── the archive ────────────────────────────────────────────────────────────────────────────────
+def test_delete_a_file_and_keep_the_rest():
+ outcome, raw = B.tar_change(_tar({"./a.txt": b"a", "./b.txt": b"b"}), "delete", "a.txt")
+ assert outcome == B.WS_OK and _files(raw) == {"./b.txt": b"b"}
+
+
+def test_delete_a_directory_with_or_without_its_own_entry():
+ for entries in ({"./d": None, "./d/x": b"x", "./dx": b"keep"}, {"d/x": b"x", "dx": b"keep"}):
+ outcome, raw = B.tar_change(_tar(entries), "delete", "d")
+ assert outcome == B.WS_OK
+ assert [n for n in _files(raw)] == [n for n in entries if n.endswith("dx")]
+
+
+def test_delete_nothing():
+ assert B.tar_change(_tar({"a": b"a"}), "delete", "b") == (B.WS_NOT_FOUND, None)
+ assert B.tar_change(None, "delete", "b") == (B.WS_NOT_FOUND, None)
+
+
+def test_mkdir():
+ outcome, raw = B.tar_change(_tar({"a": b"a"}), "mkdir", "d/e")
+ assert outcome == B.WS_OK and _files(raw) == {"a": b"a", "d/e": None}
+ assert B.tar_change(raw, "mkdir", "d/e") == (B.WS_UNCHANGED, None) # already there
+ assert B.tar_change(raw, "mkdir", "a") == (B.WS_EXISTS, None) # a file is in the way
+ outcome, raw = B.tar_change(None, "mkdir", "d") # no checkpoint yet
+ assert outcome == B.WS_OK and _files(raw) == {"d": None}
+
+
+def test_move_a_file_and_a_directory_keeping_the_bytes():
+ raw = _tar({"./a.txt": b"A", "./d": None, "./d/x": b"X", "./d/y/z": b"Z"})
+ outcome, raw = B.tar_change(raw, "move", "a.txt", "notes/a.txt")
+ assert outcome == B.WS_OK
+ outcome, raw = B.tar_change(raw, "move", "d", "e")
+ assert outcome == B.WS_OK
+ assert _files(raw) == {"./notes/a.txt": b"A", "./e": None, "./e/x": b"X", "./e/y/z": b"Z"}
+
+
+def test_move_refusals():
+ raw = _tar({"a": b"a", "b": b"b", "d/x": b"x"})
+ assert B.tar_change(raw, "move", "a", "b") == (B.WS_EXISTS, None)
+ assert B.tar_change(raw, "move", "d", "d/in") == (B.WS_BAD_PATH, None)
+ assert B.tar_change(raw, "move", "nope", "c") == (B.WS_NOT_FOUND, None)
+
+
+def test_the_nesting_base_s_files_are_found_one_level_down():
+ raw = _tar({".harness/agent-subdir": b"workspace\n", "workspace/deck.json": b"{}"})
+ outcome, raw = B.tar_change(raw, "move", "deck.json", "old/deck.json")
+ assert outcome == B.WS_OK and "workspace/old/deck.json" in _files(raw)
+ outcome, raw = B.tar_change(raw, "delete", "old")
+ assert outcome == B.WS_OK and set(_files(raw)) == {".harness/agent-subdir"}
+
+
+# ── the hosted backing ────────────────────────────────────────────────────────────────────────
+class _Blob:
+ def __init__(self, objects=None):
+ self.objects = dict(objects or {})
+
+ async def get(self, kb, key):
+ return self.objects.get((kb, key))
+
+ async def put(self, kb, key, data):
+ self.objects[(kb, key)] = data
+ return True
+
+ async def delete(self, kb, key):
+ return self.objects.pop((kb, key), None) is not None
+
+
+def _hosted(entries):
+ blob = _Blob({("kb", "sessions/s/workspace.tgz"): _tar(entries)})
+ return blob, B.CheckpointWorkspaceFiles(blob, "kb", lambda sid: f"sessions/{sid}/workspace.tgz")
+
+
+async def test_hosted_delete_mkdir_move_rewrite_the_checkpoint():
+ blob, ws = _hosted({"a.txt": b"a", "d/x": b"x"})
+ assert await ws.delete("s", "a.txt") == B.WS_OK
+ assert await ws.mkdir("s", "new") == B.WS_OK
+ assert await ws.move("s", "d", "e") == B.WS_OK
+ assert _files(blob.objects[("kb", "sessions/s/workspace.tgz")]) == {"new": None, "e/x": b"x"}
+ assert await ws.read("s", "e/x") == b"x" and await ws.read("s", "d/x") is None
+
+
+async def test_hosted_refusals_change_nothing():
+ blob, ws = _hosted({"a.txt": b"a"})
+ before = dict(blob.objects)
+ assert await ws.delete("s", "../a.txt") == B.WS_BAD_PATH
+ assert await ws.move("s", "a.txt", "../../b") == B.WS_BAD_PATH
+ assert await ws.delete("s", "missing") == B.WS_NOT_FOUND
+ # normpath leaves "." and ".." as they are; a member named ".." is one tar refuses to unpack
+ # ("Member name contains '..'", GNU tar), so every later restore of the session would fail
+ assert await ws.write("s", "..", b"x") is False
+ assert await ws.mkdir("s", "..") == B.WS_BAD_PATH and await ws.mkdir("s", ".") == B.WS_BAD_PATH
+ assert blob.objects == before
+
+
+# ── the self-hosted backing ───────────────────────────────────────────────────────────────────
+class _Resp:
+ def __init__(self, status, body=None):
+ self.status_code, self._body = status, body or {}
+
+ def json(self):
+ return self._body
+
+
+def _selfhosted(answer, entries=None):
+ blob = _Blob({("ckpt", "sessions/s/workspace.tgz"): _tar(entries)} if entries is not None else {})
+ ws = B.RunnerWorkspaceFiles("http://runner", "k", blob, "resp", "ckpt",
+ lambda sid: f"sessions/{sid}/workspace.tgz")
+ calls = []
+
+ class _Client:
+ async def request(self, method, url, params=None, headers=None):
+ calls.append((method, url, params))
+ return answer
+
+ ws._c = lambda: _Client()
+ return blob, ws, calls
+
+
+async def test_selfhosted_changes_the_live_directory_and_then_the_checkpoint():
+ blob, ws, calls = _selfhosted(_Resp(200, {"path": "d", "to": "e"}), {"d/x": b"x", "keep": b"k"})
+ assert await ws.move("s", "d", "e") == B.WS_OK
+ assert calls == [("POST", "http://runner/file/move", {"identifier": "s", "path": "d", "to": "e"})]
+ assert _files(blob.objects[("ckpt", "sessions/s/workspace.tgz")]) == {"e/x": b"x", "keep": b"k"}
+
+
+async def test_selfhosted_changes_the_nesting_base_s_checkpoint_where_its_files_are():
+ """nanobot: the runner answers the session-relative path (workspace/deck.json); the checkpoint is
+ changed by the caller's path, which tar_change looks for one level down itself. Passing the
+ runner's spelling looked for workspace/workspace/deck.json, found nothing, and changed nothing."""
+ marker = {".harness/agent-subdir": b"workspace"}
+ blob, ws, _calls = _selfhosted(_Resp(200, {"path": "workspace/deck.json"}),
+ {**marker, "workspace/deck.json": b"{}", "deck.json.bak": b""})
+ assert await ws.delete("s", "deck.json") == B.WS_OK
+ assert set(_files(blob.objects[("ckpt", "sessions/s/workspace.tgz")])) == {".harness/agent-subdir", "deck.json.bak"}
+ blob, ws, _calls = _selfhosted(_Resp(200, {"path": "workspace/a.md", "to": "workspace/b/a.md"}),
+ {**marker, "workspace/a.md": b"A"})
+ assert await ws.move("s", "a.md", "b/a.md") == B.WS_OK
+ assert _files(blob.objects[("ckpt", "sessions/s/workspace.tgz")])["workspace/b/a.md"] == b"A"
+
+
+async def test_selfhosted_a_file_made_since_the_checkpoint_leaves_it_alone():
+ blob, ws, _ = _selfhosted(_Resp(200, {"path": "new.txt"}), {"old.txt": b"o"})
+ before = dict(blob.objects)
+ assert await ws.delete("s", "new.txt") == B.WS_OK
+ assert blob.objects == before
+
+
+async def test_selfhosted_with_no_checkpoint_yet():
+ """A delete or a move has no archive to change while the session has no checkpoint (its turn ended
+ without one). A folder does: the next turn's restore empties the live directory and the captures
+ put back only files, so the archive is started with the folder, as the hosted backing starts one."""
+ blob, ws, _ = _selfhosted(_Resp(200, {"path": "x"}))
+ assert await ws.delete("s", "x") == B.WS_OK and blob.objects == {}
+ blob, ws, _ = _selfhosted(_Resp(200, {"path": "a", "created": True}))
+ assert await ws.mkdir("s", "a") == B.WS_OK
+ assert _files(blob.objects[("ckpt", "sessions/s/workspace.tgz")]) == {"a": None}
+
+
+async def test_selfhosted_a_checkpoint_that_refuses_what_the_live_directory_allowed_is_the_answer():
+ """The live directory took the change but the checkpoint has a FILE in the way: that live directory
+ is a reaped one the runner made anew, which the next turn wipes and restores from the checkpoint.
+ A 200 would be undone by that turn."""
+ blob, ws, _ = _selfhosted(_Resp(200, {"path": "notes/x", "created": True}), {"notes": b"a file"})
+ before = dict(blob.objects)
+ assert await ws.mkdir("s", "notes/x") == B.WS_EXISTS
+ assert blob.objects == before
+ blob, ws, _ = _selfhosted(_Resp(200, {"path": "a", "to": "b"}), {"a": b"a", "b": b"b"})
+ before = dict(blob.objects)
+ assert await ws.move("s", "a", "b") == B.WS_EXISTS
+ assert blob.objects == before
+
+
[email protected]("status,outcome", [(409, B.WS_EXISTS), (400, B.WS_BAD_PATH), (500, B.WS_FAILED)])
+async def test_selfhosted_a_refusal_touches_no_checkpoint(status, outcome):
+ blob, ws, _ = _selfhosted(_Resp(status), {"a": b"a"})
+ before = dict(blob.objects)
+ assert await ws.delete("s", "a") == outcome
+ assert blob.objects == before
+
+
+async def test_selfhosted_a_reaped_live_directory_leaves_the_change_to_the_checkpoint():
+ """The live directory is reaped after HR_WORKSPACE_TTL_HOURS; the checkpoint still holds the file,
+ and the next turn restores from it. The runner's 404 is not the answer then."""
+ blob, ws, _ = _selfhosted(_Resp(404), {"notes.md": b"n", "keep": b"k"})
+ assert await ws.delete("s", "notes.md") == B.WS_OK
+ assert _files(blob.objects[("ckpt", "sessions/s/workspace.tgz")]) == {"keep": b"k"}
+ assert await ws.move("s", "keep", "kept") == B.WS_OK
+ assert _files(blob.objects[("ckpt", "sessions/s/workspace.tgz")]) == {"kept": b"k"}
+ assert await ws.delete("s", "nowhere") == B.WS_NOT_FOUND
+ _blob, ws, _ = _selfhosted(_Resp(404)) # no checkpoint either
+ assert await ws.delete("s", "x") == B.WS_NOT_FOUND
+
+
+async def test_selfhosted_a_bad_path_never_reaches_the_runner():
+ _blob, ws, calls = _selfhosted(_Resp(200))
+ assert await ws.delete("s", "../x") == B.WS_BAD_PATH
+ assert await ws.move("s", "a", "/../b") == B.WS_BAD_PATH
+ assert calls == []
+
+
+# ── the durable copies ────────────────────────────────────────────────────────────────────────
+def captures(monkeypatch):
+ """The response blob store, in memory: containers/<sid>/<cfile> and its .meta."""
+ store: dict[str, bytes] = {}
+
+ async def _list(prefix, **_k):
+ return {"items": [{"file_id": k} for k in sorted(store) if k.startswith(prefix)]}
+
+ async def _get(fid, **_k):
+ return store.get(fid)
+
+ async def _put(fid, data, **_k):
+ store[fid] = data
+ return True
+
+ async def _delete(fid, **_k):
+ return store.pop(fid, None) is not None
+
+ async def _bytes(sid, cid):
+ data = store.get(f"containers/{sid}/{cid}")
+ return (data, "application/octet-stream", cid) if data is not None else None
+
+ monkeypatch.setattr(A, "_blob_list", _list)
+ monkeypatch.setattr(A, "_blob_get", _get)
+ monkeypatch.setattr(A, "_blob_put", _put)
+ monkeypatch.setattr(A, "_blob_delete", _delete)
+ monkeypatch.setattr(A, "_container_file_bytes", _bytes)
+
+ class _StoreBlob:
+ async def put(self, _kb, key, data):
+ store[key] = data
+ return True
+
+ monkeypatch.setattr(A.BACKING, "blob", _StoreBlob(), raising=False)
+
+ def add(cid, name, data, ts, app=False):
+ store[f"containers/s/{cid}"] = data
+ store[f"containers/s/{cid}.meta"] = json.dumps(
+ {"filename": name, "written_at": ts, **({"source": "app"} if app else {})}).encode()
+ store_add = add
+ store_add.store = store
+ return store_add
+
+
+async def test_a_tombstone_hides_older_captures_of_the_path_and_under_it(captures):
+ captures("cfile_agent1", "report.md", b"old", 100.0)
+ captures("cfile_agent2", "d/x.md", b"x", 100.0)
+ captures("cfile_agent3", "dx.md", b"keep", 100.0)
+ await A._put_tombstone("s", "report.md")
+ await A._put_tombstone("s", "d")
+ assert set(await A._cfile_by_name("s")) == {"dx.md"}
+
+
+async def test_a_capture_newer_than_the_tombstone_is_back(captures):
+ await A._put_tombstone("s", "report.md")
+ captures("cfile_later", "report.md", b"new", time.time() + 10)
+ assert (await A._cfile_by_name("s"))["report.md"] == "cfile_later"
+
+
+async def test_the_tombstone_replaces_the_app_s_own_copy(captures):
+ await B.put_app_capture(A.BACKING.blob, "kb", "s", "sheet.json", b"rows")
+ cid = B.app_capture_id("sheet.json")
+ assert f"containers/s/{cid}" in captures.store
+ await A._put_tombstone("s", "sheet.json")
+ assert f"containers/s/{cid}" not in captures.store and f"containers/s/{cid}.meta" not in captures.store
+ assert json.loads(captures.store[A._tomb_key("s", "sheet.json")])["filename"] == "sheet.json"
+
+
+async def test_a_restore_does_not_bring_back_what_the_app_deleted(captures, monkeypatch):
+ await B.put_app_capture(A.BACKING.blob, "kb", "s", "sheet.json", b"rows")
+ await B.put_app_capture(A.BACKING.blob, "kb", "s", "d/x.json", b"x")
+ await A._put_tombstone("s", "sheet.json")
+ await A._put_tombstone("s", "d")
+ written = {}
+
+ class _Ws:
+ async def write(self, _sid, name, data):
+ written[name] = data
+ return True
+
+ monkeypatch.setattr(A.BACKING, "workspace", _Ws(), raising=False)
+ assert await A._reapply_app_writes("s") == 0 and written == {}
+
+
+async def test_a_removed_path_does_not_come_back_by_its_bare_name(captures, monkeypatch):
+ """The read falls back to a capture named by the bare filename (captures from before paths
+ were kept). After `notes/a.md` is deleted, a capture called `a.md` is some other file."""
+ captures("cfile_legacy", "a.md", b"legacy", 50.0)
+
+ class _Gone:
+ async def read(self, *_a, **_k):
+ return None
+
+ async def _owned(*_a, **_k):
+ return ("org", {})
+
+ monkeypatch.setattr(A.BACKING, "workspace", _Gone(), raising=False)
+ monkeypatch.setattr(A, "_owned_session", _owned)
+ r = await A.read_session_file("s", "notes/a.md", request=None)
+ assert r.body == b"legacy" # before: the old fallback stands
+ await A._put_tombstone("s", "notes/a.md")
+ with pytest.raises(Exception) as e:
+ await A.read_session_file("s", "notes/a.md", request=None)
+ assert getattr(e.value, "status_code", None) == 404
+
+
+async def test_a_move_carries_the_app_s_copy_and_not_the_agent_s(captures):
+ """The app's, because a restore re-applies it. Not the agent's: carried as the app's it would be
+ re-applied on every restore, over whatever the agent later did with the file."""
+ await B.put_app_capture(A.BACKING.blob, "kb", "s", "d/sheet.json", b"typed since the checkpoint")
+ captures("cfile_agent", "d/other.md", b"agent's", 100.0)
+ await A._carry_captures("s", "d", "archive/d", await A._capture_state("s"))
+ await A._put_tombstone("s", "d")
+ by = await A._cfile_by_name("s")
+ assert set(by) == {"archive/d/sheet.json"}
+ assert captures.store[f"containers/s/{by['archive/d/sheet.json']}"] == b"typed since the checkpoint"
+
+
+# ── the routes ────────────────────────────────────────────────────────────────────────────────
+class _Fake:
+ def __init__(self, outcome=B.WS_OK):
+ self.outcome, self.calls = outcome, []
+
+ async def delete(self, sid, path):
+ self.calls.append(("delete", path))
+ return self.outcome
+
+ async def mkdir(self, sid, path):
+ self.calls.append(("mkdir", path))
+ return self.outcome
+
+ async def move(self, sid, path, to):
+ self.calls.append(("move", path, to))
+ return self.outcome
+
+
+def routes(monkeypatch, captures):
+ state = {"v": {}}
+
+ async def _owned(_r, _sid):
+ return ("org", state["v"])
+
+ async def _vget(_sid):
+ return state["v"]
+
+ fake = _Fake()
+ monkeypatch.setattr(A, "_owned_session", _owned)
+ monkeypatch.setattr(A, "_vertex_get", _vget)
+ monkeypatch.setattr(A.BACKING, "workspace", fake, raising=False)
+ monkeypatch.setattr(A, "_ws_changed", lambda sid: state.setdefault("changed", []).append(sid))
+ state["fake"] = fake
+ return state
+
+
+async def test_routes_do_the_change_and_answer(routes):
+ assert await A.delete_session_file("s", "a.txt", request=None) == \
+ {"session_id": "s", "path": "a.txt", "deleted": True}
+ assert await A.mkdir_session_file("s", "d", request=None) == \
+ {"session_id": "s", "path": "d", "dir": True, "created": True}
+ assert await A.move_session_file("s", "d", A.WorkspaceMove(to="e"), request=None) == \
+ {"session_id": "s", "path": "d", "to": "e", "moved": True}
+ assert routes["fake"].calls == [("delete", "a.txt"), ("mkdir", "d"), ("move", "d", "e")]
+ assert routes["changed"] == ["s", "s", "s"]
+
+
[email protected]("status", ["running", "starting"])
+async def test_routes_refuse_while_a_turn_runs(routes, status):
+ routes["v"] = {"turn_status": status}
+ for call in (lambda: A.delete_session_file("s", "a", request=None),
+ lambda: A.mkdir_session_file("s", "a", request=None),
+ lambda: A.move_session_file("s", "a", A.WorkspaceMove(to="b"), request=None)):
+ with pytest.raises(Exception) as e:
+ await call()
+ assert e.value.status_code == 409 and e.value.detail["code"] == "session_busy"
+ assert routes["fake"].calls == []
+
+
[email protected]("path", ["../a", "a/../../b", ".git", ".git/config", ".harness/agent-subdir",
+ "x/.git/HEAD", ".gitignore", "a\\b"])
+async def test_routes_refuse_escapes_and_internal_paths(routes, path):
+ for call in (lambda: A.delete_session_file("s", path, request=None),
+ lambda: A.mkdir_session_file("s", path, request=None),
+ lambda: A.move_session_file("s", path, A.WorkspaceMove(to="ok"), request=None),
+ lambda: A.move_session_file("s", "ok", A.WorkspaceMove(to=path), request=None)):
+ with pytest.raises(Exception) as e:
+ await call()
+ assert e.value.status_code == 400 and e.value.detail["code"] == "invalid_input"
+ assert routes["fake"].calls == []
+
+
+async def test_a_leading_slash_names_the_same_workspace_path(routes):
+ """The PUT next door treats `/a.txt` as `a.txt` (_safe_member strips it); these do the same."""
+ await A.delete_session_file("s", "/a.txt", request=None)
+ assert routes["fake"].calls == [("delete", "a.txt")]
+
+
+async def test_move_into_itself_is_refused_before_the_backing(routes):
+ with pytest.raises(Exception) as e:
+ await A.move_session_file("s", "d", A.WorkspaceMove(to="d/in"), request=None)
+ assert e.value.status_code == 400 and routes["fake"].calls == []
+
+
[email protected]("outcome,status,code", [(B.WS_NOT_FOUND, 404, "file_not_found"),
+ (B.WS_EXISTS, 409, "file_exists"),
+ (B.WS_BAD_PATH, 400, "invalid_input"),
+ (B.WS_FAILED, 502, "internal_error")])
+async def test_routes_map_the_backing_s_answer(routes, outcome, status, code):
+ routes["fake"].outcome = outcome
+ with pytest.raises(Exception) as e:
+ await A.delete_session_file("s", "a", request=None)
+ assert e.value.status_code == status and e.value.detail["code"] == code
+ assert "changed" not in routes # a refused change evicts nothing
+
+
+def test_the_action_suffix_routes_a_path_with_slashes():
+ from starlette.routing import Match
+
+ def _match(method, path):
+ scope = {"type": "http", "method": method, "path": path, "root_path": "", "query_string": b"",
+ "headers": []}
+ for r in A.app.router.routes:
+ m, child = r.matches(scope)
+ if m == Match.FULL:
+ return r.endpoint.__name__, child["path_params"]
+ return None
+
+ assert _match("POST", "/v1/sessions/s/files/notes/2026/a.md:move") == \
+ ("move_session_file", {"sid": "s", "path": "notes/2026/a.md"})
+ assert _match("POST", "/v1/sessions/s/files/a/b:mkdir") == \
+ ("mkdir_session_file", {"sid": "s", "path": "a/b"})
+ assert _match("DELETE", "/v1/sessions/s/files/a/b.txt") == \
+ ("delete_session_file", {"sid": "s", "path": "a/b.txt"})
+ # the routes beside them are untouched
+ assert _match("GET", "/v1/sessions/s/files/a/b.txt")[0] == "read_session_file"
+ assert _match("PUT", "/v1/sessions/s/files/a/b.txt")[0] == "write_session_file"
+
+
+async def test_a_capture_that_cannot_be_carried_does_not_stop_the_move(routes, monkeypatch):
+ """The backing has already moved the file; the tombstone on the source must still land, or the
+ next restore writes the app's old copy back under the old name."""
+ await B.put_app_capture(A.BACKING.blob, "kb", "s", "sheet.json", b"rows")
+
+ async def _boom(*_a, **_k):
+ raise RuntimeError("blob store flaked")
+
+ monkeypatch.setattr(A, "_container_file_bytes", _boom)
+ assert (await A.move_session_file("s", "sheet.json", A.WorkspaceMove(to="kept.json"), request=None))["moved"]
+ assert "sheet.json" not in await A._cfile_by_name("s")
+
+
+# ── what the review of #478 found ─────────────────────────────────────────────────────────────
+_NANOBOT = {".harness/agent-subdir": b"workspace\n",
+ ".git/info/exclude": b"# runner\nworkspace/.nanobot/\nworkspace/skills/\nworkspace/memory/\n"
+ b"workspace/AGENTS.md\n",
+ "workspace/AGENTS.md": b"doc", "workspace/skills/s/SKILL.md": b"skill",
+ "workspace/memory/m.md": b"mem", "workspace/deck.json": b"{}"}
+
+
+def test_the_nesting_base_s_own_directory_is_not_a_user_path():
+ """DELETE …/files/workspace on nanobot would have removed the agent's whole working directory."""
+ raw = _tar(_NANOBOT)
+ assert B.tar_change(raw, "delete", "workspace") == (B.WS_NOT_FOUND, None)
+ assert B.tar_change(raw, "move", "workspace", "w2") == (B.WS_NOT_FOUND, None)
+
+
[email protected]("op,path,to", [("delete", "skills", ""), ("delete", "AGENTS.md", ""),
+ ("delete", "memory/m.md", ""), ("move", "skills", "s2"),
+ ("move", "deck.json", "skills/deck.json"),
+ ("mkdir", "memory/new", "")])
+def test_the_base_s_excluded_machinery_is_refused(op, path, to):
+ assert B.tar_change(_tar(_NANOBOT), op, path, to) == (B.WS_BAD_PATH, None)
+
+
+def test_a_flat_base_s_excluded_files_are_refused_too():
+ """opensquilla seeds SOUL.md and memory/ at the session root and excludes them."""
+ raw = _tar({".git/info/exclude": b"SOUL.md\nmemory/\n", "SOUL.md": b"s", "memory/x": b"x",
+ "notes.md": b"n"})
+ assert B.tar_change(raw, "delete", "SOUL.md") == (B.WS_BAD_PATH, None)
+ assert B.tar_change(raw, "delete", "memory") == (B.WS_BAD_PATH, None)
+ assert B.tar_change(raw, "delete", "notes.md")[0] == B.WS_OK
+
+
+def test_a_file_written_at_the_root_between_turns_is_still_reachable_on_the_nesting_base():
+ """An app's PUT lands at the session root until the next turn adopts it."""
+ outcome, raw = B.tar_change(_tar({**_NANOBOT, "sheet.json": b"rows"}), "move", "sheet.json", "kept.json")
+ assert outcome == B.WS_OK and _files(raw)["kept.json"] == b"rows"
+
+
[email protected]("op,path,to", [("move", "a.md", "notes.txt/a.md"), ("mkdir", "notes.txt/sub", ""),
+ ("move", "a.md", "notes.txt/deeper/a.md")])
+def test_nothing_goes_under_a_file(op, path, to):
+ """Such an archive holds `notes.txt` and `notes.txt/a.md`, and no restore can unpack it."""
+ assert B.tar_change(_tar({"notes.txt": b"n", "a.md": b"a"}), op, path, to) == (B.WS_EXISTS, None)
+
+
+def test_a_rewritten_archive_unpacks(tmp_path):
+ _o, raw = B.tar_change(_tar({"./d": None, "./d/x": b"X", "./y": b"Y"}), "move", "d", "e/d")
+ with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as tf:
+ tf.extractall(tmp_path, filter="data")
+ assert (tmp_path / "e" / "d" / "x").read_bytes() == b"X" and (tmp_path / "y").read_bytes() == b"Y"
+
+
+async def test_a_hosted_rewrite_records_the_new_checkpoint_sha():
+ """A sandbox kept warm with the old workspace passes the hydrate probe on the old sha, and its next
+ checkpoint writes the deleted files back. The new sha matches no sandbox."""
+ import hashlib
+ blob, ws = _hosted({"a.txt": b"a", "b.txt": b"b"})
+ seen = []
+
+ async def _rewritten(sid, sha):
+ seen.append((sid, sha))
+ ws.on_rewrite = _rewritten
+ assert await ws.delete("s", "a.txt") == B.WS_OK
+ assert await ws.write("s", "c.txt", b"c")
+ assert await ws.delete("s", "missing") == B.WS_NOT_FOUND
+ assert len(seen) == 2
+ assert seen[-1] == ("s", hashlib.sha256(blob.objects[("kb", "sessions/s/workspace.tgz")]).hexdigest())
+
+
+async def test_two_changes_at_once_both_land():
+ import asyncio
+
+ class _Slow(_Blob):
+ async def get(self, kb, key):
+ await asyncio.sleep(0.01)
+ return await super().get(kb, key)
+
+ blob = _Slow({("kb", "sessions/s/workspace.tgz"): _tar({"a": b"a", "b": b"b", "c": b"c"})})
+ ws = B.CheckpointWorkspaceFiles(blob, "kb", lambda sid: f"sessions/{sid}/workspace.tgz")
+ assert await asyncio.gather(ws.delete("s", "a"), ws.delete("s", "b")) == [B.WS_OK, B.WS_OK]
+ assert _files(blob.objects[("kb", "sessions/s/workspace.tgz")]) == {"c": b"c"}
+
+
+async def test_selfhosted_mkdir_says_when_the_directory_was_there():
+ _blob, ws, _ = _selfhosted(_Resp(200, {"path": "d", "created": False}))
+ assert await ws.mkdir("s", "d") == B.WS_UNCHANGED
+
+
+async def test_mkdir_route_answers_created(routes):
+ routes["fake"].outcome = B.WS_UNCHANGED
+ assert (await A.mkdir_session_file("s", "d", request=None))["created"] is False
+
+
+async def test_a_move_clash_names_the_destination(routes):
+ routes["fake"].outcome = B.WS_EXISTS
+ with pytest.raises(Exception) as e:
+ await A.move_session_file("s", "a.md", A.WorkspaceMove(to="b.md"), request=None)
+ assert e.value.status_code == 409 and e.value.detail["param"] == "to"
+ assert "'b.md'" in e.value.detail["message"]
+
+
+async def test_a_tombstone_the_store_refused_is_reported_and_keeps_the_copy(routes, captures, monkeypatch):
+ await B.put_app_capture(A.BACKING.blob, "kb", "s", "sheet.json", b"rows")
+ cid = B.app_capture_id("sheet.json")
+
+ async def _refuse(fid, data, **_k):
+ return False
+ monkeypatch.setattr(A, "_blob_put", _refuse)
+ with pytest.raises(Exception) as e:
+ await A.delete_session_file("s", "sheet.json", request=None)
+ assert e.value.status_code == 502
+ assert routes.get("changed") == ["s"] # the change itself was made
+ assert f"containers/s/{cid}" in captures.store # the copy was not dropped unrecorded
+ assert "Repeat the delete" in e.value.detail["message"]
+
+
+async def test_a_delete_of_what_is_already_gone_still_records_it(routes, captures):
+ """The repeat a refused tombstone asks for: the workspace answers 404, the removal is recorded."""
+ captures("cfile_agent", "old.md", b"agent's", 100.0)
+ routes["fake"].outcome = B.WS_NOT_FOUND
+ assert (await A.delete_session_file("s", "old.md", request=None))["deleted"] is True
+ assert "old.md" not in await A._cfile_by_name("s")
+
+
+async def test_the_gateway_records_a_rewritten_checkpoint_s_sha_on_the_session(monkeypatch):
+ seen = []
+
+ class _Graph:
+ async def upsert(self, label, vid, props, *, raise_on_fail=False):
+ seen.append((label, vid, props, raise_on_fail))
+ monkeypatch.setattr(A.BACKING, "graph", _Graph(), raising=False)
+ await A._checkpoint_rewritten("s", "abc")
+ assert seen == [("HarnessSession", "s", {"ws_sha": "abc"}, True)]
+
+
+async def test_a_removed_directory_compacts_what_it_hides(captures):
+ for name in ("d/a.json", "d/b.json", "keep.json"):
+ await B.put_app_capture(A.BACKING.blob, "kb", "s", name, b"x")
+ await A._put_tombstone("s", "d/old", await A._capture_state("s"))
+ await A._put_tombstone("s", "d", await A._capture_state("s"))
+ metas = {json.loads(v)["filename"] for k, v in captures.store.items() if k.endswith(".meta")}
+ tombs = {json.loads(v)["filename"] for k, v in captures.store.items() if k.startswith("ws-removed/")}
+ assert metas == {"keep.json"} and tombs == {"d"}
+
+
+async def test_every_tombstone_is_read_beside_the_capture_window(captures):
+ """The captures stay a first page of 200 metas; the tombstones live apart and are read in full,
+ so none can fall out of a window and let the file it hid come back."""
+ for i in range(250):
+ captures(f"cfile_{i:04d}", f"f{i}.md", b"x", 100.0)
+ for i in range(30):
+ await A._put_tombstone("s", f"gone{i}.md")
+ await A._put_tombstone("s", "f0.md")
+ caps, tombs = await A._capture_state("s")
+ assert len(tombs) == 31 and "f0.md" not in caps and len(caps) == 199
+
+
+async def test_a_delete_of_a_path_that_never_existed_stores_nothing(routes, captures):
+ routes["fake"].outcome = B.WS_NOT_FOUND
+ for i in range(5):
+ with pytest.raises(Exception):
+ await A.delete_session_file("s", f"never{i}.md", request=None)
+ assert not [k for k in captures.store if k.startswith("ws-removed/")]
+
+
+async def test_the_read_fallback_scans_the_captures_once(captures, monkeypatch):
+ captures("cfile_legacy", "a.md", b"legacy", 50.0)
+ calls = []
+ real = A._capture_state
+
+ async def _counted(sid):
+ calls.append(sid)
+ return await real(sid)
+
+ class _Gone:
+ async def read(self, *_a, **_k):
+ return None
+
+ async def _owned(*_a, **_k):
+ return ("org", {})
+
+ monkeypatch.setattr(A, "_capture_state", _counted)
+ monkeypatch.setattr(A.BACKING, "workspace", _Gone(), raising=False)
+ monkeypatch.setattr(A, "_owned_session", _owned)
+ assert (await A.read_session_file("s", "notes/a.md", request=None)).body == b"legacy"
+ assert calls == ["s"]
+
+
+def test_a_hard_link_to_a_removed_file_keeps_its_bytes(tmp_path):
+ """tar stores a second hard link as a pointer to the first member's name. Removing or renaming that
+ member left a pointer to nothing, and an archive no restore can unpack."""
+ out = io.BytesIO()
+ with tarfile.open(fileobj=out, mode="w:gz") as tf:
+ info = tarfile.TarInfo("./a.txt"); info.size = 1
+ tf.addfile(info, io.BytesIO(b"A"))
+ for n in ("./b.txt", "./c.txt"):
+ ln = tarfile.TarInfo(n); ln.type, ln.linkname = tarfile.LNKTYPE, "./a.txt"
+ tf.addfile(ln)
+ outcome, raw = B.tar_change(out.getvalue(), "delete", "a.txt")
+ assert outcome == B.WS_OK
+ with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as tf:
+ tf.extractall(tmp_path / "del", filter="data")
+ assert (tmp_path / "del" / "b.txt").read_bytes() == b"A" and (tmp_path / "del" / "c.txt").read_bytes() == b"A"
+ outcome, raw = B.tar_change(out.getvalue(), "move", "a.txt", "d/a.txt")
+ with tarfile.open(fileobj=io.BytesIO(raw), mode="r:gz") as tf:
+ tf.extractall(tmp_path / "mv", filter="data")
+ assert (tmp_path / "mv" / "d" / "a.txt").read_bytes() == b"A" and (tmp_path / "mv" / "b.txt").read_bytes() == b"A"
+
+
+def test_a_new_folder_on_the_nesting_base_goes_in_the_agent_s_directory():
+ """Where the runner now makes it, so the live directory and the checkpoint agree."""
+ outcome, raw = B.tar_change(_tar(_NANOBOT), "mkdir", "slides/old")
+ assert outcome == B.WS_OK and "workspace/slides/old" in _files(raw)
+ raw = _tar({**_NANOBOT, "workspace/notes": b"a file"})
+ assert B.tar_change(raw, "mkdir", "notes/x") == (B.WS_EXISTS, None)
+
+
+async def test_a_sha_that_cannot_be_recorded_leaves_the_archive_as_it_was():
+ """Rewriting under a sha that still matches the warm sandbox would be undone by its next checkpoint."""
+ blob, ws = _hosted({"a.txt": b"a"})
+ before = dict(blob.objects)
+
+ async def _down(_sid, _sha):
+ raise RuntimeError("graph store down")
+ ws.on_rewrite = _down
+ assert await ws.delete("s", "a.txt") == B.WS_FAILED
+ assert not await ws.write("s", "b.txt", b"b")
+ assert blob.objects == before
+
+
+def test_a_delete_on_the_nesting_base_removes_every_copy():
+ """A file at the root beside the agent's copy (an app's PUT from before writes went where the file
+ is) would be adopted back next turn if only the agent's copy went."""
+ outcome, raw = B.tar_change(_tar({**_NANOBOT, "notes.md": b"app", "workspace/notes.md": b"agent"}),
+ "delete", "notes.md")
+ assert outcome == B.WS_OK and not {"notes.md", "workspace/notes.md"} & set(_files(raw))
+
+
+async def test_a_hosted_put_replaces_the_nesting_base_s_file_where_it_is():
+ blob, ws = _hosted({**_NANOBOT})
+ assert await ws.write("s", "deck.json", b'{"v":2}')
+ files = _files(blob.objects[("kb", "sessions/s/workspace.tgz")])
+ assert files["workspace/deck.json"] == b'{"v":2}' and "deck.json" not in files
+ assert await ws.read("s", "deck.json") == b'{"v":2}'
+ assert await ws.write("s", "new.json", b"n") # a new file: the root, adopted next turn
+ assert _files(blob.objects[("kb", "sessions/s/workspace.tgz")])["new.json"] == b"n"
+ assert await ws.read("s", "new.json") == b"n"
+
+
+async def test_a_put_waits_for_a_change_of_the_same_session(routes, monkeypatch):
+ """Under the same lock as delete, mkdir and move: a PUT racing a delete of its path would have
+ its fresh copy dropped by the delete's tombstone."""
+ import asyncio
+ order = []
+
+ class _Ws(_Fake):
+ async def delete(self, sid, path):
+ order.append("delete start")
+ await asyncio.sleep(0.02)
+ order.append("delete end")
+ return B.WS_OK
+
+ async def write(self, sid, path, data):
+ order.append("put")
+ return True
+
+ monkeypatch.setattr(A.BACKING, "workspace", _Ws(), raising=False)
+ await asyncio.gather(A.delete_session_file("s", "n.md", request=None),
+ A.write_session_file("s", "n.md", A.WorkspaceWrite(content="x"), request=None))
+ assert order == ["delete start", "delete end", "put"]
+
+
+def test_a_move_on_the_nesting_base_leaves_no_copy_under_the_old_name():
+ outcome, raw = B.tar_change(_tar({**_NANOBOT, "a.md": b"root", "workspace/a.md": b"agent"}),
+ "move", "a.md", "b.md")
+ files = _files(raw)
+ assert outcome == B.WS_OK and files["workspace/b.md"] == b"agent"
+ assert "a.md" not in files and "workspace/a.md" not in files
+
+
+async def test_a_hosted_put_of_a_new_file_goes_into_a_folder_the_agent_has():
+ """Adoption moves a root entry only when its name is free in the agent's directory, so a root
+ notes/new.md beside workspace/notes/ would never reach the agent."""
+ blob, ws = _hosted({**_NANOBOT, "workspace/notes/old.md": b"o"})
+ assert await ws.write("s", "notes/new.md", b"n")
+ files = _files(blob.objects[("kb", "sessions/s/workspace.tgz")])
+ assert files["workspace/notes/new.md"] == b"n" and "notes/new.md" not in files
+
+
+async def test_a_refused_move_does_not_blame_one_side(routes):
+ routes["fake"].outcome = B.WS_BAD_PATH
+ with pytest.raises(Exception) as e:
+ await A.move_session_file("s", "notes.md", A.WorkspaceMove(to="skills/notes.md"), request=None)
+ assert e.value.status_code == 400 and e.value.detail["param"] is None
+
+
+async def test_a_put_refreshes_the_cached_listing(routes, monkeypatch):
+ class _Ws(_Fake):
+ async def write(self, sid, path, data):
+ return True
+ monkeypatch.setattr(A.BACKING, "workspace", _Ws(), raising=False)
+ await A.write_session_file("s", "n.md", A.WorkspaceWrite(content="x"), request=None)
+ assert routes["changed"] == ["s"]
+
+
+# ── what the maintainer review of #478 found ──────────────────────────────────────────────────
+async def test_a_file_with_no_capture_is_read_from_the_checkpoint(captures, monkeypatch):
+ """A file the agent moved has no capture under its new name. Once the live directory is reaped
+ the checkpoint holds it there, and the by-path read finds it instead of answering 404 until the
+ next turn restores it."""
+ class _Gone:
+ async def read(self, *_a, **_k):
+ return None
+
+ async def _owned(*_a, **_k):
+ return ("org", {})
+
+ async def _bytes(sid, fid):
+ if fid == A._wf_id("archive/report.md"):
+ return b"moved", "text/markdown", "archive/report.md"
+ data = captures.store.get(f"containers/{sid}/{fid}")
+ return (data, "application/octet-stream", fid) if data is not None else None
+
+ monkeypatch.setattr(A.BACKING, "workspace", _Gone(), raising=False)
+ monkeypatch.setattr(A, "_owned_session", _owned)
+ monkeypatch.setattr(A, "_container_file_bytes", _bytes)
+ assert (await A.read_session_file("s", "archive/report.md", request=None)).body == b"moved"
+ captures("cfile_agent", "notes.md", b"captured", 100.0) # a capture still comes first
+ assert (await A.read_session_file("s", "notes.md", request=None)).body == b"captured"
+ with pytest.raises(Exception) as e:
+ await A.read_session_file("s", "report.md", request=None)
+ assert getattr(e.value, "status_code", None) == 404
+
+
[email protected]("call", ["delete", "mkdir", "move", "put"])
+async def test_a_change_that_waited_behind_a_turn_s_restore_is_refused(routes, monkeypatch, call):
+ """The route's busy check read the session before a turn was stamped running. The turn restores
+ the workspace under the change lock, so the change waits, reads the session again, and stops
+ rather than landing under the running turn."""
+ import asyncio
+
+ class _Ws(_Fake):
+ async def write(self, sid, path, data):
+ self.calls.append(("write", path))
+ return True
+ monkeypatch.setattr(A.BACKING, "workspace", _Ws(), raising=False)
+ routes["fake"] = A.BACKING.workspace
+ sid = f"busy-{call}" # a lock of its own: one bound to another test's event loop is not
+ run = {"delete": lambda: A.delete_session_file(sid, "a.txt", request=None),
+ "mkdir": lambda: A.mkdir_session_file(sid, "d", request=None),
+ "move": lambda: A.move_session_file(sid, "a.txt", A.WorkspaceMove(to="b.txt"), request=None),
+ "put": lambda: A.write_session_file(sid, "a.txt", A.WorkspaceWrite(content="x"), request=None)}[call]
+ lock = A._ws_change_lock(sid)
+ await lock.acquire() # the turn's restore, in progress
+ task = asyncio.create_task(run())
+ for _ in range(5):
+ await asyncio.sleep(0)
+ routes["v"] = {"status": "running", "turn_status": "starting"}
+ lock.release()
+ with pytest.raises(Exception) as e:
+ await task
+ assert e.value.status_code == 409 and e.value.detail["code"] == "session_busy"
+ assert routes["fake"].calls == []
+
+
+async def test_a_restore_waits_for_a_change_in_flight(monkeypatch):
+ """A change that passed its busy check before the turn was stamped running finishes before the
+ restore reads the workspace (the recycle route restores the same way a turn does)."""
+ import asyncio
+ order = []
+
+ async def _hydrate(sid, rec, force=False):
+ order.append("restore")
+ rec["hydrated"] = True
+
+ async def _vget(_sid):
+ return {"status": "idle"}
+
+ monkeypatch.setattr(A, "_hydrate", _hydrate)
+ monkeypatch.setattr(A, "_vertex_get", _vget)
+ lock = A._ws_change_lock("recycle-wait")
+ await lock.acquire() # a change, in progress
+ task = asyncio.create_task(A.recycle_session_sandbox("recycle-wait"))
+ for _ in range(5):
+ await asyncio.sleep(0)
+ order.append("change done")
+ lock.release()
+ assert (await task)["hydrated"] is True
+ assert order == ["change done", "restore"]
+
+
[email protected]("extra,path,to", [({"app.json": b"the app's"}, "deck.json", "app.json"),
+ ({"sheet.json": b"rows"}, "sheet.json", "deck.json")])
+def test_a_move_onto_a_name_taken_on_the_other_side_is_refused(extra, path, to):
+ """nanobot: a file an app put at the root waits there for adoption. Moving the agent's deck.json to
+ app.json answered 200 on the candidate (0.35.0-rc.1), and the moved file then shadowed the app's
+ app.json for good, since adoption never moves over a name the agent's directory has. The same the
+ other way round."""
+ raw = _tar({**_NANOBOT, **extra})
+ assert B.tar_change(raw, "move", path, to) == (B.WS_EXISTS, None)
diff --git a/gateway/tests/test_session_file_fallback.py b/gateway/tests/test_session_file_fallback.py
index 5d24a859..4fcbd9bf 100644
--- a/gateway/tests/test_session_file_fallback.py
+++ b/gateway/tests/test_session_file_fallback.py
@@ -36,14 +36,14 @@ async def test_by_path_falls_back_to_the_captured_blob(monkeypatch):
async def _noop():
return ("org", {})
- async def _byname(_sid):
+ async def _state(_sid):
assert _sid == sid
- return {"dashboard.json": "cfile_abc"}
+ return {"dashboard.json": ("cfile_abc", False)}, {}
async def _bytes(_sid, fid):
return (body, "application/json", "dashboard.json") if fid == "cfile_abc" else None
- monkeypatch.setattr(A, "_cfile_by_name", _byname)
+ monkeypatch.setattr(A, "_capture_state", _state)
monkeypatch.setattr(A, "_container_file_bytes", _bytes)
r = await A.read_session_file(sid, "dashboard.json", request=None)
@@ -62,10 +62,10 @@ async def test_a_file_that_never_existed_still_404s(monkeypatch):
monkeypatch.setattr(A.BACKING, "workspace", _Gone(), raising=False)
monkeypatch.setattr(A, "_owned_session", lambda *a, **k: _noop(), raising=False)
- monkeypatch.setattr(A, "_cfile_by_name", lambda _s: _empty())
+ monkeypatch.setattr(A, "_capture_state", lambda _s: _empty())
async def _empty():
- return {}
+ return {}, {}
with pytest.raises(Exception):
await A.read_session_file("hsess_x", "nope.json", request=None)
diff --git a/runner/server.py b/runner/server.py
index 95daa638..79ab9604 100644
--- a/runner/server.py
+++ b/runner/server.py
@@ -50,10 +50,12 @@ import http.client
import ipaddress
import http.server
import datetime
+import errno
import json
import mimetypes
import os
import pathlib
+import posixpath
import re
import shlex
import shutil
@@ -12799,6 +12801,62 @@ def get_file(path: str, identifier: str = "") -> Response:
return Response(content=dest.read_bytes(), media_type=media)
+_DIR_NOFOLLOW = os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW
+
+
+def _open_dir(base: pathlib.Path, parts: tuple[str, ...], uid: int | None = None, make: bool = False,
+ made: list[str] | None = None) -> int:
+ """An fd on the directory `base`/`parts`, reached one name at a time without following a symlink.
+
+ The file routes check a path and then change it, as the runner, which is root under the
+ write-wall. A process the session left running can swap a checked directory for a symlink in
+ between, and a root write, unlink, rename or chown through that link would land outside the
+ session. Opened with O_NOFOLLOW at every step, the swap fails the change instead.
+
+ `make` creates what is missing (mkdir -p), each directory owned by `uid` (a root-owned directory
+ in a workspace is one the session's own process cannot write into), and lists in `made` what it
+ created. A FILE or a symlink where a directory would be raises NotADirectoryError."""
+ fd = os.open(base, _DIR_NOFOLLOW)
+ try:
+ for name in parts:
+ try:
+ nxt = os.open(name, _DIR_NOFOLLOW, dir_fd=fd)
+ except FileNotFoundError:
+ if not make:
+ raise
+ os.mkdir(name, 0o755, dir_fd=fd)
+ if uid is not None:
+ os.chown(name, uid, uid, dir_fd=fd, follow_symlinks=False)
+ if made is not None:
+ made.append(name)
+ nxt = os.open(name, _DIR_NOFOLLOW, dir_fd=fd)
+ except OSError as e:
+ if e.errno != errno.ELOOP:
+ raise
+ # a symlink: Linux answers ELOOP, macOS ENOTDIR; one answer for both
+ raise NotADirectoryError(errno.ENOTDIR, "a symlink is where a directory would be", name) from None
+ os.close(fd)
+ fd = nxt
+ return fd
+ except BaseException:
+ os.close(fd)
+ raise
+
+
+def _remove_at(base: pathlib.Path, p: pathlib.Path) -> None:
+ """Remove `p` (a path under `base`) as itself: a directory with everything in it, a file or a
+ symlink as the entry it is. Through _open_dir, so nothing swapped in redirects it."""
+ parts = p.relative_to(base).parts
+ fd = _open_dir(base, parts[:-1])
+ try:
+ if stat.S_ISDIR(os.lstat(parts[-1], dir_fd=fd).st_mode):
+ shutil.rmtree(parts[-1], dir_fd=fd) # fd-based all the way down; links not followed
+ else:
+ os.unlink(parts[-1], dir_fd=fd)
+ finally:
+ os.close(fd)
+
+
@app.put("/file")
async def put_file(request: Request, path: str, identifier: str = "") -> dict:
"""Write ONE file into a session's workspace, as that session.
@@ -12812,34 +12870,239 @@ async def put_file(request: Request, path: str, identifier: str = "") -> dict:
dest = _safe_join(ws, path)
if dest is None:
raise HTTPException(400, "path escapes the workspace")
+ # A file that is already there is replaced WHERE it is. On a nesting base a new file lands at the
+ # session root and the next turn adopts it, but one the agent already has is in its directory:
+ # written at the root it sat beside the agent's copy, which every read found first, and which
+ # adoption would not move over.
+ have = _existing(ws, path)
+ if have is not None and have.is_file() and not have.is_symlink():
+ dest = have
+ elif have is None:
+ # A NEW file whose top folder the agent's directory already has goes into that folder:
+ # adoption moves a root entry only when its name is free in the agent's directory, so it
+ # would never reach the agent.
+ vis = _visible_root(ws)
+ top = (path or "").strip().strip("/").split("/", 1)[0]
+ if vis != ws and "/" in (path or "").strip().strip("/") and (pathlib.Path(vis) / top).is_dir():
+ inside = _safe_join(vis, path)
+ if inside is not None:
+ dest = inside
data = await request.body()
pathlib.Path(ws).mkdir(parents=True, exist_ok=True)
_isolate_session(ws)
uid = _session_uid(ws)
- missing: list[pathlib.Path] = []
- probe = dest.parent
- while not probe.exists():
- missing.append(probe)
- probe = probe.parent
- dest.parent.mkdir(parents=True, exist_ok=True)
- tmp = dest.parent / f".hr-put-{uuid.uuid4().hex[:8]}"
+ base = pathlib.Path(ws).resolve()
+ parts = dest.relative_to(base).parts
+ if not parts:
+ raise HTTPException(400, "the workspace itself is not a file")
+ tmp = f".hr-put-{uuid.uuid4().hex[:8]}"
+ try:
+ fd = _open_dir(base, parts[:-1], uid, make=True)
+ except OSError as e:
+ raise HTTPException(500, f"write failed: {str(e)[:200]}")
try:
- tmp.write_bytes(data)
- os.chmod(tmp, 0o644)
- if uid is not None:
- for d in missing:
- os.chown(d, uid, uid)
- os.chown(tmp, uid, uid)
- os.replace(tmp, dest)
+ out = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o644, dir_fd=fd)
+ with os.fdopen(out, "wb") as f:
+ f.write(data)
+ os.fchmod(f.fileno(), 0o644)
+ if uid is not None:
+ os.fchown(f.fileno(), uid, uid)
+ os.replace(tmp, parts[-1], src_dir_fd=fd, dst_dir_fd=fd)
except OSError as e:
try:
- tmp.unlink()
+ os.unlink(tmp, dir_fd=fd)
except OSError:
pass
raise HTTPException(500, f"write failed: {str(e)[:200]}")
+ finally:
+ os.close(fd)
return {"ok": True, "path": str(dest.relative_to(ws)), "bytes": len(data)}
+def _safe_leaf(root: str, rel: str) -> pathlib.Path | None:
+ """_safe_join for a path that is about to be removed or renamed: the PARENT is resolved and
+ checked, the last component is not. _safe_join resolves symlinks all the way, so a link inside
+ the workspace would name its target, and removing "the link" would remove the file it points
+ to. None for an escape, or for the root itself."""
+ rel = (rel or "").strip().strip("/")
+ name = posixpath.basename(rel)
+ if not rel or name in ("", ".", ".."):
+ return None
+ parent = _safe_join(root, posixpath.dirname(rel))
+ return None if parent is None else parent / name
+
+
+def _ws_excluded(ws: str) -> tuple[str, ...]:
+ """The base's own machinery, as _git_exclude listed it, relative to the session directory."""
+ try:
+ text = pathlib.Path(ws, ".git", "info", "exclude").read_text(errors="replace")
+ except OSError:
+ return ()
+ return tuple(ln.strip() for ln in text.splitlines() if ln.strip() and not ln.startswith("#"))
+
+
+def _ws_protected(rel: str, pats: tuple[str, ...]) -> bool:
+ """`rel` is one of those patterns' paths, under one, or a directory holding one. The gateway's
+ backing.ws_protected applies the same rule to the checkpoint; the two processes ship separately
+ (the runner image carries no gateway code), so it is written twice, and
+ runner/tests/test_file_changes.py holds the two equal over the same cases."""
+ for q in pats:
+ d = q.rstrip("/")
+ if rel == d or (q.endswith("/") and rel.startswith(q)) or d.startswith(rel + "/"):
+ return True
+ return False
+
+
+def _search_roots(ws: str, path: str) -> list[str]:
+ """Where a path the caller names may be: the visible root first, the session root second, the
+ order GET /file reads in.
+
+ The session root is searched on a nesting base only for what _adopt_into would adopt — a file an
+ app wrote there between turns. Never the runner's own entries, and never the agent's directory
+ itself: `workspace` at nanobot's root is the agent's whole working directory, its skills, memory
+ and instructions with the user's files, and a delete of it would take all of them."""
+ vis = _visible_root(ws)
+ roots = [vis]
+ if vis != ws:
+ sub = os.path.basename(vis.rstrip("/"))
+ keep = ({".git", ".gitignore", sub} | _PRODUCED_EXCLUDE_NAMES | _NOISE_SEG
+ | {q.rstrip("/") for q in _PRODUCED_EXCLUDE_PREFIX})
+ if (path or "").strip().strip("/").split("/", 1)[0] not in keep:
+ roots.append(ws)
+ return roots
+
+
+def _copies(ws: str, path: str) -> list[pathlib.Path]:
+ """Every place the path is, in that order. On a nesting base it can be in both: a file the app
+ wrote at the root between turns beside the agent's copy, which _adopt_into will not move over it."""
+ out = []
+ for root in _search_roots(ws, path):
+ p = _safe_leaf(root, path)
+ if p is not None and (p.exists() or p.is_symlink()):
+ out.append(p)
+ return out
+
+
+def _existing(ws: str, path: str) -> pathlib.Path | None:
+ """Where a path the caller names already is (the first of _copies)."""
+ found = _copies(ws, path)
+ return found[0] if found else None
+
+
+def _refuse_protected(ws: str, *paths: pathlib.Path) -> None:
+ """400 for a path the base excludes as its own. On a nesting base a path at the session root is
+ adopted into the agent's directory at the next turn, so it is also checked as it will be there."""
+ pats = _ws_excluded(ws)
+ if not pats:
+ return
+ base = pathlib.Path(ws).resolve()
+ vis = pathlib.Path(_visible_root(ws)).resolve()
+ sub = str(vis.relative_to(base)) if vis != base else ""
+ for p in paths:
+ rel = str(p.relative_to(base))
+ if _ws_protected(rel, pats) or (sub and not (rel == sub or rel.startswith(sub + "/"))
+ and _ws_protected(f"{sub}/{rel}", pats)):
+ raise HTTPException(400, "that is the agent's own machinery, not a workspace file")
+
+
+def delete_file(path: str, identifier: str = "") -> dict:
+ """Remove ONE file or directory from a session's workspace, as that session (see PUT /file).
+ A directory goes with everything in it, as an environment's source delete does."""
+ ws = _ws(identifier)
+ if _safe_leaf(ws, path) is None:
+ raise HTTPException(400, "path escapes the workspace")
+ found = _copies(ws, path)
+ if not found:
+ raise HTTPException(404, "nothing at that path")
+ _refuse_protected(ws, *found)
+ base = pathlib.Path(ws).resolve()
+ try:
+ # every copy: one left at the root of a nesting base would be adopted back next turn
+ for p in found:
+ _remove_at(base, p)
+ except OSError as e:
+ raise HTTPException(500, f"delete failed: {str(e)[:200]}")
+ return {"ok": True, "path": str(found[0].relative_to(base))}
+
+
[email protected]("/file/mkdir")
+def mkdir_file(path: str, identifier: str = "") -> dict:
+ """Create a directory (and any missing parents), owned by the session's uid. One that already
+ exists is success; a FILE in the way is 409. A new one goes in the user's root — the agent's
+ directory on a nesting base, where the checkpoint's copy of the change puts it too."""
+ ws = _ws(identifier)
+ if _safe_leaf(ws, path) is None:
+ raise HTTPException(400, "path escapes the workspace")
+ pathlib.Path(ws).mkdir(parents=True, exist_ok=True)
+ _isolate_session(ws)
+ dest = _existing(ws, path) or _safe_leaf(_visible_root(ws), path)
+ if dest is None:
+ raise HTTPException(400, "path escapes the workspace")
+ _refuse_protected(ws, dest)
+ if dest.is_symlink() or (dest.exists() and not dest.is_dir()):
+ raise HTTPException(409, "a file is at that path")
+ base = pathlib.Path(ws).resolve()
+ made: list[str] = []
+ try:
+ os.close(_open_dir(base, dest.relative_to(base).parts, _session_uid(ws), make=True, made=made))
+ except (FileExistsError, NotADirectoryError) as e:
+ raise HTTPException(409, f"mkdir failed: {str(e)[:200]}")
+ except OSError as e:
+ raise HTTPException(500, f"mkdir failed: {str(e)[:200]}")
+ return {"ok": True, "path": str(dest.relative_to(base)), "created": bool(made)}
+
+
[email protected]("/file/move")
+def move_file(path: str, to: str, identifier: str = "") -> dict:
+ """Rename a file or directory inside a session's workspace. The destination is on the same side
+ as the source (the nesting base's agent directory, or the session root), must not exist, and
+ must not be inside the source. Missing parent directories are made, as PUT /file makes them."""
+ ws = _ws(identifier)
+ if _safe_leaf(ws, path) is None or _safe_leaf(ws, to) is None:
+ raise HTTPException(400, "path escapes the workspace")
+ found = _copies(ws, path)
+ if not found:
+ raise HTTPException(404, "nothing at that path")
+ src, others = found[0], found[1:]
+ vis = pathlib.Path(_visible_root(ws)).resolve()
+ root = vis if vis != pathlib.Path(ws).resolve() and src.is_relative_to(vis) else pathlib.Path(ws).resolve()
+ dest = _safe_leaf(str(root), to)
+ if dest is None:
+ raise HTTPException(400, "path escapes the workspace")
+ if dest == src or dest.is_relative_to(src):
+ raise HTTPException(400, "a path cannot move inside itself")
+ _refuse_protected(ws, src, dest, *others)
+ # _copies(ws, to): on a nesting base the name can be taken on the OTHER side too, by a file an app
+ # put at the session root (it waits there for adoption) or by the agent's own. Moved in beside it,
+ # one of the two shadows the other for good (adoption never moves over a name the agent's
+ # directory has): an overwrite by another name.
+ if dest.exists() or dest.is_symlink() or _copies(ws, to):
+ raise HTTPException(409, "something is already at the destination")
+ base = pathlib.Path(ws).resolve()
+ sparts, dparts = src.relative_to(base).parts, dest.relative_to(base).parts
+ try:
+ sfd = _open_dir(base, sparts[:-1])
+ try:
+ dfd = _open_dir(base, dparts[:-1], _session_uid(ws), make=True)
+ try:
+ os.rename(sparts[-1], dparts[-1], src_dir_fd=sfd, dst_dir_fd=dfd)
+ finally:
+ os.close(dfd)
+ finally:
+ os.close(sfd)
+ # A second copy at the root of a nesting base (an app's PUT from before writes went where the
+ # file is) would be adopted back under the old name next turn: the old name is gone.
+ for p in others:
+ _remove_at(base, p)
+ except (FileExistsError, NotADirectoryError) as e:
+ # a FILE where one of the destination's folders would be
+ raise HTTPException(409, f"move failed: {str(e)[:200]}")
+ except OSError as e:
+ raise HTTPException(500, f"move failed: {str(e)[:200]}")
+ return {"ok": True, "path": str(src.relative_to(base)), "to": str(dest.relative_to(base))}
+
+
@app.get("/capabilities")
def capabilities(identifier: str = "") -> dict:
_wsdir = _ws(identifier)
diff --git a/runner/tests/test_file_changes.py b/runner/tests/test_file_changes.py
new file mode 100644
index 00000000..bf3886d5
--- /dev/null
+++ b/runner/tests/test_file_changes.py
@@ -0,0 +1,340 @@
+"""DELETE /file, POST /file/mkdir and POST /file/move: an app changing a session's workspace, through
+the runner, which is the one process that acts as the session (see PUT /file).
+
+What each must hold: nothing outside the workspace is reachable, by `..` or by a symlink; a symlink
+is removed or renamed as itself, never as the file it points to; the nesting base's user files
+(nanobot keeps them one level down) are found where GET /file finds them; and the answer names the
+path inside the session directory, because that is where the gateway changes the checkpoint too.
+"""
+import os
+import sys
+from pathlib import Path
+
+import pytest
+from fastapi.testclient import TestClient
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+import server # noqa: E402
+
+
+def ws(tmp_path, monkeypatch):
+ root = tmp_path / "sess"
+ root.mkdir()
+ monkeypatch.setattr(server, "_SESSION_UIDS", False)
+ monkeypatch.setattr(server, "_ws", lambda identifier="": str(root))
+ return root
+
+
+def c():
+ return TestClient(server.app)
+
+
+def test_delete_a_file(ws, c):
+ (ws / "a.txt").write_text("x")
+ r = c.delete("/file", params={"identifier": "s", "path": "a.txt"})
+ assert r.status_code == 200 and r.json()["path"] == "a.txt"
+ assert not (ws / "a.txt").exists()
+
+
+def test_delete_a_directory_with_everything_in_it(ws, c):
+ (ws / "d" / "e").mkdir(parents=True)
+ (ws / "d" / "e" / "f.txt").write_text("x")
+ assert c.delete("/file", params={"identifier": "s", "path": "d"}).status_code == 200
+ assert not (ws / "d").exists()
+
+
+def test_delete_nothing_is_404(ws, c):
+ assert c.delete("/file", params={"identifier": "s", "path": "nope.txt"}).status_code == 404
+
+
[email protected]("path", ["../x", "../../etc/passwd", "a/../../x"])
+def test_escapes_are_refused(ws, c, path):
+ (ws.parent / "x").write_text("outside")
+ assert c.delete("/file", params={"identifier": "s", "path": path}).status_code == 400
+ assert c.post("/file/mkdir", params={"identifier": "s", "path": path}).status_code == 400
+ (ws / "a.txt").write_text("x")
+ assert c.post("/file/move", params={"identifier": "s", "path": "a.txt", "to": path}).status_code == 400
+ assert (ws.parent / "x").read_text() == "outside"
+
+
+def test_the_workspace_itself_is_not_a_path(ws, c):
+ for p in ("", "/", "."):
+ assert c.delete("/file", params={"identifier": "s", "path": p}).status_code in (400, 404)
+ assert ws.is_dir()
+
+
+def test_a_symlink_is_removed_not_its_target(ws, c):
+ (ws / "real.txt").write_text("keep me")
+ os.symlink(ws / "real.txt", ws / "link.txt")
+ assert c.delete("/file", params={"identifier": "s", "path": "link.txt"}).status_code == 200
+ assert not (ws / "link.txt").is_symlink() and (ws / "real.txt").read_text() == "keep me"
+
+
+def test_a_symlink_out_of_the_workspace_is_removed_as_a_link(ws, c):
+ outside = ws.parent / "secret.txt"
+ outside.write_text("theirs")
+ os.symlink(outside, ws / "escape")
+ assert c.delete("/file", params={"identifier": "s", "path": "escape"}).status_code == 200
+ assert outside.read_text() == "theirs"
+
+
+def test_mkdir_makes_parents_and_is_idempotent(ws, c):
+ r = c.post("/file/mkdir", params={"identifier": "s", "path": "a/b/c"})
+ assert r.status_code == 200 and r.json()["created"] is True and (ws / "a" / "b" / "c").is_dir()
+ r = c.post("/file/mkdir", params={"identifier": "s", "path": "a/b/c"})
+ assert r.status_code == 200 and r.json()["created"] is False
+
+
+def test_mkdir_over_a_file_is_409(ws, c):
+ (ws / "a").write_text("x")
+ assert c.post("/file/mkdir", params={"identifier": "s", "path": "a"}).status_code == 409
+ assert c.post("/file/mkdir", params={"identifier": "s", "path": "a/b"}).status_code == 409
+
+
+def test_move_a_file_into_a_new_directory(ws, c):
+ (ws / "a.txt").write_text("x")
+ r = c.post("/file/move", params={"identifier": "s", "path": "a.txt", "to": "notes/b.txt"})
+ assert r.status_code == 200 and r.json() == {"ok": True, "path": "a.txt", "to": "notes/b.txt"}
+ assert (ws / "notes" / "b.txt").read_text() == "x" and not (ws / "a.txt").exists()
+
+
+def test_move_a_directory(ws, c):
+ (ws / "d").mkdir()
+ (ws / "d" / "f.txt").write_text("x")
+ assert c.post("/file/move", params={"identifier": "s", "path": "d", "to": "e"}).status_code == 200
+ assert (ws / "e" / "f.txt").read_text() == "x" and not (ws / "d").exists()
+
+
+def test_move_never_overwrites(ws, c):
+ (ws / "a.txt").write_text("a")
+ (ws / "b.txt").write_text("b")
+ assert c.post("/file/move", params={"identifier": "s", "path": "a.txt", "to": "b.txt"}).status_code == 409
+ assert (ws / "b.txt").read_text() == "b" and (ws / "a.txt").read_text() == "a"
+
+
+def test_move_into_itself_is_refused(ws, c):
+ (ws / "d").mkdir()
+ assert c.post("/file/move", params={"identifier": "s", "path": "d", "to": "d/inner"}).status_code == 400
+ assert (ws / "d").is_dir()
+
+
+def test_move_of_nothing_is_404(ws, c):
+ assert c.post("/file/move", params={"identifier": "s", "path": "nope", "to": "x"}).status_code == 404
+
+
+def test_the_nesting_base_finds_user_files_one_level_down(ws, c):
+ """nanobot's user files live in the agent's directory; the answer names them inside the
+ session directory, which is where they are in the checkpoint."""
+ (ws / ".harness").mkdir()
+ (ws / ".harness" / "agent-subdir").write_text("workspace")
+ (ws / "workspace").mkdir()
+ (ws / "workspace" / "deck.json").write_text("{}")
+ r = c.post("/file/move", params={"identifier": "s", "path": "deck.json", "to": "old/deck.json"})
+ assert r.status_code == 200
+ assert r.json()["path"] == "workspace/deck.json" and r.json()["to"] == "workspace/old/deck.json"
+ assert (ws / "workspace" / "old" / "deck.json").exists()
+ r = c.delete("/file", params={"identifier": "s", "path": "old"})
+ assert r.status_code == 200 and r.json()["path"] == "workspace/old"
+
+
+# ── what the review of #478 found ─────────────────────────────────────────────────────────────
+def _nanobot(ws):
+ (ws / ".harness").mkdir()
+ (ws / ".harness" / "agent-subdir").write_text("workspace")
+ (ws / ".git" / "info").mkdir(parents=True)
+ (ws / ".git" / "info" / "exclude").write_text(
+ "# runner\nworkspace/.nanobot/\nworkspace/skills/\nworkspace/memory/\nworkspace/AGENTS.md\n")
+ (ws / "workspace" / "skills" / "s").mkdir(parents=True)
+ (ws / "workspace" / "skills" / "s" / "SKILL.md").write_text("skill")
+ (ws / "workspace" / "AGENTS.md").write_text("doc")
+ (ws / "workspace" / "deck.json").write_text("{}")
+
+
+def test_the_nesting_base_s_own_directory_is_not_a_user_path(ws, c):
+ """DELETE workspace would have removed nanobot's whole working directory."""
+ _nanobot(ws)
+ assert c.delete("/file", params={"identifier": "s", "path": "workspace"}).status_code == 404
+ assert c.post("/file/move", params={"identifier": "s", "path": "workspace", "to": "w2"}).status_code == 404
+ assert (ws / "workspace" / "deck.json").exists() and (ws / "workspace" / "skills").is_dir()
+
+
+ ("delete", {"path": "skills"}), ("delete", {"path": "AGENTS.md"}),
+ ("delete", {"path": "skills/s/SKILL.md"}), ("move", {"path": "skills", "to": "s2"}),
+ ("move", {"path": "deck.json", "to": "skills/deck.json"}), ("mkdir", {"path": "memory/new"})])
+def test_the_base_s_excluded_machinery_is_refused(ws, c, call):
+ _nanobot(ws)
+ op, params = call
+ if op == "delete":
+ r = c.delete("/file", params={"identifier": "s", **params})
+ else:
+ r = c.post(f"/file/{op}", params={"identifier": "s", **params})
+ assert r.status_code == 400
+ assert (ws / "workspace" / "skills" / "s" / "SKILL.md").exists() and (ws / "workspace" / "AGENTS.md").exists()
+
+
+def test_a_flat_base_s_excluded_files_are_refused(ws, c):
+ (ws / ".git" / "info").mkdir(parents=True)
+ (ws / ".git" / "info" / "exclude").write_text("SOUL.md\nmemory/\n")
+ (ws / "SOUL.md").write_text("s")
+ (ws / "notes.md").write_text("n")
+ assert c.delete("/file", params={"identifier": "s", "path": "SOUL.md"}).status_code == 400
+ assert c.delete("/file", params={"identifier": "s", "path": "notes.md"}).status_code == 200
+ assert (ws / "SOUL.md").exists()
+
+
+def test_a_file_written_at_the_root_between_turns_is_still_reachable(ws, c):
+ """An app's PUT lands at the session root until the next turn adopts it into the agent's directory."""
+ _nanobot(ws)
+ (ws / "sheet.json").write_text("rows")
+ r = c.post("/file/move", params={"identifier": "s", "path": "sheet.json", "to": "kept.json"})
+ assert r.status_code == 200 and (ws / "kept.json").read_text() == "rows"
+
+
+def test_a_move_under_a_file_is_409(ws, c):
+ (ws / "notes.txt").write_text("n")
+ (ws / "a.md").write_text("a")
+ r = c.post("/file/move", params={"identifier": "s", "path": "a.md", "to": "notes.txt/a.md"})
+ assert r.status_code == 409 and (ws / "a.md").exists()
+
+
+def test_a_new_folder_on_the_nesting_base_goes_in_the_agent_s_directory(ws, c):
+ """Where the checkpoint's copy of the change puts it (backing.tar_change), so the two agree."""
+ _nanobot(ws)
+ r = c.post("/file/mkdir", params={"identifier": "s", "path": "slides/old"})
+ assert r.status_code == 200 and r.json()["path"] == "workspace/slides/old"
+ assert (ws / "workspace" / "slides" / "old").is_dir() and not (ws / "slides").exists()
+ (ws / "workspace" / "notes").write_text("a file")
+ assert c.post("/file/mkdir", params={"identifier": "s", "path": "notes/x"}).status_code == 409
+
+
+def test_an_existing_root_folder_is_found_before_a_new_one_is_made(ws, c):
+ """A folder an app's PUT made at the root between turns is the one mkdir answers for."""
+ _nanobot(ws)
+ (ws / "data").mkdir()
+ r = c.post("/file/mkdir", params={"identifier": "s", "path": "data"})
+ assert r.status_code == 200 and r.json()["created"] is False and not (ws / "workspace" / "data").exists()
+
+
[email protected]("rel", ["skills", "skills/x", "workspace/skills", "workspace/skills/s/SKILL.md",
+ "workspace", "workspace/deck.json", "SOUL.md", "SOUL.md.bak", "memory",
+ "memory/a", "memo", "workspace/AGENTS.md", "workspace/AGENTS.mdx"])
+def test_the_protection_rule_is_the_gateway_s(rel):
+ """The runner and the gateway's checkpoint path each apply it; the two must agree."""
+ sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "gateway"))
+ import backing # noqa: E402
+ pats = ("workspace/skills/", "workspace/AGENTS.md", "SOUL.md", "memory/")
+ assert server._ws_protected(rel, pats) == backing.ws_protected(rel, pats)
+
+
+def test_a_put_replaces_the_nesting_base_s_file_where_it_is(ws, c):
+ """Written at the root it sat beside the agent's copy, which every read found first."""
+ _nanobot(ws)
+ r = c.put("/file", params={"identifier": "s", "path": "deck.json"}, content=b'{"v":2}')
+ assert r.status_code == 200 and r.json()["path"] == "workspace/deck.json"
+ assert (ws / "workspace" / "deck.json").read_bytes() == b'{"v":2}' and not (ws / "deck.json").exists()
+ r = c.put("/file", params={"identifier": "s", "path": "new.json"}, content=b"n")
+ assert r.json()["path"] == "new.json" # a new file: the root, adopted next turn
+
+
+def test_a_delete_on_the_nesting_base_removes_every_copy(ws, c):
+ _nanobot(ws)
+ (ws / "notes.md").write_text("app") # an older PUT, beside the agent's copy
+ (ws / "workspace" / "notes.md").write_text("agent")
+ assert c.delete("/file", params={"identifier": "s", "path": "notes.md"}).status_code == 200
+ assert not (ws / "notes.md").exists() and not (ws / "workspace" / "notes.md").exists()
+
+
+def test_a_move_on_the_nesting_base_leaves_no_copy_under_the_old_name(ws, c):
+ """A root copy left behind would be adopted back under the old name next turn."""
+ _nanobot(ws)
+ (ws / "a.md").write_text("root")
+ (ws / "workspace" / "a.md").write_text("agent")
+ r = c.post("/file/move", params={"identifier": "s", "path": "a.md", "to": "b.md"})
+ assert r.status_code == 200 and (ws / "workspace" / "b.md").read_text() == "agent"
+ assert not (ws / "a.md").exists() and not (ws / "workspace" / "a.md").exists()
+
+
+def test_a_put_of_a_new_file_goes_into_a_folder_the_agent_has(ws, c):
+ _nanobot(ws)
+ (ws / "workspace" / "notes").mkdir()
+ r = c.put("/file", params={"identifier": "s", "path": "notes/new.md"}, content=b"n")
+ assert r.status_code == 200 and (ws / "workspace" / "notes" / "new.md").read_bytes() == b"n"
+ assert not (ws / "notes").exists()
+
+
+# ── what the maintainer review of #478 found ──────────────────────────────────────────────────
+# The routes check a path and then change it, as the runner, which is root under the write-wall. A
+# process the session left running between turns can swap a checked directory for a symlink in that
+# gap. Each test makes the swap at the gap (the hook the route calls between its check and its
+# change) and holds that nothing outside the workspace is touched: the change fails instead.
+def _outside(ws):
+ out = ws.parent / "outside"
+ out.mkdir()
+ (out / "x.txt").write_text("theirs")
+ return out
+
+
+def _swap_in_link(ws, name, target):
+ import shutil
+ if (ws / name).is_dir():
+ shutil.rmtree(ws / name)
+ os.symlink(target, ws / name)
+
+
+def test_a_delete_does_not_follow_a_directory_swapped_for_a_link(ws, c, monkeypatch):
+ out = _outside(ws)
+ (ws / "d").mkdir()
+ (ws / "d" / "x.txt").write_text("mine")
+ monkeypatch.setattr(server, "_refuse_protected", lambda *_a: _swap_in_link(ws, "d", out))
+ assert c.delete("/file", params={"identifier": "s", "path": "d/x.txt"}).status_code == 500
+ assert (out / "x.txt").read_text() == "theirs"
+
+
+def test_a_move_does_not_follow_a_directory_swapped_for_a_link(ws, c, monkeypatch):
+ out = _outside(ws)
+ (ws / "a.txt").write_text("mine")
+ (ws / "e").mkdir()
+ monkeypatch.setattr(server, "_refuse_protected", lambda *_a: _swap_in_link(ws, "e", out))
+ r = c.post("/file/move", params={"identifier": "s", "path": "a.txt", "to": "e/a.txt"})
+ assert r.status_code == 409 # a link where a directory would be, as a file there is
+ assert not (out / "a.txt").exists() and (ws / "a.txt").read_text() == "mine"
+
+
+def test_a_mkdir_does_not_follow_a_directory_swapped_for_a_link(ws, c, monkeypatch):
+ out = _outside(ws)
+ (ws / "d").mkdir()
+ monkeypatch.setattr(server, "_refuse_protected", lambda *_a: _swap_in_link(ws, "d", out))
+ assert c.post("/file/mkdir", params={"identifier": "s", "path": "d/new"}).status_code == 409
+ assert not (out / "new").exists()
+
+
+def test_a_put_does_not_follow_a_directory_swapped_for_a_link(ws, c, monkeypatch):
+ out = _outside(ws)
+ (ws / "d").mkdir()
+ monkeypatch.setattr(server, "_isolate_session", lambda *_a: _swap_in_link(ws, "d", out))
+ r = c.put("/file", params={"identifier": "s", "path": "d/x.txt"}, content=b"app")
+ assert r.status_code == 500
+ assert (out / "x.txt").read_text() == "theirs" and sorted(p.name for p in out.iterdir()) == ["x.txt"]
+
+
+def test_a_put_of_the_workspace_itself_is_refused(ws, c):
+ """Before, it wrote its temporary file into the directory that holds every session's workspace."""
+ for p in (".", "/"):
+ assert c.put("/file", params={"identifier": "s", "path": p}, content=b"x").status_code == 400
+ assert sorted(q.name for q in ws.parent.iterdir()) == ["sess"]
+
+
[email protected]("root_file,path,to", [("app.json", "deck.json", "app.json"),
+ ("sheet.json", "sheet.json", "deck.json")])
+def test_a_move_onto_a_name_taken_on_the_other_side_is_409(ws, c, root_file, path, to):
+ """nanobot: measured on the 0.35.0-rc.1 candidate, moving the agent's deck.json onto the app's
+ root app.json answered 200, and the moved file shadowed the app's for good (adoption never moves
+ over a name the agent's directory has)."""
+ _nanobot(ws)
+ (ws / root_file).write_text("the app's")
+ r = c.post("/file/move", params={"identifier": "s", "path": path, "to": to})
+ assert r.status_code == 409
+ assert (ws / root_file).read_text() == "the app's" and (ws / "workspace" / "deck.json").read_text() == "{}"
diff --git a/ui/src/components/ApiReference.tsx b/ui/src/components/ApiReference.tsx
index 707fe803..013bf720 100644
--- a/ui/src/components/ApiReference.tsx
+++ b/ui/src/components/ApiReference.tsx
@@ -263,6 +263,24 @@ curl "${mgmtBase}/sessions/{session_id}/files?changed=true" \\
-H "Authorization: Bearer $HARNESSROUTER_API_KEY"`} />
</Endpoint>
+ <Endpoint method="PUT" path="/v1/sessions/{session_id}/files/{path}">
+ <p className="hr-meta">Keep an app&apos;s own state in the workspace the agent also edits. <code>GET</code> reads one
+ file by path (live, while a turn runs too); <code>PUT</code> writes one, with <code>{`{"content": "…"}`}</code>
+ or <code>{`{"content_b64": "…"}`}</code>; <code>DELETE</code> removes a file or a folder and everything in it;
+ <code> POST …/files/&#123;path&#125;:mkdir</code> makes a folder (<code>created: false</code> when it was already
+ there); <code>POST …/files/&#123;path&#125;:move</code> with
+ <code> {`{"to": "archive/deck.json"}`}</code> renames a file or folder, and never overwrites (409
+ <code> file_exists</code>). The next turn sees each change. All but <code>GET</code> answer 409
+ <code> session_busy</code> while a turn runs. Paths stay inside the workspace: <code>..</code> is refused, and the three
+ that remove, make or rename also refuse what the files listing hides (<code>.git</code>, <code>.harness</code>,
+ the harness&apos;s own <code>AGENTS.md</code>, and a base&apos;s own files such as its skills folder).</p>
+ <CodeBlock lang="bash" code={`curl -X POST "${mgmtBase}/sessions/{session_id}/files/deck.json:move" \\
+ -H "Authorization: Bearer $HARNESSROUTER_API_KEY" -H "Content-Type: application/json" \\
+ -d '{"to": "archive/deck.json"}'
+curl -X DELETE "${mgmtBase}/sessions/{session_id}/files/drafts" \\
+ -H "Authorization: Bearer $HARNESSROUTER_API_KEY"`} />
+ </Endpoint>
+
{/* ── Sessions ────────────────────────────────────────────────────── */}
<h3 className="apidoc-h3">Sessions (task runs)</h3>
<Endpoint method="GET" path="/v1/sessions?harness={id}&limit=20">