The fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.
Rust agent-orchestrationai-agentsai-assistantdesktopllmlocal-firstmcppersonal-aiprivacyrustsecond-braintauri
fix(shell): sandboxed shells yield too; review fixes for the process registry
sanil-23a38aa7cdd437
diff --git a/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield.rs b/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield.rsindex 3f355fe757..c307669707 100644--- a/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield.rs+++ b/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield.rs@@ -6,8 +6,13 @@//! turn's: TinyAgents' `ShellTurnBudget` clamp writes the turn's remaining//! budget into `timeout_secs`, and the shell enforces that as the command's//! deadline. So this runs just before the clamp and moves the model's value to-//! the shell's yield argument, leaving `timeout_secs` to the clamp. Outside a-//! turn nothing moves it, and `timeout_secs` stays the caller's deadline.+//! the shell's yield argument, leaving `timeout_secs` to the clamp. The yield+//! argument's presence is what lets the call yield, so it is set (to `null`+//! when the model gave no `timeout_secs`) on every shell call of a turn that+//! registers `shell_process`. A turn without it (an agent whose tool belt names+//! `shell` alone) could not follow a process, so nothing moves and+//! `timeout_secs` stays a kill, as it was. Outside a turn nothing moves it+//! either.use async_trait::async_trait;use tinyagents_harness::context::RunContext;@@ -18,8 +23,12 @@ use tinyinference_llm::tool::ToolCall as TaToolCall;use crate::agent::tinyagents::host::OpenHumanRunContext;use crate::tools::implementations::system::YIELD_AFTER_ARG;-/// Moves a `shell` call's `timeout_secs` to its yield argument.-pub(crate) struct ShellYieldMiddleware;+/// Moves a `shell` call's `timeout_secs` to its yield argument, when the turn+/// can follow a process left running.+pub(crate) struct ShellYieldMiddleware {+ /// The turn registers `shell_process`.+ pub(crate) can_follow: bool,+}#[async_trait]impl Middleware<(), OpenHumanRunContext> for ShellYieldMiddleware {@@ -33,7 +42,7 @@ impl Middleware<(), OpenHumanRunContext> for ShellYieldMiddleware {_state: &(),call: &mut TaToolCall,) -> TaResult<()> {- if call.name == "shell" {+ if call.name == "shell" && self.can_follow {if let Some(arguments) = call.arguments.as_object_mut() {timeout_to_yield(arguments);}@@ -42,15 +51,17 @@ impl Middleware<(), OpenHumanRunContext> for ShellYieldMiddleware {}}-/// Move `timeout_secs`, when present, to [`YIELD_AFTER_ARG`].+/// Move `timeout_secs` to [`YIELD_AFTER_ARG`], `null` when absent: the call+/// may yield, at its default point unless the model gave one.pub(crate) fn timeout_to_yield(arguments: &mut serde_json::Map<String, serde_json::Value>) {- if let Some(requested) = arguments.remove("timeout_secs") {- tracing::debug!(- requested = %requested,- "[tinyagents::mw] shell_yield: timeout_secs is the call's yield point"- );- arguments.insert(YIELD_AFTER_ARG.to_owned(), requested);- }+ let requested = arguments+ .remove("timeout_secs")+ .unwrap_or(serde_json::Value::Null);+ tracing::debug!(+ requested = %requested,+ "[tinyagents::mw] shell_yield: timeout_secs is the call's yield point"+ );+ arguments.insert(YIELD_AFTER_ARG.to_owned(), requested);}#[cfg(test)]diff --git a/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield_tests.rs b/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield_tests.rsindex 25eb0cffd5..c7015aa43d 100644--- a/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield_tests.rs+++ b/crates/openhuman-core/src/agent/tinyagents/middleware/shell_yield_tests.rs@@ -16,8 +16,11 @@ fn the_models_timeout_becomes_the_yield_point() {}#[test]-fn a_call_without_a_timeout_is_left_to_the_default() {- assert_eq!(moved(json!({"command": "ls"})), json!({"command": "ls"}));+fn a_call_without_a_timeout_may_yield_at_the_default_point() {+ assert_eq!(+ moved(json!({"command": "ls"})),+ json!({"command": "ls", YIELD_AFTER_ARG: null})+ );}#[test]@@ -41,3 +44,32 @@ fn the_turn_clamp_then_sets_the_deadline() {assert_eq!(clamp.requested, None);assert_eq!(clamp.clamped_secs, 510);}++#[tokio::test]+async fn a_turn_that_cannot_follow_a_process_leaves_the_kill() {+ use tinyagents_harness::context::RunConfig;+ let mut ctx = OpenHumanRunContext::new().into_tinyagents(RunConfig::new("shell-yield"));+ let mut call = TaToolCall {+ id: "c1".into(),+ name: "shell".into(),+ arguments: json!({"command": "make", "timeout_secs": 60}),+ invalid: None,+ };+ ShellYieldMiddleware { can_follow: false }+ .before_tool(&mut ctx, &(), &mut call)+ .await+ .unwrap();+ assert_eq!(+ call.arguments,+ json!({"command": "make", "timeout_secs": 60})+ );++ ShellYieldMiddleware { can_follow: true }+ .before_tool(&mut ctx, &(), &mut call)+ .await+ .unwrap();+ assert_eq!(+ call.arguments,+ json!({"command": "make", YIELD_AFTER_ARG: 60})+ );+}diff --git a/crates/openhuman-core/src/agent/tinyagents/turn_middleware.rs b/crates/openhuman-core/src/agent/tinyagents/turn_middleware.rsindex 5e17162db0..a6338dd4ab 100644--- a/crates/openhuman-core/src/agent/tinyagents/turn_middleware.rs+++ b/crates/openhuman-core/src/agent/tinyagents/turn_middleware.rs@@ -622,7 +622,14 @@ pub(super) fn install_turn_middleware(// The model's shell `timeout_secs` is when the call comes back, not a kill:// move it aside so the clamp below writes the command's deadline there.- harness.push_middleware(Arc::new(middleware::shell_yield::ShellYieldMiddleware));+ let can_follow = harness+ .tools()+ .names()+ .iter()+ .any(|name| name == "shell_process");+ harness.push_middleware(Arc::new(middleware::shell_yield::ShellYieldMiddleware {+ can_follow,+ }));// Clamp shell `timeout_secs` to the turn remainder (#6953), on recovered args.harness.push_middleware(Arc::new(shell_turn_budget.clamp()));diff --git a/crates/openhuman-core/src/sandbox/native_jail.rs b/crates/openhuman-core/src/sandbox/native_jail.rsindex ba13f013d5..c690dc8178 100644--- a/crates/openhuman-core/src/sandbox/native_jail.rs+++ b/crates/openhuman-core/src/sandbox/native_jail.rs@@ -5,7 +5,8 @@ use std::{collections::HashMap, ffi::OsString, path::Path, time::Duration};#[cfg(feature = "modules")]use std::collections::BTreeMap;-use super::types::{SandboxExecResult, SandboxIsolation, SandboxPolicy, SandboxStatus};+use super::shell_exec::ShellRunOutcome;+use super::types::{SandboxIsolation, SandboxPolicy, SandboxStatus};use crate::config::Config;#[cfg(feature = "modules")]@@ -89,7 +90,9 @@ pub(crate) async fn require_backend(_config: &Config, _expected: &str) -> anyhowanyhow::bail!("TinyBox module support is disabled; native jail execution is unavailable")}-/// Run a command inside TinyBox's native jail and keep its resource lifecycle module-owned.+/// Run a command inside TinyBox's native jail and keep its resource lifecycle+/// module-owned. With `yield_after`, a command still running then comes back+/// as [`ShellRunOutcome::Running`] and keeps running in the jail.pub(crate) async fn execute(ctx: &crate::core::runtime::DispatchCtx,config: &Config,@@ -99,8 +102,9 @@ pub(crate) async fn execute(extra_env: &HashMap<OsString, OsString>,timeout: Duration,explicitly_requested: bool,+ yield_after: Option<Duration>,meter: Option<super::shell_progress::ShellOutputMeter>,-) -> anyhow::Result<SandboxExecResult> {+) -> anyhow::Result<ShellRunOutcome> {#[cfg(feature = "modules")]{let ctx = ctx.clone();@@ -120,6 +124,7 @@ pub(crate) async fn execute(&extra_env,timeout,explicitly_requested,+ yield_after,meter,).await@@ -138,6 +143,7 @@ pub(crate) async fn execute(extra_env,timeout,explicitly_requested,+ yield_after,meter,);anyhow::bail!("TinyBox module support is disabled; native jail execution is unavailable")@@ -154,8 +160,9 @@ async fn execute_owned(extra_env: &HashMap<OsString, OsString>,timeout: Duration,explicitly_requested: bool,+ yield_after: Option<Duration>,meter: Option<super::shell_progress::ShellOutputMeter>,-) -> anyhow::Result<SandboxExecResult> {+) -> anyhow::Result<ShellRunOutcome> {let status = tinybox::jail_status(config).await.map_err(|_| anyhow::anyhow!("TinyBox native jail status is unavailable"))?;@@ -188,7 +195,8 @@ async fn execute_owned(// TinyBox adds its own per-call scratch directory to the writable grants and// points TMPDIR, TMP and TEMP at it unless the caller overrides them.run.jail = Some(jail_request(config, policy, isolation)?);- super::shell_exec::run(ctx, config, run).await+ run.yield_after = yield_after;+ super::shell_exec::run_yielding(ctx, config, run).await}/// The caller's environment overrides, which must be UTF-8 to cross the bus.diff --git a/crates/openhuman-core/src/sandbox/native_jail_tests.rs b/crates/openhuman-core/src/sandbox/native_jail_tests.rsindex 43ba95590d..168a3f6ea1 100644--- a/crates/openhuman-core/src/sandbox/native_jail_tests.rs+++ b/crates/openhuman-core/src/sandbox/native_jail_tests.rs@@ -368,6 +368,7 @@ async fn disabled_tinybox_module_never_falls_back_to_host_execution() -> anyhow:Duration::from_secs(5),false,None,+ None,).await;@@ -481,8 +482,10 @@ async fn released_tinybox_runs_a_native_jail_through_the_pinned_loader() -> anyhDuration::from_secs(10),false,None,+ None,).await?;+ let output = finished(output);assert!(output.success());assert_eq!(output.stdout, "tinybox-jailed-ok");assert_eq!(output.stderr, "safe-error");@@ -497,8 +500,10 @@ async fn released_tinybox_runs_a_native_jail_through_the_pinned_loader() -> anyhDuration::from_secs(10),false,None,+ None,).await?;+ let output = finished(output);assert!(output.success());assert_eq!(output.stdout.len(),@@ -551,3 +556,15 @@ async fn released_tinybox_runs_a_native_jail_through_the_pinned_loader() -> anyhassert!(output.timed_out, "cron timeout is acknowledged by TinyBox");Ok(())}++/// The result of a jailed run that was not asked to yield.+fn finished(+ outcome: super::super::shell_exec::ShellRunOutcome,+) -> super::super::types::SandboxExecResult {+ match outcome {+ super::super::shell_exec::ShellRunOutcome::Finished(result) => result,+ super::super::shell_exec::ShellRunOutcome::Running(_) => {+ panic!("a run without a yield point came back while running")+ }+ }+}diff --git a/crates/openhuman-core/src/sandbox/ops.rs b/crates/openhuman-core/src/sandbox/ops.rsindex 1e0c69a4f6..0f0fb17ce5 100644--- a/crates/openhuman-core/src/sandbox/ops.rs+++ b/crates/openhuman-core/src/sandbox/ops.rs@@ -2,6 +2,7 @@//! routed execution.use super::docker;+use super::shell_exec::{ComeBack, ShellRunOutcome};use super::types::{ElevatedOp, SandboxBackendHandle, SandboxBackendKind, SandboxExecRequest, SandboxExecResult,SandboxPolicy, SandboxStatus, ELEVATED_TOOLS,@@ -321,6 +322,39 @@ pub(crate) async fn execute_in_sandbox_metered(timeout: Duration,meter: Option<super::shell_progress::ShellOutputMeter>,) -> anyhow::Result<SandboxExecResult> {+ let outcome = execute_in_sandbox_yielding(+ ctx,+ policy,+ command,+ working_dir,+ extra_env,+ timeout,+ ComeBack::default(),+ meter,+ )+ .await?;+ match outcome {+ ShellRunOutcome::Finished(result) => Ok(result),+ ShellRunOutcome::Running(_) => {+ anyhow::bail!("a sandboxed run without a yield point returned while running")+ }+ }+}++/// [`execute_in_sandbox_metered`] for a call that may come back first: on the+/// native backends a command still running at `come_back.yield_after` is+/// handed back as [`ShellRunOutcome::Running`]; Docker cannot leave a command+/// running, so there `come_back.deadline_instead` bounds it.+pub(crate) async fn execute_in_sandbox_yielding(+ ctx: &DispatchCtx,+ policy: &SandboxPolicy,+ command: &str,+ working_dir: &Path,+ extra_env: HashMap<OsString, OsString>,+ timeout: Duration,+ come_back: ComeBack,+ meter: Option<super::shell_progress::ShellOutputMeter>,+) -> anyhow::Result<ShellRunOutcome> {// The saved backend is an operator's explicit isolation requirement. It// must be checked at the spawn boundary, including callers that resolved// their policy from an older or default RuntimeConfig.@@ -377,6 +411,7 @@ pub(crate) async fn execute_in_sandbox_metered(working_dir,&extra_env,timeout,+ come_back.yield_after,meter,).await@@ -399,11 +434,15 @@ pub(crate) async fn execute_in_sandbox_metered(&extra_env,timeout,explicitly_requested,+ come_back.yield_after,meter,).await}SandboxBackendKind::Docker => {+ let timeout = come_back+ .deadline_instead+ .map_or(timeout, |deadline| timeout.min(deadline));crate::config::ensure_usable_cwd(&policy.workspace_root)?;let request = SandboxExecRequest {command: command.to_string(),@@ -417,7 +456,9 @@ pub(crate) async fn execute_in_sandbox_metered(anyhow::anyhow!("Cannot read module configuration: {error}")})?,};- docker::docker_exec(&config, policy, &request).await+ docker::docker_exec(&config, policy, &request)+ .await+ .map(ShellRunOutcome::Finished)}}}@@ -483,8 +524,9 @@ async fn execute_unsandboxed(working_dir: &Path,extra_env: &HashMap<OsString, OsString>,timeout: Duration,+ yield_after: Option<Duration>,meter: Option<super::shell_progress::ShellOutputMeter>,-) -> anyhow::Result<SandboxExecResult> {+) -> anyhow::Result<ShellRunOutcome> {let mut env = std::collections::BTreeMap::new();for (name, value) in extra_env {let (Some(name), Some(value)) = (name.to_str(), value.to_str()) else {@@ -494,18 +536,12 @@ async fn execute_unsandboxed(}std::fs::create_dir_all(&policy.state_dir).map_err(|error| anyhow::anyhow!("Cannot prepare sandbox state directory: {error}"))?;- super::shell_exec::run_host(- ctx,- config,- command,- working_dir,- &policy.state_dir,- env,- timeout,- false,- meter,- )- .await+ let mut run =+ super::shell_exec::NativeShellRun::new(command, working_dir, &policy.state_dir, timeout);+ run.meter = meter;+ run.env = env;+ run.yield_after = yield_after;+ super::shell_exec::run_yielding(ctx, config, run).await}/// Check whether a tool operation is an elevated op that must run on thediff --git a/crates/openhuman-core/src/sandbox/ops_tests.rs b/crates/openhuman-core/src/sandbox/ops_tests.rsindex c61b94a6eb..82fa3baada 100644--- a/crates/openhuman-core/src/sandbox/ops_tests.rs+++ b/crates/openhuman-core/src/sandbox/ops_tests.rs@@ -126,7 +126,7 @@ async fn run_unsandboxed() -> anyhow::Result<SandboxExecResult> {let state = tempfile::tempdir()?;let policy = unsandboxed_policy(workspace, state.path());- execute_unsandboxed(+ let outcome = execute_unsandboxed(&crate::core::runtime::DispatchCtx::host_default(),&crate::config::Config::default(),&policy,@@ -135,8 +135,13 @@ async fn run_unsandboxed(env,timeout,None,+ None,)- .await+ .await?;+ match outcome {+ ShellRunOutcome::Finished(result) => Ok(result),+ ShellRunOutcome::Running(_) => anyhow::bail!("a run without a yield point kept running"),+ }}// The `/tmp` path and Unix builtins (`false`) are Unix-only, so thesediff --git a/crates/openhuman-core/src/sandbox/shell_exec.rs b/crates/openhuman-core/src/sandbox/shell_exec.rsindex 5e1e2dd47c..341e22310d 100644--- a/crates/openhuman-core/src/sandbox/shell_exec.rs+++ b/crates/openhuman-core/src/sandbox/shell_exec.rs@@ -100,6 +100,19 @@ impl NativeShellRun {pub(crate) static USE_FIXTURE: std::sync::atomic::AtomicBool =std::sync::atomic::AtomicBool::new(true);+/// How a shell call that may come back before its command ends wants a+/// sandboxed command run.+#[derive(Clone, Copy, Debug, Default)]+pub(crate) struct ComeBack {+ /// Return with the command still running once it has run this long. Only+ /// the native backends (no sandbox, native jail) can.+ pub(crate) yield_after: Option<Duration>,+ /// For a backend that cannot leave a command running (Docker): the+ /// caller's own come-back time, enforced as a deadline instead, as the+ /// model's `timeout_secs` was before it yielded.+ pub(crate) deadline_instead: Option<Duration>,+}+/// How a run came back: the command ended, or it was still running at its/// yield point and is handed over to be followed.pub(crate) enum ShellRunOutcome {diff --git a/crates/openhuman-core/src/sandbox/shell_exec_fixture.rs b/crates/openhuman-core/src/sandbox/shell_exec_fixture.rsindex c4623b13ce..f6cdd2f76a 100644--- a/crates/openhuman-core/src/sandbox/shell_exec_fixture.rs+++ b/crates/openhuman-core/src/sandbox/shell_exec_fixture.rs@@ -178,6 +178,10 @@ impl FixtureLive {Ok(self.take(state))}+ pub(crate) fn finished(&self) -> bool {+ self.ended+ }+pub(crate) async fn kill(&mut self) -> anyhow::Result<LiveRead> {if !self.ended {self.stop().await;diff --git a/crates/openhuman-core/src/sandbox/shell_live.rs b/crates/openhuman-core/src/sandbox/shell_live.rsindex 672aac3088..e33afe61b1 100644--- a/crates/openhuman-core/src/sandbox/shell_live.rs+++ b/crates/openhuman-core/src/sandbox/shell_live.rs@@ -68,6 +68,17 @@ impl LiveHandle {}}+ /// Whether the process has stopped and its resource is released, so a+ /// failed read leaves nothing running.+ pub(crate) fn finished(&self) -> bool {+ match self {+ #[cfg(feature = "modules")]+ Self::TinyBox(live) => live.released,+ #[cfg(test)]+ Self::Fixture(live) => live.finished(),+ }+ }+/// Stop the process and return what it wrote since the previous read.pub(crate) async fn kill(&mut self) -> anyhow::Result<LiveRead> {match self {@@ -112,18 +123,26 @@ impl TinyBoxLive {self.read_until(until).await}+ /// Cancel the process and read until it reports how it ended. One still+ /// running after [`KILL_GRACE`] comes back as `Running`: the cancel did+ /// not take, and the process keeps its place until it does stop.async fn kill(&mut self) -> anyhow::Result<LiveRead> {if !self.released {tracing::debug!("[sandbox:shell] stopping a running process on request");// A process that ended on its own refuses the cancel; the read// below reports how it ended either way.- let _ = tinybox::cancel(&self.config, self.process.clone()).await;+ if tinybox::cancel(&self.config, self.process.clone())+ .await+ .is_err()+ {+ tracing::debug!("[sandbox:shell] TinyBox refused the cancel");+ }}- let mut read = self+ let read = self.read_until(tokio::time::Instant::now() + KILL_GRACE).await?;if read.state == LiveState::Running {- read.state = LiveState::Stopped;+ tracing::warn!("[sandbox:shell] a process is still running after its cancel");}Ok(read)}diff --git a/crates/openhuman-core/src/sandbox/shell_processes.rs b/crates/openhuman-core/src/sandbox/shell_processes.rsindex d9604f595d..37ce7e6594 100644--- a/crates/openhuman-core/src/sandbox/shell_processes.rs+++ b/crates/openhuman-core/src/sandbox/shell_processes.rs@@ -1,18 +1,20 @@//! The shell commands a turn left running past their yield point.//!-//! The `shell` tool adopts a [`LiveShell`] here and gives the model its id-//! (`p1`, `p2`, …); the `shell_process` tool waits on it, stops it or reports-//! its status. One registry belongs to one turn's run context, so dropping it-//! at the end of the turn stops every process still running (each-//! [`LiveHandle`] stops its own on drop). Every process also stops at its own-//! runtime budget, which the turn budget already clamps.+//! The `shell` tool reserves a [`Slot`] before it starts a command that may+//! yield, and the slot adopts the [`LiveShell`] if the command is still running+//! at its yield point; the model gets its id (`p1`, `p2`, …). The+//! `shell_process` tool waits on it, stops it or reports its status. One+//! registry belongs to one turn's run context, so dropping it at the end of the+//! turn stops every process still running (each [`LiveHandle`] stops its own on+//! drop). Every process also stops at its own runtime budget, which the turn+//! budget already clamps.use std::collections::BTreeMap;use std::sync::atomic::{AtomicU32, Ordering};use std::sync::{Arc, Mutex};use std::time::{Duration, Instant};-use super::shell_live::{LiveHandle, LiveShell, LiveState};+use super::shell_live::{LiveHandle, LiveRead, LiveShell, LiveState};/// Processes a turn may leave running at once. A further command runs to its/// end the usual way.@@ -35,7 +37,9 @@ pub(crate) enum Ended {pub(crate) enum ReportState {/// Still running.Running,- /// Ended during this call.+ /// Ended, and this answer is the first to say so. A `wait` or `kill`+ /// answer carries the rest of its output; a `status` answer leaves that+ /// output for the next `wait`.Ended(Ended),/// Had already ended, and its output was already returned.AlreadyEnded(Ended),@@ -58,13 +62,34 @@ pub(crate) struct Report {pub(crate) quiet_waits: u32,}+/// Called once with how the process ended and how long it ran, when the+/// registry learns of the end (or stops it at the end of the turn). The shell+/// tool records its audit entry here, so a yielded command is audited with its+/// real outcome.+pub(crate) type OnEnd = Box<dyn FnOnce(Ended, Duration) + Send>;+struct Entry {handle: LiveHandle,command: String,started: Instant,timeout: Duration,ended: Option<Ended>,+ /// Whether an answer has said the process ended.+ end_reported: bool,+ /// Output a `status` read that no answer has returned yet.+ pending_stdout: String,+ pending_stderr: String,quiet_waits: u32,+ on_end: Option<OnEnd>,+}++impl Drop for Entry {+ fn drop(&mut self) {+ // The turn ended with the process still running; its handle stops it.+ if let Some(on_end) = self.on_end.take() {+ on_end(self.ended.unwrap_or(Ended::Stopped), self.started.elapsed());+ }+ }}/// The running processes of one turn.@@ -72,6 +97,7 @@ struct Entry {pub(crate) struct ShellProcesses {next: AtomicU32,entries: Mutex<BTreeMap<String, Arc<tokio::sync::Mutex<Entry>>>>,+ /// Processes running or reserved: a slot counts from [`Self::reserve`].running: Mutex<usize>,}@@ -84,27 +110,28 @@ impl PartialEq for ShellProcesses {impl Eq for ShellProcesses {}-impl ShellProcesses {- pub(crate) fn new() -> Arc<Self> {- Arc::new(Self::default())- }-- /// Whether another process may be left running.- pub(crate) fn has_room(&self) -> bool {- *self.running() < MAX_RUNNING- }+/// A place for one more running process, held while its command starts. It+/// adopts the command if it is still running at its yield point; dropped+/// unused (the command finished first, or failed), it frees the place.+pub(crate) struct Slot {+ processes: Arc<ShellProcesses>,+ held: bool,+}+impl Slot {/// Take over a process still running at its yield point and name it.pub(crate) fn adopt(- &self,+ mut self,live: LiveShell,command: &str,started: Instant,timeout: Duration,+ on_end: Option<OnEnd>,) -> String {- let id = format!("p{}", self.next.fetch_add(1, Ordering::SeqCst) + 1);- *self.running() += 1;- self.entries().insert(+ self.held = false;+ let processes = &self.processes;+ let id = format!("p{}", processes.next.fetch_add(1, Ordering::SeqCst) + 1);+ processes.entries().insert(id.clone(),Arc::new(tokio::sync::Mutex::new(Entry {handle: live.handle,@@ -112,72 +139,151 @@ impl ShellProcesses {started,timeout,ended: None,+ end_reported: false,+ pending_stdout: String::new(),+ pending_stderr: String::new(),quiet_waits: 0,+ on_end,})),);tracing::debug!(process = %id, "[shell] process left running past its yield point");id}+}++impl Drop for Slot {+ fn drop(&mut self) {+ if self.held {+ let mut running = self.processes.running();+ *running = running.saturating_sub(1);+ }+ }+}++impl ShellProcesses {+ pub(crate) fn new() -> Arc<Self> {+ Arc::new(Self::default())+ }++ /// Reserve a place for a command that may be left running, or `None`+ /// when the turn already holds [`MAX_RUNNING`]. Reserving before the+ /// command starts keeps parallel calls from all seeing room.+ pub(crate) fn reserve(self: &Arc<Self>) -> Option<Slot> {+ let mut running = self.running();+ if *running >= MAX_RUNNING {+ return None;+ }+ *running += 1;+ Some(Slot {+ processes: Arc::clone(self),+ held: true,+ })+ }/// Wait up to `wait` for new output or the end of the process.pub(crate) async fn wait(&self, id: &str, wait: Duration) -> Result<Report, String> {let entry = self.entry(id)?;let mut entry = entry.lock().await;- if let Some(ended) = entry.ended {- return Ok(report(- id,- &entry,- String::new(),- String::new(),- ReportState::AlreadyEnded(ended),- ));+ if entry.ended.is_some() {+ return Ok(self.answer_ended(id, &mut entry));}- let read = entry- .handle- .read(wait)- .await- .map_err(|error| format!("Could not read process {id}: {error}"))?;- let quiet = read.stdout.is_empty() && read.stderr.is_empty();+ let read = match entry.handle.read(wait).await {+ Ok(read) => read,+ Err(error) => {+ return Err(self.failed(&mut entry, format!("Could not read process {id}: {error}")))+ }+ };+ let (stdout, stderr) = take_pending(&mut entry, read.stdout, read.stderr);+ let quiet = stdout.is_empty() && stderr.is_empty();entry.quiet_waits = if quiet { entry.quiet_waits + 1 } else { 0 };let state = self.settle(&mut entry, read.state);- Ok(report(id, &entry, read.stdout, read.stderr, state))+ returned(&mut entry, &state);+ Ok(report(id, &entry, stdout, stderr, state))}/// Stop the process and return what it wrote since the previous answer.+ /// A process that does not stop is reported as still running.pub(crate) async fn kill(&self, id: &str) -> Result<Report, String> {let entry = self.entry(id)?;let mut entry = entry.lock().await;- if let Some(ended) = entry.ended {- return Ok(report(- id,- &entry,- String::new(),- String::new(),- ReportState::AlreadyEnded(ended),- ));+ if entry.ended.is_some() {+ return Ok(self.answer_ended(id, &mut entry));}tracing::debug!(process = %id, "[shell] stopping a running process on request");- let read = entry- .handle- .kill()- .await- .map_err(|error| format!("Could not stop process {id}: {error}"))?;+ let read = match entry.handle.kill().await {+ Ok(read) => read,+ Err(error) => {+ return Err(self.failed(&mut entry, format!("Could not stop process {id}: {error}")))+ }+ };+ let (stdout, stderr) = take_pending(&mut entry, read.stdout, read.stderr);let state = self.settle(&mut entry, read.state);- Ok(report(id, &entry, read.stdout, read.stderr, state))+ returned(&mut entry, &state);+ Ok(report(id, &entry, stdout, stderr, state))}- /// Where the process is, without its output.+ /// Where the process is now, without its output: a read that returns at+ /// once refreshes its state, and what it read is kept for the next+ /// `wait`, so a process that has ended says so and frees its place.pub(crate) async fn status(&self, id: &str) -> Result<Report, String> {let entry = self.entry(id)?;- let entry = entry.lock().await;+ let mut entry = entry.lock().await;+ if entry.ended.is_none() {+ let LiveRead {+ stdout,+ stderr,+ state,+ } = entry+ .handle+ .read(Duration::ZERO)+ .await+ .map_err(|error| format!("Could not read process {id}: {error}"))?;+ entry.pending_stdout.push_str(&stdout);+ entry.pending_stderr.push_str(&stderr);+ self.settle(&mut entry, state);+ }let state = match entry.ended {- Some(ended) => ReportState::AlreadyEnded(ended),None => ReportState::Running,+ Some(ended) if entry.end_reported => ReportState::AlreadyEnded(ended),+ Some(ended) => ReportState::Ended(ended),};Ok(report(id, &entry, String::new(), String::new(), state))}- /// Record an ending, once.+ /// A read that failed. When the handle has already stopped the process and+ /// released it (TinyBox's output limit, a failed cleanup), the process is+ /// over: it frees its place, and later answers say it stopped. Otherwise+ /// it may still be running and keeps its place.+ fn failed(&self, entry: &mut Entry, message: String) -> String {+ if entry.handle.finished() {+ tracing::debug!("[shell] a process ended in a failed read; freeing its place");+ self.settle(entry, LiveState::Stopped);+ entry.end_reported = true;+ }+ message+ }++ /// The answer for a process already known to have ended: its last output+ /// the first time, "already ended" after that.+ fn answer_ended(&self, id: &str, entry: &mut Entry) -> Report {+ let ended = entry.ended.unwrap_or(Ended::Stopped);+ if entry.end_reported {+ return report(+ id,+ entry,+ String::new(),+ String::new(),+ ReportState::AlreadyEnded(ended),+ );+ }+ entry.end_reported = true;+ let (stdout, stderr) = take_pending(entry, String::new(), String::new());+ report(id, entry, stdout, stderr, ReportState::Ended(ended))+ }++ /// Record an ending, once: free the place and report the outcome. An+ /// answer that returns the process's last output then marks the end+ /// reported (`returned`).fn settle(&self, entry: &mut Entry, state: LiveState) -> ReportState {let ended = match state {LiveState::Running => return ReportState::Running,@@ -189,6 +295,10 @@ impl ShellProcesses {entry.ended = Some(ended);let mut running = self.running();*running = running.saturating_sub(1);+ drop(running);+ if let Some(on_end) = entry.on_end.take() {+ on_end(ended, entry.started.elapsed());+ }}ReportState::Ended(ended)}@@ -223,6 +333,22 @@ impl ShellProcesses {}}+/// An answer that says the process ended has returned its last output.+fn returned(entry: &mut Entry, state: &ReportState) {+ if matches!(state, ReportState::Ended(_)) {+ entry.end_reported = true;+ }+}++/// Output a `status` read kept, followed by what this answer read.+fn take_pending(entry: &mut Entry, stdout: String, stderr: String) -> (String, String) {+ let mut out = std::mem::take(&mut entry.pending_stdout);+ out.push_str(&stdout);+ let mut err = std::mem::take(&mut entry.pending_stderr);+ err.push_str(&stderr);+ (out, err)+}+fn report(id: &str, entry: &Entry, stdout: String, stderr: String, state: ReportState) -> Report {Report {id: id.to_owned(),diff --git a/crates/openhuman-core/src/sandbox/shell_processes_tests.rs b/crates/openhuman-core/src/sandbox/shell_processes_tests.rsindex 32d46e5660..8b04cff9dc 100644--- a/crates/openhuman-core/src/sandbox/shell_processes_tests.rs+++ b/crates/openhuman-core/src/sandbox/shell_processes_tests.rs@@ -18,19 +18,21 @@ async fn start(command: &str, yield_after_ms: u64, timeout_secs: u64) -> ShellRu}async fn adopted(- processes: &ShellProcesses,+ processes: &Arc<ShellProcesses>,command: &str,yield_after_ms: u64,timeout_secs: u64,) -> (String, String) {+ let slot = processes.reserve().expect("room for another process");match start(command, yield_after_ms, timeout_secs).await {ShellRunOutcome::Running(live) => {let printed = live.stdout.clone();- let id = processes.adopt(+ let id = slot.adopt(live,command,Instant::now(),Duration::from_secs(timeout_secs),+ None,);(id, printed)}@@ -114,17 +116,110 @@ async fn at_most_four_processes_are_left_running() {let processes = ShellProcesses::new();let mut ids = Vec::new();for _ in 0..MAX_RUNNING {- assert!(processes.has_room());ids.push(adopted(&processes, "sleep 30", 100, 60).await.0);}- assert!(!processes.has_room());+ assert!(processes.reserve().is_none());processes.kill(&ids[0]).await.unwrap();- assert!(processes.has_room(), "a stopped process frees its place");+ assert!(+ processes.reserve().is_some(),+ "a stopped process frees its place"+ );for id in &ids[1..] {processes.kill(id).await.unwrap();}}+#[test]+fn parallel_calls_cannot_reserve_more_than_four_places() {+ let processes = ShellProcesses::new();+ let slots: Vec<_> = (0..MAX_RUNNING)+ .map(|_| processes.reserve().expect("a place"))+ .collect();+ assert!(+ processes.reserve().is_none(),+ "a fifth call started before any was adopted"+ );+ drop(slots);+ assert!(+ processes.reserve().is_some(),+ "a command that finished first frees its place"+ );+}++#[cfg(not(windows))]+#[tokio::test]+async fn status_notices_an_end_and_keeps_its_output_for_the_next_wait() {+ let processes = ShellProcesses::new();+ let (id, _) = adopted(&processes, "echo a; sleep 1; echo b", 300, 30).await;+ tokio::time::sleep(Duration::from_millis(1500)).await;++ let status = processes.status(&id).await.unwrap();+ assert_eq!(status.state, ReportState::Ended(Ended::Exited(0)));+ assert!(status.stdout.is_empty(), "status carries no output");+ let freed: Vec<_> = (0..MAX_RUNNING)+ .map(|_| {+ processes+ .reserve()+ .expect("the ended process freed its place")+ })+ .collect();+ drop(freed);++ let wait = processes.wait(&id, Duration::from_secs(1)).await.unwrap();+ assert_eq!(wait.state, ReportState::Ended(Ended::Exited(0)));+ assert_eq!(+ wait.stdout.trim(),+ "b",+ "the output status read is returned"+ );+ let again = processes.wait(&id, Duration::from_secs(1)).await.unwrap();+ assert_eq!(again.state, ReportState::AlreadyEnded(Ended::Exited(0)));+}++#[cfg(not(windows))]+#[tokio::test]+async fn the_end_is_reported_to_its_hook_once() {+ let processes = ShellProcesses::new();+ let ends = Arc::new(Mutex::new(Vec::new()));+ let seen = Arc::clone(&ends);+ let slot = processes.reserve().unwrap();+ let ShellRunOutcome::Running(live) = start("echo a; sleep 1; exit 3", 300, 30).await else {+ panic!("expected a running process");+ };+ let id = slot.adopt(+ live,+ "exit 3",+ Instant::now(),+ Duration::from_secs(30),+ Some(Box::new(move |ended, _| seen.lock().unwrap().push(ended))),+ );+ processes.wait(&id, Duration::from_secs(10)).await.unwrap();+ processes.wait(&id, Duration::from_secs(1)).await.unwrap();+ drop(processes);+ assert_eq!(*ends.lock().unwrap(), vec![Ended::Exited(3)]);+}++#[cfg(not(windows))]+#[tokio::test]+async fn a_process_still_running_at_turn_end_is_reported_stopped() {+ let processes = ShellProcesses::new();+ let ends = Arc::new(Mutex::new(Vec::new()));+ let seen = Arc::clone(&ends);+ let slot = processes.reserve().unwrap();+ let ShellRunOutcome::Running(live) = start("sleep 30", 100, 60).await else {+ panic!("expected a running process");+ };+ slot.adopt(+ live,+ "sleep 30",+ Instant::now(),+ Duration::from_secs(60),+ Some(Box::new(move |ended, _| seen.lock().unwrap().push(ended))),+ );+ drop(processes);+ assert_eq!(*ends.lock().unwrap(), vec![Ended::Stopped]);+}+#[cfg(not(windows))]#[tokio::test]async fn dropping_the_registry_stops_its_processes() {diff --git a/crates/openhuman-core/src/tools/impl/system/mod.rs b/crates/openhuman-core/src/tools/impl/system/mod.rsindex e17be2e422..9a61f25611 100644--- a/crates/openhuman-core/src/tools/impl/system/mod.rs+++ b/crates/openhuman-core/src/tools/impl/system/mod.rs@@ -12,6 +12,7 @@ mod schedule;mod shell;mod shell_description;mod shell_process;+mod shell_run;mod shell_saas;mod shell_support;mod update_apply;diff --git a/crates/openhuman-core/src/tools/impl/system/shell.rs b/crates/openhuman-core/src/tools/impl/system/shell.rsindex 5c32074add..aa2424b680 100644--- a/crates/openhuman-core/src/tools/impl/system/shell.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell.rs@@ -1,7 +1,8 @@use super::shell_description::{command_param_description, shell_description};+use super::shell_run::{audit_once, run_in_policy, run_native, AuditOnce, YieldRequest};#[cfg(feature = "modules")]use super::shell_support::container_environment;-use super::shell_support::{command_failure, exec_result, tool_timeout_message};+use super::shell_support::{command_failure, tool_timeout_message};use crate::agent::tinyagents::host::run_context::{command_environment_for_tool, dispatch_ctx_for_tool,};@@ -12,7 +13,7 @@ use crate::security::{AuditLogger, CommandExecutionLog, SecurityPolicy};use async_trait::async_trait;use serde_json::json;use std::path::{Path, PathBuf};-use std::sync::Arc;+use std::sync::{Arc, PoisonError};use std::time::{Duration, Instant};use tinytools::ToolRunContext;use tinytools::{PermissionLevel, Tool, ToolCallOptions, ToolResult, ToolTimeout};@@ -98,35 +99,6 @@ impl ShellTool {}}- /// Emit a single `CommandExecution` audit event. A write failure is logged- /// as a structured warning but not propagated — audit must never block or- /// fail a tool call, yet a silently broken audit trail must not go- /// unnoticed.- fn emit_audit(- &self,- command: &str,- approved: bool,- allowed: bool,- success: bool,- duration_ms: u64,- ) {- if let Err(error) = self.audit.log_command_event(CommandExecutionLog {- channel: "tool:shell",- command,- risk_level: "unknown",- approved,- allowed,- success,- duration_ms,- }) {- tracing::warn!(- error = %error,- channel = "tool:shell",- "[shell] failed to persist command execution audit event"- );- }- }-/// Resolve the working directory for this shell invocation.////// Returns the per-worker git-worktree checkout when the tinyagents harness@@ -193,7 +165,7 @@ impl Tool for ShellTool {"type": "integer","minimum": 0,"maximum": 3600,- "description": "Optional: how long (seconds) to wait before coming back. A command still running then returns its output so far and keeps running as a process you follow with `shell_process`. Default 120, at most 600; 0 waits for the command to end. It does not kill the command: put `timeout N` in the command for that."+ "description": "Optional: how long (seconds) to wait before coming back. A command still running then returns its output so far and keeps running as a process you follow with `shell_process`. Default 120, at most 600; 0 waits for the command to end. Only where a command cannot be left running (four already are, or a Docker sandbox) is it stopped at this time instead. To stop a command after a time, put `timeout N` in the command."}},"required": ["command"]@@ -299,9 +271,16 @@ impl ShellTool {// `YIELD_AFTER_ARG` by `middleware::shell_yield`); outside a turn, the// caller's. `None`/`0` ⇒ unbounded (issue #4023).let requested_timeout = args.get("timeout_secs").and_then(|v| v.as_u64());- let requested_yield = args- .get(super::shell_support::YIELD_AFTER_ARG)- .and_then(|v| v.as_u64());+ let come_back = YieldRequest::from_args(&args);+ // `approved` = the command is Prompt-class (required human approval) and+ // thus went through the harness ApprovalGate to reach here. Reads and+ // Full-mode writes run without a prompt, so they audit as approved=false+ // rather than over-claiming a human approval that never happened. (The+ // gate's exact yes/no isn't threaded into tools; this is the accurate+ // "required approval" proxy.) A command left running is audited when it+ // ends, with its real outcome.+ let approved = self.external_effect_with_args(&args);+ let audit = audit_once(self.audit.clone(), command.to_owned(), approved);let start = Instant::now();let (allowed, result) = crate::sandbox::shell_progress::with_shell_progress(context,@@ -311,24 +290,20 @@ impl ShellTool {ctx,command,requested_timeout,- requested_yield,+ come_back,context,meter,+ &audit,)},).await;let duration_ms = u64::try_from(start.elapsed().as_millis()).unwrap_or(u64::MAX);- // `allowed` = passed the in-tool security checks. `approved` = the command- // is Prompt-class (required human approval) and thus went through the- // harness ApprovalGate to reach here — distinct from `allowed`. Reads and- // Full-mode writes run without a prompt, so they audit as approved=false- // rather than over-claiming a human approval that never happened. (The- // gate's exact yes/no isn't threaded into tools; this is the accurate- // "required approval" proxy.)- let approved = self.external_effect_with_args(&args);- // emit_audit signature is (command, approved, allowed, …) — keep that order.- self.emit_audit(command, approved, allowed, !result.is_error, duration_ms);+ // `allowed` = passed the in-tool security checks.+ let hook = audit.lock().unwrap_or_else(PoisonError::into_inner).take();+ if let Some(hook) = hook {+ hook(allowed, !result.is_error, duration_ms);+ }Ok(result)}}@@ -348,8 +323,18 @@ impl ShellTool {command: &str,requested_timeout: Option<u64>,) -> (bool, ToolResult) {- self.run_with_security_in_context(ctx, command, requested_timeout, None, None, None)- .await+ let come_back = YieldRequest::default();+ let audit = std::sync::Mutex::new(None);+ self.run_with_security_in_context(+ ctx,+ command,+ requested_timeout,+ come_back,+ None,+ None,+ &audit,+ )+ .await}async fn run_with_security_in_context(@@ -357,9 +342,10 @@ impl ShellTool {ctx: &DispatchCtx,command: &str,requested_timeout: Option<u64>,- requested_yield: Option<u64>,+ come_back: YieldRequest,context: Option<&dyn ToolRunContext>,meter: Option<crate::sandbox::shell_progress::ShellOutputMeter>,+ audit: &AuditOnce,) -> (bool, ToolResult) {let command_environment = command_environment_for_tool(context);// Revalidate facts immediately before execution. The authorization@@ -431,14 +417,9 @@ impl ShellTool {let action_dir = saas_action_dir;return match resolved {Ok(policy) => {- self.run_in_policy(- ctx,- policy,- command,- requested_timeout,- &action_dir,- context,- meter,+ let (timeout, dir) = (requested_timeout, &action_dir);+ run_in_policy(+ ctx, policy, command, timeout, come_back, dir, context, meter, audit,).await}@@ -468,8 +449,18 @@ impl ShellTool {};if sandbox_required {let action_dir = self.effective_action_dir_for_context(context);+ let timeout = requested_timeout;return self- .run_sandboxed(ctx, command, requested_timeout, &action_dir, context, meter)+ .run_sandboxed(+ ctx,+ command,+ timeout,+ come_back,+ &action_dir,+ context,+ meter,+ audit,+ ).await;}@@ -487,7 +478,10 @@ impl ShellTool {);};- let explicit_timeout = self.explicit_timeout(requested_timeout);+ // Docker cannot leave a command running: the model's come-back+ // time stays its deadline here.+ let explicit_timeout =+ self.explicit_timeout(come_back.deadline_instead(requested_timeout));if let Some(timeout) = explicit_timeout {let timeout_ms = timeout.as_nanos()@@ -637,9 +631,10 @@ impl ShellTool {run.python_utf8 = true;run.hide_window = self.runtime.hide_window();run.meter = meter;- let result =- super::shell_support::run_native(ctx, &config, run, requested_yield, context).await;- (true, result)+ (+ true,+ run_native(ctx, &config, run, come_back, context, audit).await,+ )}/// Execute a command through the sandbox backend. Called when the@@ -649,9 +644,11 @@ impl ShellTool {ctx: &DispatchCtx,command: &str,requested_timeout: Option<u64>,+ come_back: YieldRequest,action_dir: &Path,context: Option<&dyn ToolRunContext>,meter: Option<crate::sandbox::shell_progress::ShellOutputMeter>,+ audit: &AuditOnce,) -> (bool, ToolResult) {let config = crate::config::RuntimeConfig::default();let policy = crate::sandbox::resolve_sandbox_policy(@@ -662,80 +659,39 @@ impl ShellTool {&config,false,);- self.run_in_policy(- ctx,- policy,- command,- requested_timeout,- action_dir,- context,- meter,+ let timeout = requested_timeout;+ run_in_policy(+ ctx, policy, command, timeout, come_back, action_dir, context, meter, audit,).await}+}- /// Execute a command under a resolved sandbox `policy`.- async fn run_in_policy(- &self,- ctx: &DispatchCtx,- policy: crate::sandbox::SandboxPolicy,- command: &str,- requested_timeout: Option<u64>,- action_dir: &Path,- context: Option<&dyn ToolRunContext>,- meter: Option<crate::sandbox::shell_progress::ShellOutputMeter>,- ) -> (bool, ToolResult) {- use crate::sandbox;-- tracing::debug!(- backend = ?policy.backend,- command = command,- "[shell] routing to sandbox backend"- );-- let mut extra_env =- std::collections::HashMap::<std::ffi::OsString, std::ffi::OsString>::new();- let command_environment = command_environment_for_tool(context);- super::shell_support::extend_explicit_environment(- &mut extra_env,- command_environment.as_deref(),- );- // Apply the same Git config hardening to local and sandboxed shells.- extra_env.extend(tinytools_std::filesystem::shell_git_env());- // And the same Python defaults the native path sets.- for (var, val) in super::shell_support::python_env_defaults(|name| {- command_environment- .as_ref()- .and_then(|overrides| overrides.get(name).map(std::ffi::OsString::from))- .or_else(|| {- command_environment- .is_none()- .then(|| std::env::var_os(name))- .flatten()- })- }) {- extra_env.insert(var.into(), val);- }-- // Sandbox backends require a finite deadline. Without an explicit- // `timeout_secs`, substitute the generous effective-unbounded cap so a- // long command isn't killed while still bounding a wedged sandbox.- let explicit_timeout = self.explicit_timeout(requested_timeout);- let effective = explicit_timeout.unwrap_or_else(|| {- Duration::from_secs(tinytools_std::timeout::SANDBOX_UNBOUNDED_CAP_SECS)- });- tracing::debug!(- timeout_secs = effective.as_secs(),- requested_timeout_secs = ?requested_timeout,- unbounded = explicit_timeout.is_none(),- "[shell] starting sandboxed command"+/// Emit a single `CommandExecution` audit event. A write failure is logged as+/// a structured warning but not propagated — audit must never block or fail a+/// tool call, yet a silently broken audit trail must not go unnoticed.+pub(super) fn record_audit(+ audit: &AuditLogger,+ command: &str,+ approved: bool,+ allowed: bool,+ success: bool,+ duration_ms: u64,+) {+ if let Err(error) = audit.log_command_event(CommandExecutionLog {+ channel: "tool:shell",+ command,+ risk_level: "unknown",+ approved,+ allowed,+ success,+ duration_ms,+ }) {+ tracing::warn!(+ error = %error,+ channel = "tool:shell",+ "[shell] failed to persist command execution audit event");-- let result = sandbox::ops::execute_in_sandbox_metered(- ctx, &policy, command, action_dir, extra_env, effective, meter,- )- .await;- (true, exec_result(result, effective))}}diff --git a/crates/openhuman-core/src/tools/impl/system/shell_description.rs b/crates/openhuman-core/src/tools/impl/system/shell_description.rsindex 8b849bf1ef..44b34ca84e 100644--- a/crates/openhuman-core/src/tools/impl/system/shell_description.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell_description.rs@@ -21,10 +21,10 @@ pub(super) fn shell_description(flavor: ShellFlavor) -> &'static str {commands chain with `&&` (`;` is not a separator). For PowerShell run \`powershell -NoProfile -Command \"...\"`. Only stdout/stderr comes back, so a script \that computes silently or only writes a file returns nothing — print what you need, \- or read the file afterwards. A command still running after `timeout_secs` (default 120) comes back with its output so far and keeps running as a process: follow it with `shell_process` (wait for more, or kill it) instead of starting it again; it is stopped when the turn's time runs out. `timeout_secs` never kills: to stop a command after a time, put `timeout N` in it. Don't pipe a long command through `tail`/`head` to shorten it: long output is kept and capped for you, and a pipe holds everything back until the command ends."+ or read the file afterwards. A command still running after `timeout_secs` (default 120) comes back with its output so far and keeps running as a process: follow it with `shell_process` (wait for more, or kill it) instead of starting it again; it is stopped when the turn's time runs out. `timeout_secs` is when the call comes back, not a kill: to stop a command after a time, put `timeout N` in it. Don't pipe a long command through `tail`/`head` to shorten it: long output is kept and capped for you, and a pipe holds everything back until the command ends."}ShellFlavor::Posix => {- "Execute a shell command: run code, manipulate workspace files, or launch applications (`open -a Music`, `xdg-open music://`). Only stdout/stderr comes back, so a script that computes silently or only writes a file returns nothing — print what you need, or read the file afterwards. A command still running after `timeout_secs` (default 120) comes back with its output so far and keeps running as a process: follow it with `shell_process` (wait for more, or kill it) instead of starting it again; it is stopped when the turn's time runs out. `timeout_secs` never kills: to stop a command after a time, put `timeout N` in it. Don't pipe a long command through `tail`/`head` to shorten it: long output is kept and capped for you, and a pipe holds everything back until the command ends."+ "Execute a shell command: run code, manipulate workspace files, or launch applications (`open -a Music`, `xdg-open music://`). Only stdout/stderr comes back, so a script that computes silently or only writes a file returns nothing — print what you need, or read the file afterwards. A command still running after `timeout_secs` (default 120) comes back with its output so far and keeps running as a process: follow it with `shell_process` (wait for more, or kill it) instead of starting it again; it is stopped when the turn's time runs out. `timeout_secs` is when the call comes back, not a kill: to stop a command after a time, put `timeout N` in it. Don't pipe a long command through `tail`/`head` to shorten it: long output is kept and capped for you, and a pipe holds everything back until the command ends."}}}diff --git a/crates/openhuman-core/src/tools/impl/system/shell_process.rs b/crates/openhuman-core/src/tools/impl/system/shell_process.rsindex b8760d8fef..1bd51adbc1 100644--- a/crates/openhuman-core/src/tools/impl/system/shell_process.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell_process.rs@@ -148,6 +148,16 @@ pub(super) fn render(report: &Report, action: &str) -> ToolResult {"Process {id} is still running ({elapsed}; it stops at its {} timeout).",human_duration(report.timeout))),+ ReportState::Running if action == "kill" => ToolResult::error(format!(+ "Process {id} did not stop: it is still running ({elapsed}) after the stop request. \+ Check it with {{\"process\": \"{id}\", \"action\": \"status\"}}; it is stopped \+ at its {} timeout or when the turn ends.",+ human_duration(report.timeout)+ )),+ ReportState::Ended(ended) if action == "status" => ToolResult::success(format!(+ "Process {id} has {} ({elapsed}); {{\"process\": \"{id}\"}} returns its last output.",+ ended_phrase(*ended)+ )),ReportState::Running => {let mut out = format!("Process {id} is still running ({elapsed}).");out.push_str(&printed(@@ -200,15 +210,20 @@ pub(super) fn render(report: &Report, action: &str) -> ToolResult {}ReportState::AlreadyEnded(ended) => ToolResult::success(format!("Process {id} already {}; its output was returned earlier.",- match ended {- Ended::Exited(code) => format!("exited with code {code}"),- Ended::TimedOut => "stopped at its timeout".to_owned(),- Ended::Stopped => "stopped".to_owned(),- }+ ended_phrase(*ended))),}}+/// How a process ended, as a verb phrase.+fn ended_phrase(ended: Ended) -> String {+ match ended {+ Ended::Exited(code) => format!("exited with code {code}"),+ Ended::TimedOut => "stopped at its timeout".to_owned(),+ Ended::Stopped => "stopped".to_owned(),+ }+}+#[cfg(test)]#[path = "shell_process_tests.rs"]mod tests;diff --git a/crates/openhuman-core/src/tools/impl/system/shell_process_tests.rs b/crates/openhuman-core/src/tools/impl/system/shell_process_tests.rsindex dd4e4a0626..91037a0051 100644--- a/crates/openhuman-core/src/tools/impl/system/shell_process_tests.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell_process_tests.rs@@ -88,3 +88,26 @@ fn status_reports_without_output() {"Process p2 is still running (3m 5s; it stops at its 30m 0s timeout).");}++#[test]+fn a_stop_that_did_not_take_says_so() {+ let result = render(&report(ReportState::Running, "", 0), "kill");+ assert!(result.is_error);+ assert!(+ result.output().contains("did not stop"),+ "{}",+ result.output()+ );+}++#[test]+fn status_reports_an_end_and_points_at_wait_for_the_output() {+ let result = render(+ &report(ReportState::Ended(Ended::Exited(2)), "", 0),+ "status",+ );+ assert!(!result.is_error);+ let out = result.output();+ assert!(out.contains("has exited with code 2"), "{out}");+ assert!(out.contains("returns its last output"), "{out}");+}diff --git a/crates/openhuman-core/src/tools/impl/system/shell_run.rs b/crates/openhuman-core/src/tools/impl/system/shell_run.rsnew file mode 100644index 0000000000..5297eb8eb1--- /dev/null+++ b/crates/openhuman-core/src/tools/impl/system/shell_run.rs@@ -0,0 +1,259 @@+//! Running a `shell` call's command, and handing it to the turn when it is+//! still running at its yield point.+//!+//! Both the unsandboxed native shell and the sandbox backends come through+//! here. The native ones (no sandbox, TinyBox's native jail) can leave a+//! command running; Docker cannot, so there the model's own come-back time is+//! a deadline instead, as `timeout_secs` was before it yielded. A call yields+//! only when the turn moved the model's `timeout_secs` to [`YIELD_AFTER_ARG`]+//! (`agent::tinyagents::middleware::shell_yield`, which does so only when the+//! turn can follow a process with `shell_process`) and the turn has room for+//! another process.++use std::sync::{Arc, Mutex};+use std::time::{Duration, Instant};++use tinytools::{ToolResult, ToolRunContext};++use super::shell_support::{exec_result, yield_after, yielded_output, YIELD_AFTER_ARG};+use crate::sandbox::shell_exec::{ComeBack, ShellRunOutcome};+use crate::sandbox::shell_processes::{Ended, Slot};++/// Records the call's audit entry: whether the command passed the in-tool+/// checks, whether it succeeded, and how long it ran, in milliseconds.+pub(super) type AuditHook = Box<dyn FnOnce(bool, bool, u64) + Send>;++/// The call's audit entry, written once: right after the call, or, when the+/// command is left running, when the turn's registry sees it end.+pub(super) type AuditOnce = Mutex<Option<AuditHook>>;++/// What a call asked about coming back before its command ends.+#[derive(Clone, Copy, Debug, Default)]+pub(super) struct YieldRequest {+ /// The turn lets this call yield: it moved the model's `timeout_secs` to+ /// [`YIELD_AFTER_ARG`].+ pub(super) allowed: bool,+ /// The model's own come-back time, if it gave one.+ pub(super) requested: Option<u64>,+}++impl YieldRequest {+ pub(super) fn from_args(args: &serde_json::Value) -> Self {+ let value = args.get(YIELD_AFTER_ARG);+ Self {+ allowed: value.is_some(),+ requested: value.and_then(serde_json::Value::as_u64),+ }+ }++ /// The deadline for a backend that cannot leave a command running: the+ /// command's own deadline, or the model's come-back time if that is+ /// sooner, so the model's `timeout_secs` still bounds the command there.+ pub(super) fn deadline_instead(&self, deadline: Option<u64>) -> Option<u64> {+ match (+ deadline.filter(|secs| *secs > 0),+ self.requested.filter(|secs| *secs > 0),+ ) {+ (Some(deadline), Some(requested)) => Some(deadline.min(requested)),+ (deadline, requested) => deadline.or(requested),+ }+ }+}++/// A place in the turn's registry for this call, when it may yield.+fn slot(request: YieldRequest, context: Option<&dyn ToolRunContext>) -> Option<Slot> {+ if !request.allowed {+ return None;+ }+ let slot =+ crate::agent::tinyagents::host::run_context::shell_processes_for_tool(context)?.reserve();+ if slot.is_none() {+ tracing::debug!(+ "[shell] the turn is following its limit of processes; waiting for the end"+ );+ }+ slot+}++/// The yield point for this call with a slot reserved for it, and the+/// command's deadline. A call that cannot yield (the turn has no place left,+/// or the command is a deliberate wait) keeps the model's come-back time as+/// its deadline, as `timeout_secs` was, so it still comes back by then.+fn come_back(+ request: YieldRequest,+ context: Option<&dyn ToolRunContext>,+ command: &str,+ deadline: Duration,+) -> (Option<Slot>, Option<Duration>, Duration) {+ let slot = slot(request, context);+ let after = slot+ .as_ref()+ .and_then(|_| yield_after(request.requested, deadline, command));+ if after.is_some() {+ return (slot, after, deadline);+ }+ let deadline = request+ .requested+ .filter(|secs| *secs > 0)+ .map_or(deadline, |secs| deadline.min(Duration::from_secs(secs)));+ (None, None, deadline)+}++/// The tool result for a run, adopting a command still running into the+/// turn's registry; its audit entry then waits for the end.+fn finish(+ outcome: anyhow::Result<ShellRunOutcome>,+ slot: Option<Slot>,+ command: &str,+ started: Instant,+ timeout: Duration,+ audit: &AuditOnce,+) -> ToolResult {+ let live = match outcome {+ Ok(ShellRunOutcome::Running(live)) => live,+ Ok(ShellRunOutcome::Finished(result)) => return exec_result(Ok(result), timeout),+ Err(error) => return exec_result(Err(error), timeout),+ };+ let Some(slot) = slot else {+ // Unreachable: no yield point without a slot. The dropped handle+ // stops the process.+ return ToolResult::error("Command was left running unexpectedly.");+ };+ let (stdout, stderr) = (live.stdout.clone(), live.stderr.clone());+ let on_end = audit+ .lock()+ .unwrap_or_else(std::sync::PoisonError::into_inner)+ .take()+ .map(|hook| -> crate::sandbox::shell_processes::OnEnd {+ Box::new(move |ended: Ended, ran: Duration| {+ hook(+ true,+ ended == Ended::Exited(0),+ u64::try_from(ran.as_millis()).unwrap_or(u64::MAX),+ );+ })+ });+ let id = slot.adopt(live, command, started, timeout, on_end);+ ToolResult::success(yielded_output(+ &id,+ started.elapsed(),+ timeout,+ &stdout,+ &stderr,+ ))+}++/// Run a native shell command for a `shell` call.+pub(super) async fn run_native(+ ctx: &crate::core::runtime::DispatchCtx,+ config: &crate::config::Config,+ mut run: crate::sandbox::shell_exec::NativeShellRun,+ request: YieldRequest,+ context: Option<&dyn ToolRunContext>,+ audit: &AuditOnce,+) -> ToolResult {+ let (slot, after, deadline) = come_back(request, context, &run.command, run.timeout);+ run.yield_after = after;+ run.timeout = deadline;+ let (command, timeout) = (run.command.clone(), run.timeout);+ let started = Instant::now();+ let outcome = crate::sandbox::shell_exec::run_yielding(ctx, config, run).await;+ finish(outcome, slot, &command, started, timeout, audit)+}++/// Run a command under a resolved sandbox `policy` for a `shell` call.+pub(super) async fn run_in_policy(+ ctx: &crate::core::runtime::DispatchCtx,+ policy: crate::sandbox::SandboxPolicy,+ command: &str,+ requested_timeout: Option<u64>,+ request: YieldRequest,+ action_dir: &std::path::Path,+ context: Option<&dyn ToolRunContext>,+ meter: Option<crate::sandbox::shell_progress::ShellOutputMeter>,+ audit: &AuditOnce,+) -> (bool, ToolResult) {+ tracing::debug!(+ backend = ?policy.backend,+ command = command,+ "[shell] routing to sandbox backend"+ );++ let mut extra_env = std::collections::HashMap::<std::ffi::OsString, std::ffi::OsString>::new();+ let command_environment =+ crate::agent::tinyagents::host::run_context::command_environment_for_tool(context);+ super::shell_support::extend_explicit_environment(+ &mut extra_env,+ command_environment.as_deref(),+ );+ // Apply the same Git config hardening to local and sandboxed shells.+ extra_env.extend(tinytools_std::filesystem::shell_git_env());+ // And the same Python defaults the native path sets.+ for (var, val) in super::shell_support::python_env_defaults(|name| {+ command_environment+ .as_ref()+ .and_then(|overrides| overrides.get(name).map(std::ffi::OsString::from))+ .or_else(|| {+ command_environment+ .is_none()+ .then(|| std::env::var_os(name))+ .flatten()+ })+ }) {+ extra_env.insert(var.into(), val);+ }++ // Sandbox backends require a finite deadline. Without an explicit+ // `timeout_secs`, substitute the generous effective-unbounded cap so a+ // long command isn't killed while still bounding a wedged sandbox.+ let explicit_timeout = tinytools_std::timeout::explicit_call_timeout_duration(+ requested_timeout,+ tinytools_std::timeout::MAX_TIMEOUT_SECS,+ );+ let effective = explicit_timeout+ .unwrap_or_else(|| Duration::from_secs(tinytools_std::timeout::SANDBOX_UNBOUNDED_CAP_SECS));+ let (slot, yield_after, effective) = come_back(request, context, command, effective);+ let deadline_instead = request+ .deadline_instead(requested_timeout)+ .map(Duration::from_secs);+ tracing::debug!(+ timeout_secs = effective.as_secs(),+ requested_timeout_secs = ?requested_timeout,+ unbounded = explicit_timeout.is_none(),+ yield_after_secs = ?yield_after.map(|after| after.as_secs()),+ "[shell] starting sandboxed command"+ );++ let started = Instant::now();+ let outcome = crate::sandbox::ops::execute_in_sandbox_yielding(+ ctx,+ &policy,+ command,+ action_dir,+ extra_env,+ effective,+ ComeBack {+ yield_after,+ deadline_instead,+ },+ meter,+ )+ .await;+ let result = finish(outcome, slot, command, started, effective, audit);+ (true, result)+}++/// The call's audit entry, to be written once by whoever learns the outcome.+pub(super) fn audit_once(+ audit: Arc<crate::security::AuditLogger>,+ command: String,+ approved: bool,+) -> AuditOnce {+ Mutex::new(Some(Box::new(move |allowed, success, duration_ms| {+ super::shell::record_audit(&audit, &command, approved, allowed, success, duration_ms);+ })))+}++#[cfg(test)]+#[path = "shell_run_tests.rs"]+mod tests;diff --git a/crates/openhuman-core/src/tools/impl/system/shell_run_tests.rs b/crates/openhuman-core/src/tools/impl/system/shell_run_tests.rsnew file mode 100644index 0000000000..469c1683c5--- /dev/null+++ b/crates/openhuman-core/src/tools/impl/system/shell_run_tests.rs@@ -0,0 +1,199 @@+//! A `shell` call's run: what it asked about coming back, the sandboxed path's+//! yield, and when its audit entry is written.++use super::*;+use crate::agent::tinyagents::host::run_context::test_turn_run;+use crate::security::{AuditLogger, AutonomyLevel, SecurityPolicy};+use crate::tools::implementations::system::{ShellProcessTool, ShellTool};+use serde_json::json;+use tinyagents_harness::ids::CallId;+use tinyagents_harness::tool::ToolExecutionContext;+use tinytools::{Tool, ToolCallOptions};++#[test]+fn a_call_yields_only_when_the_turn_moved_its_timeout() {+ let none = YieldRequest::from_args(&json!({"command": "ls", "timeout_secs": 60}));+ assert!(!none.allowed);+ let default = YieldRequest::from_args(&json!({"command": "ls", YIELD_AFTER_ARG: null}));+ assert!(default.allowed);+ assert_eq!(default.requested, None);+ let given = YieldRequest::from_args(&json!({"command": "ls", YIELD_AFTER_ARG: 45}));+ assert!(given.allowed);+ assert_eq!(given.requested, Some(45));+}++#[test]+fn where_a_command_cannot_be_left_running_the_models_time_is_its_deadline() {+ let asked = YieldRequest {+ allowed: true,+ requested: Some(60),+ };+ assert_eq!(asked.deadline_instead(Some(600)), Some(60));+ assert_eq!(+ asked.deadline_instead(Some(30)),+ Some(30),+ "the turn's is sooner"+ );+ assert_eq!(asked.deadline_instead(None), Some(60));+ let default = YieldRequest {+ allowed: true,+ requested: None,+ };+ assert_eq!(default.deadline_instead(Some(600)), Some(600));+ assert_eq!(default.deadline_instead(None), None);+ let wait = YieldRequest {+ allowed: true,+ requested: Some(0),+ };+ assert_eq!(+ wait.deadline_instead(None),+ None,+ "0 means wait for the end"+ );+}++fn unsandboxed_policy() -> crate::sandbox::SandboxPolicy {+ crate::sandbox::resolve_sandbox_policy(+ &crate::core::runtime::DispatchCtx::unbound(),+ tinyagents_definition::SandboxMode::None,+ &std::env::temp_dir(),+ &std::env::temp_dir().join("oh-shell-run-state"),+ &crate::config::RuntimeConfig::default(),+ false,+ )+}++#[cfg(not(windows))]+#[tokio::test]+async fn a_sandboxed_command_yields_and_is_followed_to_its_end() {+ let run = test_turn_run();+ let call = ToolExecutionContext::from_run_context(&run, CallId::new("c1"));+ let audit = Mutex::new(None);+ let (allowed, result) = run_in_policy(+ &crate::core::runtime::DispatchCtx::unbound(),+ unsandboxed_policy(),+ "echo first; sleep 2; echo second",+ None,+ YieldRequest {+ allowed: true,+ requested: Some(1),+ },+ &std::env::temp_dir(),+ Some(&call),+ None,+ &audit,+ )+ .await;+ assert!(allowed);+ let out = result.output();+ assert!(out.starts_with("Still running as process p1"), "{out}");+ assert!(out.contains("first"), "{out}");++ let follow = ToolExecutionContext::from_run_context(&run, CallId::new("c2"));+ let done = ShellProcessTool::new()+ .execute_with_context(+ json!({"process": "p1", "wait_secs": 10}),+ ToolCallOptions::default(),+ Some(&follow),+ )+ .await+ .unwrap();+ let out = done.output();+ assert!(+ out.contains("exited with code 0") && out.contains("second"),+ "{out}"+ );+}++#[cfg(not(windows))]+#[tokio::test]+async fn without_a_turn_a_sandboxed_command_runs_to_its_end() {+ let (_, result) = run_in_policy(+ &crate::core::runtime::DispatchCtx::unbound(),+ unsandboxed_policy(),+ "echo start; sleep 2; echo done",+ None,+ YieldRequest {+ allowed: true,+ requested: Some(30),+ },+ &std::env::temp_dir(),+ None,+ None,+ &Mutex::new(None),+ )+ .await;+ assert_eq!(+ result.output().trim(),+ "start\ndone",+ "nothing to follow it with"+ );+}++fn audit_log() -> (Arc<AuditLogger>, tempfile::TempDir) {+ let tmp = tempfile::tempdir().expect("create tempdir");+ let logger = AuditLogger::new(+ crate::config::AuditConfig {+ enabled: true,+ log_path: "audit.log".into(),+ max_size_mb: 10,+ },+ tmp.path().to_path_buf(),+ )+ .expect("create audit logger");+ (Arc::new(logger), tmp)+}++fn audit_lines(dir: &tempfile::TempDir) -> Vec<crate::security::AuditEvent> {+ std::fs::read_to_string(dir.path().join("audit.log"))+ .unwrap_or_default()+ .lines()+ .map(|line| serde_json::from_str(line).expect("audit event JSON parses"))+ .collect()+}++#[cfg(not(windows))]+#[tokio::test]+async fn a_command_left_running_is_audited_when_it_ends() {+ let (audit, dir) = audit_log();+ let shell = ShellTool::new(+ Arc::new(SecurityPolicy {+ autonomy: AutonomyLevel::Full,+ workspace_dir: std::env::temp_dir(),+ action_dir: std::env::temp_dir(),+ ..SecurityPolicy::default()+ }),+ Arc::new(crate::modules::tinybox::host_runtime::NativeRuntime::new()),+ audit,+ );+ let run = test_turn_run();+ let call = ToolExecutionContext::from_run_context(&run, CallId::new("c1"));+ let yielded = shell+ .execute_with_context(+ json!({"command": "echo a; sleep 2; exit 3", YIELD_AFTER_ARG: 1}),+ ToolCallOptions::default(),+ Some(&call),+ )+ .await+ .unwrap();+ assert!(+ yielded.output().starts_with("Still running"),+ "{}",+ yielded.output()+ );+ assert!(audit_lines(&dir).is_empty(), "no outcome is known yet");++ let follow = ToolExecutionContext::from_run_context(&run, CallId::new("c2"));+ ShellProcessTool::new()+ .execute_with_context(+ json!({"process": "p1", "wait_secs": 10}),+ ToolCallOptions::default(),+ Some(&follow),+ )+ .await+ .unwrap();+ let lines = audit_lines(&dir);+ assert_eq!(lines.len(), 1, "one entry, written at the end");+ let result = lines[0].result.as_ref().expect("result present");+ assert!(!result.success, "exit 3 is audited as a failure");+}diff --git a/crates/openhuman-core/src/tools/impl/system/shell_support.rs b/crates/openhuman-core/src/tools/impl/system/shell_support.rsindex f0db41868f..2e19f2acd4 100644--- a/crates/openhuman-core/src/tools/impl/system/shell_support.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell_support.rs@@ -247,10 +247,10 @@ pub(super) const MIN_YIELD_AFTER_SECS: u64 = 30;/// The yield point for `command` with `deadline` to run: the requested time/// (default 120 s, at most 600 s), but no later than an eighth of the deadline-/// (and not earlier than 30 s for that cap), so a command started late in a-/// turn still comes back before the turn's deadline kills it. None for `0`, and-/// none for a deliberate wait (a command that starts with `sleep`), which would-/// only come back with nothing to show.+/// (and not earlier than 30 s for that cap), and always before the deadline+/// itself, so a command started late in a turn still comes back before the+/// turn's deadline kills it. None for `0`, and none for a deliberate wait+/// ([`is_deliberate_wait`]), which would only come back with nothing to show.pub(super) fn yield_after(requested: Option<u64>,deadline: Duration,@@ -259,12 +259,35 @@ pub(super) fn yield_after(let secs = requested.unwrap_or(DEFAULT_YIELD_AFTER_SECS).min(MAX_YIELD_AFTER_SECS);- let first = command.split_whitespace().next();- if secs == 0 || first == Some("sleep") {+ if secs == 0 || is_deliberate_wait(command) {return None;}- let cap = (deadline.as_secs() / 8).max(MIN_YIELD_AFTER_SECS);- Some(Duration::from_secs(secs.min(cap)))+ let deadline = deadline.as_secs();+ let cap = (deadline / 8).max(MIN_YIELD_AFTER_SECS).min(deadline / 2);+ (cap > 0).then(|| Duration::from_secs(secs.min(cap)))+}++/// Commands that only look at the state a wait led to: reading a log,+/// listing files, checking a process.+const QUICK_LOOKS: &[&str] = &[+ "cat", "date", "df", "du", "echo", "grep", "head", "ls", "pgrep", "printf", "ps", "stat",+ "tail", "test", "wc",+];++/// A command that starts with `sleep` and then only looks at what happened+/// (`sleep 60; tail -20 build.log`). It is meant to wait, so it is not cut+/// short. `sleep 2; make -j8` starts real work after the sleep and yields as+/// usual.+pub(super) fn is_deliberate_wait(command: &str) -> bool {+ let mut steps = command+ .split(['\n', ';', '&', '|'])+ .map(str::trim)+ .filter(|step| !step.is_empty());+ let first_word = |step: &str| step.split_whitespace().next().unwrap_or("").to_owned();+ if steps.next().map(first_word).as_deref() != Some("sleep") {+ return false;+ }+ steps.all(|step| QUICK_LOOKS.contains(&first_word(step).as_str()))}/// The result of a call that came back with its command still running.@@ -289,48 +312,6 @@ pub(super) fn yielded_output(out}-/// Run a native shell command for a `shell` call. In a turn with room for-/// another process, a command still running at its yield point is handed to-/// the turn's registry and comes back as a process id; direct dispatch (no run-/// context) and a turn already following its limit of processes wait for the-/// end as before.-pub(super) async fn run_native(- ctx: &crate::core::runtime::DispatchCtx,- config: &crate::config::Config,- mut run: crate::sandbox::shell_exec::NativeShellRun,- requested_yield: Option<u64>,- context: Option<&dyn tinytools::ToolRunContext>,-) -> ToolResult {- use crate::sandbox::shell_exec::{run_yielding, ShellRunOutcome};- let processes = crate::agent::tinyagents::host::run_context::shell_processes_for_tool(context)- .filter(|processes| processes.has_room());- if processes.is_some() {- run.yield_after = yield_after(requested_yield, run.timeout, &run.command);- }- let (command, timeout) = (run.command.clone(), run.timeout);- let started = std::time::Instant::now();- match run_yielding(ctx, config, run).await {- Ok(ShellRunOutcome::Running(live)) => {- let Some(processes) = processes else {- // Unreachable: no yield point without a registry. The dropped- // handle stops the process.- return ToolResult::error("Command was left running unexpectedly.");- };- let (stdout, stderr) = (live.stdout.clone(), live.stderr.clone());- let id = processes.adopt(live, &command, started, timeout);- ToolResult::success(yielded_output(- &id,- started.elapsed(),- timeout,- &stdout,- &stderr,- ))- }- Ok(ShellRunOutcome::Finished(result)) => exec_result(Ok(result), timeout),- Err(error) => exec_result(Err(error), timeout),- }-}-/// `label` and the non-empty streams, or `empty` when both are empty.pub(super) fn printed(stdout: &str, stderr: &str, label: &str, empty: &str) -> String {let (stdout, stderr) = (stdout.trim_end(), stderr.trim_end());diff --git a/crates/openhuman-core/src/tools/impl/system/shell_support_tests.rs b/crates/openhuman-core/src/tools/impl/system/shell_support_tests.rsindex e1cd5f65e7..994c4178c6 100644--- a/crates/openhuman-core/src/tools/impl/system/shell_support_tests.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell_support_tests.rs@@ -31,8 +31,25 @@ fn zero_waits_for_the_end() {fn a_deliberate_wait_is_not_cut_short() {assert_eq!(yield_after(None, LONG, "sleep 100; tail run.log"), None);assert_eq!(yield_after(Some(30), LONG, " sleep 5"), None);+ assert_eq!(+ yield_after(None, LONG, "sleep 240 && tail -3 /app/run.log; ls -la /app"),+ None+ );assert!(yield_after(None, LONG, "sleeper --run").is_some());assert!(yield_after(None, LONG, "make; sleep 5").is_some());+ assert!(+ yield_after(None, LONG, "sleep 2; make -j8").is_some(),+ "real work after the sleep still yields"+ );+}++#[test]+fn the_yield_point_stays_before_a_short_deadline() {+ assert_eq!(+ yield_after(None, Duration::from_secs(20), "make"),+ Some(Duration::from_secs(10))+ );+ assert_eq!(yield_after(None, Duration::from_secs(1), "make"), None);}#[test]diff --git a/crates/openhuman-core/src/tools/impl/system/shell_tests_keep_running_tests.rs b/crates/openhuman-core/src/tools/impl/system/shell_tests_keep_running_tests.rsindex a17e5c3f44..ad68617e94 100644--- a/crates/openhuman-core/src/tools/impl/system/shell_tests_keep_running_tests.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell_tests_keep_running_tests.rs@@ -52,10 +52,14 @@ async fn a_long_command_yields_and_shell_process_follows_it_to_its_end() {#[tokio::test]async fn without_a_turn_a_command_runs_to_its_end() {let result = shell()- .execute(json!({"command": "sleep 2; echo done", "yield_after_secs": 1}))+ .execute(json!({"command": "echo start; sleep 2; echo done", "yield_after_secs": 30})).await.unwrap();- assert_eq!(result.output().trim(), "done");+ assert_eq!(+ result.output().trim(),+ "start\ndone",+ "nothing to follow it with"+ );}#[cfg(not(windows))]@@ -76,18 +80,25 @@ async fn yield_after_zero_waits_for_the_end() {#[cfg(not(windows))]#[tokio::test]-async fn a_deliberate_wait_runs_to_its_end() {+async fn a_call_that_cannot_yield_comes_back_by_the_models_time() {+ // A deliberate wait does not yield, so the model's `timeout_secs` stops it+ // instead, as `timeout_secs` did before it yielded.let run = test_turn_run();let call = ToolExecutionContext::from_run_context(&run, CallId::new("c1"));let result = shell().execute_with_context(- json!({"command": "sleep 2; echo done", "yield_after_secs": 1}),+ json!({"command": "sleep 5; tail -1 /dev/null", "yield_after_secs": 1}),ToolCallOptions::default(),Some(&call),).await.unwrap();- assert_eq!(result.output().trim(), "done");+ assert!(result.is_error);+ assert!(+ result.output().starts_with("Command timed out after 1s"),+ "{}",+ result.output()+ );}#[cfg(not(windows))]diff --git a/crates/openhuman-core/src/tools/impl/system/shell_tests_timeout_tests.rs b/crates/openhuman-core/src/tools/impl/system/shell_tests_timeout_tests.rsindex f49abc3e5f..c4403cc304 100644--- a/crates/openhuman-core/src/tools/impl/system/shell_tests_timeout_tests.rs+++ b/crates/openhuman-core/src/tools/impl/system/shell_tests_timeout_tests.rs@@ -2,6 +2,7 @@//! tool's `timeout_secs` (#6953).use super::*;+use crate::tools::implementations::system::shell_support::exec_result;#[cfg(not(windows))]fn process_is_running(pid: i32) -> bool {diff --git a/docs/specs/shell-keep-running.md b/docs/specs/shell-keep-running.mdindex 1047902812..48ec4369ce 100644--- a/docs/specs/shell-keep-running.md+++ b/docs/specs/shell-keep-running.md@@ -89,11 +89,20 @@ In a turn, `shell`'s `timeout_secs` is a **yield point**, not a kill.The result is not an error. A command that finishes before the yield pointreturns exactly as today.- The yield point comes no later than an eighth of the command's deadline- (not earlier than 30 s for that cap), so a command started late in a turn- still yields before the turn kills it.-- A command that starts with `sleep` is a deliberate wait: it runs to its end- instead of yielding with nothing to show (Claude Code makes the same- exception).+ (not earlier than 30 s for that cap) and always before the deadline itself,+ so a command started late in a turn still yields before the turn kills it.+- A deliberate wait, `sleep` followed only by commands that look at the+ result (`sleep 60; tail -20 build.log`), runs to its end instead of+ yielding with nothing to show. Claude Code makes a similar exception.+ `sleep 2; make -j8` yields as usual.+- **A call that cannot yield is still bounded by the model's time.** When the+ turn already follows four processes, when the command is a deliberate wait,+ or on a backend that cannot leave a command running (Docker), the model's+ `timeout_secs` stays the command's deadline, as before.+- **Only where the model can follow.** The host middleware moves+ `timeout_secs` aside only in a turn that registers `shell_process`. An+ agent whose tool belt names `shell` alone keeps `timeout_secs` as a kill+ and never yields.- **The kill is the turn's.** TinyAgents' `ShellTurnBudget` clamp writes theturn's remaining budget into `timeout_secs`, and the shell passes that toTinyBox as the command's `timeout_ms`. So a host middleware@@ -119,13 +128,17 @@ A new tool, `shell_process`:|---|---|| `process` | The id a yield returned. || `action` | `wait` (default), `kill` or `status`. |-| `wait_secs` | For `wait`: how long to block before returning, default 120, capped by the hard deadline and the turn budget. |+| `wait_secs` | For `wait`: how long to block before returning, default 120, clamped to 1–600. The process's own deadline still stops it. |- `wait` blocks host-side until the process exits or `wait_secs` passes, thenreturns only the output written since the last read, plus "exited withcode N" or "still running (Nm Ns)".- `kill` cancels it and returns its remaining output.-- `status` returns running or exited and the byte count, without output.+- `status` says whether it is still running, without output. It refreshes+ the state with a read that returns at once and keeps that output for the+ next `wait`, so a process that has ended says so and frees its place.+- A `kill` that did not take within 10 s is reported as such; the process+ keeps its place until it stops.- A `wait` on a process that has already exited and been read returns"already exited with code N" at once, so a loop cannot spin on it.@@ -133,11 +146,18 @@ A new tool, `shell_process`:A per-turn registry, e.g. `sandbox/shell_processes.rs`:-- An entry holds the TinyBox resource and process, the read cursor, the start- time, the command line (for display), the hard deadline and the meter.+- An entry holds the live handle (TinyBox resource, process and read cursor),+ the start time, the command line (for display) and the hard deadline.+ Output read after the yield goes to the model in `shell_process` replies,+ not through the shell's progress meter.- Ids are short and turn-local (`p1`, `p2`, …).-- At most 4 live processes per turn; a fifth command waits for its end as- before.+- At most 4 live processes per turn. A call reserves its place before the+ command starts, so parallel calls cannot overshoot; a command that finishes+ first frees it. A fifth command waits for its end, bounded by the model's+ `timeout_secs`.+- **Audit:** a command left running is audited when it ends (or is stopped+ at turn end), with its real outcome and duration, not as a success at the+ yield.- When a process has stopped and its output has been read, the registryreleases the output and closes the resource.- **Turn end:** the registry lives on the turn's run context@@ -165,8 +185,9 @@ A per-turn registry, e.g. `sandbox/shell_processes.rs`:The shell tool's description says that a command still running after`timeout_secs` comes back with its output so far and keeps running as aprocess, to follow it with `shell_process` rather than start a second copy,-that it is stopped when the turn's time runs out, that `timeout_secs` never-kills (put `timeout N` in the command for that), and not to pipe a long+that it is stopped when the turn's time runs out, that `timeout_secs` is when+the call comes back rather than a kill (put `timeout N` in the command for+that), and not to pipe a longcommand through `tail` or `head` (a pipeline holds its output until thecommand ends, so a yield shows nothing).@@ -186,8 +207,8 @@ command ends, so a yield shows nothing).|---|---|| Native TinyBox shell (default) | yes || Native jail | yes (same process API) |-| Docker module (`run_docker`) | no: it is one-shot and has no process handle; behaves as today |-| Sandbox-policy path | yes where it reaches `shell_exec::run_host`, otherwise as today |+| Docker module (`run_docker`) | no: it is one-shot and has no process handle; the model's `timeout_secs` is its deadline, as before |+| Sandbox-policy path (sandboxed agents, SaaS) | yes on the no-sandbox and native-jail backends; Docker as above || Cron jobs | unchanged; they keep blocking semantics || Windows | yes, nothing here is Unix-specific |