Skip to content

Quickly sandbox your AI agent.

Python

Add streaming stdin and improve mounted-file behavior

Tyler Paynef95aff394182
diff --git a/docs/contributor-guide/03-testing.md b/docs/contributor-guide/03-testing.md
index 354aab6..b52608d 100644
--- a/docs/contributor-guide/03-testing.md
+++ b/docs/contributor-guide/03-testing.md
@@ -47,3 +47,14 @@ async def test_my_feature(tmp_dir):
| `@pytest.mark.docker` | Test needs Docker daemon |
Skip conditions (`skip_no_qemu`, `skip_no_docker`, `skip_no_mke2fs`) are in `tests/conftest.py`.
+
+## Streaming stdin integration
+
+The stdin integration tests exercise both virtio-serial and HTTP. They require
+an image rebuilt with a guest agent that advertises `stdin_streaming`; older
+images skip these cases. To use a local save containing the updated agent:
+
+```bash
+QUICKSAND_STDIN_TEST_IMAGE=/path/to/updated-save \
+ uv run pytest tests/integration/execution/test_stdin.py -v
+```
diff --git a/docs/under-the-hood/03-running-commands.md b/docs/under-the-hood/03-running-commands.md
index 41b5d3f..9aab076 100644
--- a/docs/under-the-hood/03-running-commands.md
+++ b/docs/under-the-hood/03-running-commands.md
@@ -17,7 +17,7 @@ This skips the entire firmware boot sequence and starts the Linux kernel immedia
## Agent token injection
-The guest agent is a minimal Rust HTTP server baked into the initrd. It reads its auth token and port from the kernel command line (`/proc/cmdline`):
+The guest agent is a minimal Rust server bundled in the VM image. It reads its auth token and port from the kernel command line (`/proc/cmdline`):
```
quicksand_token=abc123 quicksand_port=8080
@@ -39,7 +39,12 @@ The VM's serial port is connected to QEMU's stdin/stdout. Combined with `console
result = await sb.execute("ls /")
```
-The `execute()` call sends an HTTP POST to the guest agent. But the guest is behind QEMU's NAT. It doesn't have a real IP on the host network. The connection is routed via a port forward in the `-netdev` flags:
+Quicksand normally sends length-prefixed JSON frames over a virtio-serial channel.
+Request IDs allow command replies and output events to be routed independently.
+This transport does not require guest networking.
+
+The HTTP fallback sends a POST to the guest agent. The guest is behind QEMU's
+NAT, so this connection is routed via a port forward in the `-netdev` flags:
```bash
-netdev user,id=net0,...,hostfwd=tcp:127.0.0.1:8080-:8080
@@ -61,12 +66,42 @@ sb.execute("ls /")
result = await sb.execute("apt install -y python3", on_stdout=callback)
```
-Streaming uses the `/execute_stream` endpoint instead of `/execute`. The guest agent responds with Server-Sent Events (SSE). Each event contains one output chunk.
+Streaming uses the `execute_stream` method over virtio-serial, or the
+`/execute_stream` endpoint over HTTP. HTTP responses use Server-Sent Events
+(SSE); serial output events additionally carry the execution request's `id`.
```
-data: {"type": "stdout", "data": "Reading package lists..."}
-data: {"type": "stdout", "data": "Building dependency tree..."}
-data: {"type": "exit", "exit_code": 0}
+data: {"stream": "stdout", "data": "Reading package lists..."}
+data: {"stream": "stdout", "data": "Building dependency tree..."}
+data: {"stream": "exit", "exit_code": 0}
```
The HTTP connection stays open until the command finishes. Callbacks fire for each chunk as it arrives.
+
+## Incremental stdin
+
+Authentication responses advertise `capabilities: ["stdin_streaming"]` when the
+guest supports input streaming. The host requires this capability before
+starting a command with stdin, so old agents cannot silently ignore the input.
+
+The host adds a unique `stdin_id` to `execute_stream`. Once the process and its
+input channel are ready, the agent sends `{"stream": "ready"}`. The host then
+sends separate `stdin` requests (HTTP `POST /stdin` or serial method `stdin`):
+
+```json
+{"stdin_id": "<execution>", "data": "<base64-encoded bytes>"}
+```
+
+Each chunk contains at most 64 KiB of decoded input. The agent acknowledges it
+after writing to the child pipe, and the host waits for that acknowledgement
+before requesting more input. Exhausting the input iterable sends
+`{"stdin_id": "<execution>", "eof": true}` to close the pipe.
+
+Input and cancellation requests bypass the HTTP execution lock and remain
+available during exclusive commands. They are not retried: replaying a write
+after a lost acknowledgement could duplicate input. Output continues on the
+original stream while input is being supplied.
+
+Cancellation uses HTTP `POST /cancel` or serial method `cancel`, with the same
+`stdin_id`. The agent terminates the execution's process group and releases its
+input and exclusive-command state. Guest timeouts perform the same cleanup.
diff --git a/docs/under-the-hood/04-file-exchange.md b/docs/under-the-hood/04-file-exchange.md
index 2719c37..9cedf30 100644
--- a/docs/under-the-hood/04-file-exchange.md
+++ b/docs/under-the-hood/04-file-exchange.md
@@ -30,6 +30,12 @@ mount -t cifs //10.0.2.100/QUICKSAND0 /mnt/code \
`nosharesock` forces a dedicated TCP connection per mount. Without it, a mount/unmount/mount cycle wedges because the kernel CIFS client tries to resume a session the server already closed.
+For the bundled SMB server, Quicksand also appends `nobrl`: SMB byte-range locks
+are not implemented, so the guest kernel handles locking locally instead.
+This applies to both boot-time and hot mounts. Locks do not coordinate access
+with the host or other mounts/sandboxes. Servers that support byte-range locks,
+such as the optional native SMB backend, retain server-side locking.
+
Each mount gets its own SMB share name (`QUICKSAND0`, `QUICKSAND1`, ...). The SMB server maps each share name to the corresponding host directory.
## Hot-mounts
diff --git a/docs/user-guide/03-running-commands.md b/docs/user-guide/03-running-commands.md
index ad3b006..4d47ac5 100644
--- a/docs/user-guide/03-running-commands.md
+++ b/docs/user-guide/03-running-commands.md
@@ -52,7 +52,50 @@ result = await sb.execute(
)
```
-The callbacks receive chunks of output in real time via Server-Sent Events. The final `ExecuteResult` still contains the complete stdout/stderr.
+The callbacks receive chunks of output in real time over virtio-serial, or via
+Server-Sent Events when using the HTTP fallback. The final `ExecuteResult` still
+contains the complete stdout/stderr.
+
+## Streaming stdin
+
+Pass an async iterable of byte chunks to feed a running command incrementally:
+
+```python
+async def input_chunks():
+ yield b"first line\n"
+ yield b"second line\n"
+
+result = await sb.execute(
+ "cat",
+ stdin=input_chunks(),
+ on_stdout=lambda chunk: print(chunk, end=""),
+)
+```
+
+The producer is advanced as the guest accepts input, rather than being buffered
+in full. Output callbacks run concurrently, so the producer can wait for output
+before supplying its next chunk. Finishing the iterable closes stdin (EOF).
+
+For small inputs, `stdin` also accepts `bytes` or a UTF-8 string:
+
+```python
+result = await sb.execute("wc -c", stdin=b"hello")
+result = await sb.execute("cat", stdin="hello\n")
+result = await sb.execute("cat", stdin=b"") # Send EOF immediately
+```
+
+Async iterables must yield `bytes`, including for binary data. Quicksand splits
+large chunks automatically. Omitting `stdin` preserves the usual execution
+behavior.
+
+Input consumption stops when the command exits. Cancelling the call or reaching
+its timeout terminates the stdin-enabled command's process group. Exceptions
+from the input producer propagate to the caller after the command is cancelled.
+
+Streaming stdin requires a guest image whose agent advertises the
+`stdin_streaming` capability. Older images return an explicit error without
+starting the command or consuming input. Update or rebuild the image with the
+new guest agent; upgrading the host Python package alone is not sufficient.
## Multi-step workflows
diff --git a/docs/user-guide/04-file-exchange.md b/docs/user-guide/04-file-exchange.md
index 4fd6022..7f26582 100644
--- a/docs/user-guide/04-file-exchange.md
+++ b/docs/user-guide/04-file-exchange.md
@@ -25,6 +25,35 @@ async with Sandbox(
Files appear inside the VM at the specified guest path. Changes to non-readonly mounts are visible on both sides immediately.
+### Git repositories on CIFS mounts
+
+CIFS mounts present synthesized Unix permissions rather than each host file's
+executable bit. As a result, `git status` inside the guest can report mode-only
+changes for otherwise unchanged files.
+
+Disable executable-bit comparisons for Git commands against the mounted
+repository:
+
+```python
+result = await sb.execute("git -c core.filemode=false -C /mnt/code status")
+```
+
+The `-c` option applies only to that command; it does not change the host
+repository's Git configuration. Content changes are still reported, but
+executable-bit changes are ignored.
+
+### File locking on CIFS mounts
+
+The bundled SMB server does not implement server-side byte-range locks. Quicksand
+mounts its shares with `nobrl`, allowing `flock` and SQLite to use local locking
+inside the guest instead.
+
+These locks coordinate processes using the same mount in one sandbox. They do
+not coordinate with host processes, other sandboxes, or independent mounts of
+the same host directory. Do not concurrently access a writable SQLite database
+from those locations. The optional native SMB backend retains server-side
+locking.
+
## Dynamic hot-mounts
Share a directory into an already-running sandbox:
diff --git a/packages/dev/quicksand-image-tools/quicksand_image_tools/agent-openapi.yaml b/packages/dev/quicksand-image-tools/quicksand_image_tools/agent-openapi.yaml
index eb6ae2b..381b3d3 100644
--- a/packages/dev/quicksand-image-tools/quicksand_image_tools/agent-openapi.yaml
+++ b/packages/dev/quicksand-image-tools/quicksand_image_tools/agent-openapi.yaml
@@ -7,6 +7,20 @@ info:
## Authentication
- Token and port are passed via kernel command line (`quicksand_token`, `quicksand_port`)
- All endpoints except `/authenticate` require `Authorization: Bearer <token>` header
+
+ ## Optional stdin streaming
+ Successful authentication advertises `capabilities: ["stdin_streaming"]`.
+ Released images without this capability do not support stdin streaming;
+ clients must check the capability before sending `stdin_id`.
+
+ The virtio-serial transport uses the same execute_stream, stdin, and cancel
+ parameters. Its successful authenticate result includes the same capabilities.
+ Control replies are `{"id": <control request id>, "result": <response>}`.
+ Every streaming event includes the original execute_stream request `id`.
+ Invalid serial requests receive `{"id": <request id>, "error": {"message": ...}}`;
+ rejected stdin-enabled execute_stream requests also receive an exit event with
+ exit_code -1. Stdin and cancel controls require an authenticated serial session
+ and are processed independently of exclusive commands and blocked stdin writes.
version: 1.0.0
servers:
@@ -44,6 +58,13 @@ components:
properties:
authenticated:
type: boolean
+ capabilities:
+ type: array
+ items:
+ type: string
+ description: |
+ Successful authentication includes "stdin_streaming". Older guest
+ images omit this field; clients must not assume support when absent.
ExecuteRequest:
type: object
@@ -62,6 +83,88 @@ components:
type: string
nullable: true
description: Working directory for command execution
+ exclusive:
+ type: boolean
+ default: false
+ description: Reject other execution requests while this command runs
+
+ StdinId:
+ type: string
+ pattern: '^[0-9a-fA-F]{32}$'
+ description: Random UUID hex string identifying one stdin-enabled execution
+
+ ExecuteStreamRequest:
+ allOf:
+ - $ref: '#/components/schemas/ExecuteRequest'
+ - type: object
+ properties:
+ stdin_id:
+ $ref: '#/components/schemas/StdinId'
+ shell:
+ type: string
+ default: /bin/sh
+ description: |
+ Shell executable for stdin-enabled execution, invoked with -c.
+ Without stdin_id, execution retains the legacy /bin/sh behavior.
+
+ StdinRequest:
+ oneOf:
+ - type: object
+ additionalProperties: false
+ required: [stdin_id, data]
+ properties:
+ stdin_id:
+ $ref: '#/components/schemas/StdinId'
+ data:
+ type: string
+ contentEncoding: base64
+ maxLength: 87384
+ description: |
+ Standard padded base64, decoding to at most 65536 bytes.
+ Empty data is allowed and is not EOF. Invalid base64, oversized
+ decoded chunks, and data combined with eof are rejected.
+ - type: object
+ additionalProperties: false
+ required: [stdin_id, eof]
+ properties:
+ stdin_id:
+ $ref: '#/components/schemas/StdinId'
+ eof:
+ type: boolean
+ const: true
+ description: Close the child's stdin pipe; false or null is invalid
+
+ StdinResponse:
+ type: object
+ required: [closed]
+ properties:
+ closed:
+ type: boolean
+ description: |
+ False only after the entire chunk has actually been written to the
+ child pipe, not merely queued. True after EOF or if input is already
+ closed, the child exited, or stdin_id is absent/completed. A broken
+ pipe is closed input, not a failure of an otherwise successful command.
+ EOF/BrokenPipe closes only stdin: stdin_id remains cancellable while
+ the command continues processing, until execution cleanup completes.
+
+ CancelRequest:
+ type: object
+ additionalProperties: false
+ required: [stdin_id]
+ properties:
+ stdin_id:
+ $ref: '#/components/schemas/StdinId'
+
+ CancelResponse:
+ type: object
+ required: [cancelled]
+ properties:
+ cancelled:
+ type: boolean
+ description: |
+ True after cancelling a live stdin execution and completing its
+ process cleanup. False when absent, completed, or already cancelled.
ExecuteResponse:
type: object
@@ -87,10 +190,18 @@ components:
properties:
stream:
type: string
- enum: [stdout, stderr, exit]
+ enum: [ready, stdout, stderr, exit]
+ description: |
+ ready occurs only for stdin-enabled execution, after registration
+ and process startup. Startup failure instead emits stderr and exit.
data:
type: string
- description: Output text (for stdout/stderr events)
+ description: |
+ Output text (stdout/stderr events). With stdin_id, output is
+ incremental even without a newline and does not add newlines.
+ UTF-8 characters split across reads are preserved; invalid UTF-8 is
+ replaced with U+FFFD. Startup, timeout, and cancellation failures
+ emit a stderr message followed by exit_code -1.
exit_code:
type: integer
description: Process exit code (for exit events only)
@@ -176,15 +287,28 @@ paths:
summary: Execute a shell command with streaming output
description: |
Run a shell command and stream stdout/stderr as SSE events in real-time.
- Same request body as /execute. The response is a text/event-stream where
- each event has a JSON payload with "stream" (stdout/stderr/exit) and
- "data" or "exit_code" fields.
+ Without stdin_id, preserve legacy execution and line-based output.
+ With stdin_id, pipe stdin and emit {"stream":"ready"} after the process
+ and registry entry are ready. Hosts wait for ready before sending input
+ via /stdin. An active stdin_id cannot be reused or overwritten.
+
+ The response is a text/event-stream with JSON in ordinary SSE data
+ events: ready, stdout/stderr with data strings, and exit with exit_code.
+ Command output is incremental, including text without a trailing newline.
+ Stdout/stderr events can arrive before any input is sent. A startup
+ failure emits a stderr message and exit_code -1 without ready.
+
+ Timeout, /cancel, or output transport disconnection terminates the
+ command's process group and descendants, reaps the child, closes stdin,
+ removes the registry entry, and releases exclusivity. Timeout/cancel
+ emits a stderr message followed by exit_code -1 when connected.
+ Cleanup does not wait for the host to consume final output.
requestBody:
required: true
content:
application/json:
schema:
- $ref: '#/components/schemas/ExecuteRequest'
+ $ref: '#/components/schemas/ExecuteStreamRequest'
responses:
'200':
description: SSE stream of command output
@@ -198,6 +322,107 @@ paths:
application/json:
schema:
$ref: '#/components/schemas/ErrorResponse'
+ '400':
+ description: Invalid stdin_id or stdin-enabled execution parameters
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+ '409':
+ description: stdin_id is already active
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+ '503':
+ description: An exclusive command is already running
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+
+ /stdin:
+ post:
+ summary: Write an acknowledged binary stdin chunk or close stdin
+ description: |
+ Requires bearer authentication, including while an exclusive command
+ runs. Submit either a base64 data chunk or eof: true, never both.
+ Hosts send one chunk at a time and await its acknowledgement. Do not
+ retry requests: chunks are not deduplicated. Buffering is bounded;
+ excess concurrent writes are rejected rather than queued indefinitely.
+ On closed: true, stop producing input and await the execution's final
+ output. When the input iterable is exhausted, send eof: true.
+ requestBody:
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/StdinRequest'
+ responses:
+ '200':
+ description: Chunk written, EOF applied, or input already closed
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/StdinResponse'
+ '400':
+ description: Malformed request, invalid base64, or decoded chunk over 65536 bytes
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+ '401':
+ description: Invalid or missing token
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+ '409':
+ description: A stdin request is already pending and the bounded queue is full
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+ '500':
+ description: Unexpected pipe write error
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+
+ /cancel:
+ post:
+ summary: Cancel a stdin-enabled execution and clean up its process group
+ description: |
+ Requires bearer authentication. Works during exclusive execution and
+ while a stdin write is blocked. Hosts cancel when their input producer
+ raises or the execute operation is cancelled. Unrelated executions
+ are not cancelled.
+ requestBody:
+ required: true
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/CancelRequest'
+ responses:
+ '200':
+ description: Execution cancelled and reaped, or already absent/completed
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/CancelResponse'
+ '400':
+ description: Malformed cancel request or invalid stdin_id
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
+ '401':
+ description: Invalid or missing token
+ content:
+ application/json:
+ schema:
+ $ref: '#/components/schemas/ErrorResponse'
/ping:
get:
diff --git a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.lock b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.lock
index bfb8a88..d856e25 100644
--- a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.lock
+++ b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.lock
@@ -71,6 +71,12 @@ dependencies = [
"tower-service",
]
+[[package]]
+name = "base64"
+version = "0.22.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
+
[[package]]
name = "bytes"
version = "1.11.1"
@@ -274,11 +280,13 @@ name = "quicksand-guest-agent"
version = "0.1.0"
dependencies = [
"axum",
+ "base64",
"libc",
"serde",
"serde_json",
"tokio",
"tokio-stream",
+ "tower",
]
[[package]]
diff --git a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.toml b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.toml
index 0c3b3b1..17bbb91 100644
--- a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.toml
+++ b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/Cargo.toml
@@ -7,7 +7,7 @@ license = "MIT"
[dependencies]
# Async runtime - multi-threaded for concurrent request handling
-tokio = { version = "1", features = ["rt-multi-thread", "net", "io-util", "process", "time", "fs", "macros"] }
+tokio = { version = "1", features = ["rt-multi-thread", "net", "io-util", "process", "time", "fs", "macros", "sync"] }
# HTTP server - lightweight
axum = { version = "0.7", default-features = false, features = ["json", "tokio", "http1"] }
@@ -15,6 +15,7 @@ axum = { version = "0.7", default-features = false, features = ["json", "tokio",
# JSON serialization
serde = { version = "1", features = ["derive"] }
serde_json = "1"
+base64 = "0.22"
# SSE streaming support
tokio-stream = "0.1"
@@ -25,6 +26,9 @@ tokio-stream = "0.1"
# drives one non-blocking fd through `tokio::io::unix::AsyncFd` instead.
libc = "0.2"
+[dev-dependencies]
+tower = { version = "0.5", features = ["util"] }
+
[profile.release]
opt-level = "z" # Optimize for size
lto = true # Link-time optimization
diff --git a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/main.rs b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/main.rs
index 570a775..baecc02 100644
--- a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/main.rs
+++ b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/main.rs
@@ -3,6 +3,8 @@
//! Minimal HTTP API running inside the guest VM to handle commands from the host.
//! Reads configuration (token, port) from kernel command line.
+mod stdin_execution;
+
use axum::{
extract::State,
http::{header, StatusCode},
@@ -28,7 +30,9 @@ use std::{
};
use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::time::timeout;
-use tokio_stream::wrappers::ReceiverStream;
+use tokio_stream::{wrappers::ReceiverStream, StreamExt};
+
+use stdin_execution::{ExecutionRequest, InputRequest, RequestError, StdinExecutions};
// ============================================================================
// Logging
@@ -68,6 +72,7 @@ struct AppState {
/// True while an exclusive command (sync, fstrim, etc.) is running.
/// Other execute requests are rejected with 503 while this is set.
exclusive_busy: Arc<AtomicBool>,
+ stdin_executions: StdinExecutions,
}
// ============================================================================
@@ -82,6 +87,21 @@ struct AuthRequest {
#[derive(Serialize)]
struct AuthResponse {
authenticated: bool,
+ #[serde(skip_serializing_if = "Vec::is_empty")]
+ capabilities: Vec<&'static str>,
+}
+
+impl AuthResponse {
+ fn new(authenticated: bool) -> Self {
+ Self {
+ authenticated,
+ capabilities: if authenticated {
+ vec!["stdin_streaming"]
+ } else {
+ Vec::new()
+ },
+ }
+ }
}
#[derive(Deserialize)]
@@ -93,6 +113,21 @@ struct ExecuteRequest {
/// When true, reject all other execute requests while this one is running.
#[serde(default)]
exclusive: bool,
+ shell: Option<String>,
+ stdin_id: Option<String>,
+}
+
+impl ExecuteRequest {
+ fn into_stdin_execution(self) -> ExecutionRequest {
+ ExecutionRequest {
+ stdin_id: self.stdin_id.expect("stdin_id was checked"),
+ command: self.command,
+ shell: self.shell.unwrap_or_else(|| "/bin/sh".into()),
+ cwd: self.cwd,
+ timeout: self.timeout,
+ exclusive: self.exclusive,
+ }
+ }
}
fn default_timeout() -> f64 {
@@ -150,9 +185,9 @@ async fn authenticate(
Json(req): Json<AuthRequest>,
) -> impl IntoResponse {
if req.token == *state.token {
- (StatusCode::OK, Json(AuthResponse { authenticated: true }))
+ (StatusCode::OK, Json(AuthResponse::new(true)))
} else {
- (StatusCode::UNAUTHORIZED, Json(AuthResponse { authenticated: false }))
+ (StatusCode::UNAUTHORIZED, Json(AuthResponse::new(false)))
}
}
@@ -244,6 +279,22 @@ async fn execute_stream(
return e.into_response();
}
+ if req.stdin_id.is_some() {
+ let output = match state.stdin_executions.start(
+ req.into_stdin_execution(),
+ Arc::clone(&state.exclusive_busy),
+ ) {
+ Ok(output) => output,
+ Err(error) => return stdin_request_error(error),
+ };
+ let stream = ReceiverStream::new(output).map(|event| {
+ Ok::<_, Infallible>(
+ Event::default().data(serde_json::to_string(&event).expect("serializable output")),
+ )
+ });
+ return Sse::new(stream).into_response();
+ }
+
// Reject if an exclusive command is already running.
if state.exclusive_busy.load(Ordering::SeqCst) {
return (
@@ -361,6 +412,56 @@ async fn execute_stream(
Sse::new(stream).into_response()
}
+fn stdin_request_error(error: RequestError) -> axum::response::Response {
+ let status = match &error {
+ RequestError::Invalid(_) => StatusCode::BAD_REQUEST,
+ RequestError::Conflict(_) => StatusCode::CONFLICT,
+ RequestError::ExclusiveBusy => StatusCode::SERVICE_UNAVAILABLE,
+ RequestError::Io(_) => StatusCode::INTERNAL_SERVER_ERROR,
+ };
+ (
+ status,
+ Json(ErrorResponse {
+ detail: error.to_string(),
+ }),
+ )
+ .into_response()
+}
+
+async fn stdin(
+ State(state): State<AppState>,
+ headers: axum::http::HeaderMap,
+ Json(value): Json<serde_json::Value>,
+) -> axum::response::Response {
+ if let Err(error) = verify_token(&headers, &state.token) {
+ return error.into_response();
+ }
+ let ack = match InputRequest::parse(value).and_then(|input| state.stdin_executions.submit(input)) {
+ Ok(ack) => ack,
+ Err(error) => return stdin_request_error(error),
+ };
+ match ack.wait().await {
+ Ok(closed) => Json(serde_json::json!({"closed": closed})).into_response(),
+ Err(error) => stdin_request_error(error),
+ }
+}
+
+async fn cancel(
+ State(state): State<AppState>,
+ headers: axum::http::HeaderMap,
+ Json(value): Json<serde_json::Value>,
+) -> axum::response::Response {
+ if let Err(error) = verify_token(&headers, &state.token) {
+ return error.into_response();
+ }
+ let stdin_id = match stdin_execution::parse_cancel(value) {
+ Ok(stdin_id) => stdin_id,
+ Err(error) => return stdin_request_error(error),
+ };
+ let cancelled = state.stdin_executions.cancel(&stdin_id).await;
+ Json(serde_json::json!({"cancelled": cancelled})).into_response()
+}
+
async fn ping(
State(state): State<AppState>,
headers: axum::http::HeaderMap,
@@ -570,15 +671,31 @@ async fn handle_virtio_serial(token: String, exclusive_busy: Arc<AtomicBool>) {
log(&format!("Virtio-serial channel open: {}", VIRTIO_SERIAL_PATH));
+ serve_virtio_serial(chan, token, exclusive_busy, StdinExecutions::default()).await;
+}
+
+async fn serve_virtio_serial(
+ chan: Arc<SerialChannel>,
+ token: String,
+ exclusive_busy: Arc<AtomicBool>,
+ stdin_executions: StdinExecutions,
+) {
// Dedicated writer task: request handlers (including spawned command tasks)
- // send frames through this channel; the writer drains it and serialises the
- // frames onto the wire in arrival order, so responses for different request
- // ids never interleave. The reader loop below shares the same fd via
+ // send frames through output channels. One writer serialises whole frames,
+ // preserving each stream's order without interleaving bytes from different
+ // request ids. The reader loop below shares the same fd via
// `AsyncFd`, which tracks read/write readiness independently.
let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel::<serde_json::Value>();
+ // Keep legacy output unchanged, but backpressure the new stdin executions.
+ let (stream_tx, mut stream_rx) = tokio::sync::mpsc::channel::<serde_json::Value>(64);
let writer_chan = Arc::clone(&chan);
let writer_task = tokio::spawn(async move {
- while let Some(msg) = rx.recv().await {
+ loop {
+ let msg = tokio::select! {
+ Some(msg) = rx.recv() => msg,
+ Some(msg) = stream_rx.recv() => msg,
+ else => break,
+ };
if let Err(e) = write_frame_serial(&writer_chan, &msg).await {
log(&format!("Write error: {}", e));
break;
@@ -589,10 +706,16 @@ async fn handle_virtio_serial(token: String, exclusive_busy: Arc<AtomicBool>) {
let mut authenticated = false;
loop {
- let frame = match read_frame_serial(&chan).await {
- Ok(f) => f,
- Err(e) => {
- log(&format!("Virtio-serial read error: {}", e));
+ let frame = tokio::select! {
+ frame = read_frame_serial(&chan) => match frame {
+ Ok(frame) => frame,
+ Err(error) => {
+ log(&format!("Virtio-serial read error: {error}"));
+ break;
+ }
+ },
+ _ = tx.closed() => {
+ log("Virtio-serial writer disconnected");
break;
}
};
@@ -605,7 +728,7 @@ async fn handle_virtio_serial(token: String, exclusive_busy: Arc<AtomicBool>) {
"authenticate" => {
let req_token = params.get("token").and_then(|v| v.as_str()).unwrap_or("");
authenticated = req_token == token;
- send_frame(&tx, serde_json::json!({"id": id, "result": {"authenticated": authenticated}}));
+ send_frame(&tx, serde_json::json!({"id": id, "result": AuthResponse::new(authenticated)}));
}
"ping" if authenticated => {
send_frame(&tx, serde_json::json!({"id": id, "result": {"pong": true, "pid": std::process::id()}}));
@@ -649,6 +772,36 @@ async fn handle_virtio_serial(token: String, exclusive_busy: Arc<AtomicBool>) {
});
}
"execute_stream" if authenticated => {
+ if params.get("stdin_id").is_some_and(|value| !value.is_null()) {
+ let result = serde_json::from_value::<ExecuteRequest>(params)
+ .map_err(|error| RequestError::Invalid(format!("Invalid stdin execution request: {error}")))
+ .and_then(|request| {
+ stdin_executions.start(
+ request.into_stdin_execution(),
+ Arc::clone(&exclusive_busy),
+ )
+ });
+ match result {
+ Ok(mut output) => {
+ let tx = stream_tx.clone();
+ tokio::spawn(async move {
+ while let Some(event) = output.recv().await {
+ let mut frame = serde_json::to_value(event).expect("serializable output");
+ frame["id"] = serde_json::json!(id);
+ if tx.send(frame).await.is_err() {
+ break;
+ }
+ }
+ });
+ }
+ Err(error) => {
+ send_frame(&tx, serde_json::json!({"id": id, "error": {"message": error.to_string()}}));
+ send_frame(&tx, serde_json::json!({"id": id, "stream": "exit", "exit_code": -1}));
+ }
+ }
+ continue;
+ }
+
let command = params.get("command").and_then(|v| v.as_str()).unwrap_or("").to_string();
let timeout_secs = params.get("timeout").and_then(|v| v.as_f64()).unwrap_or(30.0);
let cwd = params.get("cwd").and_then(|v| v.as_str()).map(|s| s.to_string());
@@ -738,6 +891,33 @@ async fn handle_virtio_serial(token: String, exclusive_busy: Arc<AtomicBool>) {
}
});
}
+ "stdin" if authenticated => {
+ match InputRequest::parse(params).and_then(|input| stdin_executions.submit(input)) {
+ Ok(ack) => {
+ let tx = tx.clone();
+ tokio::spawn(async move {
+ match ack.wait().await {
+ Ok(closed) => send_frame(&tx, serde_json::json!({"id": id, "result": {"closed": closed}})),
+ Err(error) => send_frame(&tx, serde_json::json!({"id": id, "error": {"message": error.to_string()}})),
+ }
+ });
+ }
+ Err(error) => send_frame(&tx, serde_json::json!({"id": id, "error": {"message": error.to_string()}})),
+ }
+ }
+ "cancel" if authenticated => {
+ match stdin_execution::parse_cancel(params) {
+ Ok(stdin_id) => {
+ let executions = stdin_executions.clone();
+ let tx = tx.clone();
+ tokio::spawn(async move {
+ let cancelled = executions.cancel(&stdin_id).await;
+ send_frame(&tx, serde_json::json!({"id": id, "result": {"cancelled": cancelled}}));
+ });
+ }
+ Err(error) => send_frame(&tx, serde_json::json!({"id": id, "error": {"message": error.to_string()}})),
+ }
+ }
_ if !authenticated => {
send_frame(&tx, serde_json::json!({"id": id, "error": {"message": "Not authenticated"}}));
}
@@ -747,16 +927,32 @@ async fn handle_virtio_serial(token: String, exclusive_busy: Arc<AtomicBool>) {
}
}
- // Read loop ended (peer closed or read error). Drop the last sender so the
- // writer task's channel closes and it exits, then join it.
- drop(tx);
- let _ = writer_task.await;
+ stdin_executions.cancel_all().await;
+ // A disconnected peer may leave the device writer blocked on writability.
+ // Do not wait for legacy command senders to close before ending transport I/O.
+ writer_task.abort();
+ if let Err(error) = writer_task.await {
+ if !error.is_cancelled() {
+ log(&format!("Virtio-serial writer task failed: {error}"));
+ }
+ }
}
// ============================================================================
// Main
// ============================================================================
+fn http_router(state: AppState) -> Router {
+ Router::new()
+ .route("/authenticate", post(authenticate))
+ .route("/execute", post(execute))
+ .route("/execute_stream", post(execute_stream))
+ .route("/stdin", post(stdin))
+ .route("/cancel", post(cancel))
+ .route("/ping", get(ping))
+ .with_state(state)
+}
+
#[tokio::main]
async fn main() {
log("Rust agent starting...");
@@ -802,14 +998,10 @@ async fn main() {
let state = AppState {
token: Arc::new(token),
exclusive_busy,
+ stdin_executions: StdinExecutions::default(),
};
- let app = Router::new()
- .route("/authenticate", post(authenticate))
- .route("/execute", post(execute))
- .route("/execute_stream", post(execute_stream))
- .route("/ping", get(ping))
- .with_state(state);
+ let app = http_router(state);
let addr = SocketAddr::from(([0, 0, 0, 0], port));
log(&format!("Listening on {}", addr));
@@ -817,3 +1009,6 @@ async fn main() {
let listener = tokio::net::TcpListener::bind(addr).await.unwrap();
axum::serve(listener, app).await.unwrap();
}
+
+#[cfg(test)]
+mod transport_tests;
diff --git a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/stdin_execution.rs b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/stdin_execution.rs
new file mode 100644
index 0000000..abb4a7b
--- /dev/null
+++ b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/stdin_execution.rs
@@ -0,0 +1,655 @@
+use base64::{engine::general_purpose::STANDARD, Engine};
+use serde::{Deserialize, Serialize};
+use std::{
+ collections::HashMap,
+ fmt, io,
+ process::Stdio,
+ sync::{
+ atomic::{AtomicBool, Ordering},
+ Arc, Mutex,
+ },
+ time::Duration,
+};
+use tokio::{
+ io::{AsyncRead, AsyncReadExt, AsyncWriteExt},
+ process::{Child, ChildStdin, Command},
+ sync::{mpsc, oneshot, watch},
+ time::{sleep_until, Instant},
+};
+
+pub const MAX_STDIN_CHUNK: usize = 65_536;
+const MAX_ENCODED_CHUNK: usize = MAX_STDIN_CHUNK.div_ceil(3) * 4;
+const OUTPUT_CHUNK: usize = 8192;
+
+#[derive(Debug)]
+pub enum RequestError {
+ Invalid(String),
+ Conflict(&'static str),
+ ExclusiveBusy,
+ Io(String),
+}
+
+impl fmt::Display for RequestError {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ match self {
+ Self::Invalid(message) | Self::Io(message) => f.write_str(message),
+ Self::Conflict(message) => f.write_str(message),
+ Self::ExclusiveBusy => f.write_str("Exclusive command in progress"),
+ }
+ }
+}
+
+#[derive(Debug, Serialize)]
+#[serde(tag = "stream", rename_all = "lowercase")]
+pub enum OutputEvent {
+ Ready,
+ Stdout { data: String },
+ Stderr { data: String },
+ Exit { exit_code: i32 },
+}
+
+pub struct ExecutionRequest {
+ pub stdin_id: String,
+ pub command: String,
+ pub shell: String,
+ pub cwd: Option<String>,
+ pub timeout: f64,
+ pub exclusive: bool,
+}
+
+#[derive(Deserialize)]
+#[serde(untagged, deny_unknown_fields)]
+enum StdinWireRequest {
+ Data { stdin_id: String, data: String },
+ Eof { stdin_id: String, eof: bool },
+}
+
+pub struct InputRequest {
+ stdin_id: String,
+ input: Input,
+}
+
+enum Input {
+ Data(Vec<u8>),
+ Eof,
+}
+
+fn validate_id(stdin_id: &str) -> Result<(), RequestError> {
+ if stdin_id.len() != 32 || !stdin_id.bytes().all(|byte| byte.is_ascii_hexdigit()) {
+ return Err(RequestError::Invalid(
+ "stdin_id must be a 32-character UUID hex string".into(),
+ ));
+ }
+ Ok(())
+}
+
+impl InputRequest {
+ pub fn parse(value: serde_json::Value) -> Result<Self, RequestError> {
+ let request = serde_json::from_value(value).map_err(|error| {
+ RequestError::Invalid(format!(
+ "Expected stdin_id and either base64 data or eof: true: {error}"
+ ))
+ })?;
+ let (stdin_id, input) = match request {
+ StdinWireRequest::Data { stdin_id, data } => {
+ if data.len() > MAX_ENCODED_CHUNK {
+ return Err(RequestError::Invalid(
+ "stdin data exceeds the maximum decoded chunk size of 65536 bytes".into(),
+ ));
+ }
+ let bytes = STANDARD.decode(data).map_err(|error| {
+ RequestError::Invalid(format!("Invalid base64 stdin data: {error}"))
+ })?;
+ if bytes.len() > MAX_STDIN_CHUNK {
+ return Err(RequestError::Invalid(
+ "stdin data exceeds the maximum decoded chunk size of 65536 bytes".into(),
+ ));
+ }
+ (stdin_id, Input::Data(bytes))
+ }
+ StdinWireRequest::Eof { stdin_id, eof } => {
+ if !eof {
+ return Err(RequestError::Invalid("eof must be true".into()));
+ }
+ (stdin_id, Input::Eof)
+ }
+ };
+ validate_id(&stdin_id)?;
+ Ok(Self { stdin_id, input })
+ }
+}
+
+pub fn parse_cancel(value: serde_json::Value) -> Result<String, RequestError> {
+ #[derive(Deserialize)]
+ #[serde(deny_unknown_fields)]
+ struct CancelRequest {
+ stdin_id: String,
+ }
+
+ let request: CancelRequest = serde_json::from_value(value)
+ .map_err(|error| RequestError::Invalid(format!("Invalid cancel request: {error}")))?;
+ validate_id(&request.stdin_id)?;
+ Ok(request.stdin_id)
+}
+
+struct InputWrite {
+ input: Input,
+ ack: oneshot::Sender<Result<bool, String>>,
+}
+
+pub struct InputAck {
+ pending: Option<oneshot::Receiver<Result<bool, String>>>,
+}
+
+impl InputAck {
+ pub async fn wait(self) -> Result<bool, RequestError> {
+ match self.pending {
+ Some(ack) => match ack.await {
+ Ok(result) => result.map_err(RequestError::Io),
+ // Dropping the writer closes its pipe and any outstanding ACKs.
+ Err(_) => Ok(true),
+ },
+ None => Ok(true),
+ }
+ }
+}
+
+struct Control {
+ input: mpsc::Sender<InputWrite>,
+ input_closed: Arc<AtomicBool>,
+ cancel: watch::Sender<bool>,
+ done: watch::Receiver<bool>,
+}
+
+#[derive(Clone, Default)]
+pub struct StdinExecutions {
+ entries: Arc<Mutex<HashMap<String, Arc<Control>>>>,
+}
+
+impl StdinExecutions {
+ pub fn start(
+ &self,
+ request: ExecutionRequest,
+ exclusive_busy: Arc<AtomicBool>,
+ ) -> Result<mpsc::Receiver<OutputEvent>, RequestError> {
+ validate_id(&request.stdin_id)?;
+ let duration = Duration::try_from_secs_f64(request.timeout)
+ .map_err(|_| RequestError::Invalid("timeout must be finite and nonnegative".into()))?;
+ let deadline = Instant::now()
+ .checked_add(duration)
+ .ok_or_else(|| RequestError::Invalid("timeout is too large".into()))?;
+
+ let mut entries = self.entries.lock().expect("stdin registry poisoned");
+ if entries.contains_key(&request.stdin_id) {
+ return Err(RequestError::Conflict("stdin_id is already active"));
+ }
+ let exclusive = ExclusiveGuard::acquire(exclusive_busy, request.exclusive)?;
+ let (input_tx, input_rx) = mpsc::channel(1);
+ let (cancel_tx, cancel_rx) = watch::channel(false);
+ let (done_tx, done_rx) = watch::channel(false);
+ let control = Arc::new(Control {
+ input: input_tx,
+ input_closed: Arc::new(AtomicBool::new(false)),
+ cancel: cancel_tx,
+ done: done_rx,
+ });
+ // A following serial cancel frame must find the ID even before the
+ // execution task runs or emits ready.
+ entries.insert(request.stdin_id.clone(), Arc::clone(&control));
+ let registration = Registration {
+ executions: self.clone(),
+ stdin_id: request.stdin_id.clone(),
+ control,
+ exclusive,
+ done: done_tx,
+ };
+ drop(entries);
+
+ let (output_tx, output_rx) = mpsc::channel(16);
+ tokio::spawn(run(
+ request,
+ deadline,
+ input_rx,
+ cancel_rx,
+ output_tx,
+ registration,
+ ));
+ Ok(output_rx)
+ }
+
+ pub fn submit(&self, request: InputRequest) -> Result<InputAck, RequestError> {
+ let control = self
+ .entries
+ .lock()
+ .expect("stdin registry poisoned")
+ .get(&request.stdin_id)
+ .cloned();
+ let Some(control) = control else {
+ return Ok(InputAck { pending: None });
+ };
+ if control.input_closed.load(Ordering::SeqCst) || *control.cancel.borrow() {
+ return Ok(InputAck { pending: None });
+ }
+ let (ack_tx, ack_rx) = oneshot::channel();
+ // Never retain an unbounded number of chunks in pending HTTP/serial tasks.
+ // There is one in-flight write and at most one queued request.
+ match control.input.try_send(InputWrite {
+ input: request.input,
+ ack: ack_tx,
+ }) {
+ Ok(()) => Ok(InputAck {
+ pending: Some(ack_rx),
+ }),
+ Err(mpsc::error::TrySendError::Closed(_)) => Ok(InputAck { pending: None }),
+ Err(mpsc::error::TrySendError::Full(_)) => {
+ Err(RequestError::Conflict("A stdin request is already pending"))
+ }
+ }
+ }
+
+ pub async fn cancel(&self, stdin_id: &str) -> bool {
+ let control = self
+ .entries
+ .lock()
+ .expect("stdin registry poisoned")
+ .get(stdin_id)
+ .cloned();
+ let Some(control) = control else {
+ return false;
+ };
+ // EOF/BrokenPipe can precede command completion; input_closed must not
+ // disable cancellation of a command that is still processing.
+ if *control.done.borrow() {
+ return false;
+ }
+ let already_cancelled = control.cancel.send_replace(true);
+ let mut done = control.done.clone();
+ // The registration owns the sender, so closure also means it was dropped.
+ let _ = done.wait_for(|done| *done).await;
+ !already_cancelled
+ }
+
+ pub async fn cancel_all(&self) {
+ let controls: Vec<_> = self
+ .entries
+ .lock()
+ .expect("stdin registry poisoned")
+ .values()
+ .cloned()
+ .collect();
+ for control in &controls {
+ control.cancel.send_replace(true);
+ }
+ for control in controls {
+ let mut done = control.done.clone();
+ let _ = done.wait_for(|done| *done).await;
+ }
+ }
+}
+
+struct ExclusiveGuard {
+ busy: Arc<AtomicBool>,
+}
+
+impl ExclusiveGuard {
+ fn acquire(busy: Arc<AtomicBool>, exclusive: bool) -> Result<Option<Self>, RequestError> {
+ if busy.load(Ordering::SeqCst) {
+ return Err(RequestError::ExclusiveBusy);
+ }
+ if exclusive {
+ busy.compare_exchange(false, true, Ordering::SeqCst, Ordering::SeqCst)
+ .map_err(|_| RequestError::ExclusiveBusy)?;
+ Ok(Some(Self { busy }))
+ } else {
+ Ok(None)
+ }
+ }
+}
+
+impl Drop for ExclusiveGuard {
+ fn drop(&mut self) {
+ self.busy.store(false, Ordering::SeqCst);
+ }
+}
+
+struct Registration {
+ executions: StdinExecutions,
+ stdin_id: String,
+ control: Arc<Control>,
+ exclusive: Option<ExclusiveGuard>,
+ done: watch::Sender<bool>,
+}
+
+impl Drop for Registration {
+ fn drop(&mut self) {
+ self.control.input_closed.store(true, Ordering::SeqCst);
+ let mut entries = self
+ .executions
+ .entries
+ .lock()
+ .expect("stdin registry poisoned");
+ if entries
+ .get(&self.stdin_id)
+ .is_some_and(|entry| Arc::ptr_eq(entry, &self.control))
+ {
+ entries.remove(&self.stdin_id);
+ }
+ drop(entries);
+ self.exclusive.take();
+ self.done.send_replace(true);
+ }
+}
+
+struct ProcessGroup {
+ child: Child,
+ pgid: i32,
+ armed: bool,
+}
+
+impl ProcessGroup {
+ fn new(child: Child) -> Self {
+ Self {
+ pgid: child.id().expect("new child has no PID") as i32,
+ child,
+ armed: true,
+ }
+ }
+
+ fn kill_group(&mut self) -> io::Result<()> {
+ if self.armed {
+ // Each stdin execution is a new process group, never the agent's group.
+ let result = unsafe { libc::kill(-self.pgid, libc::SIGKILL) };
+ if result < 0 {
+ let error = io::Error::last_os_error();
+ if error.raw_os_error() != Some(libc::ESRCH) {
+ return Err(error);
+ }
+ }
+ self.armed = false;
+ }
+ Ok(())
+ }
+
+ async fn terminate_and_reap(&mut self) -> io::Result<()> {
+ if let Err(group_error) = self.kill_group() {
+ // Still reap the direct child if signalling the group failed.
+ self.child.kill().await?;
+ return Err(group_error);
+ }
+ self.child.wait().await?;
+ Ok(())
+ }
+}
+
+impl Drop for ProcessGroup {
+ fn drop(&mut self) {
+ if let Err(error) = self.kill_group() {
+ crate::log(&format!("Failed to kill stdin process group: {error}"));
+ }
+ // Child::kill_on_drop is a second guard; normal paths explicitly reap it.
+ }
+}
+
+enum Outcome {
+ Exit(i32),
+ Failed(String),
+ Disconnected,
+}
+
+async fn run(
+ request: ExecutionRequest,
+ deadline: Instant,
+ input: mpsc::Receiver<InputWrite>,
+ mut cancel: watch::Receiver<bool>,
+ output: mpsc::Sender<OutputEvent>,
+ registration: Registration,
+) {
+ let outcome = {
+ // On startup failure, drop any queued input ACKs before terminal output.
+ let input_requests = input;
+ if output.is_closed() {
+ Outcome::Disconnected
+ } else if *cancel.borrow() {
+ Outcome::Failed("Command cancelled".into())
+ } else {
+ let mut command = Command::new(&request.shell);
+ command
+ .arg("-c")
+ .arg(&request.command)
+ .stdin(Stdio::piped())
+ .stdout(Stdio::piped())
+ .stderr(Stdio::piped())
+ .process_group(0)
+ .kill_on_drop(true);
+ if let Some(cwd) = &request.cwd {
+ command.current_dir(cwd);
+ }
+ match command.spawn() {
+ Err(error) => Outcome::Failed(format!("Failed to start command: {error}")),
+ Ok(child) => {
+ let mut process = ProcessGroup::new(child);
+ let input_closed = Arc::clone(&registration.control.input_closed);
+ let result = tokio::select! {
+ biased;
+ _ = cancel.wait_for(|cancelled| *cancelled) => {
+ Outcome::Failed("Command cancelled".into())
+ }
+ _ = output.closed() => Outcome::Disconnected,
+ _ = sleep_until(deadline) => {
+ Outcome::Failed(format!("Command timed out after {} seconds", request.timeout))
+ }
+ result = drive(&mut process, input_requests, input_closed, &output) => {
+ match result {
+ Ok(code) => Outcome::Exit(code),
+ Err(error) => Outcome::Failed(error),
+ }
+ }
+ };
+ match process.terminate_and_reap().await {
+ Ok(()) => result,
+ Err(error) => {
+ Outcome::Failed(format!("Failed to terminate command: {error}"))
+ }
+ }
+ }
+ }
+ }
+ };
+
+ // Reaping, closing stdin and releasing exclusivity must not depend on a slow
+ // output consumer accepting the final events.
+ drop(registration);
+ let exit_code = match outcome {
+ Outcome::Disconnected => return,
+ Outcome::Exit(code) => code,
+ Outcome::Failed(message) => {
+ if output
+ .send(OutputEvent::Stderr { data: message })
+ .await
+ .is_err()
+ {
+ return;
+ }
+ -1
+ }
+ };
+ // The receiver disappearing here is an ordinary transport disconnection.
+ let _ = output.send(OutputEvent::Exit { exit_code }).await;
+}
+
+async fn drive(
+ process: &mut ProcessGroup,
+ input: mpsc::Receiver<InputWrite>,
+ input_closed: Arc<AtomicBool>,
+ output: &mpsc::Sender<OutputEvent>,
+) -> Result<i32, String> {
+ let stdin = process.child.stdin.take().expect("stdin was piped");
+ let stdout = process.child.stdout.take().expect("stdout was piped");
+ let stderr = process.child.stderr.take().expect("stderr was piped");
+ let mut writer = Some(Box::pin(write_input(
+ stdin,
+ input,
+ Arc::clone(&input_closed),
+ )));
+ let stdout = read_output(stdout, false, output);
+ let stderr = read_output(stderr, true, output);
+ tokio::pin!(stdout, stderr);
+ output
+ .send(OutputEvent::Ready)
+ .await
+ .map_err(|_| "Output transport closed".to_string())?;
+
+ let mut exit_code = None;
+ let mut stdout_done = false;
+ let mut stderr_done = false;
+ loop {
+ if stdout_done && stderr_done {
+ if let Some(code) = exit_code {
+ return Ok(code);
+ }
+ }
+ // These are independent futures: a blocked pipe write or output send
+ // cannot keep the child wait (or the outer cancellation) from progressing.
+ tokio::select! {
+ result = process.child.wait(), if exit_code.is_none() => {
+ input_closed.store(true, Ordering::SeqCst);
+ writer.take();
+ exit_code = Some(result.map_err(|error| format!("Failed to wait for command: {error}"))?
+ .code().unwrap_or(-1));
+ // Background descendants must not hold the pipes open after the
+ // command exits, or outlive an otherwise successful execution.
+ process.kill_group().map_err(|error| format!("Failed to kill command descendants: {error}"))?;
+ }
+ result = async { writer.as_mut().expect("input writer exists").await }, if writer.is_some() => {
+ writer.take();
+ result?;
+ }
+ result = &mut stdout, if !stdout_done => {
+ stdout_done = true;
+ result?;
+ }
+ result = &mut stderr, if !stderr_done => {
+ stderr_done = true;
+ result?;
+ }
+ }
+ }
+}
+
+struct InputClosed(Arc<AtomicBool>);
+
+impl Drop for InputClosed {
+ fn drop(&mut self) {
+ self.0.store(true, Ordering::SeqCst);
+ }
+}
+
+async fn write_input(
+ mut stdin: ChildStdin,
+ mut input: mpsc::Receiver<InputWrite>,
+ closed: Arc<AtomicBool>,
+) -> Result<(), String> {
+ let _closed = InputClosed(closed);
+ while let Some(request) = input.recv().await {
+ match request.input {
+ Input::Eof => {
+ drop(stdin);
+ // A lost ACK receiver means the host abandoned its control request.
+ let _ = request.ack.send(Ok(true));
+ return Ok(());
+ }
+ Input::Data(bytes) => match stdin.write_all(&bytes).await {
+ Ok(()) => {
+ let _ = request.ack.send(Ok(false));
+ }
+ Err(error) if error.kind() == io::ErrorKind::BrokenPipe => {
+ let _ = request.ack.send(Ok(true));
+ return Ok(());
+ }
+ Err(error) => {
+ let message = format!("Failed to write stdin: {error}");
+ let _ = request.ack.send(Err(message.clone()));
+ return Err(message);
+ }
+ },
+ }
+ }
+ Ok(())
+}
+
+#[derive(Default)]
+struct Utf8Decoder {
+ pending: Vec<u8>,
+}
+
+impl Utf8Decoder {
+ fn decode(&mut self, bytes: &[u8], eof: bool) -> String {
+ self.pending.extend_from_slice(bytes);
+ let mut text = String::new();
+ let mut consumed = 0;
+ while consumed < self.pending.len() {
+ match std::str::from_utf8(&self.pending[consumed..]) {
+ Ok(valid) => {
+ text.push_str(valid);
+ consumed = self.pending.len();
+ }
+ Err(error) => {
+ let valid_end = consumed + error.valid_up_to();
+ text.push_str(
+ std::str::from_utf8(&self.pending[consumed..valid_end])
+ .expect("UTF-8 valid prefix"),
+ );
+ consumed = valid_end;
+ match error.error_len() {
+ Some(length) => {
+ text.push('\u{fffd}');
+ consumed += length;
+ }
+ None => {
+ if eof {
+ text.push('\u{fffd}');
+ consumed = self.pending.len();
+ }
+ break;
+ }
+ }
+ }
+ }
+ }
+ self.pending.drain(..consumed);
+ text
+ }
+}
+
+async fn read_output(
+ mut reader: impl AsyncRead + Unpin,
+ is_stderr: bool,
+ output: &mpsc::Sender<OutputEvent>,
+) -> Result<(), String> {
+ let mut buffer = [0; OUTPUT_CHUNK];
+ let mut decoder = Utf8Decoder::default();
+ loop {
+ let length = reader
+ .read(&mut buffer)
+ .await
+ .map_err(|error| format!("Failed to read command output: {error}"))?;
+ let data = decoder.decode(&buffer[..length], length == 0);
+ if !data.is_empty() {
+ let event = if is_stderr {
+ OutputEvent::Stderr { data }
+ } else {
+ OutputEvent::Stdout { data }
+ };
+ output
+ .send(event)
+ .await
+ .map_err(|_| "Output transport closed".to_string())?;
+ }
+ if length == 0 {
+ return Ok(());
+ }
+ }
+}
+
+#[cfg(test)]
+mod tests;
diff --git a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/stdin_execution/tests.rs b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/stdin_execution/tests.rs
new file mode 100644
index 0000000..7cfb152
--- /dev/null
+++ b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/stdin_execution/tests.rs
@@ -0,0 +1,513 @@
+use super::*;
+use serde_json::json;
+use tokio::time::{sleep, timeout};
+
+const ID: &str = "0123456789abcdef0123456789abcdef";
+const OTHER_ID: &str = "fedcba9876543210fedcba9876543210";
+
+fn request(command: &str) -> ExecutionRequest {
+ ExecutionRequest {
+ stdin_id: ID.into(),
+ command: command.into(),
+ shell: "/bin/sh".into(),
+ cwd: None,
+ timeout: 5.0,
+ exclusive: false,
+ }
+}
+
+fn data(stdin_id: &str, bytes: &[u8]) -> InputRequest {
+ InputRequest::parse(json!({"stdin_id": stdin_id, "data": STANDARD.encode(bytes)})).unwrap()
+}
+
+async fn write(executions: &StdinExecutions, stdin_id: &str, bytes: &[u8]) -> bool {
+ executions
+ .submit(data(stdin_id, bytes))
+ .unwrap()
+ .wait()
+ .await
+ .unwrap()
+}
+
+async fn eof(executions: &StdinExecutions, stdin_id: &str) -> bool {
+ executions
+ .submit(InputRequest::parse(json!({"stdin_id": stdin_id, "eof": true})).unwrap())
+ .unwrap()
+ .wait()
+ .await
+ .unwrap()
+}
+
+async fn next(output: &mut mpsc::Receiver<OutputEvent>) -> OutputEvent {
+ timeout(Duration::from_secs(3), output.recv())
+ .await
+ .expect("output timed out")
+ .expect("output closed before exit")
+}
+
+async fn ready(output: &mut mpsc::Receiver<OutputEvent>) {
+ assert!(matches!(next(output).await, OutputEvent::Ready));
+}
+
+async fn finish(mut output: mpsc::Receiver<OutputEvent>) -> (String, String, i32) {
+ let mut stdout = String::new();
+ let mut stderr = String::new();
+ loop {
+ match next(&mut output).await {
+ OutputEvent::Stdout { data } => stdout.push_str(&data),
+ OutputEvent::Stderr { data } => stderr.push_str(&data),
+ OutputEvent::Exit { exit_code } => return (stdout, stderr, exit_code),
+ OutputEvent::Ready => panic!("unexpected ready event"),
+ }
+ }
+}
+
+fn assert_clean(executions: &StdinExecutions, busy: &AtomicBool) {
+ assert!(executions.entries.lock().unwrap().is_empty());
+ assert!(!busy.load(Ordering::SeqCst));
+}
+
+pub(super) async fn assert_process_stopped(pid: i32) {
+ timeout(Duration::from_secs(3), async {
+ loop {
+ if unsafe { libc::kill(pid, 0) } < 0 {
+ assert_eq!(io::Error::last_os_error().raw_os_error(), Some(libc::ESRCH));
+ return;
+ }
+ // Orphan descendants may be zombies until the platform's init reaps
+ // them. They are terminated; the directly owned child is reaped here.
+ let status = std::process::Command::new("ps")
+ .args(["-o", "stat=", "-p", &pid.to_string()])
+ .output()
+ .unwrap();
+ if String::from_utf8_lossy(&status.stdout)
+ .trim_start()
+ .starts_with('Z')
+ {
+ return;
+ }
+ sleep(Duration::from_millis(10)).await;
+ }
+ })
+ .await
+ .expect("command process survived cleanup");
+}
+
+#[test]
+fn input_validation_is_strict_and_bounded() {
+ for value in [
+ json!({}),
+ json!({"stdin_id": ID}),
+ json!({"stdin_id": "invalid", "data": ""}),
+ json!({"stdin_id": ID, "data": null}),
+ json!({"stdin_id": ID, "data": "%%%"}),
+ json!({"stdin_id": ID, "data": "Zg"}),
+ json!({"stdin_id": ID, "data": "Zg==", "eof": true}),
+ json!({"stdin_id": ID, "data": "Zg==", "eof": null}),
+ json!({"stdin_id": ID, "eof": false}),
+ json!({"stdin_id": ID, "eof": null}),
+ json!({"stdin_id": ID, "eof": "true"}),
+ json!({"stdin_id": ID, "eof": true, "extra": 1}),
+ json!({"stdin_id": ID, "data": STANDARD.encode(vec![0; MAX_STDIN_CHUNK + 1])}),
+ json!({"stdin_id": ID, "data": "A".repeat(MAX_ENCODED_CHUNK + 4)}),
+ ] {
+ assert!(
+ matches!(
+ InputRequest::parse(value.clone()),
+ Err(RequestError::Invalid(_))
+ ),
+ "accepted malformed input: {value}"
+ );
+ }
+ assert!(InputRequest::parse(json!({"stdin_id": ID, "data": ""})).is_ok());
+ assert!(InputRequest::parse(json!({"stdin_id": ID, "eof": true})).is_ok());
+ assert!(InputRequest::parse(json!({
+ "stdin_id": ID, "data": STANDARD.encode(vec![255; MAX_STDIN_CHUNK])
+ }))
+ .is_ok());
+ assert_eq!(parse_cancel(json!({"stdin_id": ID})).unwrap(), ID);
+ for value in [
+ json!({}),
+ json!({"stdin_id": null}),
+ json!({"stdin_id": "invalid"}),
+ json!({"stdin_id": ID, "eof": true}),
+ ] {
+ assert!(parse_cancel(value).is_err());
+ }
+}
+
+#[test]
+fn utf8_decoder_preserves_split_characters_and_replaces_invalid_bytes() {
+ let mut decoder = Utf8Decoder::default();
+ assert_eq!(decoder.decode(b"prompt:\xf0", false), "prompt:");
+ assert_eq!(decoder.decode(b"\x9f\x98", false), "");
+ assert_eq!(decoder.decode(b"\x80!", false), "😀!");
+ assert_eq!(decoder.decode(b"\xffx\xe2\x82", false), "�x");
+ assert_eq!(decoder.decode(b"", true), "�");
+ assert!(decoder.pending.is_empty());
+
+ let bytes = "hello € 😀 world".as_bytes();
+ for split in 0..=bytes.len() {
+ let mut decoder = Utf8Decoder::default();
+ let actual =
+ decoder.decode(&bytes[..split], false) + &decoder.decode(&bytes[split..], true);
+ assert_eq!(actual, "hello € 😀 world");
+ }
+}
+
+#[tokio::test]
+async fn output_is_incremental_without_newlines_or_utf8_corruption() {
+ let (mut producer, reader) = tokio::io::duplex(32);
+ let (tx, mut rx) = mpsc::channel(4);
+ let reader = tokio::spawn(async move { read_output(reader, false, &tx).await });
+ producer.write_all(b"prompt:\xe2").await.unwrap();
+ assert!(matches!(next(&mut rx).await, OutputEvent::Stdout { data } if data == "prompt:"));
+ producer.write_all(b"\x82").await.unwrap();
+ assert!(timeout(Duration::from_millis(20), rx.recv()).await.is_err());
+ producer.write_all(b"\xac!").await.unwrap();
+ assert!(matches!(next(&mut rx).await, OutputEvent::Stdout { data } if data == "€!"));
+ producer.write_all(b"\xf0\x9f").await.unwrap();
+ drop(producer);
+ assert!(matches!(next(&mut rx).await, OutputEvent::Stdout { data } if data == "�"));
+ reader.await.unwrap().unwrap();
+ assert!(rx.recv().await.is_none());
+}
+
+#[tokio::test]
+async fn binary_chunks_are_written_exactly_once_and_eof_finishes() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut output = executions
+ .start(request("od -An -v -tx1"), Arc::clone(&busy))
+ .unwrap();
+ ready(&mut output).await;
+ let output = tokio::spawn(finish(output));
+ let maximum: Vec<_> = (0u8..=255).cycle().take(MAX_STDIN_CHUNK).collect();
+ let chunks: [&[u8]; 4] = [b"\0\xff\n", &maximum, b"", b"\x80\r\0end"];
+ for chunk in chunks {
+ assert!(!write(&executions, ID, chunk).await);
+ }
+ assert!(eof(&executions, ID).await);
+ let (stdout, stderr, code) = output.await.unwrap();
+ let actual: Vec<_> = stdout
+ .split_whitespace()
+ .map(|hex| u8::from_str_radix(hex, 16).unwrap())
+ .collect();
+ assert_eq!(actual, chunks.concat());
+ assert_eq!((stderr.as_str(), code), ("", 0));
+ assert_clean(&executions, &busy);
+ assert!(write(&executions, ID, b"late").await);
+ assert!(eof(&executions, ID).await);
+ assert!(!executions.cancel(ID).await);
+}
+
+#[tokio::test]
+async fn command_can_prompt_before_input_is_produced() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut output = executions
+ .start(
+ request("printf 'prompt:'; IFS= read -r line; printf '<%s>' \"$line\"; printf err >&2"),
+ Arc::clone(&busy),
+ )
+ .unwrap();
+ ready(&mut output).await;
+ assert!(matches!(next(&mut output).await, OutputEvent::Stdout { data } if data == "prompt:"));
+ assert!(!write(&executions, ID, b"hello\n").await);
+ assert!(eof(&executions, ID).await);
+ assert_eq!(finish(output).await, ("<hello>".into(), "err".into(), 0));
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn requested_shell_and_cwd_are_honored() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut request =
+ request("test -n \"$BASH_VERSION\" || exit 42; printf '%s' \"$PWD\"; cat >/dev/null");
+ request.shell = "/bin/bash".into();
+ request.cwd = Some("src".into());
+ let mut output = executions.start(request, Arc::clone(&busy)).unwrap();
+ ready(&mut output).await;
+ assert!(eof(&executions, ID).await);
+ let (stdout, stderr, code) = finish(output).await;
+ assert_eq!(
+ stdout,
+ std::env::current_dir()
+ .unwrap()
+ .join("src")
+ .to_string_lossy()
+ );
+ assert_eq!((stderr.as_str(), code), ("", 0));
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn broken_pipe_is_closed_input_not_a_command_failure() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut output = executions
+ .start(
+ request("exec 0<&-; printf done; sleep 0.1"),
+ Arc::clone(&busy),
+ )
+ .unwrap();
+ ready(&mut output).await;
+ assert!(matches!(next(&mut output).await, OutputEvent::Stdout { data } if data == "done"));
+ assert!(write(&executions, ID, b"not consumed").await);
+ assert_eq!(finish(output).await, ("".into(), "".into(), 0));
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn closed_input_remains_cancellable_while_the_command_is_processing() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ for (command, close_with_eof) in [
+ ("cat >/dev/null; printf processing; sleep 30", true),
+ ("exec 0<&-; printf processing; sleep 30", false),
+ ] {
+ let mut request = request(command);
+ request.exclusive = true;
+ let mut output = executions.start(request, Arc::clone(&busy)).unwrap();
+ ready(&mut output).await;
+ if close_with_eof {
+ assert!(!write(&executions, ID, b"input").await);
+ assert!(eof(&executions, ID).await);
+ }
+ assert!(
+ matches!(next(&mut output).await, OutputEvent::Stdout { data } if data == "processing")
+ );
+ if !close_with_eof {
+ assert!(write(&executions, ID, b"broken pipe").await);
+ }
+ assert!(write(&executions, ID, b"already closed").await);
+ assert!(executions.entries.lock().unwrap().contains_key(ID));
+ assert!(busy.load(Ordering::SeqCst));
+ assert!(timeout(Duration::from_secs(2), executions.cancel(ID))
+ .await
+ .unwrap());
+ let (stdout, stderr, code) = finish(output).await;
+ assert_eq!((stdout.as_str(), code), ("", -1));
+ assert!(stderr.contains("Command cancelled"));
+ assert_clean(&executions, &busy);
+ }
+}
+
+#[tokio::test]
+async fn early_exit_does_not_wait_for_input_or_eof() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut output = executions
+ .start(request("printf done"), Arc::clone(&busy))
+ .unwrap();
+ ready(&mut output).await;
+ assert_eq!(finish(output).await, ("done".into(), "".into(), 0));
+ assert!(write(&executions, ID, b"late").await);
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn duplicate_ids_and_exclusive_commands_are_rejected_without_overwriting() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut first = request("cat >/dev/null");
+ first.exclusive = true;
+ let mut output = executions.start(first, Arc::clone(&busy)).unwrap();
+ ready(&mut output).await;
+ assert!(matches!(
+ executions.start(request("printf wrong"), Arc::clone(&busy)),
+ Err(RequestError::Conflict(_))
+ ));
+ let mut other = request("printf wrong");
+ other.stdin_id = OTHER_ID.into();
+ assert!(matches!(
+ executions.start(other, Arc::clone(&busy)),
+ Err(RequestError::ExclusiveBusy)
+ ));
+ assert!(!write(&executions, ID, b"data").await);
+ assert!(eof(&executions, ID).await);
+ assert_eq!(finish(output).await, ("".into(), "".into(), 0));
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn queued_input_has_no_ack_until_written_and_is_bounded() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let output = executions
+ .start(request("cat >/dev/null"), Arc::clone(&busy))
+ .unwrap();
+ let mut ack = executions.submit(data(ID, b"one")).unwrap();
+ assert!(matches!(
+ ack.pending.as_mut().unwrap().try_recv(),
+ Err(oneshot::error::TryRecvError::Empty)
+ ));
+ assert!(matches!(
+ executions.submit(data(ID, b"two")),
+ Err(RequestError::Conflict(_))
+ ));
+ assert!(executions.cancel(ID).await);
+ assert!(ack.wait().await.unwrap());
+ drop(output);
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn startup_failures_are_terminal_and_release_exclusivity() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ for invalid_shell in [true, false] {
+ let mut request = request("cat");
+ request.exclusive = true;
+ if invalid_shell {
+ request.shell = "./quicksand-no-such-shell".into();
+ } else {
+ request.cwd = Some("./quicksand-no-such-working-directory".into());
+ }
+ let output = executions.start(request, Arc::clone(&busy)).unwrap();
+ let (stdout, stderr, code) = finish(output).await;
+ assert_eq!((stdout.as_str(), code), ("", -1));
+ assert!(stderr.contains("Failed to start command"));
+ assert_clean(&executions, &busy);
+ assert!(write(&executions, ID, b"late").await);
+ }
+}
+
+#[tokio::test]
+async fn invalid_timeouts_do_not_create_executions() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ for seconds in [f64::NAN, f64::INFINITY, -1.0, f64::MAX] {
+ let mut request = request("cat");
+ request.timeout = seconds;
+ request.exclusive = true;
+ assert!(matches!(
+ executions.start(request, Arc::clone(&busy)),
+ Err(RequestError::Invalid(_))
+ ));
+ assert_clean(&executions, &busy);
+ }
+}
+
+#[tokio::test]
+async fn timeout_kills_the_process_group_and_releases_exclusivity() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut request = request("sleep 30 & printf '%s %s\\n' $$ $!; wait");
+ request.timeout = 0.3;
+ request.exclusive = true;
+ let mut output = executions.start(request, Arc::clone(&busy)).unwrap();
+ ready(&mut output).await;
+ let (stdout, stderr, code) = finish(output).await;
+ assert_eq!(code, -1);
+ assert!(stderr.contains("Command timed out after 0.3 seconds"));
+ for pid in stdout.split_whitespace() {
+ assert_process_stopped(pid.parse().unwrap()).await;
+ }
+ assert_eq!(stdout.split_whitespace().count(), 2);
+ assert!(write(&executions, ID, b"late").await);
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn cancellation_interrupts_a_blocked_write_and_preserves_unrelated_execution() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut other = request("IFS= read -r line; printf '%s' \"$line\"");
+ other.stdin_id = OTHER_ID.into();
+ let mut other_output = executions.start(other, Arc::clone(&busy)).unwrap();
+ ready(&mut other_output).await;
+
+ let mut blocked = request("sleep 30 & printf '%s %s\\n' $$ $!; wait");
+ blocked.exclusive = true;
+ let mut output = executions.start(blocked, Arc::clone(&busy)).unwrap();
+ ready(&mut output).await;
+ let writer_executions = executions.clone();
+ let writer = tokio::spawn(async move {
+ for _ in 0..128 {
+ if write(&writer_executions, ID, &vec![0; MAX_STDIN_CHUNK]).await {
+ return true;
+ }
+ }
+ false
+ });
+ sleep(Duration::from_millis(100)).await;
+ assert!(
+ !writer.is_finished(),
+ "a non-reading child should backpressure input"
+ );
+ assert!(timeout(Duration::from_secs(2), executions.cancel(ID))
+ .await
+ .unwrap());
+ assert!(timeout(Duration::from_secs(2), writer)
+ .await
+ .unwrap()
+ .unwrap());
+ let (stdout, stderr, code) = finish(output).await;
+ assert_eq!(code, -1);
+ assert!(stderr.contains("Command cancelled"));
+ for pid in stdout.split_whitespace() {
+ assert_process_stopped(pid.parse().unwrap()).await;
+ }
+ assert!(!busy.load(Ordering::SeqCst));
+ assert!(!write(&executions, OTHER_ID, b"survived\n").await);
+ assert!(eof(&executions, OTHER_ID).await);
+ assert_eq!(
+ finish(other_output).await,
+ ("survived".into(), "".into(), 0)
+ );
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn cancellation_does_not_wait_for_a_blocked_output_consumer() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut request = request("while :; do printf 'output without newline'; done");
+ request.exclusive = true;
+ let mut output = executions.start(request, Arc::clone(&busy)).unwrap();
+ ready(&mut output).await;
+ sleep(Duration::from_millis(50)).await;
+ assert!(timeout(Duration::from_secs(2), executions.cancel(ID))
+ .await
+ .unwrap());
+ assert_clean(&executions, &busy);
+ drop(output);
+}
+
+#[tokio::test]
+async fn output_disconnection_cancels_and_reaps_execution() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut request = request("printf '%s' $$; sleep 30");
+ request.exclusive = true;
+ let mut output = executions.start(request, Arc::clone(&busy)).unwrap();
+ ready(&mut output).await;
+ let pid = match next(&mut output).await {
+ OutputEvent::Stdout { data } => data.parse().unwrap(),
+ event => panic!("unexpected event: {event:?}"),
+ };
+ let mut done = executions.entries.lock().unwrap()[ID].done.clone();
+ drop(output);
+ timeout(Duration::from_secs(2), done.wait_for(|done| *done))
+ .await
+ .unwrap()
+ .unwrap();
+ assert_process_stopped(pid).await;
+ assert_clean(&executions, &busy);
+}
+
+#[tokio::test]
+async fn successful_exit_kills_background_descendants() {
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let mut output = executions
+ .start(request("sleep 30 & printf '%s' $!"), Arc::clone(&busy))
+ .unwrap();
+ ready(&mut output).await;
+ let (stdout, stderr, code) = finish(output).await;
+ assert_eq!((stderr.as_str(), code), ("", 0));
+ assert_process_stopped(stdout.parse().unwrap()).await;
+ assert_clean(&executions, &busy);
+}
diff --git a/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/transport_tests.rs b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/transport_tests.rs
new file mode 100644
index 0000000..f135352
--- /dev/null
+++ b/packages/dev/quicksand-image-tools/quicksand_image_tools/quicksand-guest-agent/src/transport_tests.rs
@@ -0,0 +1,641 @@
+use super::*;
+use axum::{
+ body::{to_bytes, Body, BodyDataStream},
+ http::Request,
+ response::Response,
+};
+use base64::{engine::general_purpose::STANDARD, Engine};
+use serde_json::{json, Value};
+use tokio::time::sleep;
+use tower::ServiceExt;
+
+const ID: &str = "0123456789abcdef0123456789abcdef";
+const TOKEN: &str = "test-token";
+
+fn state() -> AppState {
+ AppState {
+ token: Arc::new(TOKEN.into()),
+ exclusive_busy: Arc::new(AtomicBool::new(false)),
+ stdin_executions: StdinExecutions::default(),
+ }
+}
+
+async fn post_json(app: &Router, path: &str, value: Value, authenticated: bool) -> Response {
+ let mut request = Request::builder()
+ .method("POST")
+ .uri(path)
+ .header(header::CONTENT_TYPE, "application/json");
+ if authenticated {
+ request = request.header(header::AUTHORIZATION, format!("Bearer {TOKEN}"));
+ }
+ app.clone()
+ .oneshot(request.body(Body::from(value.to_string())).unwrap())
+ .await
+ .unwrap()
+}
+
+async fn response_json(response: Response) -> Value {
+ let bytes = to_bytes(response.into_body(), 1024 * 1024).await.unwrap();
+ serde_json::from_slice(&bytes).unwrap()
+}
+
+async fn sse_next(stream: &mut BodyDataStream) -> Value {
+ let frame = timeout(Duration::from_secs(3), stream.next())
+ .await
+ .expect("SSE event timed out")
+ .expect("SSE ended before exit")
+ .unwrap();
+ let frame = std::str::from_utf8(&frame).unwrap();
+ serde_json::from_str(frame.trim().strip_prefix("data:").unwrap().trim_start()).unwrap()
+}
+
+async fn wait_until_idle(busy: &AtomicBool) {
+ timeout(Duration::from_secs(3), async {
+ while busy.load(Ordering::SeqCst) {
+ sleep(Duration::from_millis(5)).await;
+ }
+ })
+ .await
+ .expect("exclusive state was not released");
+}
+
+#[tokio::test]
+async fn http_capability_gating_and_control_authentication() {
+ let app = http_router(state());
+ let response = post_json(&app, "/authenticate", json!({"token": TOKEN}), false).await;
+ assert_eq!(response.status(), StatusCode::OK);
+ assert_eq!(
+ response_json(response).await,
+ json!({"authenticated": true, "capabilities": ["stdin_streaming"]})
+ );
+ let response = post_json(&app, "/authenticate", json!({"token": "wrong"}), false).await;
+ assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
+ assert_eq!(
+ response_json(response).await,
+ json!({"authenticated": false})
+ );
+ for (path, params) in [
+ ("/stdin", json!({"stdin_id": ID, "data": ""})),
+ ("/stdin", json!({"stdin_id": ID, "eof": true})),
+ ("/cancel", json!({"stdin_id": ID})),
+ ] {
+ let response = post_json(&app, path, params, false).await;
+ assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
+ assert_eq!(
+ response_json(response).await,
+ json!({"detail": "Invalid token"})
+ );
+ }
+}
+
+#[tokio::test]
+async fn http_binary_input_eof_and_exclusive_controls() {
+ let state = state();
+ let app = http_router(state.clone());
+ let response = post_json(
+ &app,
+ "/execute_stream",
+ json!({"stdin_id": ID, "command": "od -An -v -tx1", "exclusive": true}),
+ true,
+ )
+ .await;
+ assert_eq!(response.status(), StatusCode::OK);
+ assert_eq!(
+ response.headers()[header::CONTENT_TYPE],
+ "text/event-stream"
+ );
+ let mut output = response.into_body().into_data_stream();
+ assert_eq!(sse_next(&mut output).await, json!({"stream": "ready"}));
+ let response = post_json(&app, "/execute", json!({"command": "printf wrong"}), true).await;
+ assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
+ for bytes in [&b"\0\xff"[..], &b"\x80\n\r"[..]] {
+ let response = post_json(
+ &app,
+ "/stdin",
+ json!({"stdin_id": ID, "data": STANDARD.encode(bytes)}),
+ true,
+ )
+ .await;
+ assert_eq!(response.status(), StatusCode::OK);
+ assert_eq!(response_json(response).await, json!({"closed": false}));
+ }
+ let response = post_json(&app, "/stdin", json!({"stdin_id": ID, "eof": true}), true).await;
+ assert_eq!(response_json(response).await, json!({"closed": true}));
+ let mut stdout = String::new();
+ loop {
+ let event = sse_next(&mut output).await;
+ assert!(event.get("id").is_none());
+ match event["stream"].as_str().unwrap() {
+ "stdout" => stdout.push_str(event["data"].as_str().unwrap()),
+ "exit" => {
+ assert_eq!(event["exit_code"], 0);
+ break;
+ }
+ stream => panic!("unexpected stream {stream}: {event}"),
+ }
+ }
+ assert_eq!(
+ stdout.split_whitespace().collect::<Vec<_>>(),
+ ["00", "ff", "80", "0a", "0d"]
+ );
+ assert!(!state.exclusive_busy.load(Ordering::SeqCst));
+ let response = post_json(&app, "/stdin", json!({"stdin_id": ID, "data": ""}), true).await;
+ assert_eq!(response_json(response).await, json!({"closed": true}));
+ let response = post_json(&app, "/cancel", json!({"stdin_id": ID}), true).await;
+ assert_eq!(response_json(response).await, json!({"cancelled": false}));
+}
+
+#[tokio::test]
+async fn http_validation_duplicate_ids_and_startup_errors_are_explicit() {
+ let state = state();
+ let app = http_router(state.clone());
+ for params in [
+ json!({"stdin_id": ID, "eof": false}),
+ json!({"stdin_id": ID, "eof": true, "data": ""}),
+ json!({"stdin_id": ID, "data": "!!!"}),
+ json!({"stdin_id": ID, "data": STANDARD.encode(vec![0; 65537])}),
+ ] {
+ let response = post_json(&app, "/stdin", params, true).await;
+ assert_eq!(response.status(), StatusCode::BAD_REQUEST);
+ assert!(response_json(response).await["detail"].is_string());
+ }
+ for params in [
+ json!({"stdin_id": "invalid", "command": "cat"}),
+ json!({"stdin_id": ID, "command": "cat", "timeout": -1}),
+ ] {
+ let response = post_json(&app, "/execute_stream", params, true).await;
+ assert_eq!(response.status(), StatusCode::BAD_REQUEST);
+ }
+ let params = json!({"stdin_id": ID, "command": "cat", "exclusive": true});
+ let response = post_json(&app, "/execute_stream", params.clone(), true).await;
+ let mut output = response.into_body().into_data_stream();
+ assert_eq!(sse_next(&mut output).await["stream"], "ready");
+ let duplicate = post_json(&app, "/execute_stream", params, true).await;
+ assert_eq!(duplicate.status(), StatusCode::CONFLICT);
+ let response = post_json(&app, "/cancel", json!({"stdin_id": ID}), true).await;
+ assert_eq!(response_json(response).await, json!({"cancelled": true}));
+ assert!(!state.exclusive_busy.load(Ordering::SeqCst));
+ assert_eq!(sse_next(&mut output).await["stream"], "stderr");
+ assert_eq!(
+ sse_next(&mut output).await,
+ json!({"stream": "exit", "exit_code": -1})
+ );
+ let response = post_json(
+ &app,
+ "/execute_stream",
+ json!({"stdin_id": ID, "command": "cat", "shell": "./quicksand-no-such-shell", "exclusive": true}),
+ true,
+ )
+ .await;
+ let mut output = response.into_body().into_data_stream();
+ let error = sse_next(&mut output).await;
+ assert_eq!(error["stream"], "stderr");
+ assert!(error["data"]
+ .as_str()
+ .unwrap()
+ .contains("Failed to start command"));
+ assert_eq!(
+ sse_next(&mut output).await,
+ json!({"stream": "exit", "exit_code": -1})
+ );
+ assert!(!state.exclusive_busy.load(Ordering::SeqCst));
+}
+
+#[tokio::test]
+async fn http_sse_disconnection_cancels_the_execution() {
+ let state = state();
+ let app = http_router(state.clone());
+ let response = post_json(
+ &app,
+ "/execute_stream",
+ json!({"stdin_id": ID, "command": "sleep 30", "exclusive": true}),
+ true,
+ )
+ .await;
+ let mut output = response.into_body().into_data_stream();
+ assert_eq!(sse_next(&mut output).await["stream"], "ready");
+ drop(output);
+ wait_until_idle(&state.exclusive_busy).await;
+ assert!(!state.stdin_executions.cancel(ID).await);
+}
+
+#[tokio::test]
+async fn http_disconnection_before_ready_cancels_execution() {
+ let state = state();
+ let app = http_router(state.clone());
+ let response = post_json(
+ &app,
+ "/execute_stream",
+ json!({"stdin_id": ID, "command": "sleep 30", "exclusive": true}),
+ true,
+ )
+ .await;
+ assert!(state.exclusive_busy.load(Ordering::SeqCst));
+ // Abandon the response without ever polling its SSE body.
+ drop(response);
+ wait_until_idle(&state.exclusive_busy).await;
+ assert!(!state.stdin_executions.cancel(ID).await);
+}
+
+#[tokio::test]
+async fn http_legacy_execution_shapes_and_line_streaming_are_unchanged() {
+ let app = http_router(state());
+ let response = post_json(&app, "/execute", json!({"command": "printf legacy"}), true).await;
+ assert_eq!(
+ response_json(response).await,
+ json!({"stdout": "legacy", "stderr": "", "exit_code": 0})
+ );
+ let response = post_json(
+ &app,
+ "/execute_stream",
+ json!({"command": "printf legacy"}),
+ true,
+ )
+ .await;
+ let mut output = response.into_body().into_data_stream();
+ assert_eq!(
+ sse_next(&mut output).await,
+ json!({"stream": "stdout", "data": "legacy\n"})
+ );
+ assert_eq!(
+ sse_next(&mut output).await,
+ json!({"stream": "exit", "exit_code": 0})
+ );
+}
+
+struct SerialTest {
+ peer: Arc<SerialChannel>,
+ server: tokio::task::JoinHandle<()>,
+ executions: StdinExecutions,
+ busy: Arc<AtomicBool>,
+}
+
+impl SerialTest {
+ fn new() -> Self {
+ let (server, peer) = std::os::unix::net::UnixStream::pair().unwrap();
+ server.set_nonblocking(true).unwrap();
+ peer.set_nonblocking(true).unwrap();
+ let channel = |socket: std::os::unix::net::UnixStream| {
+ Arc::new(SerialChannel {
+ fd: tokio::io::unix::AsyncFd::new(std::os::fd::OwnedFd::from(socket)).unwrap(),
+ })
+ };
+ let executions = StdinExecutions::default();
+ let busy = Arc::new(AtomicBool::new(false));
+ let server = tokio::spawn(serve_virtio_serial(
+ channel(server),
+ TOKEN.into(),
+ Arc::clone(&busy),
+ executions.clone(),
+ ));
+ Self {
+ peer: channel(peer),
+ server,
+ executions,
+ busy,
+ }
+ }
+
+ async fn send(&self, id: u64, method: &str, params: Value) {
+ write_frame_serial(
+ &self.peer,
+ &json!({"id": id, "method": method, "params": params}),
+ )
+ .await
+ .unwrap();
+ }
+
+ async fn recv(&self) -> Value {
+ timeout(Duration::from_secs(3), read_frame_serial(&self.peer))
+ .await
+ .expect("serial response timed out")
+ .unwrap()
+ }
+
+ async fn authenticate(&self) {
+ self.send(1, "authenticate", json!({"token": TOKEN})).await;
+ assert_eq!(
+ self.recv().await,
+ json!({"id": 1, "result": {"authenticated": true, "capabilities": ["stdin_streaming"]}})
+ );
+ }
+
+ async fn disconnect(self) {
+ drop(self.peer);
+ timeout(Duration::from_secs(3), self.server)
+ .await
+ .expect("serial server did not shut down")
+ .unwrap();
+ assert!(!self.busy.load(Ordering::SeqCst));
+ assert!(!self.executions.cancel(ID).await);
+ }
+}
+
+#[tokio::test]
+async fn serial_controls_require_authentication_and_legacy_execution_still_works() {
+ let serial = SerialTest::new();
+ for (id, method, params) in [
+ (10, "stdin", json!({"stdin_id": ID, "data": ""})),
+ (11, "cancel", json!({"stdin_id": ID})),
+ ] {
+ serial.send(id, method, params).await;
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": id, "error": {"message": "Not authenticated"}})
+ );
+ }
+ serial
+ .send(12, "authenticate", json!({"token": "wrong"}))
+ .await;
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 12, "result": {"authenticated": false}})
+ );
+ serial.authenticate().await;
+ serial
+ .send(13, "execute", json!({"command": "printf legacy"}))
+ .await;
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 13, "result": {"stdout": "legacy", "stderr": "", "exit_code": 0}})
+ );
+ serial
+ .send(14, "execute_stream", json!({"command": "printf legacy"}))
+ .await;
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 14, "stream": "stdout", "data": "legacy\n"})
+ );
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 14, "stream": "exit", "exit_code": 0})
+ );
+ serial.disconnect().await;
+}
+
+#[tokio::test]
+async fn serial_ready_incremental_output_shell_and_control_ack_ids() {
+ let serial = SerialTest::new();
+ serial.authenticate().await;
+ serial
+ .send(
+ 2,
+ "execute_stream",
+ json!({
+ "stdin_id": ID,
+ "command": "test -n \"$BASH_VERSION\" || exit 42; printf 'prompt:'; IFS= read -r line; printf '<%s>' \"$line\"; printf err >&2",
+ "shell": "/bin/bash",
+ "exclusive": true
+ }),
+ )
+ .await;
+ assert_eq!(serial.recv().await, json!({"id": 2, "stream": "ready"}));
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 2, "stream": "stdout", "data": "prompt:"})
+ );
+ serial
+ .send(
+ 3,
+ "stdin",
+ json!({"stdin_id": ID, "data": STANDARD.encode(b"hello\n")}),
+ )
+ .await;
+ let mut events = Vec::new();
+ loop {
+ let frame = serial.recv().await;
+ if frame["id"] == 3 {
+ assert_eq!(frame, json!({"id": 3, "result": {"closed": false}}));
+ break;
+ }
+ events.push(frame);
+ }
+ serial
+ .send(4, "stdin", json!({"stdin_id": ID, "eof": true}))
+ .await;
+ let mut eof_ack = false;
+ while !eof_ack || !events.iter().any(|event| event["stream"] == "exit") {
+ let frame = serial.recv().await;
+ if frame["id"] == 4 {
+ assert_eq!(frame, json!({"id": 4, "result": {"closed": true}}));
+ eof_ack = true;
+ } else {
+ events.push(frame);
+ }
+ }
+ let mut stdout = String::new();
+ let mut stderr = String::new();
+ for event in events {
+ assert_eq!(event["id"], 2);
+ match event["stream"].as_str().unwrap() {
+ "stdout" => stdout.push_str(event["data"].as_str().unwrap()),
+ "stderr" => stderr.push_str(event["data"].as_str().unwrap()),
+ "exit" => assert_eq!(event["exit_code"], 0),
+ stream => panic!("unexpected stream {stream}"),
+ }
+ }
+ assert_eq!((stdout.as_str(), stderr.as_str()), ("<hello>", "err"));
+ serial.disconnect().await;
+}
+
+#[tokio::test]
+async fn serial_cancel_before_ready_observation_finds_registered_execution() {
+ let serial = SerialTest::new();
+ serial.authenticate().await;
+ let mut frames = Vec::new();
+ for frame in [
+ json!({
+ "id": 2,
+ "method": "execute_stream",
+ "params": {"stdin_id": ID, "command": "sleep 30", "exclusive": true}
+ }),
+ json!({"id": 3, "method": "cancel", "params": {"stdin_id": ID}}),
+ ] {
+ let payload = serde_json::to_vec(&frame).unwrap();
+ frames.extend_from_slice(&(payload.len() as u32).to_be_bytes());
+ frames.extend_from_slice(&payload);
+ }
+ // Queue both frames in order, without waiting for or reading ready.
+ serial.peer.write_all(&frames).await.unwrap();
+ let mut cancelled = false;
+ let mut exited = false;
+ while !cancelled || !exited {
+ let response = serial.recv().await;
+ if response["id"] == 3 {
+ assert_eq!(response, json!({"id": 3, "result": {"cancelled": true}}));
+ cancelled = true;
+ } else {
+ assert_eq!(response["id"], 2);
+ match response["stream"].as_str().unwrap() {
+ "ready" => {}
+ "stderr" => assert!(response["data"].as_str().unwrap().contains("cancelled")),
+ "exit" => {
+ assert_eq!(response["exit_code"], -1);
+ exited = true;
+ }
+ stream => panic!("unexpected stream {stream}"),
+ }
+ }
+ }
+ assert!(!serial.busy.load(Ordering::SeqCst));
+ serial
+ .send(4, "stdin", json!({"stdin_id": ID, "eof": true}))
+ .await;
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 4, "result": {"closed": true}})
+ );
+ serial.disconnect().await;
+}
+
+#[tokio::test]
+async fn serial_cancel_is_dispatched_while_a_pipe_write_is_blocked() {
+ let serial = SerialTest::new();
+ serial.authenticate().await;
+ serial
+ .send(
+ 2,
+ "execute_stream",
+ json!({"stdin_id": ID, "command": "sleep 30", "exclusive": true}),
+ )
+ .await;
+ assert_eq!(serial.recv().await, json!({"id": 2, "stream": "ready"}));
+ let mut blocked_id = None;
+ let mut events = Vec::new();
+ for id in 10..42 {
+ serial
+ .send(
+ id,
+ "stdin",
+ json!({"stdin_id": ID, "data": STANDARD.encode(vec![0; 65536])}),
+ )
+ .await;
+ let response = read_frame_serial(&serial.peer);
+ tokio::pin!(response);
+ match timeout(Duration::from_millis(50), &mut response).await {
+ Ok(frame) => assert_eq!(
+ frame.unwrap(),
+ json!({"id": id, "result": {"closed": false}})
+ ),
+ Err(_) => {
+ serial.send(100, "cancel", json!({"stdin_id": ID})).await;
+ events.push(
+ timeout(Duration::from_secs(3), &mut response)
+ .await
+ .unwrap()
+ .unwrap(),
+ );
+ blocked_id = Some(id);
+ break;
+ }
+ }
+ }
+ let blocked_id = blocked_id.expect("non-reading child did not backpressure stdin");
+ while !events.iter().any(|event| event["id"] == 100)
+ || !events.iter().any(|event| event["id"] == blocked_id)
+ || !events.iter().any(|event| event["stream"] == "exit")
+ {
+ events.push(serial.recv().await);
+ }
+ assert!(events.contains(&json!({"id": 100, "result": {"cancelled": true}})));
+ assert!(events.contains(&json!({"id": blocked_id, "result": {"closed": true}})));
+ assert!(events.contains(&json!({"id": 2, "stream": "exit", "exit_code": -1})));
+ assert!(!serial.busy.load(Ordering::SeqCst));
+ serial.send(101, "ping", json!({})).await;
+ let ping = serial.recv().await;
+ assert_eq!(ping["id"], 101);
+ assert_eq!(ping["result"]["pong"], true);
+ serial.disconnect().await;
+}
+
+#[tokio::test]
+async fn serial_validation_duplicate_ids_and_startup_errors_are_terminal() {
+ let serial = SerialTest::new();
+ serial.authenticate().await;
+ for (id, method, params) in [
+ (10, "stdin", json!({"stdin_id": ID, "eof": false})),
+ (11, "cancel", json!({"stdin_id": ID, "data": ""})),
+ ] {
+ serial.send(id, method, params).await;
+ let response = serial.recv().await;
+ assert_eq!(response["id"], id);
+ assert!(response["error"]["message"].is_string());
+ }
+ serial
+ .send(
+ 20,
+ "execute_stream",
+ json!({"stdin_id": ID, "command": "cat >/dev/null"}),
+ )
+ .await;
+ assert_eq!(serial.recv().await, json!({"id": 20, "stream": "ready"}));
+ serial
+ .send(
+ 21,
+ "execute_stream",
+ json!({"stdin_id": ID, "command": "printf wrong"}),
+ )
+ .await;
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 21, "error": {"message": "stdin_id is already active"}})
+ );
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 21, "stream": "exit", "exit_code": -1})
+ );
+ serial
+ .send(22, "stdin", json!({"stdin_id": ID, "eof": true}))
+ .await;
+ let mut completed = false;
+ let mut acknowledged = false;
+ while !completed || !acknowledged {
+ let response = serial.recv().await;
+ if response["id"] == 20 {
+ assert_eq!(
+ response,
+ json!({"id": 20, "stream": "exit", "exit_code": 0})
+ );
+ completed = true;
+ } else {
+ assert_eq!(response, json!({"id": 22, "result": {"closed": true}}));
+ acknowledged = true;
+ }
+ }
+ serial
+ .send(
+ 30,
+ "execute_stream",
+ json!({"stdin_id": ID, "command": "cat", "shell": "./quicksand-no-such-shell", "exclusive": true}),
+ )
+ .await;
+ let error = serial.recv().await;
+ assert_eq!(error["id"], 30);
+ assert_eq!(error["stream"], "stderr");
+ assert!(error["data"]
+ .as_str()
+ .unwrap()
+ .contains("Failed to start command"));
+ assert_eq!(
+ serial.recv().await,
+ json!({"id": 30, "stream": "exit", "exit_code": -1})
+ );
+ assert!(!serial.busy.load(Ordering::SeqCst));
+ serial.disconnect().await;
+}
+
+#[tokio::test]
+async fn serial_disconnect_cancels_active_stdin_executions() {
+ let serial = SerialTest::new();
+ serial.authenticate().await;
+ serial
+ .send(
+ 2,
+ "execute_stream",
+ json!({"stdin_id": ID, "command": "sleep 30", "exclusive": true}),
+ )
+ .await;
+ assert_eq!(serial.recv().await, json!({"id": 2, "stream": "ready"}));
+ assert!(serial.busy.load(Ordering::SeqCst));
+ serial.disconnect().await;
+}
diff --git a/packages/quicksand-core/quicksand_core/_types.py b/packages/quicksand-core/quicksand_core/_types.py
index 0624b71..79b0e51 100644
--- a/packages/quicksand-core/quicksand_core/_types.py
+++ b/packages/quicksand-core/quicksand_core/_types.py
@@ -14,6 +14,7 @@ from __future__ import annotations
import subprocess
import sys
+from collections.abc import AsyncIterable
from dataclasses import dataclass
from enum import StrEnum
from pathlib import Path
@@ -242,6 +243,8 @@ class QuicksandGuestAgentMethod(StrEnum):
EXECUTE = "execute"
EXECUTE_STREAM = "execute_stream"
+ STDIN = "stdin"
+ CANCEL = "cancel"
PING = "ping"
AUTHENTICATE = "authenticate"
@@ -401,6 +404,8 @@ class GuestCommands:
# Execute Protocol Types
# =============================================================================
+StdinSource = AsyncIterable[bytes] | bytes | str
+
@dataclass
class ExecuteParams:
diff --git a/packages/quicksand-core/quicksand_core/host/_stdin.py b/packages/quicksand-core/quicksand_core/host/_stdin.py
new file mode 100644
index 0000000..146c7a7
--- /dev/null
+++ b/packages/quicksand-core/quicksand_core/host/_stdin.py
@@ -0,0 +1,138 @@
+"""Shared, backpressured stdin handling for both guest-agent transports."""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+import contextlib
+import logging
+from collections.abc import AsyncGenerator, AsyncIterable, Awaitable, Callable, Coroutine
+from typing import Any
+
+from .._types import QuicksandGuestAgentMethod, StdinSource
+
+logger = logging.getLogger("quicksand.stdin")
+
+STDIN_CHUNK_SIZE = 64 * 1024
+STDIN_CAPABILITY = "stdin_streaming"
+STDIN_UNSUPPORTED = (
+ "This guest image does not support streaming stdin. "
+ "Update or rebuild the image with a guest agent that supports stdin_streaming."
+)
+_CANCEL_TIMEOUT = 2.0
+
+SendRequest = Callable[
+ [QuicksandGuestAgentMethod, dict[str, Any], float],
+ Awaitable[dict[str, Any]],
+]
+
+
+def validate_stdin(source: StdinSource) -> None:
+ if not isinstance(source, (str, bytes, AsyncIterable)):
+ raise TypeError("stdin must be str, bytes, or an async iterable of bytes")
+
+
+async def _stdin_chunks(source: StdinSource) -> AsyncGenerator[bytes, None]:
+ if isinstance(source, str):
+ source = source.encode("utf-8")
+ if isinstance(source, bytes):
+ for offset in range(0, len(source), STDIN_CHUNK_SIZE):
+ yield source[offset : offset + STDIN_CHUNK_SIZE]
+ return
+
+ iterator = aiter(source)
+ try:
+ async for chunk in iterator:
+ if not isinstance(chunk, bytes):
+ raise TypeError("stdin chunks must be bytes")
+ for offset in range(0, len(chunk), STDIN_CHUNK_SIZE):
+ yield chunk[offset : offset + STDIN_CHUNK_SIZE]
+ finally:
+ close = getattr(iterator, "aclose", None)
+ if close is not None:
+ await close()
+
+
+async def _cancel_execution(stdin_id: str, send_request: SendRequest) -> None:
+ try:
+ response = await asyncio.wait_for(
+ send_request(
+ QuicksandGuestAgentMethod.CANCEL,
+ {"stdin_id": stdin_id},
+ _CANCEL_TIMEOUT,
+ ),
+ timeout=_CANCEL_TIMEOUT,
+ )
+ if "error" in response:
+ logger.warning("Failed to cancel stdin execution: %s", response["error"])
+ else:
+ result = response.get("result")
+ if not isinstance(result, dict) or not isinstance(result.get("cancelled"), bool):
+ logger.warning("Invalid cancellation acknowledgement from guest agent")
+ except (OSError, RuntimeError, TimeoutError) as error:
+ logger.warning("Failed to cancel stdin execution: %s", error)
+
+
+async def run_with_stdin(
+ response: Coroutine[Any, Any, dict[str, Any]],
+ source: StdinSource,
+ *,
+ stdin_id: str,
+ ready: asyncio.Event,
+ send_request: SendRequest,
+ timeout: float,
+) -> dict[str, Any]:
+ async def send_input(data: bytes | None) -> bool:
+ params: dict[str, Any] = {"stdin_id": stdin_id}
+ if data is None:
+ params["eof"] = True
+ else:
+ params["data"] = base64.b64encode(data).decode("ascii")
+ reply = await send_request(QuicksandGuestAgentMethod.STDIN, params, timeout)
+ if "error" in reply:
+ raise RuntimeError(f"Failed to send stdin: {reply['error']['message']}")
+ result = reply.get("result")
+ if not isinstance(result, dict) or not isinstance(result.get("closed"), bool):
+ raise RuntimeError("Invalid stdin acknowledgement from guest agent")
+ if data is None and not result["closed"]:
+ raise RuntimeError("Guest agent did not acknowledge stdin EOF")
+ return result["closed"]
+
+ async def feed() -> None:
+ await ready.wait()
+ async with contextlib.aclosing(_stdin_chunks(source)) as chunks:
+ while not output_task.done():
+ chunk = await anext(chunks, None)
+ if output_task.done():
+ return
+ if await send_input(chunk):
+ return
+
+ output_task = asyncio.create_task(response)
+ input_task = asyncio.create_task(feed())
+ deadline = asyncio.timeout(timeout)
+ result: dict[str, Any] | None = None
+ try:
+ async with deadline:
+ done, _ = await asyncio.wait(
+ (output_task, input_task), return_when=asyncio.FIRST_COMPLETED
+ )
+ if input_task in done:
+ await input_task
+ result = await output_task
+ return result
+ except TimeoutError:
+ if not deadline.expired():
+ raise
+ result = {"error": {"message": f"Stream timed out after {timeout}s"}}
+ return result
+ finally:
+ input_task.cancel()
+ output_task.cancel()
+ if result is None or "error" in result:
+ await _cancel_execution(stdin_id, send_request)
+ cleanup = await asyncio.gather(input_task, output_task, return_exceptions=True)
+ if result is not None:
+ for error in cleanup:
+ if isinstance(error, Exception):
+ raise error
diff --git a/packages/quicksand-core/quicksand_core/host/quicksand_guest_agent_client.py b/packages/quicksand-core/quicksand_core/host/quicksand_guest_agent_client.py
index fa17c9e..43190f5 100644
--- a/packages/quicksand-core/quicksand_core/host/quicksand_guest_agent_client.py
+++ b/packages/quicksand-core/quicksand_core/host/quicksand_guest_agent_client.py
@@ -8,10 +8,12 @@ import logging
import random
from collections.abc import Callable
from typing import Any, TypeVar
+from uuid import uuid4
import httpx
-from .._types import NetworkConstants, QuicksandGuestAgentMethod, Timeouts
+from .._types import NetworkConstants, QuicksandGuestAgentMethod, StdinSource, Timeouts
+from ._stdin import STDIN_CAPABILITY, STDIN_UNSUPPORTED, run_with_stdin, validate_stdin
logger = logging.getLogger("quicksand.quicksand_guest_agent")
@@ -71,6 +73,7 @@ class QuicksandGuestAgentClient:
self._port = port
self._token = token
self._client: httpx.AsyncClient | None = None
+ self._supports_stdin = False
# Lazy-init: asyncio.Lock() binds to the current event loop at creation
# time, so we must not create it in __init__ (which may run outside any loop).
self._lock: asyncio.Lock | None = None
@@ -80,6 +83,10 @@ class QuicksandGuestAgentClient:
"""Check if the client is connected to the agent."""
return self._client is not None
+ @property
+ def supports_stdin(self) -> bool:
+ return self._supports_stdin
+
async def connect(
self,
timeout: float = Timeouts.BOOT_DEFAULT,
@@ -131,6 +138,10 @@ class QuicksandGuestAgentClient:
if response.status_code == 200:
data = response.json()
if data.get("authenticated"):
+ capabilities = data.get("capabilities", [])
+ self._supports_stdin = (
+ isinstance(capabilities, list) and STDIN_CAPABILITY in capabilities
+ )
logger.debug("Quicksand guest agent authentication successful")
return # Success!
raise RuntimeError("Authentication rejected by quicksand guest agent")
@@ -152,6 +163,7 @@ class QuicksandGuestAgentClient:
async def close(self) -> None:
"""Close the HTTP client connection."""
+ self._supports_stdin = False
if self._client:
try:
await self._client.aclose()
@@ -179,6 +191,9 @@ class QuicksandGuestAgentClient:
if self._client is None:
raise RuntimeError("Not connected to quicksand guest agent")
+ if method in (QuicksandGuestAgentMethod.STDIN, QuicksandGuestAgentMethod.CANCEL):
+ return await self._send_control_request(method, params, timeout)
+
if self._lock is None:
self._lock = asyncio.Lock()
async with self._lock:
@@ -215,12 +230,31 @@ class QuicksandGuestAgentClient:
except httpx.HTTPStatusError as e:
return {"error": {"message": str(e)}}
+ async def _send_control_request(
+ self,
+ method: QuicksandGuestAgentMethod,
+ params: dict[str, Any],
+ timeout: float,
+ ) -> dict[str, Any]:
+ assert self._client is not None
+ # A streaming response holds _lock. Input/cancel requests must bypass it
+ # and must never be retried, since replaying a chunk would duplicate bytes.
+ try:
+ response = await self._client.post(f"/{method.value}", json=params, timeout=timeout)
+ if response.status_code != 200:
+ return {"error": {"message": f"HTTP {response.status_code}: {response.text}"}}
+ return {"result": response.json()}
+ except (httpx.TransportError, json.JSONDecodeError) as error:
+ return {"error": {"message": f"Input connection error: {error}"}}
+
async def send_stream_request(
self,
params: dict[str, Any],
timeout: float,
on_stdout: Callable[[str], None] | None = None,
on_stderr: Callable[[str], None] | None = None,
+ *,
+ stdin: StdinSource | None = None,
) -> dict[str, Any]:
"""POST to /execute_stream and parse SSE events, invoking callbacks.
@@ -230,60 +264,102 @@ class QuicksandGuestAgentClient:
if self._client is None:
raise RuntimeError("Not connected to quicksand guest agent")
+ stdin_id = None
+ ready = None
+ if stdin is not None:
+ validate_stdin(stdin)
+ if not self.supports_stdin:
+ return {"error": {"message": STDIN_UNSUPPORTED}}
+ stdin_id = uuid4().hex
+ ready = asyncio.Event()
+ params = {**params, "stdin_id": stdin_id}
+
if self._lock is None:
self._lock = asyncio.Lock()
async with self._lock:
- stdout_parts: list[str] = []
- stderr_parts: list[str] = []
- exit_code = -1
+ response = self._receive_stream(params, timeout, on_stdout, on_stderr, ready)
+ if stdin is not None:
+ assert stdin_id is not None and ready is not None
+ return await run_with_stdin(
+ response,
+ stdin,
+ stdin_id=stdin_id,
+ ready=ready,
+ send_request=self.send_request,
+ timeout=timeout,
+ )
+ return await response
- try:
- async with self._client.stream(
- "POST",
- "/execute_stream",
- json=params,
- timeout=httpx.Timeout(timeout, connect=Timeouts.GUEST_AGENT_CONNECT),
- ) as response:
- if response.status_code == 401:
- return {"error": {"message": "Authentication failed"}}
- if response.status_code != 200:
- return {"error": {"message": f"HTTP {response.status_code}"}}
-
- async for line in response.aiter_lines():
- if not line.startswith("data: "):
- continue
- payload = line[6:] # strip "data: " prefix
- try:
- event = json.loads(payload)
- except json.JSONDecodeError:
- continue
-
- stream = event.get("stream")
- if stream == "stdout":
- data = event.get("data", "")
- stdout_parts.append(data)
- if on_stdout:
- on_stdout(data)
- elif stream == "stderr":
- data = event.get("data", "")
- stderr_parts.append(data)
- if on_stderr:
- on_stderr(data)
- elif stream == "exit":
- exit_code = event.get("exit_code", -1)
-
- return {
- "result": {
- "stdout": "".join(stdout_parts),
- "stderr": "".join(stderr_parts),
- "exit_code": exit_code,
- }
- }
+ async def _receive_stream(
+ self,
+ params: dict[str, Any],
+ timeout: float,
+ on_stdout: Callable[[str], None] | None,
+ on_stderr: Callable[[str], None] | None,
+ ready: asyncio.Event | None,
+ ) -> dict[str, Any]:
+ assert self._client is not None
+ stdout_parts: list[str] = []
+ stderr_parts: list[str] = []
+ exit_code = -1
+ exited = False
+ try:
+ async with self._client.stream(
+ "POST",
+ "/execute_stream",
+ json=params,
+ timeout=httpx.Timeout(timeout, connect=Timeouts.GUEST_AGENT_CONNECT),
+ ) as response:
+ if response.status_code == 401:
+ return {"error": {"message": "Authentication failed"}}
+ if response.status_code != 200:
+ return {"error": {"message": f"HTTP {response.status_code}"}}
- except httpx.TransportError as e:
- return {"error": {"message": f"Stream connection error: {e}"}}
- except httpx.HTTPStatusError as e:
- return {"error": {"message": str(e)}}
+ async for line in response.aiter_lines():
+ if not line.startswith("data: "):
+ continue
+ try:
+ event = json.loads(line[6:])
+ except json.JSONDecodeError:
+ if ready is not None:
+ return {"error": {"message": "Invalid guest stream event"}}
+ continue
+ if not isinstance(event, dict):
+ return {"error": {"message": "Invalid guest stream event"}}
+ if "error" in event:
+ return {"error": event["error"]}
+ stream = event.get("stream")
+ if stream == "ready":
+ if ready is not None:
+ ready.set()
+ elif stream == "stdout":
+ data = event.get("data", "")
+ stdout_parts.append(data)
+ if on_stdout:
+ on_stdout(data)
+ elif stream == "stderr":
+ data = event.get("data", "")
+ stderr_parts.append(data)
+ if on_stderr:
+ on_stderr(data)
+ elif stream == "exit":
+ exit_code = event.get("exit_code", -1)
+ exited = True
+ break
+
+ if ready is not None and not exited:
+ return {"error": {"message": "Guest stream ended without an exit status"}}
+ return {
+ "result": {
+ "stdout": "".join(stdout_parts),
+ "stderr": "".join(stderr_parts),
+ "exit_code": exit_code,
+ }
+ }
+ except httpx.TransportError as error:
+ return {"error": {"message": f"Stream connection error: {error}"}}
+ except httpx.HTTPStatusError as error:
+ return {"error": {"message": str(error)}}
# Alias for backwards compatibility
diff --git a/packages/quicksand-core/quicksand_core/host/smb.py b/packages/quicksand-core/quicksand_core/host/smb.py
index b29c683..2ea04d7 100644
--- a/packages/quicksand-core/quicksand_core/host/smb.py
+++ b/packages/quicksand-core/quicksand_core/host/smb.py
@@ -78,6 +78,11 @@ class SMBServer(ABC):
"""Returns (username, password) for CIFS mount authentication."""
...
+ @property
+ def supports_byte_range_locks(self) -> bool:
+ """Whether the server implements SMB byte-range locking."""
+ return True
+
@abstractmethod
def list_shares(self) -> list[dict]:
"""List all active shares.
@@ -116,6 +121,10 @@ class _ConfigBackedSMBServer(SMBServer):
def credentials(self) -> tuple[str, str]:
return ("guest", "")
+ @property
+ def supports_byte_range_locks(self) -> bool:
+ return False
+
def start(self) -> None:
self._temp_dir = Path(tempfile.mkdtemp(prefix="quicksand-smb-"))
self._config_path = self._temp_dir / "smb_config.json"
diff --git a/packages/quicksand-core/quicksand_core/host/virtio_serial_agent_client.py b/packages/quicksand-core/quicksand_core/host/virtio_serial_agent_client.py
index cfc09e4..c853816 100644
--- a/packages/quicksand-core/quicksand_core/host/virtio_serial_agent_client.py
+++ b/packages/quicksand-core/quicksand_core/host/virtio_serial_agent_client.py
@@ -23,8 +23,10 @@ import struct
from collections.abc import Callable
from pathlib import Path
from typing import Any
+from uuid import uuid4
-from .._types import QuicksandGuestAgentMethod, Timeouts
+from .._types import QuicksandGuestAgentMethod, StdinSource, Timeouts
+from ._stdin import STDIN_CAPABILITY, STDIN_UNSUPPORTED, run_with_stdin, validate_stdin
logger = logging.getLogger("quicksand.virtio_agent")
@@ -82,11 +84,16 @@ class VirtioSerialAgentClient:
self._pending: dict[int, asyncio.Future[dict]] = {}
self._pending_streams: dict[int, asyncio.Queue[dict]] = {}
self._request_id = 0
+ self._supports_stdin = False
@property
def is_connected(self) -> bool:
return self._writer is not None and self._reader_task is not None
+ @property
+ def supports_stdin(self) -> bool:
+ return self._supports_stdin
+
def _next_id(self) -> int:
self._request_id += 1
return self._request_id
@@ -169,6 +176,10 @@ class VirtioSerialAgentClient:
auth_writes_pending -= 1
result = response.get("result", {})
if result.get("authenticated"):
+ capabilities = result.get("capabilities", [])
+ self._supports_stdin = (
+ isinstance(capabilities, list) and STDIN_CAPABILITY in capabilities
+ )
logger.debug("Virtio-serial agent authentication successful")
# Drain replies to any earlier auth writes that timed out.
# By the time we got our successful reply, those replies
@@ -268,6 +279,7 @@ class VirtioSerialAgentClient:
self._pending_streams.clear()
def _close_transport(self) -> None:
+ self._supports_stdin = False
if self._writer:
with contextlib.suppress(Exception):
self._writer.close()
@@ -294,6 +306,8 @@ class VirtioSerialAgentClient:
method_map = {
QuicksandGuestAgentMethod.EXECUTE: "execute",
QuicksandGuestAgentMethod.PING: "ping",
+ QuicksandGuestAgentMethod.STDIN: "stdin",
+ QuicksandGuestAgentMethod.CANCEL: "cancel",
}
method_name = method_map.get(method)
if method_name is None:
@@ -309,26 +323,25 @@ class VirtioSerialAgentClient:
msg = {"id": request_id, "method": method_name, "params": params}
try:
- async with self._writer_lock:
- self._writer.write(_encode_frame(msg))
- await self._writer.drain()
- except (ConnectionResetError, BrokenPipeError, OSError) as e:
- self._pending.pop(request_id, None)
- return {"error": {"message": f"Connection error: {e}"}}
-
- try:
- response = await asyncio.wait_for(future, timeout=timeout)
- except TimeoutError:
- # Caller gives up: pop our slot so a late reply is dropped at
- # the reader instead of poisoning the next call's read.
+ try:
+ async with self._writer_lock:
+ self._writer.write(_encode_frame(msg))
+ await self._writer.drain()
+ response = await asyncio.wait_for(future, timeout=timeout)
+ except TimeoutError:
+ return {"error": {"message": f"Request timed out after {timeout}s"}}
+ except (ConnectionResetError, OSError) as error:
+ return {"error": {"message": f"Connection error: {error}"}}
+ if "error" in response:
+ return {"error": response["error"]}
+ return {"result": response.get("result", response)}
+ finally:
self._pending.pop(request_id, None)
- return {"error": {"message": f"Request timed out after {timeout}s"}}
- except (ConnectionResetError, OSError) as e:
- return {"error": {"message": f"Connection error: {e}"}}
-
- if "error" in response:
- return {"error": response["error"]}
- return {"result": response.get("result", response)}
+ if not future.done():
+ future.cancel()
+ elif not future.cancelled():
+ # A disconnect may fail the future before the writer finishes.
+ future.exception()
async def send_stream_request(
self,
@@ -336,31 +349,55 @@ class VirtioSerialAgentClient:
timeout: float,
on_stdout: Callable[[str], None] | None = None,
on_stderr: Callable[[str], None] | None = None,
+ *,
+ stdin: StdinSource | None = None,
) -> dict[str, Any]:
if self._writer is None or self._reader_task is None or self._writer_lock is None:
raise RuntimeError("Not connected to agent")
+ stdin_id = None
+ ready = None
+ if stdin is not None:
+ validate_stdin(stdin)
+ if not self.supports_stdin:
+ return {"error": {"message": STDIN_UNSUPPORTED}}
+ stdin_id = uuid4().hex
+ ready = asyncio.Event()
+ params = {**params, "stdin_id": stdin_id}
+
request_id = self._next_id()
queue: asyncio.Queue[dict] = asyncio.Queue()
self._pending_streams[request_id] = queue
msg = {"id": request_id, "method": "execute_stream", "params": params}
- try:
- async with self._writer_lock:
- self._writer.write(_encode_frame(msg))
- await self._writer.drain()
- except (ConnectionResetError, BrokenPipeError, OSError) as e:
- self._pending_streams.pop(request_id, None)
- return {"error": {"message": f"Stream connection error: {e}"}}
+ writer = self._writer
+ writer_lock = self._writer_lock
+
+ async def receive() -> dict[str, Any]:
+ try:
+ async with writer_lock:
+ writer.write(_encode_frame(msg))
+ await writer.drain()
+ except (ConnectionResetError, OSError) as error:
+ return {"error": {"message": f"Stream connection error: {error}"}}
+ return await self._consume_stream(queue, on_stdout, on_stderr, ready)
try:
- return await asyncio.wait_for(
- self._consume_stream(queue, on_stdout, on_stderr),
- timeout=timeout,
- )
- except TimeoutError:
- return {"error": {"message": f"Stream timed out after {timeout}s"}}
+ if stdin is not None:
+ assert stdin_id is not None and ready is not None
+ return await run_with_stdin(
+ receive(),
+ stdin,
+ stdin_id=stdin_id,
+ ready=ready,
+ send_request=self.send_request,
+ timeout=timeout,
+ )
+ try:
+ return await asyncio.wait_for(receive(), timeout=timeout)
+ except TimeoutError:
+ return {"error": {"message": f"Stream timed out after {timeout}s"}}
finally:
# Drop any further frames for this id (the reader will log+skip
# them once we're no longer registered).
@@ -371,6 +408,7 @@ class VirtioSerialAgentClient:
queue: asyncio.Queue[dict],
on_stdout: Callable[[str], None] | None,
on_stderr: Callable[[str], None] | None,
+ ready: asyncio.Event | None = None,
) -> dict[str, Any]:
stdout_parts: list[str] = []
stderr_parts: list[str] = []
@@ -379,7 +417,10 @@ class VirtioSerialAgentClient:
if "error" in frame:
return {"error": frame["error"]}
stream = frame.get("stream")
- if stream == "stdout":
+ if stream == "ready":
+ if ready is not None:
+ ready.set()
+ elif stream == "stdout":
data = frame.get("data", "")
stdout_parts.append(data)
if on_stdout:
diff --git a/packages/quicksand-core/quicksand_core/sandbox/_execution.py b/packages/quicksand-core/quicksand_core/sandbox/_execution.py
index c0472ab..639c3a8 100644
--- a/packages/quicksand-core/quicksand_core/sandbox/_execution.py
+++ b/packages/quicksand-core/quicksand_core/sandbox/_execution.py
@@ -15,6 +15,7 @@ from .._types import (
ExecuteResult,
GuestCommands,
QuicksandGuestAgentMethod,
+ StdinSource,
Timeouts,
)
from ._protocol import _SandboxProtocol
@@ -36,6 +37,8 @@ class _ExecutionMixin(_SandboxProtocol):
on_stdout: Callable[[str], None] | None = None,
on_stderr: Callable[[str], None] | None = None,
exclusive: bool = False,
+ *,
+ stdin: StdinSource | None = None,
) -> ExecuteResult:
"""
Execute a shell command inside the sandbox.
@@ -48,13 +51,17 @@ class _ExecutionMixin(_SandboxProtocol):
Common options: /bin/sh, /bin/bash, /bin/zsh
on_stdout: Optional callback invoked with each chunk of stdout
as it arrives. When provided, the command is executed
- in streaming mode via SSE.
+ in streaming mode over the active agent transport.
on_stderr: Optional callback invoked with each chunk of stderr
as it arrives. When provided, the command is executed
- in streaming mode via SSE.
+ in streaming mode over the active agent transport.
exclusive: If True, the guest agent will reject other requests
while this command is running. Used for system commands
like sync/fstrim that need exclusive access.
+ stdin: Input bytes, a UTF-8 string, or an async iterable of byte
+ chunks. Iterable exhaustion closes stdin. Input is consumed
+ incrementally and stops when the command exits. Requires
+ an image with a stdin-capable guest agent.
Returns:
ExecuteResult with stdout, stderr, and exit_code.
@@ -67,7 +74,7 @@ class _ExecutionMixin(_SandboxProtocol):
)
params_dict = {k: v for k, v in asdict(params).items() if v is not None}
- if on_stdout is not None or on_stderr is not None:
+ if stdin is not None or on_stdout is not None or on_stderr is not None:
client = self._agent_client
if client is None:
raise RuntimeError("Not connected to guest agent")
@@ -76,6 +83,7 @@ class _ExecutionMixin(_SandboxProtocol):
timeout=timeout + 5,
on_stdout=on_stdout,
on_stderr=on_stderr,
+ stdin=stdin,
)
else:
response = await self._send_request(
diff --git a/packages/quicksand-core/quicksand_core/sandbox/_mounts.py b/packages/quicksand-core/quicksand_core/sandbox/_mounts.py
index 297e5b5..bc111d1 100644
--- a/packages/quicksand-core/quicksand_core/sandbox/_mounts.py
+++ b/packages/quicksand-core/quicksand_core/sandbox/_mounts.py
@@ -81,6 +81,8 @@ class _MountMixin(_SandboxProtocol):
username, password = self._smb_server.credentials
cifs_opts = MountOptions.cifs_opts(username, password)
+ if not self._smb_server.supports_byte_range_locks:
+ cifs_opts += ",nobrl"
mount_cmd = (
f"sudo mount -t cifs //{gateway}/{shlex.quote(share_name)}"
f" {shlex.quote(guest_path)}"
diff --git a/packages/quicksand-core/quicksand_core/sandbox/_protocol.py b/packages/quicksand-core/quicksand_core/sandbox/_protocol.py
index 63ed8d6..2eccfb6 100644
--- a/packages/quicksand-core/quicksand_core/sandbox/_protocol.py
+++ b/packages/quicksand-core/quicksand_core/sandbox/_protocol.py
@@ -14,6 +14,7 @@ from .._types import (
ResolvedImage,
SandboxConfig,
SaveManifest,
+ StdinSource,
Timeouts,
)
from ..host.dns_proxy import HostDnsProxy
@@ -77,6 +78,8 @@ class _SandboxProtocol(Protocol):
on_stdout: Callable[[str], None] | None = None,
on_stderr: Callable[[str], None] | None = None,
exclusive: bool = False,
+ *,
+ stdin: StdinSource | None = None,
) -> ExecuteResult: ...
async def save(
diff --git a/tests/integration/execution/test_stdin.py b/tests/integration/execution/test_stdin.py
new file mode 100644
index 0000000..07e0e1c
--- /dev/null
+++ b/tests/integration/execution/test_stdin.py
@@ -0,0 +1,178 @@
+"""End-to-end stdin behavior with a stdin-capable guest image."""
+
+from __future__ import annotations
+
+import asyncio
+import contextlib
+import hashlib
+import os
+
+import pytest
+import pytest_asyncio
+from quicksand import Sandbox
+from quicksand_core.host.quicksand_guest_agent_client import QuicksandGuestAgentClient
+from quicksand_core.host.virtio_serial_agent_client import VirtioSerialAgentClient
+
+from tests.conftest import has_qemu
+from tests.integration.conftest import get_vm_type
+
+
+class _HttpSandbox(Sandbox):
+ def _build_vm_command(self) -> list[str]:
+ self._agent_socket_path = None
+ self._agent_socket_port = None
+ return super()._build_vm_command()
+
+
+@pytest_asyncio.fixture(scope="module", params=["virtio", "http"])
+async def stdin_sandbox(request):
+ if not has_qemu():
+ pytest.skip("QEMU not installed")
+ image = os.environ.get("QUICKSAND_STDIN_TEST_IMAGE", get_vm_type())
+ sandbox_class = _HttpSandbox if request.param == "http" else Sandbox
+ sandbox = sandbox_class(image=image)
+ try:
+ await sandbox.start()
+ assert sandbox._agent_client is not None
+ if not sandbox._agent_client.supports_stdin:
+ pytest.skip("Guest image must be rebuilt with stdin_streaming support")
+ expected_client = (
+ QuicksandGuestAgentClient if request.param == "http" else VirtioSerialAgentClient
+ )
+ assert isinstance(sandbox._agent_client, expected_client)
+ yield sandbox
+ finally:
+ await sandbox.stop()
+
+
+async def test_binary_stdin_is_streamed_without_corruption(stdin_sandbox):
+ payload = bytes(range(256)) * 4096
+
+ async def chunks():
+ yield b""
+ yield payload[:1]
+ yield payload[1:100001]
+ yield payload[100001:]
+
+ result = await stdin_sandbox.execute("sha256sum", stdin=chunks())
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout.split()[0] == hashlib.sha256(payload).hexdigest()
+
+
+async def test_stdin_can_wait_for_output_without_a_newline(stdin_sandbox):
+ prompt = asyncio.Event()
+ reply = asyncio.Event()
+ output = []
+
+ async def chunks():
+ await prompt.wait()
+ yield b"hello\n"
+ await reply.wait()
+ yield b"world\n"
+
+ def on_stdout(chunk):
+ output.append(chunk)
+ text = "".join(output)
+ if "prompt>" in text:
+ prompt.set()
+ if "received:hello" in text:
+ reply.set()
+
+ result = await stdin_sandbox.execute(
+ "printf 'prompt>'; IFS= read -r line; printf 'received:%s' \"$line\"; cat",
+ stdin=chunks(),
+ on_stdout=on_stdout,
+ timeout=10,
+ exclusive=True,
+ )
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout == "prompt>received:helloworld\n"
+
+
+async def test_empty_stdin_sends_eof(stdin_sandbox):
+ result = await stdin_sandbox.execute("wc -c", stdin=b"")
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout.strip() == "0"
+
+
+async def test_early_exit_stops_stdin_producer(stdin_sandbox):
+ closed = asyncio.Event()
+
+ async def chunks():
+ try:
+ yield b"first"
+ await asyncio.Event().wait()
+ finally:
+ closed.set()
+
+ result = await stdin_sandbox.execute("head -c 1", stdin=chunks())
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout == "f"
+ assert closed.is_set()
+
+
+async def test_stdin_cancellation_kills_process_group(stdin_sandbox):
+ started = asyncio.Event()
+ output = []
+
+ def on_stdout(chunk):
+ output.append(chunk)
+ if "\n" in "".join(output):
+ started.set()
+
+ async def chunks():
+ await asyncio.Event().wait()
+ yield b"unused"
+
+ task = asyncio.create_task(
+ stdin_sandbox.execute(
+ 'sleep 30 & printf \'%s %s\\n\' "$$" "$!"; wait',
+ stdin=chunks(),
+ on_stdout=on_stdout,
+ exclusive=True,
+ )
+ )
+ try:
+ await asyncio.wait_for(started.wait(), timeout=5)
+ pids = [int(part) for part in "".join(output).strip().split()]
+ assert len(pids) == 2
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+ finally:
+ if not task.done():
+ task.cancel()
+ with contextlib.suppress(asyncio.CancelledError):
+ await task
+
+ for _ in range(50):
+ result = await stdin_sandbox.execute(" && ".join(f"test ! -d /proc/{pid}" for pid in pids))
+ if result.exit_code == 0:
+ break
+ await asyncio.sleep(0.05)
+ assert result.exit_code == 0, "Cancelled command or descendant is still running"
+
+
+async def test_stdin_timeout_releases_exclusive_command(stdin_sandbox):
+ async def chunks():
+ await asyncio.Event().wait()
+ yield b"unused"
+
+ result = await stdin_sandbox.execute("sleep 30", stdin=chunks(), timeout=0.2, exclusive=True)
+ assert result.exit_code == -1
+ assert "timed out" in result.stderr
+ result = await stdin_sandbox.execute("printf available")
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout == "available"
diff --git a/tests/integration/mounts/readwrite/test_git.py b/tests/integration/mounts/readwrite/test_git.py
new file mode 100644
index 0000000..846b95b
--- /dev/null
+++ b/tests/integration/mounts/readwrite/test_git.py
@@ -0,0 +1,91 @@
+"""Regression coverage for Git index updates through CIFS mounts."""
+
+from __future__ import annotations
+
+import os
+import shlex
+import subprocess
+from pathlib import Path
+
+import pytest
+import pytest_asyncio
+
+
+def host_git(repo: Path, *args: str) -> str:
+ return subprocess.run(
+ [
+ "git",
+ "-C",
+ str(repo),
+ "-c",
+ "user.name=Quicksand Tests",
+ "-c",
+ "-c",
+ "commit.gpgsign=false",
+ "-c",
+ f"core.hooksPath={os.devnull}",
+ "-c",
+ "core.autocrlf=false",
+ *args,
+ ],
+ check=True,
+ capture_output=True,
+ text=True,
+ ).stdout
+
+
+@pytest_asyncio.fixture(scope="module")
+async def git_sandbox(shared_sandbox):
+ result = await shared_sandbox.execute(
+ "command -v git || (sudo apt-get update -qq && sudo apt-get install -y -qq git)",
+ timeout=180,
+ )
+ assert result.exit_code == 0, result.stderr or result.stdout
+ return shared_sandbox
+
+
[email protected]("staged_changes", [False, True])
[email protected]("ignore_filemode", [False, True])
+async def test_git_status_preserves_host_index(
+ git_sandbox, mount_dir, staged_changes, ignore_filemode
+):
+ repo = mount_dir / f"git-index-{staged_changes}-{ignore_filemode}"
+ repo.mkdir()
+ host_git(repo, "init", "--quiet")
+ tracked_file = repo / "a.txt"
+ tracked_file.write_text("committed\n")
+ host_git(repo, "add", "a.txt")
+ host_git(
+ repo,
+ "commit",
+ "--quiet",
+ "-m",
+ "Initial fixture\n\n"
+ "Co-authored-by: Copilot App <[email protected]>",
+ )
+
+ if staged_changes:
+ tracked_file.write_text("staged\n")
+ host_git(repo, "add", "a.txt")
+ tracked_file.write_text("unstaged\n")
+
+ expected_status = "MM a.txt\n" if staged_changes else ""
+ entries_before = host_git(repo, "ls-files", "--stage")
+ guest_repo = shlex.quote(f"/mnt/host/{repo.name}")
+ guest_git = f"git -c safe.directory={guest_repo}"
+ if ignore_filemode:
+ # CIFS synthesizes execute bits; also exercise an otherwise clean index refresh.
+ guest_git += " -c core.filemode=false"
+
+ for _ in range(2):
+ result = await git_sandbox.execute(f"{guest_git} -C {guest_repo} status --porcelain=v1")
+ assert result.exit_code == 0, result.stderr
+ if ignore_filemode:
+ assert result.stdout == expected_status
+ assert (repo / ".git" / "index").is_file()
+ assert host_git(repo, "ls-files", "--stage") == entries_before
+ assert host_git(repo, "status", "--porcelain=v1") == expected_status
diff --git a/tests/integration/mounts/readwrite/test_locking.py b/tests/integration/mounts/readwrite/test_locking.py
new file mode 100644
index 0000000..0292051
--- /dev/null
+++ b/tests/integration/mounts/readwrite/test_locking.py
@@ -0,0 +1,70 @@
+"""File locking within a guest using the bundled SMB server."""
+
+from __future__ import annotations
+
+import pytest
+import pytest_asyncio
+
+
+async def test_flock_acquires_lock(shared_sandbox):
+ result = await shared_sandbox.execute("flock -n /mnt/host/flock.lock -c 'printf locked'")
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout == "locked"
+
+
+async def test_flock_on_dynamic_mount(shared_sandbox, mount_dir):
+ directory = mount_dir / "dynamic-locking"
+ directory.mkdir()
+ handle = await shared_sandbox.mount(str(directory), "/mnt/dynamic-locking")
+ try:
+ result = await shared_sandbox.execute(
+ "flock -n /mnt/dynamic-locking/flock.lock -c 'printf locked'"
+ )
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout == "locked"
+ finally:
+ await shared_sandbox.unmount(handle)
+
+
+async def test_flock_excludes_another_process_and_releases(shared_sandbox):
+ result = await shared_sandbox.execute(
+ "flock -n /mnt/host/exclusive.lock sh -c "
+ "'flock -n /mnt/host/exclusive.lock true; test \"$?\" -eq 1'"
+ )
+ assert result.exit_code == 0, result.stderr
+
+ result = await shared_sandbox.execute("flock -n /mnt/host/exclusive.lock true")
+ assert result.exit_code == 0, result.stderr
+
+
+@pytest_asyncio.fixture(scope="module")
+async def sqlite_sandbox(shared_sandbox):
+ result = await shared_sandbox.execute(
+ "command -v sqlite3 || (sudo apt-get update -qq && sudo apt-get install -y -qq sqlite3)",
+ timeout=180,
+ )
+ assert result.exit_code == 0, result.stderr or result.stdout
+ return shared_sandbox
+
+
+async def test_sqlite_can_commit_on_mount(sqlite_sandbox):
+ result = await sqlite_sandbox.execute(
+ "sqlite3 /mnt/host/locking.sqlite "
+ "'CREATE TABLE records (value TEXT);"
+ ' BEGIN IMMEDIATE; INSERT INTO records VALUES ("committed"); COMMIT;\''
+ )
+ assert result.exit_code == 0, result.stderr
+
+ result = await sqlite_sandbox.execute(
+ "sqlite3 /mnt/host/locking.sqlite 'SELECT value FROM records;'"
+ )
+ assert result.exit_code == 0, result.stderr
+ assert result.stdout == "committed\n"
diff --git a/tests/unit/test_checkpoint_ops.py b/tests/unit/test_checkpoint_ops.py
index 18125ef..ad3ca3f 100644
--- a/tests/unit/test_checkpoint_ops.py
+++ b/tests/unit/test_checkpoint_ops.py
@@ -206,6 +206,8 @@ class TestHotSave:
on_stdout=None,
on_stderr=None,
exclusive=False,
+ *,
+ stdin=None,
):
return MagicMock(stdout="", stderr="", exit_code=0)
@@ -372,6 +374,8 @@ class TestCheckpointRevert:
on_stdout=None,
on_stderr=None,
exclusive=False,
+ *,
+ stdin=None,
):
return MagicMock(stdout="", stderr="", exit_code=0)
diff --git a/tests/unit/test_disk.py b/tests/unit/test_disk.py
index b043d44..d706b19 100644
--- a/tests/unit/test_disk.py
+++ b/tests/unit/test_disk.py
@@ -54,6 +54,8 @@ class _MockSandbox(_LifecycleMixin):
on_stdout=None,
on_stderr=None,
exclusive=False,
+ *,
+ stdin=None,
):
return self._execute_fn(command, timeout)
diff --git a/tests/unit/test_http_agent_stdin.py b/tests/unit/test_http_agent_stdin.py
new file mode 100644
index 0000000..a4a99fa
--- /dev/null
+++ b/tests/unit/test_http_agent_stdin.py
@@ -0,0 +1,182 @@
+"""HTTP stdin controls must run concurrently with the SSE response."""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+import json
+from contextlib import asynccontextmanager
+from unittest.mock import patch
+
+import httpx
+import pytest
+from quicksand_core.host.quicksand_guest_agent_client import QuicksandGuestAgentClient
+
+
+class EventStream(httpx.AsyncByteStream):
+ def __init__(self):
+ self.events: asyncio.Queue[dict | None] = asyncio.Queue()
+ self.closed = False
+
+ async def __aiter__(self):
+ while (event := await self.events.get()) is not None:
+ yield f"data: {json.dumps(event)}\n\n".encode()
+
+ async def aclose(self):
+ self.closed = True
+
+
+class HttpAgent:
+ def __init__(self, *, supports_stdin=True, fail_input=False, omit_exit=False):
+ self.supports_stdin = supports_stdin
+ self.fail_input = fail_input
+ self.omit_exit = omit_exit
+ self.requests = []
+ self.received = bytearray()
+ self.stream = EventStream()
+ self.stdin_id = None
+
+ async def handle(self, request: httpx.Request):
+ self.requests.append(request)
+ params = json.loads(request.content)
+ assert request.headers["Authorization"] == "Bearer test-token"
+ endpoint = request.url.path
+ if endpoint == "/authenticate":
+ data: dict[str, bool | list[str]] = {"authenticated": True}
+ if self.supports_stdin:
+ data["capabilities"] = ["stdin_streaming"]
+ return httpx.Response(200, json=data)
+ if endpoint == "/execute_stream":
+ self.stdin_id = params.get("stdin_id")
+ if self.stdin_id is None:
+ self.stream.events.put_nowait({"stream": "stdout", "data": "legacy"})
+ self.stream.events.put_nowait({"stream": "exit", "exit_code": 0})
+ else:
+ self.stream.events.put_nowait({"stream": "ready"})
+ self.stream.events.put_nowait({"stream": "stdout", "data": "prompt>"})
+ if self.omit_exit:
+ self.stream.events.put_nowait(None)
+ return httpx.Response(200, stream=self.stream)
+ assert params["stdin_id"] == self.stdin_id
+ if endpoint == "/cancel":
+ return httpx.Response(200, json={"cancelled": True})
+ assert endpoint == "/stdin"
+ if self.fail_input:
+ raise httpx.ReadError("lost acknowledgement")
+ if params.get("eof"):
+ self.stream.events.put_nowait({"stream": "exit", "exit_code": 0})
+ self.stream.events.put_nowait(None)
+ return httpx.Response(200, json={"closed": True})
+ self.received.extend(base64.b64decode(params["data"], validate=True))
+ self.stream.events.put_nowait({"stream": "stdout", "data": "accepted"})
+ return httpx.Response(200, json={"closed": False})
+
+
+@asynccontextmanager
+async def connected_client(agent):
+ original = httpx.AsyncClient
+
+ def client_factory(**kwargs):
+ return original(transport=httpx.MockTransport(agent.handle), **kwargs)
+
+ client = QuicksandGuestAgentClient(port=1, token="test-token")
+ with patch(
+ "quicksand_core.host.quicksand_guest_agent_client.httpx.AsyncClient",
+ side_effect=client_factory,
+ ):
+ await client.connect(timeout=2)
+ try:
+ yield client
+ finally:
+ await client.close()
+
+
+async def test_http_stdin_progresses_while_sse_holds_execution_lock():
+ agent = HttpAgent()
+ prompt = asyncio.Event()
+ echoed = asyncio.Event()
+ output = []
+
+ async def source():
+ await prompt.wait()
+ yield b"first"
+ await echoed.wait()
+ yield b"\x00\xfflast"
+
+ def on_stdout(chunk):
+ output.append(chunk)
+ if "prompt>" in "".join(output):
+ prompt.set()
+ if "accepted" in "".join(output):
+ echoed.set()
+
+ async with connected_client(agent) as client:
+ assert client.supports_stdin
+ result = await client.send_stream_request(
+ {"command": "cat"}, timeout=2, stdin=source(), on_stdout=on_stdout
+ )
+ assert result["result"]["stdout"] == "prompt>acceptedaccepted"
+ assert agent.received == b"first\x00\xfflast"
+ assert agent.stream.closed
+
+
+async def test_http_legacy_agent_rejects_input_without_consuming_it():
+ agent = HttpAgent(supports_stdin=False)
+ produced = []
+
+ async def source():
+ produced.append(True)
+ yield b"unused"
+
+ async with connected_client(agent) as client:
+ assert not client.supports_stdin
+ result = await client.send_stream_request({"command": "cat"}, timeout=2, stdin=source())
+ assert "does not support streaming stdin" in result["error"]["message"]
+ assert not produced
+ assert [request.url.path for request in agent.requests] == ["/authenticate"]
+
+
+async def test_http_legacy_output_streaming_is_unchanged():
+ agent = HttpAgent(supports_stdin=False)
+ async with connected_client(agent) as client:
+ result = await client.send_stream_request({"command": "echo legacy"}, timeout=2)
+ assert result["result"]["stdout"] == "legacy"
+ assert result["result"]["exit_code"] == 0
+
+
+async def test_http_input_is_never_retried_after_lost_acknowledgement():
+ agent = HttpAgent(fail_input=True)
+ async with connected_client(agent) as client:
+ with pytest.raises(RuntimeError, match="lost acknowledgement"):
+ await client.send_stream_request({"command": "cat"}, timeout=2, stdin=b"once")
+ endpoints = [request.url.path for request in agent.requests]
+ assert endpoints.count("/stdin") == 1
+ assert endpoints.count("/cancel") == 1
+ assert agent.stream.closed
+
+
+async def test_http_timeout_cancels_command_and_closes_producer():
+ agent = HttpAgent()
+ closed = asyncio.Event()
+
+ async def source():
+ try:
+ yield b"first"
+ await asyncio.Event().wait()
+ finally:
+ closed.set()
+
+ async with connected_client(agent) as client:
+ result = await client.send_stream_request({"command": "cat"}, timeout=0.1, stdin=source())
+ assert "timed out" in result["error"]["message"]
+ assert closed.is_set()
+ assert any(request.url.path == "/cancel" for request in agent.requests)
+ assert agent.stream.closed
+
+
+async def test_http_missing_exit_status_is_an_error():
+ agent = HttpAgent(omit_exit=True)
+ async with connected_client(agent) as client:
+ result = await client.send_stream_request({"command": "cat"}, timeout=2, stdin=b"unused")
+ assert "without an exit status" in result["error"]["message"]
+ assert any(request.url.path == "/cancel" for request in agent.requests)
diff --git a/tests/unit/test_mounts.py b/tests/unit/test_mounts.py
index b269b48..ab9302e 100644
--- a/tests/unit/test_mounts.py
+++ b/tests/unit/test_mounts.py
@@ -8,6 +8,12 @@ from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from quicksand_core._types import Mount, MountHandle, MountOptions, NetworkMode
+from quicksand_core.host.smb import (
+ QuicksandSMBServer,
+ QuicksandSMBTCPServer,
+ SMBServer,
+ WindowsSMBServer,
+)
from quicksand_core.sandbox import ExecuteResult, SandboxConfig
from quicksand_core.sandbox._mounts import _MountMixin
@@ -58,6 +64,8 @@ class _MockSandbox(_MountMixin):
on_stdout=None,
on_stderr=None,
exclusive=False,
+ *,
+ stdin=None,
):
return self._execute_fn(command, timeout)
@@ -113,6 +121,7 @@ class TestMountShares:
mock_server = MagicMock()
mock_server.add_share.return_value = "QUICKSAND0"
mock_server.credentials = ("guest", "")
+ mock_server.supports_byte_range_locks = False
mock_create_server.return_value = mock_server
mounts = [Mount(host="/host/data", guest="/mnt/data")]
@@ -124,6 +133,8 @@ class TestMountShares:
# Should have mkdir + CIFS mount commands
assert any("mkdir -p /mnt/data" in c for c in commands)
assert any("mount -t cifs" in c and "QUICKSAND0" in c for c in commands)
+ mount_cmd = next(c for c in commands if "mount -t cifs" in c)
+ assert "nobrl" in shlex.split(mount_cmd)[-1].split(",")
@pytest.mark.asyncio
@patch("quicksand_core.sandbox._mounts.create_smb_server")
@@ -266,6 +277,7 @@ class TestDynamicMount:
mock_server = MagicMock()
mock_server.add_share.return_value = "QUICKSAND0"
mock_server.credentials = ("guest", "")
+ mock_server.supports_byte_range_locks = False
mock_create_server.return_value = mock_server
sb = _MockSandbox(mock_execute, [])
@@ -276,6 +288,8 @@ class TestDynamicMount:
assert handle.guest == "/mnt/data"
assert handle.readonly is False
assert handle in sb.active_mounts
+ mount_cmd = next(c for c in commands if "mount -t cifs" in c)
+ assert "nobrl" in shlex.split(mount_cmd)[-1].split(",")
@pytest.mark.asyncio
@patch("quicksand_core.sandbox._mounts.create_smb_server")
@@ -357,6 +371,39 @@ class TestCleanupMounts:
assert errors[0][0] == "SMB server"
+class TestCifsLocking:
+ @pytest.mark.parametrize("server", [QuicksandSMBServer(), QuicksandSMBTCPServer()])
+ def test_bundled_servers_require_local_locks(self, server):
+ assert not server.supports_byte_range_locks
+
+ def test_native_server_keeps_remote_locks(self):
+ assert WindowsSMBServer(username="test").supports_byte_range_locks
+
+ @pytest.mark.parametrize("supports_locks", [False, True])
+ @pytest.mark.parametrize("readonly", [False, True])
+ @pytest.mark.asyncio
+ async def test_local_locks_depend_on_server_capability(self, supports_locks, readonly):
+ commands = []
+
+ def mock_execute(cmd: str, timeout: float) -> ExecuteResult:
+ commands.append(cmd)
+ return ExecuteResult(stdout="", stderr="", exit_code=0)
+
+ server = MagicMock(spec=SMBServer)
+ server.credentials = ("guest", "")
+ server.supports_byte_range_locks = supports_locks
+ server.get_guestfwd_cmd.return_value = "python -m quicksand_smb"
+
+ sb = _MockSandbox(mock_execute, [])
+ sb._smb_server = server
+ await sb._mount_cifs_share("QUICKSAND0", "/mnt/data", readonly)
+
+ mount_cmd = next(c for c in commands if "mount -t cifs" in c)
+ options = shlex.split(mount_cmd)[-1].split(",")
+ assert ("nobrl" in options) is not supports_locks
+ assert ("ro" in options) is readonly
+
+
class TestCifsOpts:
"""Tests for MountOptions.cifs_opts static method."""
diff --git a/tests/unit/test_stdin.py b/tests/unit/test_stdin.py
new file mode 100644
index 0000000..86ef1c7
--- /dev/null
+++ b/tests/unit/test_stdin.py
@@ -0,0 +1,286 @@
+"""Shared stdin flow control and public execute() routing."""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+from quicksand_core import Sandbox
+from quicksand_core._types import QuicksandGuestAgentMethod
+from quicksand_core.host._stdin import STDIN_CHUNK_SIZE, run_with_stdin
+from quicksand_core.host.virtio_serial_agent_client import VirtioSerialAgentClient
+
+_RESULT = {"result": {"stdout": "done", "stderr": "", "exit_code": 0}}
+
+
+async def test_input_waits_for_acknowledgement_and_bounds_chunks():
+ ready = asyncio.Event()
+ writing = asyncio.Event()
+ release = asyncio.Event()
+ finished = asyncio.Event()
+ produced = []
+ chunks = []
+ payload = bytes(range(256)) * 1024
+
+ async def source():
+ produced.append(1)
+ yield payload
+ produced.append(2)
+ yield b"last"
+
+ async def response():
+ ready.set()
+ await finished.wait()
+ return _RESULT
+
+ async def send(method, params, timeout):
+ assert method == QuicksandGuestAgentMethod.STDIN
+ if params.get("eof"):
+ finished.set()
+ return {"result": {"closed": True}}
+ chunk = base64.b64decode(params["data"], validate=True)
+ chunks.append(chunk)
+ if len(chunks) == 1:
+ writing.set()
+ await release.wait()
+ return {"result": {"closed": False}}
+
+ task = asyncio.create_task(
+ run_with_stdin(
+ response(), source(), stdin_id="test", ready=ready, send_request=send, timeout=5
+ )
+ )
+ try:
+ await asyncio.wait_for(writing.wait(), timeout=2)
+ assert produced == [1]
+ assert len(chunks) == 1
+ finally:
+ release.set()
+ assert await task == _RESULT
+ assert b"".join(chunks) == payload + b"last"
+ assert all(len(chunk) <= STDIN_CHUNK_SIZE for chunk in chunks)
+
+
+ ("source", "expected"),
+ [(b"", b""), (b"\x00\xff\n", b"\x00\xff\n"), ("hello \u2603", b"hello \xe2\x98\x83")],
+)
+async def test_static_input_preserves_bytes_and_closes_stdin(source, expected):
+ ready = asyncio.Event()
+ finished = asyncio.Event()
+ requests = []
+
+ async def response():
+ ready.set()
+ await finished.wait()
+ return _RESULT
+
+ async def send(method, params, timeout):
+ requests.append(params)
+ if params.get("eof"):
+ finished.set()
+ return {"result": {"closed": params.get("eof", False)}}
+
+ assert (
+ await run_with_stdin(
+ response(), source, stdin_id="test", ready=ready, send_request=send, timeout=2
+ )
+ == _RESULT
+ )
+ assert requests[-1] == {"stdin_id": "test", "eof": True}
+ assert b"".join(base64.b64decode(r["data"]) for r in requests if "data" in r) == expected
+
+
+async def test_exit_before_input_does_not_consume_producer():
+ ready = asyncio.Event()
+ produced = []
+ send = AsyncMock()
+
+ async def source():
+ produced.append(True)
+ yield b"unused"
+
+ async def response():
+ ready.set()
+ return _RESULT
+
+ assert (
+ await run_with_stdin(
+ response(), source(), stdin_id="test", ready=ready, send_request=send, timeout=2
+ )
+ == _RESULT
+ )
+ assert not produced
+ send.assert_not_awaited()
+
+
+async def test_early_exit_closes_a_blocked_producer():
+ ready = asyncio.Event()
+ producing = asyncio.Event()
+ closed = asyncio.Event()
+
+ async def source():
+ try:
+ yield b"first"
+ producing.set()
+ await asyncio.Event().wait()
+ finally:
+ closed.set()
+
+ async def response():
+ ready.set()
+ await producing.wait()
+ return _RESULT
+
+ send = AsyncMock(return_value={"result": {"closed": False}})
+ assert (
+ await run_with_stdin(
+ response(), source(), stdin_id="test", ready=ready, send_request=send, timeout=2
+ )
+ == _RESULT
+ )
+ assert closed.is_set()
+ assert send.await_count == 1
+
+
[email protected]("error_type", [ValueError, TimeoutError])
+async def test_producer_errors_cancel_command_and_propagate(error_type):
+ ready = asyncio.Event()
+ send = AsyncMock(return_value={"result": {"closed": False}})
+
+ async def source():
+ yield b"first"
+ raise error_type("producer failed")
+
+ async def response():
+ ready.set()
+ await asyncio.Event().wait()
+ return _RESULT
+
+ with pytest.raises(error_type, match="producer failed"):
+ await run_with_stdin(
+ response(), source(), stdin_id="test", ready=ready, send_request=send, timeout=2
+ )
+ assert send.await_args_list[-1].args[:2] == (
+ QuicksandGuestAgentMethod.CANCEL,
+ {"stdin_id": "test"},
+ )
+
+
+async def test_invalid_producer_chunk_cancels_command():
+ ready = asyncio.Event()
+ send = AsyncMock(return_value={"result": {"cancelled": True}})
+
+ async def source():
+ yield "not bytes"
+
+ async def response():
+ ready.set()
+ await asyncio.Event().wait()
+ return _RESULT
+
+ with pytest.raises(TypeError, match="stdin chunks must be bytes"):
+ await run_with_stdin(
+ response(), source(), stdin_id="test", ready=ready, send_request=send, timeout=2
+ )
+ assert send.await_args is not None
+ assert send.await_args.args[0] == QuicksandGuestAgentMethod.CANCEL
+
+
+async def test_cancellation_before_ready_still_cancels_remote_command():
+ started = asyncio.Event()
+ ready = asyncio.Event()
+ send = AsyncMock(return_value={"result": {"cancelled": True}})
+
+ async def response():
+ started.set()
+ await asyncio.Event().wait()
+ return _RESULT
+
+ task = asyncio.create_task(
+ run_with_stdin(
+ response(), b"unused", stdin_id="test", ready=ready, send_request=send, timeout=2
+ )
+ )
+ await started.wait()
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+ assert send.await_args is not None
+ assert send.await_args.args[0] == QuicksandGuestAgentMethod.CANCEL
+
+
+async def test_timeout_cancels_command_and_stops_producer():
+ ready = asyncio.Event()
+ closed = asyncio.Event()
+ send = AsyncMock(return_value={"result": {"closed": False}})
+
+ async def source():
+ try:
+ yield b"first"
+ await asyncio.Event().wait()
+ finally:
+ closed.set()
+
+ async def response():
+ ready.set()
+ await asyncio.Event().wait()
+ return _RESULT
+
+ result = await run_with_stdin(
+ response(), source(), stdin_id="test", ready=ready, send_request=send, timeout=0.1
+ )
+ assert "timed out" in result["error"]["message"]
+ assert closed.is_set()
+ assert send.await_args is not None
+ assert send.await_args.args[0] == QuicksandGuestAgentMethod.CANCEL
+
+
[email protected]("ack", [{}, {"closed": "false"}, {"closed": False}])
+async def test_invalid_eof_acknowledgement_is_not_ignored(ack):
+ ready = asyncio.Event()
+
+ async def response():
+ ready.set()
+ await asyncio.Event().wait()
+ return _RESULT
+
+ send = AsyncMock(return_value={"result": ack})
+ with pytest.raises(RuntimeError, match="acknowledge"):
+ await run_with_stdin(
+ response(), b"", stdin_id="test", ready=ready, send_request=send, timeout=2
+ )
+
+
[email protected]("stdin", [b"", b"hello", "hello"])
+async def test_execute_routes_input_to_streaming_client(stdin):
+ sandbox = Sandbox(image="ubuntu")
+ sandbox._is_running = True
+ sandbox._process_manager = MagicMock(is_running=True)
+ client = AsyncMock(spec=VirtioSerialAgentClient)
+ client.send_stream_request.return_value = _RESULT
+ sandbox._agent_client = client
+
+ result = await sandbox.execute("cat", stdin=stdin)
+
+ assert result.stdout == "done"
+ client.send_request.assert_not_awaited()
+ assert client.send_stream_request.await_args.kwargs["stdin"] is stdin
+ assert "stdin" not in client.send_stream_request.await_args.args[0]
+
+
+async def test_execute_without_input_retains_request_path():
+ sandbox = Sandbox(image="ubuntu")
+ sandbox._is_running = True
+ sandbox._process_manager = MagicMock(is_running=True)
+ client = AsyncMock(spec=VirtioSerialAgentClient)
+ client.send_request.return_value = _RESULT
+ sandbox._agent_client = client
+
+ result = await sandbox.execute("true")
+
+ assert result.exit_code == 0
+ client.send_request.assert_awaited_once()
+ client.send_stream_request.assert_not_awaited()
diff --git a/tests/unit/test_virtio_serial_agent_client.py b/tests/unit/test_virtio_serial_agent_client.py
index 6c2158e..74707f8 100644
--- a/tests/unit/test_virtio_serial_agent_client.py
+++ b/tests/unit/test_virtio_serial_agent_client.py
@@ -3,6 +3,7 @@
from __future__ import annotations
import asyncio
+import base64
import contextlib
import json
import socket
@@ -11,6 +12,7 @@ import sys
from collections.abc import Awaitable, Callable
from unittest.mock import patch
+import pytest
from quicksand_core._types import QuicksandGuestAgentMethod
from quicksand_core.host.virtio_serial_agent_client import VirtioSerialAgentClient
@@ -101,8 +103,9 @@ class FakeAgent:
returned frames back to the same connection.
"""
- def __init__(self, handler: Handler):
+ def __init__(self, handler: Handler, *, capabilities: list[str] | None = None):
self._handler = handler
+ self._capabilities = capabilities
self._client_sock: socket.socket | None = None
self._srv_writer: asyncio.StreamWriter | None = None
self._serve_task: asyncio.Task | None = None
@@ -150,7 +153,10 @@ class FakeAgent:
msg = await _read_frame(reader)
self.received_requests.append(msg)
if msg.get("method") == "authenticate":
- writer.write(_encode_frame({"result": {"authenticated": True}}))
+ result: dict = {"authenticated": True}
+ if self._capabilities is not None:
+ result["capabilities"] = self._capabilities
+ writer.write(_encode_frame({"result": result}))
await writer.drain()
continue
self._handler_tasks.append(asyncio.create_task(self._handle(msg, writer)))
@@ -350,3 +356,181 @@ async def test_stream_request_accumulates_multi_frame_output() -> None:
("out", "world\n"),
("err", "warn\n"),
]
+
+
+async def test_legacy_agent_rejects_stdin_without_consuming_it():
+ produced = []
+
+ async def source():
+ produced.append(True)
+ yield b"unused"
+
+ async def handler(msg):
+ raise AssertionError(f"Unexpected execution: {msg}")
+
+ async with FakeAgent(handler) as agent:
+ client = await _connected_client(agent)
+ try:
+ assert not client.supports_stdin
+ result = await client.send_stream_request({"command": "cat"}, timeout=2, stdin=source())
+ finally:
+ await client.close()
+ assert "does not support streaming stdin" in result["error"]["message"]
+ assert not produced
+ assert [r["method"] for r in agent.received_requests] == ["authenticate"]
+
+
+async def test_stdin_and_output_progress_in_both_directions():
+ prompt = asyncio.Event()
+ echoed = asyncio.Event()
+ output = []
+ received = bytearray()
+ execution_id = None
+
+ async def source():
+ await prompt.wait()
+ yield b"first\n"
+ await echoed.wait()
+ yield b"\x00\xfflast"
+
+ async def handler(msg):
+ nonlocal execution_id
+ request_id = msg["id"]
+ if msg["method"] == "execute_stream":
+ execution_id = request_id
+ return [
+ {"id": request_id, "stream": "ready"},
+ {"id": request_id, "stream": "stdout", "data": "prompt>"},
+ ]
+ assert msg["method"] == "stdin"
+ if msg["params"].get("eof"):
+ return [
+ {"id": request_id, "result": {"closed": True}},
+ {"id": execution_id, "stream": "exit", "exit_code": 0},
+ ]
+ received.extend(base64.b64decode(msg["params"]["data"], validate=True))
+ return [
+ {"id": request_id, "result": {"closed": False}},
+ {"id": execution_id, "stream": "stdout", "data": "accepted"},
+ ]
+
+ def on_stdout(chunk):
+ output.append(chunk)
+ if "prompt>" in "".join(output):
+ prompt.set()
+ if "accepted" in "".join(output):
+ echoed.set()
+
+ async with FakeAgent(handler, capabilities=["stdin_streaming"]) as agent:
+ client = await _connected_client(agent)
+ try:
+ assert client.supports_stdin
+ result = await client.send_stream_request(
+ {"command": "cat"}, timeout=2, stdin=source(), on_stdout=on_stdout
+ )
+ finally:
+ await client.close()
+ assert result["result"]["stdout"] == "prompt>acceptedaccepted"
+ assert received == b"first\n\x00\xfflast"
+ assert not client.supports_stdin
+
+
+async def test_concurrent_stdin_streams_do_not_mix_input():
+ executions = {}
+
+ async def handler(msg):
+ request_id = msg["id"]
+ stdin_id = msg["params"]["stdin_id"]
+ if msg["method"] == "execute_stream":
+ executions[stdin_id] = (request_id, bytearray())
+ return [{"id": request_id, "stream": "ready"}]
+ execution_id, data = executions[stdin_id]
+ if msg["params"].get("eof"):
+ return [
+ {"id": request_id, "result": {"closed": True}},
+ {"id": execution_id, "stream": "stdout", "data": data.decode()},
+ {"id": execution_id, "stream": "exit", "exit_code": 0},
+ ]
+ data.extend(base64.b64decode(msg["params"]["data"]))
+ return [{"id": request_id, "result": {"closed": False}}]
+
+ async with FakeAgent(handler, capabilities=["stdin_streaming"]) as agent:
+ client = await _connected_client(agent)
+ try:
+ results = await asyncio.gather(
+ client.send_stream_request({"command": "cat"}, timeout=2, stdin=b"one"),
+ client.send_stream_request({"command": "cat"}, timeout=2, stdin=b"two"),
+ )
+ assert not client._pending
+ assert not client._pending_streams
+ finally:
+ await client.close()
+ assert [result["result"]["stdout"] for result in results] == ["one", "two"]
+ assert len(executions) == 2
+
+
+async def test_cancel_during_stdin_ack_cleans_up_pending_requests():
+ writing = asyncio.Event()
+ cancelled = asyncio.Event()
+ closed = asyncio.Event()
+
+ async def source():
+ try:
+ yield b"first"
+ finally:
+ closed.set()
+
+ async def handler(msg):
+ request_id = msg["id"]
+ if msg["method"] == "execute_stream":
+ return [{"id": request_id, "stream": "ready"}]
+ if msg["method"] == "stdin":
+ writing.set()
+ await asyncio.Event().wait()
+ if msg["method"] == "cancel":
+ cancelled.set()
+ return [{"id": request_id, "result": {"cancelled": True}}]
+ return [_ok(request_id, "still connected")]
+
+ async with FakeAgent(handler, capabilities=["stdin_streaming"]) as agent:
+ client = await _connected_client(agent)
+ try:
+ task = asyncio.create_task(
+ client.send_stream_request({"command": "cat"}, timeout=5, stdin=source())
+ )
+ await asyncio.wait_for(writing.wait(), timeout=2)
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+ assert cancelled.is_set()
+ assert closed.is_set()
+ assert not client._pending
+ assert not client._pending_streams
+ result = await client.send_request(QuicksandGuestAgentMethod.PING, {}, timeout=2)
+ assert result["result"]["stdout"] == "still connected"
+ finally:
+ await client.close()
+
+
+async def test_cancel_before_write_cancels_registered_future():
+ async def handler(msg):
+ return [_ok(msg["id"])]
+
+ async with FakeAgent(handler) as agent:
+ client = await _connected_client(agent)
+ assert client._writer_lock is not None
+ await client._writer_lock.acquire()
+ try:
+ task = asyncio.create_task(
+ client.send_request(QuicksandGuestAgentMethod.PING, {}, timeout=2)
+ )
+ await asyncio.sleep(0)
+ future = next(iter(client._pending.values()))
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+ assert future.cancelled()
+ assert not client._pending
+ finally:
+ client._writer_lock.release()
+ await client.close()