Skip to content

Secure, Fast, and Extensible Sandbox runtime for AI agents.

32.1 KB
Raw Blame

title: Secure Container Runtime description: Use gVisor, Kata Containers, and Firecracker microVMs for stronger sandbox isolation.

Secure Container Runtime Guide

This guide explains how to use secure container runtimes with OpenSandbox to provide hardware-level isolation for executing untrusted AI-generated code.

Table of Contents


Overview

What are Secure Container Runtimes?

Secure container runtimes provide stronger isolation than the standard runc runtime used by Docker and containerd. They add additional security layers through different mechanisms:

Runtime Isolation Mechanism Startup Overhead Memory Overhead Best For
runc (default) Process-level cgroups ~0ms Minimal Trusted workloads, local development
gVisor User-space kernel (syscall interception) ~10-50ms ~50MB General workloads with low overhead
Kata (QEMU) Full VM with QEMU hypervisor ~500ms ~20-50MB Maximum compatibility and isolation
Kata (Firecracker) MicroVM with Firecracker hypervisor ~125ms ~5MB High density, minimal footprint
Kata (CLH) Cloud Hypervisor ~200ms ~10-20MB Balanced performance and isolation

Why Use Secure Runtimes?

OpenSandbox is designed to execute untrusted code generated by AI models (Claude, GPT-4, Gemini, etc.). Secure runtimes provide:

  1. Container Escape Protection: Prevents malicious code from breaking out of the container
  2. Kernel-Level Isolation: Each sandbox gets its own kernel context
  3. Multi-Tenant Safety: Different users' sandboxes are strongly isolated
  4. Compliance: Meets security requirements for regulated industries

Supported Runtime Types

OpenSandbox supports the following secure runtime types through server-level configuration:

  • "gvisor" - gVisor with runsc
  • "kata" - Kata Containers with QEMU hypervisor (default)
  • "firecracker" - Kata Containers with Firecracker hypervisor
  • "" (empty) - Standard runc (default, no secure runtime)

Key Design Principle

Server-Level Configuration: The secure runtime is configured once at the server level by administrators. All sandboxes on that server transparently use the configured runtime. SDK users and API callers require no code changes.


Server Configuration

Secure runtimes are configured through the ~/.sandbox.toml configuration file. The server validates the configured runtime at startup and will refuse to start if the runtime is unavailable.

Configuration File

Edit ~/.sandbox.toml:

[runtime]
type = "docker"  # or "kubernetes"
execd_image = "opensandbox/execd:latest"

# Secure container runtime configuration
# When enabled, ALL sandboxes on this server use the specified runtime
[secure_runtime]
# Runtime type: "", "gvisor", "kata", "firecracker"
type = ""

# Docker mode: OCI runtime name (e.g., "runsc" for gVisor, "kata-runtime" for Kata)
# Required when runtime.type = "docker" and type is not empty
docker_runtime = "runsc"

# Kubernetes mode: RuntimeClass name (e.g., "gvisor", "kata-qemu", "kata-fc")
# Required when runtime.type = "kubernetes" and type is not empty
k8s_runtime_class = "gvisor"

Configuration Examples

Example 1: gVisor on Docker

[runtime]
type = "docker"
execd_image = "opensandbox/execd:latest"

[secure_runtime]
type = "gvisor"
docker_runtime = "runsc"
k8s_runtime_class = "gvisor"

Example 2: Kata Containers on Kubernetes

[runtime]
type = "kubernetes"
execd_image = "opensandbox/execd:latest"

[secure_runtime]
type = "kata"
docker_runtime = "kata-runtime"
k8s_runtime_class = "kata-qemu"

Example 3: Kata + Firecracker on Kubernetes

[runtime]
type = "kubernetes"
execd_image = "opensandbox/execd:latest"

[secure_runtime]
type = "firecracker"
docker_runtime = ""  # Not supported in Docker mode
k8s_runtime_class = "kata-fc"

Startup Validation

When the server starts, it automatically validates that the configured secure runtime is available:

$ opensandbox-server
INFO     Validating secure runtime for Docker backend
INFO     Docker OCI runtime 'runsc' is available: {...}
INFO     Application startup complete.

If the runtime is not available, the server will refuse to start with a clear error message:

ERROR    Configured Docker runtime 'runsc' is not available.
        Available runtimes: runc.
        Please install and configure it in /etc/docker/daemon.json.

Multiple Runtime Profiles on Kubernetes

[secure_runtime] is a server startup setting, not a sandbox creation field. One Lifecycle Server process therefore applies one runtime profile to every template-mode sandbox it creates. To offer runc and a stronger runtime such as Kata in the same Kubernetes cluster, run one independently addressed Lifecycle Server release per profile and route each client to the appropriate Server API.

Server profile API Service [secure_runtime] Resulting Pod field
Standard opensandbox-runc type = "" runtimeClassName omitted
Isolated opensandbox-kata type = "kata", k8s_runtime_class = "kata-qemu" runtimeClassName: kata-qemu

Both profiles can use the same OpenSandbox controller and sandbox workload namespace. The RuntimeClass should carry its own scheduling constraints so Kata Pods land only on nodes where Kata is installed.

::: warning Resource names must be unique The Server chart defaults fullnameOverride to opensandbox-server, so changing only the Helm release name is not sufficient: two default releases would render the same Deployment, ServiceAccount, RBAC, ConfigMap, and Service names. Set a different fullnameOverride for every profile. The standalone ingress-gateway chart also defaults to a fixed fullnameOverride (opensandbox-ingress-gateway); give each gateway a distinct value, such as opensandbox-runc-gateway and opensandbox-kata-gateway. Its ClusterRole and ClusterRoleBinding names derive from this value and must be unique across the cluster. Separate control-plane namespaces alone do not prevent those RBAC collisions. :::

Deploy two Server profiles

Create the opensandbox workload namespace and one control-plane namespace per profile. Create the API-key Secrets in their corresponding control-plane namespaces as described in Kubernetes Deployment. Use opensandbox-runc-api-key in opensandbox-runc-system and opensandbox-kata-api-key in opensandbox-kata-system to match the references below.

Use the values files below for both releases. They replace configToml in full, including the [egress] image and mode needed for sandbox network policies. The examples use direct sandbox endpoints: clients must be able to reach the sandbox Pod IPs inside the cluster. For external clients, configure a gateway as described below. Match the execd and egress image tags to the checked-out release.

::: code-group

fullnameOverride: opensandbox-runc
namespaceOverride: opensandbox-runc-system

server:
  replicaCount: 1
  gateway:
    enabled: false
  env:
    - name: OPENSANDBOX_SERVER_API_KEY
      valueFrom:
        secretKeyRef:
          name: opensandbox-runc-api-key
          key: api-key

configToml: |
  [server]
  host = "0.0.0.0"
  port = 80
  api_key = ""

  [log]
  level = "INFO"

  [runtime]
  type = "kubernetes"
  execd_image = "sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/execd:release-1.1.1-rc.1"

  [kubernetes]
  namespace = "opensandbox"
  workload_provider = "batchsandbox"

  [egress]
  image = "sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/egress:release-1.1.1-rc.1"
  mode = "dns+nft"

  [secure_runtime]
  type = ""
fullnameOverride: opensandbox-kata
namespaceOverride: opensandbox-kata-system

server:
  replicaCount: 1
  gateway:
    enabled: false
  env:
    - name: OPENSANDBOX_SERVER_API_KEY
      valueFrom:
        secretKeyRef:
          name: opensandbox-kata-api-key
          key: api-key

configToml: |
  [server]
  host = "0.0.0.0"
  port = 80
  api_key = ""

  [log]
  level = "INFO"

  [runtime]
  type = "kubernetes"
  execd_image = "sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/execd:release-1.1.1-rc.1"

  [kubernetes]
  namespace = "opensandbox"
  workload_provider = "batchsandbox"

  [egress]
  image = "sandbox-registry.cn-zhangjiakou.cr.aliyuncs.com/opensandbox/egress:release-1.1.1-rc.1"
  mode = "dns+nft"

  [secure_runtime]
  type = "kata"
  k8s_runtime_class = "kata-qemu"

:::

From an OpenSandbox checkout with the base CRDs and controller already installed, install both profiles using the Server chart at manifests/charts/server. The chart supplies its matching Server image tag by default:

kubectl create namespace opensandbox --dry-run=client -o yaml | kubectl apply -f -

helm upgrade --install opensandbox-runc manifests/charts/server \
  --namespace opensandbox-runc-system \
  --create-namespace \
  --values values-runc.yaml

helm upgrade --install opensandbox-kata manifests/charts/server \
  --namespace opensandbox-kata-system \
  --create-namespace \
  --values values-kata.yaml

Expose the two Services under different internal DNS names, hostnames, or API gateway routes. Do not put them behind an unqualified round-robin route: runtime selection is made by choosing the Server endpoint. Keep subsequent get, renew, pause, resume, snapshot, and delete calls on the same profile endpoint that created the sandbox.

Gateway access

server.gateway.enabled=true only makes the Server announce a gateway address; it does not deploy the gateway. Install one standalone manifests/charts/ingress-gateway release per profile with the distinct fullnameOverride values above. Set gateway.dataplaneNamespace to that profile's workload namespace (opensandbox in these examples).

Expose each gateway through its own client-reachable host, and set the matching Server's server.gateway.host to that host. Match server.gateway.gatewayRouteMode to the gateway's gateway.gatewayRouteMode. If signed access is enabled, configure the same signing key ring on each Server/gateway pair as described in Secure Access.

For example, install the two gateways and update the existing Server releases:

helm upgrade --install opensandbox-runc-gateway manifests/charts/ingress-gateway \
  --namespace opensandbox-runc-system \
  --set namespaceOverride=opensandbox-runc-system \
  --set fullnameOverride=opensandbox-runc-gateway \
  --set gateway.dataplaneNamespace=opensandbox \
  --set gateway.gatewayRouteMode=header

helm upgrade --install opensandbox-kata-gateway manifests/charts/ingress-gateway \
  --namespace opensandbox-kata-system \
  --set namespaceOverride=opensandbox-kata-system \
  --set fullnameOverride=opensandbox-kata-gateway \
  --set gateway.dataplaneNamespace=opensandbox \
  --set gateway.gatewayRouteMode=header

helm upgrade --install opensandbox-runc manifests/charts/server \
  --namespace opensandbox-runc-system --values values-runc.yaml \
  --set server.gateway.enabled=true \
  --set-string server.gateway.host=runc-gateway.example.com \
  --set server.gateway.gatewayRouteMode=header

helm upgrade --install opensandbox-kata manifests/charts/server \
  --namespace opensandbox-kata-system --values values-kata.yaml \
  --set server.gateway.enabled=true \
  --set-string server.gateway.host=kata-gateway.example.com \
  --set server.gateway.gatewayRouteMode=header

The gateway chart uses namespaceOverride for its resource namespace; setting only Helm's --namespace does not change that default. The gateway Services above are ClusterIP by default. Expose them through your cluster's ingress or load balancer and replace the *.example.com hosts with those reachable addresses before running the Server upgrades. A gateway host routes sandbox traffic; clients still use the corresponding Server API endpoint for lifecycle operations.

Verify profile selection

Create one sandbox through each Server, then inspect the generated BatchSandbox Pod template:

kubectl get batchsandbox --namespace opensandbox <runc-sandbox-id> \
  -o jsonpath='{.spec.template.spec.runtimeClassName}'
# Empty output: the cluster's default OCI runtime is used.

kubectl get batchsandbox --namespace opensandbox <kata-sandbox-id> \
  -o jsonpath='{.spec.template.spec.runtimeClassName}'
# kata-qemu

::: tip Pool mode Pool-backed sandboxes reuse Pods that already exist, so the Server's [secure_runtime] setting does not rewrite their runtime. Create a separate Pool per profile, set spec.template.spec.runtimeClassName on each Pool explicitly, and expose only the matching poolRef through that profile. :::

::: info Operational boundary Multiple Server profiles provide deterministic runtime selection, not a security or tenancy boundary. Servers targeting the same workload namespace can observe the same Kubernetes resources. Use separate workload namespaces (and suitably scoped RBAC or separate clusters) when profiles must also be isolated from one another. Separate gateway names and hosts avoid resource and routing collisions; they do not restrict access to workloads in a shared namespace. :::


Docker Mode

Docker mode is fully supported for secure container runtimes.

Prerequisites

  • Docker daemon installed and running
  • Secure runtime installed on the host

gVisor Setup for Docker

Step 1: Install gVisor runsc

For Docker mode, you only need to install the runsc OCI runtime:

# Ubuntu/Debian
curl -fsSL https://gvisor.dev/archive.key | sudo gpg --dearmor -o /usr/share/keyrings/gvisor-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases release main" | \
  sudo tee /etc/apt/sources.list.d/gvisor.list
sudo apt-get update && sudo apt-get install -y runsc

# Verify installation
runsc --version

Note: For Docker mode, only runsc is required. The containerd-shim-runsc-v1 is only needed for Kubernetes/containerd.

Reference: See gVisor Installation Guide for other distributions and installation methods.

Step 2: Configure Docker daemon

Use the runsc install command to automatically configure Docker daemon:

sudo runsc install

To keep Docker snapshots working (see the warning below), pass --overlay2=none through to runsc:

sudo runsc install -- --overlay2=none

Or manually edit /etc/docker/daemon.json:

{
  "runtimes": {
    "runsc": {
      "path": "/usr/bin/runsc",
      "runtimeArgs": [
        "--overlay2=none"
      ]
    }
  }
}

::: warning Docker snapshots require --overlay2=none runsc normally keeps root filesystem changes in its own root:self overlay. Docker cannot include those changes in docker commit, so a Docker-backed OpenSandbox snapshot can report success while restoring only the original image. Keep --overlay2=none in the runtime arguments when you use snapshot, fork, or restore workflows. This makes sandbox writes visible to Docker's storage driver and therefore available to the snapshot image.

Existing runsc installations created without this argument must update daemon.json and restart Docker before taking snapshots. See gVisor's filesystem documentation for the root filesystem overlay behavior. :::

Restart Docker:

sudo systemctl restart docker

Reference: See gVisor Docker Quick Start for more details.

Step 3: Configure OpenSandbox Server

Edit ~/.sandbox.toml:

[runtime]
type = "docker"
execd_image = "opensandbox/execd:latest"

[secure_runtime]
type = "gvisor"
docker_runtime = "runsc"

Step 4: Start Server and Verify

opensandbox-server

Create a test sandbox:

curl -X POST http://localhost:8080/v1/sandboxes \
  -H "Content-Type: application/json" \
  -d '{
    "image": {"uri": "python:3.11"},
    "timeout": 3600,
    "resourceLimits": {"cpu": "500m", "memory": "512Mi"},
    "entrypoint": ["python", "-u", "-c", "import time\nwhile True: print('hello from gVisor!'); time.sleep(1)"],
    "metadata": {
      "name": "gvisor-docker-sandbox"
    }
  }'

Verify the runtime:

docker ps --format "{{.ID}}\t{{.Image}}\t{{.Names}}"
docker inspect <container_id> | grep -A2 Runtime
# Expected output:
# "Runtime": "runsc",

Kata Containers Setup for Docker

System Requirements

Kata Containers requires hardware virtualization support. Verify your system meets the following requirements:

Hardware Virtualization Support:

# Check if CPU supports hardware virtualization (VT-x for Intel, AMD-V for AMD)
lscpu | grep Virtualization
# Expected output: Virtualization: VT-x (Intel) or AMD-V (AMD)

# Alternatively on Intel
grep -E --color=auto 'vmx|svm' /proc/cpuinfo
# Expected: vmx (Intel) or svm (AMD) flags present

KVM Module:

# Check if KVM module is loaded
lsmod | grep kvm
# Expected: kvm_intel (Intel) or kvm_amd (AMD)

# If not loaded, load KVM module
sudo modprobe kvm_intel  # For Intel
# or
sudo modprobe kvm_amd    # For AMD

Kernel Requirements:

  • Linux kernel 5.10 or later recommended
  • KVM enabled in kernel config

Docker Requirements:

  • Docker 20.10 or later
  • /etc/docker/daemon.json configured for Kata runtime

Installation

Download and install Kata Containers static binaries from GitHub releases:

# Find the latest release at https://github.com/kata-containers/kata-containers/releases
KATA_VERSION="3.27.0"
wget https://github.com/kata-containers/kata-containers/releases/download/${KATA_VERSION}/kata-static-${KATA_VERSION}-amd64.tar.zst

# Extract to root directory - Kata will be installed in /opt/kata
zstd -d kata-static-${KATA_VERSION}-amd64.tar.zst
tar -xvf kata-static-${KATA_VERSION}-amd64.tar -C /

# Create symbolic links for PATH access
sudo ln -sf /opt/kata/bin/kata-runtime /usr/local/bin/kata-runtime
sudo ln -sf /opt/kata/bin/containerd-shim-kata-v2 /usr/local/bin/containerd-shim-kata-v2

# Verify installation
kata-runtime --version

Configure Docker Daemon

Edit /etc/docker/daemon.json to register Kata as a runtime:

{
  "default-runtime": "runc",
  "runtimes": {
    "kata": {
      "runtimeType": "io.containerd.kata.v2"
    }
  }
}

Restart Docker to apply changes:

sudo systemctl restart docker

# Verify Kata is available in Docker
docker info | grep -A5 Runtimes
# Expected output should include "io.containerd.runc.v2 kata"

Configure OpenSandbox Server

Edit ~/.sandbox.toml:

[runtime]
type = "docker"
execd_image = "opensandbox/execd:latest"

[secure_runtime]
type = "kata"
docker_runtime = "kata"

Verify Installation

Test with OpenSandbox API

Create a sandbox and verify it's running in a VM by checking the kernel:

# Create a test sandbox
curl --location 'http://127.0.0.1:8080/v1/sandboxes' \
  --header 'Content-Type: application/json' \
  --data '{
    "image": {"uri": "ubuntu:latest"},
    "timeout": 3600,
    "resourceLimits": {"cpu": "500m", "memory": "512Mi"},
    "entrypoint": ["/bin/bash", "-c", "while true; do uname -a; sleep 1; done"],
    "metadata": {
      "name": "kata-sandbox"
    }
  }'

Check the container's kernel to verify VM isolation:

# Get the container ID
docker ps | grep kata-sandbox

# Check the kernel inside the container (should be different from host)
docker exec <container_id> uname -a
# Expected output: Linux <hostname> 5.10.x-generic #x86_64 ... (Kata VM kernel)

# Compare with host kernel
uname -a
# Host kernel might be different version or have different hostname

Key Indicators of Kata VM:

  • Container runs in a separate kernel with different hostname
  • Kernel version is typically 5.10.x (Kata's guest kernel)
  • Host process list shows qemu-system-x86_64 or similar hypervisor process

Kubernetes Mode

Kubernetes mode supports secure runtimes through RuntimeClass resources.

Prerequisites

  • Kubernetes cluster with containerd runtime
  • Secure runtime installed on all nodes
  • RuntimeClass CRDs created

gVisor Setup for Kubernetes

Step 1: Install gVisor Components on All Nodes

For Kubernetes with containerd, every node needs two binaries, both part of the runsc package:

  1. runsc - the gVisor OCI runtime
  2. containerd-shim-runsc-v1 - the containerd shim for gVisor
# On each node - Ubuntu/Debian
curl -fsSL https://gvisor.dev/archive.key | sudo gpg --dearmor -o /usr/share/keyrings/gvisor-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases release main" | \
  sudo tee /etc/apt/sources.list.d/gvisor.list
sudo apt-get update

# Install gVisor
sudo apt-get install -y runsc

# Verify installation
runsc --version
containerd-shim-runsc-v1 --version

Reference: See gVisor Installation Guide for complete installation instructions and other distributions.

Step 2: Configure containerd

Edit /etc/containerd/config.toml (containerd 1.x, config version = 2):

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"

For containerd 2.x (config version = 3), the section is [plugins."io.containerd.cri.v1.runtime".containerd.runtimes.runsc].

To pass flags to runsc, point the shim at a config file and put the flags under [runsc_config]. The shim silently ignores other section names. For example:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc.options]
    TypeUrl = "io.containerd.runsc.v1.options"
    ConfigPath = "/etc/containerd/runsc.toml"
sudo tee /etc/containerd/runsc.toml > /dev/null <<'EOF'
[runsc_config]
  # Each key is passed to runsc as --<key>=<value>.
  # For example, here is how to enable nvproxy for GPU functionality.
  # This is equivalent to `--nvproxy=true`.
  nvproxy = "true"
EOF

Restart containerd:

sudo systemctl restart containerd

Step 3: Create RuntimeClass CRD

# gvisor-runtimeclass.yaml
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
kubectl apply -f gvisor-runtimeclass.yaml

Step 4: Configure OpenSandbox Server

Edit ~/.sandbox.toml:

[runtime]
type = "kubernetes"
execd_image = "opensandbox/execd:latest"

[secure_runtime]
type = "gvisor"
k8s_runtime_class = "gvisor"

Step 5: Verify Installation

# Test the RuntimeClass
kubectl run test-gvisor --restart=Never --image=hello-world --runtime-class=gvisor
kubectl logs test-gvisor
kubectl delete pod test-gvisor

Kata Containers Setup for Kubernetes

Step 1: Install Kata Containers

Follow the official Kata Containers installation guide.

Quick installation using Helm:

# Install kata-deploy which will set up Kata Containers via DaemonSet
helm install kata-deploy "oci://ghcr.io/kata-containers/kata-deploy-charts/kata-deploy" --version "3.27.0" --namespace kube-system --create-namespace

# Wait for kata-deploy pods to be ready
kubectl wait --for=condition=ready pod -l name=kata-deploy -n kube-system --timeout=300s

Note: The kata-deploy DaemonSet will automatically configure containerd on all nodes. Manual containerd configuration is not required when using kata-deploy.

Step 2: Verify Installation

Check that Kata Containers is installed and RuntimeClasses are created:

# Check RuntimeClasses
kubectl get runtimeclass

# Expected output:
# NAME         HANDLER     AGE
# kata         kata-qemu   10m
# kata-qemu    kata-qemu   10m
# kata-clh     kata-clh    10m
# kata-fc      kata-fc     10m

# Test Kata with a simple pod
kubectl run test-kata --restart=Never --image=hello-world --runtime-class=kata-qemu
kubectl logs test-kata
kubectl delete pod test-kata

Creating Pools for Different Runtimes (Optional)

When using Pool CRDs for pre-warmed sandboxes, create separate pools for each runtime type:

# gvisor-pool.yaml
apiVersion: sandbox.opensandbox.io/v1alpha1
kind: Pool
metadata:
  name: gvisor-pool
  labels:
    runtime: gvisor
spec:
  template:
    spec:
      runtimeClassName: gvisor
      containers:
        - name: sandbox-container
          image: opensandbox/code-interpreter:v1.1.0
  capacitySpec:
    bufferMax: 10
    bufferMin: 2
    poolMax: 20
    poolMin: 5

User Guide

This section is for AI application developers using OpenSandbox.

No Code Changes Required

Important: The secure runtime is configured at the server level. Your code does not need to change.

Simply create a sandbox using the OpenSandbox Lifecycle API - the server automatically applies the configured secure runtime:

Create a test sandbox:

curl -X POST http://localhost:8080/v1/sandboxes \
  -H "Content-Type: application/json" \
  -d '{
    "image": {"uri": "python:3.11"},
    "timeout": 3600,
    "resourceLimits": {"cpu": "500m", "memory": "512Mi"},
    "entrypoint": ["python", "-u", "-c", "import time\nwhile True: print(\"hello from secure sandbox!\"); time.sleep(1)"],
    "metadata": {
      "name": "my-secure-sandbox"
    }
  }'

Response:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "status": "running"
}

The sandbox will automatically use the secure runtime configured on the server (gVisor, Kata, or runc).

How It Works

  1. Administrator configures the secure runtime in ~/.sandbox.toml
  2. Server validates the runtime at startup
  3. Server automatically injects the runtime into each sandbox:
    • Docker mode: Adds runtime to HostConfig
    • Kubernetes mode: Adds runtimeClassName to Pod spec
  4. User creates sandboxes via API - no runtime parameter needed

Verifying Runtime Isolation

After creating a sandbox, verify the runtime being used:

Docker mode:

docker ps --format "{{.ID}}\t{{.Image}}\t{{.Names}}"
docker inspect <container_id> | grep -A2 Runtime
# Expected output for gVisor:
# "Runtime": "runsc",

Kubernetes mode:

kubectl get pod <pod-name> -o jsonpath='{.spec.runtimeClassName}'
# Expected output for gVisor:
# gvisor

Administrator Guide

This section is for platform operators and SREs managing secure runtime infrastructure.

Prerequisites

Secure runtimes must be installed and configured on your infrastructure before configuring OpenSandbox. OpenSandbox does not install runtimes automatically.

Installation Summary

Runtime Docker Kubernetes
gVisor Install runsc → Configure daemon.json Install runsc → Configure containerd → Create RuntimeClass
Kata (QEMU) Install kata-runtime → Configure daemon.json Install Kata → Configure containerd → Create RuntimeClass
Kata (Firecracker) Not supported Install Kata → Configure containerd → Create RuntimeClass

Configuration Validation

The server validates secure runtime configuration at startup:

  1. Docker mode: Checks if the runtime exists in Docker daemon's runtime list
  2. Kubernetes mode: Checks if the RuntimeClass exists in the cluster

If validation fails, the server refuses to start with a clear error message.

Security Best Practices

  1. Default to gVisor: Provides good security with acceptable performance for most workloads
  2. Use Kata for Untrusted Code: Maximum isolation for completely unknown code
  3. Regular Updates: Keep runtimes updated for security patches
  4. Test Compatibility: Validate your workloads with the chosen runtime before production
  5. Monitor Resources: Secure runtimes have higher memory overhead

Runtime Selection Guidelines

Use Case Recommended Runtime Reasoning
Development/Testing runc (default) Fastest startup, lowest overhead
Production AI Code Execution gVisor Good balance of security and performance
High-Security Requirements Kata (QEMU) Maximum isolation, full compatibility
High-Density Multi-Tenant Kata (Firecracker) Minimal memory overhead per sandbox
Untrusted Network Code gVisor or Kata Network traffic is handled by gVisor's netstack or a guest Linux kernel, not the host's TCP/IP stack

Troubleshooting and Best Practices

Common Issues

1. Runtime Not Found (Docker)

Error: Configured Docker runtime 'runsc' is not available.

Solution: Ensure the runtime is configured in /etc/docker/daemon.json and Docker has been restarted:

sudo systemctl restart docker
docker info | grep -A5 Runtimes

2. RuntimeClass Not Found (Kubernetes)

Error: RuntimeClass 'gvisor' does not exist.

Solution: Create the RuntimeClass CRD:

kubectl get runtimeclass
kubectl apply -f gvisor-runtimeclass.yaml

3. Syscall Compatibility Issues

Error: Container exits with code 1, no logs

Cause: gVisor doesn't implement all syscalls. Some applications may not be compatible.

Solution: Check the gVisor compatibility guide. To find the failing syscall, temporarily add --debug --debug-log=/tmp/runsc/ --strace to the runsc flags and look for unsupported syscall messages in the logs (gVisor debugging guide). If the workload needs unimplemented kernel features, use Kata (QEMU), which runs a full Linux guest kernel.

4. Pod Stuck in ContainerCreating

Cause: RuntimeClass handler not configured on the node.

Solution: Verify containerd configuration:

# On the node
sudo containerd config dump
sudo systemctl restart containerd

5. Egress Sidecar Incompatible with gVisor

Error: Sandbox pods CrashLoopBackOff with egress container log:

iptables: Failed to initialize nft: Protocol not supported

Or with iptables-legacy:

iptables v1.8.9 (legacy): can't initialize iptables table 'nat': Table does not exist (do you need to insmod?)

Cause: gVisor's netstack implements the filter and mangle iptables tables but does not implement the nat table. The egress sidecar uses a REDIRECT rule in the nat table to intercept DNS queries (port 53 → 15353), so it cannot start under gVisor. This is an upstream gVisor limitation (gvisor#170).

Solution:

  • Use secure_runtime.type = "kata" with k8s_runtime_class = "kata-qemu" — Kata provides a full Linux kernel per pod, so the nat table is available and the egress sidecar works unchanged.
  • Use a CNI-level FQDN policy (e.g., Cilium toFQDNs) instead of the egress sidecar for network isolation under gVisor.
  • Remove network_policy from sandbox creation requests if egress control is not required.

Note: The server validates this combination at request time and returns HTTP 400 with a clear error message when secure_runtime.type = "gvisor" and network_policy are used together.

Compatibility Matrix

Feature runc gVisor Kata (QEMU) Kata (CLH) Kata (FC)
Syscall Compatibility Full Partial Full Full Limited
GPU Support Yes Yes* Yes Yes No
IPv6 Yes Yes Yes Yes Yes
Privileged Mode Yes Yes Yes Yes No
Docker Volume Yes Yes Yes Yes Yes
Systemd Yes Yes* Yes Yes No

*: Requires configuration.

Getting Help