Secure, Fast, and Extensible Sandbox runtime for AI agents.
feat(charts/fast-sandbox): OverlayBD OCI node stack behind runtime.oci.enabled
高然a62cb99fbf57
diff --git a/manifests/charts/fast-sandbox/README.md b/manifests/charts/fast-sandbox/README.mdindex 959a11f59..f504c13bd 100644--- a/manifests/charts/fast-sandbox/README.md+++ b/manifests/charts/fast-sandbox/README.md@@ -24,6 +24,7 @@ boxlite, containerd-based runtimes and the upstream central sandbox-proxy are ou```- Nodes that should serve Firecracker sandboxes need bare-metal KVM (`/dev/kvm`). There is NO manual labeling step: the firecracker-runtime readiness loop verifies each host, installs the Firecracker assets, and applies the `sandbox.fast.io/kvm` + `fast-sandbox.io/firecracker-node` labels plus the `FirecrackerReady` condition itself (rechecking every 5 minutes).+- OverlayBD OCI on-demand loading (`runtime.oci.enabled=true`) additionally needs kernel ublk support on every runtime node (`CONFIG_BLK_DEV_UBLK`, `/dev/ublkc*`; `modprobe ublk_drv` when built as a module) and an OCI registry reachable from the cluster: SandboxTemplates publish `oci://` artifacts there and nodes read blocks on demand. The SandboxPool's fastlet containers must set `FAST_SANDBOX_OCI_ENABLED=true` separately — pools are not managed by this chart. See the fast-sandbox [OverlayBD OCI guide](https://github.com/opensandbox-group/fast-sandbox/blob/main/docs/guides/overlaybd-oci-sandboxes.md) for the credential (registry Secret + `fast-sandbox-registry` ConfigMap) and StateRoot-sharing requirements.- The agent registry Secret with artifact-store pull credentials (compiled `registry.json`):@@ -84,6 +85,9 @@ The following table lists the configurable parameters of the chart and their def| runtime.image.repository | string | `"opensandbox/fsb-firecracker-runtime"` | Runtime image repository (built by manifests/release/build-fast-sandbox.sh) || runtime.image.tag | string | `""` | Image tag. Empty uses the release-<appVersion> companion image tag. || runtime.nodeSelector | object | `{}` | Node selector. Empty by default: the runtime applies the firecracker scheduling labels itself, so it must run on every candidate node. Pin it with your own coarse selector only if the cluster hosts unrelated node pools. |+| runtime.oci | object | `{"cacheSizeGB":16,"enabled":false}` | OverlayBD OCI on-demand loading (fast-sandbox PR #127 contract). Enabling installs the fsb-vm + OverlayBD tools bundle under /opt/fast-sandbox/oci (init container), sets FAST_SANDBOX_OCI_ENABLED on the agent (relaxing the XFS/reflink readiness requirement), and runs one overlaybd-ublkd sidecar per node serving the shared ublk daemon socket. Nodes need KVM plus kernel ublk support (CONFIG_BLK_DEV_UBLK, /dev/ublkc*). The SandboxPool's fastlet containers need FAST_SANDBOX_OCI_ENABLED=true separately — pools are not managed by this chart. |+| runtime.oci.cacheSizeGB | int | `16` | Node cache capacity for the shared overlaybd block cache (OVERLAYBD_CACHE_SIZE_GB); independent of VM upper layers. |+| runtime.oci.enabled | bool | `false` | Whether the node OCI stack (tools init + overlaybd-ublkd sidecar + agent flag + janitor oci-state-root) is installed || runtime.registrySecret | string | `"fast-sandbox-agent-registry"` | Secret carrying the compiled agent registry configuration (registry.json key with artifact-store pull credentials); must be provisioned by the operator. || runtime.resources | object | `{"agent":{},"janitor":{}}` | Container resources for the DaemonSet. Setting requests matters more than limits here: without them the pod is BestEffort and is evicted first under node pressure, taking down the management API for every fastlet on the node. Limits stay off by default so artifact pulls and the readiness loop are not throttled or OOM-killed at their spikes. || runtime.resources.agent | object | `{}` | Resources for the firecracker-runtime agent container. |diff --git a/manifests/charts/fast-sandbox/README.md.gotmpl b/manifests/charts/fast-sandbox/README.md.gotmplindex 2eb28b6fc..417ad8cd3 100644--- a/manifests/charts/fast-sandbox/README.md.gotmpl+++ b/manifests/charts/fast-sandbox/README.md.gotmpl@@ -24,6 +24,7 @@ boxlite, containerd-based runtimes and the upstream central sandbox-proxy are ou```- Nodes that should serve Firecracker sandboxes need bare-metal KVM (`/dev/kvm`). There is NO manual labeling step: the firecracker-runtime readiness loop verifies each host, installs the Firecracker assets, and applies the `sandbox.fast.io/kvm` + `fast-sandbox.io/firecracker-node` labels plus the `FirecrackerReady` condition itself (rechecking every 5 minutes).+- OverlayBD OCI on-demand loading (`runtime.oci.enabled=true`) additionally needs kernel ublk support on every runtime node (`CONFIG_BLK_DEV_UBLK`, `/dev/ublkc*`; `modprobe ublk_drv` when built as a module) and an OCI registry reachable from the cluster: SandboxTemplates publish `oci://` artifacts there and nodes read blocks on demand. The SandboxPool's fastlet containers must set `FAST_SANDBOX_OCI_ENABLED=true` separately — pools are not managed by this chart. See the fast-sandbox [OverlayBD OCI guide](https://github.com/opensandbox-group/fast-sandbox/blob/main/docs/guides/overlaybd-oci-sandboxes.md) for the credential (registry Secret + `fast-sandbox-registry` ConfigMap) and StateRoot-sharing requirements.- The agent registry Secret with artifact-store pull credentials (compiled `registry.json`):diff --git a/manifests/charts/fast-sandbox/templates/runtime-daemonset.yaml b/manifests/charts/fast-sandbox/templates/runtime-daemonset.yamlindex 3d311741f..bad21afbc 100644--- a/manifests/charts/fast-sandbox/templates/runtime-daemonset.yaml+++ b/manifests/charts/fast-sandbox/templates/runtime-daemonset.yaml@@ -178,6 +178,19 @@ spec:# The runtime runs on the cluster network (NOT hostNetwork): its dart# child needs the cluster DNS to resolve the headless Service peers,# and a regular Pod identity for the stable HRW node hostname file.+ {{- if .Values.runtime.oci.enabled }}+ # OCI tools bundle (fsb-vm, OverlayBD tools, convertor) installed+ # once under the shared node hostPath; the agent, the overlaybd-ublkd+ # sidecar, the janitor and Fastlet Pods all consume it read-only.+ initContainers:+ - name: install-oci-tools+ image: {{ include "fast-sandbox.image" (dict "repository" .Values.runtime.image.repository "tag" .Values.runtime.image.tag "appVersion" .Chart.AppVersion) }}+ imagePullPolicy: {{ .Values.runtime.image.pullPolicy }}+ command: ["/usr/local/bin/install-oci-tools"]+ volumeMounts:+ - name: oci-tools+ mountPath: /opt/fast-sandbox/oci+ {{- end }}containers:- name: firecracker-agentimage: {{ include "fast-sandbox.image" (dict "repository" .Values.runtime.image.repository "tag" .Values.runtime.image.tag "appVersion" .Chart.AppVersion) }}@@ -189,6 +202,12 @@ spec:valueFrom:fieldRef:fieldPath: metadata.uid+ {{- if .Values.runtime.oci.enabled }}+ # Relaxes the XFS/reflink node readiness requirement; native+ # templates stay usable (rootfs copies just take longer).+ - name: FAST_SANDBOX_OCI_ENABLED+ value: "true"+ {{- end }}- name: FAST_SANDBOX_AGENT_CONFIGvalue: /etc/fast-sandbox/agent-config/agent.yaml# Node readiness: spec.nodeName enables the host-check loop (the@@ -244,17 +263,68 @@ spec:readOnly: true- name: fc-assetsmountPath: /opt/fast-sandbox/firecracker+ {{- if .Values.runtime.oci.enabled }}+ - name: oci-tools+ mountPath: /opt/fast-sandbox/oci+ readOnly: true+ - name: oci-socket+ mountPath: /run/fast-sandbox/overlaybd+ {{- end }}+ {{- if .Values.runtime.oci.enabled }}+ # One shared overlaybd-ublkd per node: hosts every sandbox's ublk+ # devices behind /run/fast-sandbox/overlaybd/ublkd.sock (fsb-vm+ # connects directly; the agent does not proxy device ops). Shares the+ # agent's StateRoot at the same absolute path and the host /dev so+ # devices created here are visible to the VMs.+ - name: overlaybd-ublkd+ image: {{ include "fast-sandbox.image" (dict "repository" .Values.runtime.image.repository "tag" .Values.runtime.image.tag "appVersion" .Chart.AppVersion) }}+ imagePullPolicy: {{ .Values.runtime.image.pullPolicy }}+ command: ["/usr/local/bin/overlaybd-node"]+ env:+ - name: FAST_SANDBOX_OCI_ENABLED+ value: "true"+ - name: FAST_SANDBOX_STATE_ROOT+ value: {{ .Values.runtime.stateRoot }}+ - name: OVERLAYBD_CACHE_SIZE_GB+ value: {{ .Values.runtime.oci.cacheSizeGB | quote }}+ securityContext:+ privileged: true+ readinessProbe:+ exec:+ command:+ - /bin/sh+ - -ec+ - curl -fsS --max-time 2 --unix-socket /run/fast-sandbox/overlaybd/ublkd.sock http://localhost/v1/ping+ periodSeconds: 5+ volumeMounts:+ - name: oci-tools+ mountPath: /opt/fast-sandbox/oci+ readOnly: true+ - name: oci-socket+ mountPath: /run/fast-sandbox/overlaybd+ - name: agent-state+ mountPath: {{ .Values.runtime.stateRoot }}+ - name: host-dev+ mountPath: /dev+ {{- end }}# Node janitor sidecar: sweeps orphaned fastlet network state (netns/# tap/veth), boxlite leftovers, and — the reason for hostPID — kills# leaked VMM processes fastlets delegate over the node-cleanup UDS# socket. The containerd backend is DISABLED here (--containerd-socket=):# this pod targets the firecracker runtime; the standalone janitor# DaemonSet keeps covering the containerd-based runtimes.+ {{- $janitorArgs := list+ (printf "--scan-interval=%s" .Values.janitor.scanInterval)+ (printf "--orphan-timeout=%s" .Values.janitor.orphanTimeout)+ "--containerd-socket=" }}+ {{- if .Values.runtime.oci.enabled }}+ {{- $janitorArgs = append $janitorArgs (printf "--oci-state-root=%s" .Values.runtime.stateRoot) }}+ {{- end }}- name: janitorimage: {{ include "fast-sandbox.image" (dict "repository" .Values.janitor.image.repository "tag" .Values.janitor.image.tag "appVersion" .Chart.AppVersion) }}imagePullPolicy: {{ .Values.janitor.image.pullPolicy }}command: ["/workspace/janitor"]- args: ["--scan-interval={{ .Values.janitor.scanInterval }}", "--orphan-timeout={{ .Values.janitor.orphanTimeout }}", "--containerd-socket="]+ args: {{ $janitorArgs | toJson }}ports:- name: metricscontainerPort: 9092@@ -282,6 +352,20 @@ spec:- name: runtime-environmentsmountPath: /etc/fast-sandbox/runtime-environmentsreadOnly: true+ {{- if .Values.runtime.oci.enabled }}+ # fsb-vm cleanup of retained ublk devices and checkpoint pack+ # resources left by deleted Pods (revalidated against Sandbox+ # absence first).+ - name: oci-tools+ mountPath: /opt/fast-sandbox/oci+ readOnly: true+ - name: oci-socket+ mountPath: /run/fast-sandbox/overlaybd+ - name: host-dev+ mountPath: /dev+ - name: agent-state+ mountPath: {{ .Values.runtime.stateRoot }}+ {{- end }}volumes:- name: agent-sockethostPath:@@ -339,4 +423,25 @@ spec:# janitor falls back to its defaults (containerd backend is# disabled via --containerd-socket= anyway).optional: true+ {{- if .Values.runtime.oci.enabled }}+ # Immutable OCI tools release installed by the init container and+ # atomically switched under current/; shared read-only with the+ # overlaybd-ublkd sidecar, the janitor, and Fastlet Pods.+ - name: oci-tools+ hostPath:+ path: /opt/fast-sandbox/oci+ type: DirectoryOrCreate+ # Root-only ublkd control socket + credential directory shared with+ # Fastlet Pods (fast-sandbox-registry ConfigMap supplies credentials).+ - name: oci-socket+ hostPath:+ path: /run/fast-sandbox/overlaybd+ type: DirectoryOrCreate+ # Host device tree: ublk devices created by the sidecar must be+ # visible to the Firecracker VMs.+ - name: host-dev+ hostPath:+ path: /dev+ type: Directory+ {{- end }}{{- end }}diff --git a/manifests/charts/fast-sandbox/values.yaml b/manifests/charts/fast-sandbox/values.yamlindex d465394f2..f92fa7485 100644--- a/manifests/charts/fast-sandbox/values.yaml+++ b/manifests/charts/fast-sandbox/values.yaml@@ -231,3 +231,19 @@ runtime:# startup-only; the nodeReadiness section (fcVersion, interval,# minFree, minMemory) hot-reloads on every readiness pass.config: ""+ # -- OverlayBD OCI on-demand loading (fast-sandbox PR #127 contract).+ # Enabling installs the fsb-vm + OverlayBD tools bundle under+ # /opt/fast-sandbox/oci (init container), sets FAST_SANDBOX_OCI_ENABLED+ # on the agent (relaxing the XFS/reflink readiness requirement), and+ # runs one overlaybd-ublkd sidecar per node serving the shared ublk+ # daemon socket. Nodes need KVM plus kernel ublk support+ # (CONFIG_BLK_DEV_UBLK, /dev/ublkc*). The SandboxPool's fastlet+ # containers need FAST_SANDBOX_OCI_ENABLED=true separately — pools are+ # not managed by this chart.+ oci:+ # -- Whether the node OCI stack (tools init + overlaybd-ublkd+ # sidecar + agent flag + janitor oci-state-root) is installed+ enabled: false+ # -- Node cache capacity for the shared overlaybd block cache+ # (OVERLAYBD_CACHE_SIZE_GB); independent of VM upper layers.+ cacheSizeGB: 16