Skip to content

Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.

Put the node's IAM role out of a Bot's reach in the EKS recipe (#620)

CamoZeroDay77c84da0fd0f
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 26a7b9a1..57abdeb2 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -156,6 +156,28 @@ a deployment.
`docs/architecture.md`, `docs/deployment.md` and `docs/configuration.md` all described the old rule
and now describe this one.
+### The EKS cluster recipe puts the node's IAM role out of a Bot's reach
+
+A Bot has a shell, and a shell reaches whatever its pod reaches, including the instance metadata
+service at 169.254.169.254 that hands out the node's IAM role. The NetworkPolicy excepts that
+address, but it is off by default and does nothing on EKS until the VPC CNI is told to enforce it,
+so the cluster config in the chart README now sets `disableIMDSv1` and `disablePodIMDS` on the node
+group, which closes it at the node whatever the CNI is doing.
+
+`disablePodIMDS` is the line that does the work: it sets the hop limit to 1, and that governs the
+IMDSv2 token PUT, so a pod one hop further out than the node cannot get a token. `disableIMDSv1` is
+written beside it to say so out loud rather than to change anything, because eksctl defaults it to
+true and sets `httpTokens: required` for either flag. Pods on the cluster network are covered; one
+running with `hostNetwork: true` is on the node and is not.
+
+**It costs something, and the recipe says so.** The EBS CSI driver reads instance metadata from
+IMDS, and its own documentation asks for a hop limit of 2 or greater in a containerized environment.
+A hop limit of 1 also rules out `MutableCSINodeAllocatableCount`, which requires IMDS to be the
+driver's metadata source. Take these two lines out if you need that.
+
+Nothing in OpenBot wants pod-level IMDS: the chart reaches AWS through IRSA. Documentation only; no
+chart template changed, and an existing cluster is unaffected until its node group is recreated.
+
### A coworker can be a file of its own
The example package declared every coworker in one `agents.yaml`, so adding one meant editing a file
diff --git a/charts/openbot/README.md b/charts/openbot/README.md
index c359a413..3427df48 100644
--- a/charts/openbot/README.md
+++ b/charts/openbot/README.md
@@ -82,8 +82,44 @@ managedNodeGroups:
maxSize: 4
volumeSize: 60
volumeType: gp3
+ # A Bot has a shell, and a shell reaches whatever its pod reaches. These two put the node's
+ # own IAM role out of that reach. See below.
+ disableIMDSv1: true
+ disablePodIMDS: true
```
+**Why the two IMDS lines.** A node's instance metadata service, at 169.254.169.254, hands out that
+node's IAM role to anything that asks from the node. A Bot's computer is a pod on that node running
+commands a model chose, so `curl` against that address is one `run_command` away, and no rule in
+OpenBot's own policy engine is consulted: the engine decides browser navigation and tool calls, and
+the shell reaches the network directly.
+
+`disablePodIMDS` sets the hop limit to 1, so a packet from a pod, which has one more hop to make
+than one from the node itself, no longer arrives. That hop limit applies to the IMDSv2 token PUT,
+and IMDSv1 is a bare GET with no PUT to limit, so the hop limit only means anything once a token is
+mandatory. It already is: eksctl documents `disableIMDSv1` as defaulting to true, and sets
+`httpTokens: required` for either flag. `disableIMDSv1: true` is written out here to state that
+rather than to change it, since a reader deciding whether this recipe is safe should not have to
+know an upstream default. Note this covers pods on the cluster network; a pod running with
+`hostNetwork: true` is on the node and still reaches the address.
+
+**What the hop limit costs.** The EBS CSI driver reads EC2 instance metadata from IMDS, and its own
+FAQ asks for "the hop limit for IMDSv2 responses ... set to 2 or greater" in a containerized
+environment like EKS. A hop limit of 1 also rules out `MutableCSINodeAllocatableCount`, whose
+documented prerequisites are that the driver "must be using IMDS Metadata" and that IMDS be its only
+enabled or preferred metadata source. Drop these two lines if your cluster needs either, and lean on
+the NetworkPolicy below instead.
+
+Two things eksctl will refuse rather than warn about, both worth knowing before the next edit to
+this file: `disablePodIMDS` cannot be combined with `iam.withAddonPolicies` on the same node group,
+and neither field may be set on a node group that supplies its own `launchTemplate`.
+
+The NetworkPolicy below also excepts `169.254.0.0/16`, and is the better control where it is
+enforced. It is off by default and does nothing on EKS until the VPC CNI is told to enforce it, so
+these two lines are what holds in the configuration a cluster actually starts in. Nothing in OpenBot
+wants pod-level IMDS either way: the chart reaches AWS through IRSA, which is a ServiceAccount
+annotation and a projected token rather than the node's role.
+
```sh
eksctl create cluster -f cluster.yaml
@@ -401,7 +437,9 @@ reach. If your release has `networkPolicy.enabled` and `computers.mode: sandbox`
set the range and run it again, and the policy is narrow for the first time.
A Bot's computer is allowed 80 and 443 to public addresses and nothing else, which is what stops a
-browser reaching the cluster, the database, or the cloud's credential endpoint. A per-Bot egress
+browser reaching the cluster, the database, or the cloud's credential endpoint. That last one is
+worth closing at the node as well, because this policy is off until somebody turns it on: see
+`disablePodIMDS` in the EKS cluster config above. A per-Bot egress
proxy is therefore two settings rather than one: the variable that names it, and the rule that lets
the computer reach it.