Skip to content

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

feat(platform): cap the swap proxy's credentialed requests per box and per user (#14874)

Krzysztof Czerwinski2b9079481ed2
diff --git a/autogpt_platform/swap_proxy/README.md b/autogpt_platform/swap_proxy/README.md
index 2f80fd70b3..7631d8c075 100644
--- a/autogpt_platform/swap_proxy/README.md
+++ b/autogpt_platform/swap_proxy/README.md
@@ -51,6 +51,12 @@ users at once.
box's egress is pinned): a provider outside it comes back with no host it
may be sent to, scrubbed but never swapped.
(`swap_proxy/swap.py`, `swap_proxy/source.py`)
+ A request that gets a value swapped in counts against a quota per box and
+ per user, in a fixed window shared by every replica through Redis. Past
+ either limit, or if the count cannot be taken, the request is not sent:
+ the box gets a `429` (or `503`) whose body says why and when the limit
+ resets, and the audit says `refused-request` / `quota-exceeded` or
+ `quota-unavailable`. (`swap_proxy/quota.py`)
5. **Scrub.** A value echoed back in a text response, or in a websocket message
from the server, is turned back into its placeholder before the box sees it,
and so is the exact base64 of an HTTP Basic pair the proxy built, which a
@@ -180,7 +186,9 @@ poetry run swap-proxy
| `SWAP_PROXY_CONFDIR` | `~/.mitmproxy` | directory the signing CA is mounted into; see below |
| `SWAP_PROXY_GENERATE_CA` | `false` | local runs only: generate a CA if the directory has none |
| `SWAP_PROXY_EGRESS_ALLOW` | empty | comma-separated private hosts or CIDRs boxes may reach anyway |
-| `REDIS_HOST`, `REDIS_PORT`, `REDIS_PASSWORD`, `REDIS_CLUSTER_HOST`, `REDIS_CLUSTER_PORT`, `REDIS_USE_ANNOUNCED_ADDRESS` | | same meaning as in the backend |
+| `SWAP_PROXY_QUOTA_PER_BOX` / `SWAP_PROXY_QUOTA_PER_USER` | `1000` / `3000` | credentialed requests per box (session or expert) and per user in each window; `0` turns one off |
+| `SWAP_PROXY_QUOTA_WINDOW_SECONDS` | `3600` | the quota's fixed window |
+| `REDIS_HOST`, `REDIS_PORT`, `REDIS_PASSWORD`, `REDIS_CLUSTER_HOST`, `REDIS_CLUSTER_PORT`, `REDIS_USE_ANNOUNCED_ADDRESS` | | same meaning as in the backend. The quota runs one script, under the proxy's own Redis user: besides what the credential lookup needs, that user needs `+eval +get +incr +expire ~swap:quota:*`. Without them every count fails and every credentialed request gets a `503` |
### The signing CA
@@ -364,6 +372,13 @@ run it by hand: `gh workflow run platform-swap-proxy-ci.yml --ref <branch>`.
reconnect rotates it), but the backend no longer knows its box: from then on
nothing is swapped into it, and its text responses from bound hosts are
refused as while the backend is down, until the client opens a new one.
+- The quota counts requests that get a value swapped in, nothing else. A
+ request over it whose body turns out to stream (a large `git push`) cannot
+ be answered with the `429`: it goes out with its placeholder, no value in
+ it, and the provider refuses it; the audit still says why. A request with no
+ body to stream (an HTTP/2 `GET` has no length either) is answered. A body
+ swapped while it streams, and a websocket message a box sends after its
+ upgrade was counted, are not counted (`swap_anywhere` only, for both).
- NAT64 (`64:ff9b::/96`, and its local-use prefix) and 6to4 addresses are judged
by the IPv4 address they carry. Teredo and operator-chosen NAT64 prefixes are
not recognised.
diff --git a/autogpt_platform/swap_proxy/poetry.lock b/autogpt_platform/swap_proxy/poetry.lock
index 27869848e2..5349a799cc 100644
--- a/autogpt_platform/swap_proxy/poetry.lock
+++ b/autogpt_platform/swap_proxy/poetry.lock
@@ -620,6 +620,32 @@ cffi = {version = ">=2.0.0", markers = "platform_python_implementation != \"PyPy
[package.extras]
ssh = ["bcrypt (>=3.1.5)"]
+[[package]]
+name = "fakeredis"
+version = "2.38.0"
+description = "Python implementation of redis API, can be used for testing purposes."
+optional = false
+python-versions = ">=3.8"
+groups = ["dev"]
+files = [
+ {file = "fakeredis-2.38.0-py3-none-any.whl", hash = "sha256:d9fb0518c4eaa35f1f2c94df6b4a4c97ff3ca9f43d6cc8112317a9037d244301"},
+ {file = "fakeredis-2.38.0.tar.gz", hash = "sha256:d2abfd24652f86501044499bf08c9d639db050f695eefc06b8b8b6f0bb24dbd6"},
+]
+
+[package.dependencies]
+lupa = {version = ">=2.1", optional = true, markers = "extra == \"lua\""}
+redis = ">=4.3"
+sortedcontainers = ">=2"
+
+[package.extras]
+bf = ["pyprobables (>=0.6)"]
+cf = ["pyprobables (>=0.6)"]
+json = ["jsonpath-ng (>=1.6)"]
+lua = ["lupa (>=2.1)"]
+probabilistic = ["pyprobables (>=0.6)"]
+valkey = ["valkey (>=6)"]
+vectorset = ["jsonpath-ng (>=1.6) ; python_version >= \"3.11\"", "numpy (>=2.4.0) ; python_version >= \"3.11\""]
+
[[package]]
name = "flask"
version = "3.1.3"
@@ -842,6 +868,83 @@ files = [
[package.dependencies]
pyasn1 = ">=0.4.6"
+[[package]]
+name = "lupa"
+version = "2.8"
+description = "Python wrapper around Lua and LuaJIT"
+optional = false
+python-versions = ">=3.8"
+groups = ["dev"]
+files = [
+ {file = "lupa-2.8-cp310-abi3-win32.whl", hash = "sha256:c2a5fd15dc62374e1661a55f01744c9ec1c56f291ba4a0749d3af2174556e78f"},
+ {file = "lupa-2.8-cp310-abi3-win_arm64.whl", hash = "sha256:9e304fb1c50cf23fd8882afbe1aa87525ef8a72667bcab3b37b2bbb2bc542269"},
+ {file = "lupa-2.8-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:97bd01e90b8031e56a5fd5bb70605aea09f1dba675c1140308a52780f93d06f1"},
+ {file = "lupa-2.8-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b5ebe1a13c45767919c86750b84fe2da9f6288b6f3cea4ce7660bb2abc9d921"},
+ {file = "lupa-2.8-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:097e7d0f1719a88020b67c82e05d53d7973c166952393afcecfd8434c7e19a15"},
+ {file = "lupa-2.8-cp310-cp310-win_amd64.whl", hash = "sha256:7bb223ee8f72d0dc076b0d65296ee72f1c69450f9d2fed5315f7707d98c4a03d"},
+ {file = "lupa-2.8-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b12e43c1fb787189dfc28cd604aef0baa2cb95e27da19498d520361d0ace070a"},
+ {file = "lupa-2.8-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f6f603391dffb256e36a79fd2044084d5f4b8a0a4c0e5ad291cd3ab3aaf1fd0a"},
+ {file = "lupa-2.8-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9f6f41c91366e7d0d474f87d81c1274af861f40812bf729c9f97ab4c8f3c7ac8"},
+ {file = "lupa-2.8-cp311-cp311-win_amd64.whl", hash = "sha256:f5a6af145b0ea818f01d27bfe2583a4b538570bef61d22c8773e0eccf011234c"},
+ {file = "lupa-2.8-cp312-abi3-macosx_10_13_x86_64.whl", hash = "sha256:f4342f4de76ae7ce2ab0672d36003bdb7e1a33252f293b569298ddd792e70e33"},
+ {file = "lupa-2.8-cp312-abi3-manylinux2010_i686.manylinux_2_12_i686.manylinux_2_28_i686.whl", hash = "sha256:4203fa1659315e939a5304e75001b8cc14234fb3cbb3ed86c049b0cc5d90fcee"},
+ {file = "lupa-2.8-cp312-abi3-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:81f2d843ce668b653146c007467570210ae44be51dac6926666c51d49536f307"},
+ {file = "lupa-2.8-cp312-abi3-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d3d0cde2c77588d1c60875a4f34f059513476c6e1775351897195b51e0f3df08"},
+ {file = "lupa-2.8-cp312-abi3-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9e0d11b8f3a8dac6413f704fef7161d048bb10c58bdac6cbffa5e60efa56e9a3"},
+ {file = "lupa-2.8-cp312-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:54cff414f21f8cd8c6be4aae52541f3b9cd39602b59e3a3db9b5c9f9f674ff18"},
+ {file = "lupa-2.8-cp312-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:24b4d8af5558e549b70daf1547f5c1c1d664ecea9fc790f83efe5d75e9a93797"},
+ {file = "lupa-2.8-cp312-abi3-musllinux_1_2_i686.whl", hash = "sha256:ce86dff1ee7f7cf45f5622065ae991949dd7bb1703581cbc58a630137bb7ccf9"},
+ {file = "lupa-2.8-cp312-abi3-musllinux_1_2_ppc64le.whl", hash = "sha256:f4d01b2a08c70bbb883a9e082b6b36b89121ed5910b710f1ba11c73295ff4fba"},
+ {file = "lupa-2.8-cp312-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:7f210d5a8353e510ea1199c42cf3cbdd630553bf2bc8fb4c00fea06fdec7c798"},
+ {file = "lupa-2.8-cp312-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:4f81a02806e7c7ad26d8c6fa222c8bef1b0c1b124347c879be880b41339d41e4"},
+ {file = "lupa-2.8-cp312-abi3-win32.whl", hash = "sha256:360056453a7a4eaa4ac5a204c31a5a014b1eb2ee5490603234d2ba831684f1f2"},
+ {file = "lupa-2.8-cp312-abi3-win_arm64.whl", hash = "sha256:1628371c6592a6d5650497a9e31fb2bb3a7e9883c1f301d1111265e484045af9"},
+ {file = "lupa-2.8-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:450650f91c48c2415b0d59ab3abfcfda3b6efb5b858205f4d4bda8ad141fa529"},
+ {file = "lupa-2.8-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:27044f3363047f946b3d3aab9157cbd172b3538ada9ec1baef43432bf7d03a78"},
+ {file = "lupa-2.8-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8cf4f064a0e5531afce2d7d750120c10c10f9529139af6ca6150d13151034398"},
+ {file = "lupa-2.8-cp312-cp312-win_amd64.whl", hash = "sha256:281bedc5deb92d31e649a3552edd662449365a635904fa4d5cb4509c7245e34e"},
+ {file = "lupa-2.8-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45fc9da0145ecb0083ef5ff9975116cc784bd0258bdc2bd131ba15483ce18398"},
+ {file = "lupa-2.8-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:58e18afed57955b41130e269c78f53d4123ab86e236b53816f4cbffa25cb5d30"},
+ {file = "lupa-2.8-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fc47f536ac13a79cef47d29a2b205576a22841f042a2bcec1676b95806e7706a"},
+ {file = "lupa-2.8-cp313-cp313-win_amd64.whl", hash = "sha256:ce9404c661dbac65cc9bed351ad45e797af93d30d70be309a3fa8209ac86d93b"},
+ {file = "lupa-2.8-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:348c3f8ecabb6324dcbc05c2740d762ef8fcec7b06c79e45262ab97a217684e3"},
+ {file = "lupa-2.8-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:951496471056061598a7d1729a6cdf48d662fec777a9f2d8aa5a1e62fd30e5a5"},
+ {file = "lupa-2.8-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a591b9947ca347b41a63370e121d6e2b1458fe6dde9ae065029ec10a37f25ff4"},
+ {file = "lupa-2.8-cp314-cp314-win_amd64.whl", hash = "sha256:3903c9cf628dae2f56405503247b77a61a3a61bd2dda470e336950c74776d55d"},
+ {file = "lupa-2.8-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f711a8ab0486b9ac6fdda94a22ddcfbc9f0d4a27e3a8cf1bf79c6e48b33017c1"},
+ {file = "lupa-2.8-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:dc51250e76367a3e27fcd01dc769b9bfcbbc34f48df48dde53d6af6e75b7eaa5"},
+ {file = "lupa-2.8-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f8a22088a552828958603323f0a5c4b3e11e03b75d0bf4c965ef879de9b60a8d"},
+ {file = "lupa-2.8-cp314-cp314t-win32.whl", hash = "sha256:4f7c553c1d8cfffbe85d81daef730d12cae4b6002d457542914da0ac8a1145b3"},
+ {file = "lupa-2.8-cp314-cp314t-win_amd64.whl", hash = "sha256:d8766aff03a78c80ad2d188a8bdb216de5ec838359cd87e05bbdfa56394a6105"},
+ {file = "lupa-2.8-cp314-cp314t-win_arm64.whl", hash = "sha256:91d622777febda3ab1bed1d45295f2f32a4680c7b3d7caf8c669998ed5c44118"},
+ {file = "lupa-2.8-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:81b283bfb13cc43fa4910fc98ec110ab861bcb39680f48b266f99d6e3be1049e"},
+ {file = "lupa-2.8-cp38-cp38-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5caf45d15d424cee52fd67341e96e2b1dde0658ae90eb156ac56aa0d8330bc38"},
+ {file = "lupa-2.8-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:33e7e5aebca64b154b0a1679caf79e19254ff37bba51e87abab6848f97cb2de1"},
+ {file = "lupa-2.8-cp38-cp38-win32.whl", hash = "sha256:e8d4f4dd4acf4a0e42adc6b1ad220e1c86fe3028402c2f78bd0728a6d241bbe9"},
+ {file = "lupa-2.8-cp38-cp38-win_amd64.whl", hash = "sha256:1ac2b1ec7504e6148cba1bc35ac36c74d18a0ca6d367ffe7e78a3773c2694c0e"},
+ {file = "lupa-2.8-cp39-abi3-macosx_10_9_x86_64.whl", hash = "sha256:b036738282a5acd2e71fdddb317c9df8b87c1673aa57f403d05fcc2be8abc4ba"},
+ {file = "lupa-2.8-cp39-abi3-manylinux2010_i686.manylinux_2_12_i686.manylinux_2_28_i686.whl", hash = "sha256:ac6b6e8d0e617e26a98cbb44880bcd75de5d32b3ad7b3b3793583909292b47ed"},
+ {file = "lupa-2.8-cp39-abi3-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:ba3a7dd839f90c3d2e53bebe3c192b1f3f9fd720a6781256405123211fd0dce6"},
+ {file = "lupa-2.8-cp39-abi3-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d7edb13a7a5250b5c6c22d1495d9e842b5c9fc5081c8fe6b5efe2112fe3e41f9"},
+ {file = "lupa-2.8-cp39-abi3-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:891f72e0bffbed1e4175f975aeb2a083956586a100066525e1be485f617f7b25"},
+ {file = "lupa-2.8-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:a295f87b5b7ebbfd5191932e8cb0e51df3c7769101ac6b6c7d7c9fb27bfd1307"},
+ {file = "lupa-2.8-cp39-abi3-musllinux_1_2_armv7l.whl", hash = "sha256:4fe5d7a810b64ea8511eb885fc8cdde042ee5ff7b7d08ae78f32449756acb177"},
+ {file = "lupa-2.8-cp39-abi3-musllinux_1_2_i686.whl", hash = "sha256:bfc470012ef66ad064c7bd77416af03a3452ef630b04b9012595ea13f2e54518"},
+ {file = "lupa-2.8-cp39-abi3-musllinux_1_2_ppc64le.whl", hash = "sha256:250e035fdaffe8c87093e3ebc206ac29a26131b1568ea711d780c26001ce96e7"},
+ {file = "lupa-2.8-cp39-abi3-musllinux_1_2_riscv64.whl", hash = "sha256:b9bddb09acfffb4f828f790f444b11dc0cca591afea1a244d9329eea2d20c003"},
+ {file = "lupa-2.8-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:2e64acbbd47e9b82a64405a39e0d2b36a5a7dad8ab41c0f3437f572f7d282ba3"},
+ {file = "lupa-2.8-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f6ddca4774d5ca451768a95e378a3aa041076e29f4613b8562f8e98efb6690fd"},
+ {file = "lupa-2.8-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3ffcfd8e19f943ad459136b3f60f085ae4948f024192a93ca4b4ac3023ec88d8"},
+ {file = "lupa-2.8-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9f3f3955f65f9fde2dc6eda3041ccd394cf54d4bf083f0cdf6feb3d58e5f38d3"},
+ {file = "lupa-2.8-cp39-cp39-win32.whl", hash = "sha256:9e76e45057cfcaa20ee3422c2289a91f9d51783d020da3570ee226de8f6e71cd"},
+ {file = "lupa-2.8-cp39-cp39-win_amd64.whl", hash = "sha256:6fbcc9911f05c67affbd225fc024268e61e98a18ad1b1c2aed6c8796e4056554"},
+ {file = "lupa-2.8-cp39-cp39-win_arm64.whl", hash = "sha256:6c817d5421094507662e5f8feb8cd1e154c10879921c06079b6063be9d8f33c5"},
+ {file = "lupa-2.8-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:32e4e5103bbddcdd2458fb2ccae6c8ba11c9997c711d7e379e0d45551d109c76"},
+ {file = "lupa-2.8-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7667001804657496dee9feced2daae5000b4604a3218dd8e6b7b754982ba88b8"},
+ {file = "lupa-2.8-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:86f6f668966965b15247dc32d064cfe7be67b71e584ccfacbe2f637575296878"},
+ {file = "lupa-2.8.tar.gz", hash = "sha256:d8022641b9ec8ecf2c5ecbe9f47e5a70e0b87c4b5ae921b92cb02a638e0acd08"},
+]
+
[[package]]
name = "markupsafe"
version = "3.0.3"
@@ -1399,7 +1502,7 @@ version = "8.1.0"
description = "Python client for Redis database and key-value store"
optional = false
python-versions = ">=3.10"
-groups = ["main"]
+groups = ["main", "dev"]
files = [
{file = "redis-8.1.0-py3-none-any.whl", hash = "sha256:a4fe1aac3d3b3cc791d4b3d5931c5a956045dc951ee74d1c913ee3ac4d2ee9fb"},
{file = "redis-8.1.0.tar.gz", hash = "sha256:6e1a19beef9225c83efd689c7e6b7da2d5215b1f42cd13b7fc3714d0a09c7b25"},
@@ -1484,7 +1587,7 @@ version = "2.4.0"
description = "Sorted Containers -- Sorted List, Sorted Dict, Sorted Set"
optional = false
python-versions = "*"
-groups = ["main"]
+groups = ["main", "dev"]
files = [
{file = "sortedcontainers-2.4.0-py2.py3-none-any.whl", hash = "sha256:a163dcaede0f1c021485e957a39245190e74249897e2ae4b2aa38595db237ee0"},
{file = "sortedcontainers-2.4.0.tar.gz", hash = "sha256:25caa5a06cc30b6b83d11423433f65d1f9d76c4c6a0c90e3379eaa43b9bfdb88"},
@@ -1708,4 +1811,4 @@ cffi = ["cffi (>=1.17,<2.0) ; platform_python_implementation != \"PyPy\" and pyt
[metadata]
lock-version = "2.1"
python-versions = ">=3.12,<3.14"
-content-hash = "fd37f95c016409583ca7748f27b59d7b8cef351765825b38096134657149be4e"
+content-hash = "5e46dd347aac1ebc071a3d0fd807a8bb194111245b61d8d6c0f790228fc32844"
diff --git a/autogpt_platform/swap_proxy/pyproject.toml b/autogpt_platform/swap_proxy/pyproject.toml
index a0559413f6..a6baa67c86 100644
--- a/autogpt_platform/swap_proxy/pyproject.toml
+++ b/autogpt_platform/swap_proxy/pyproject.toml
@@ -19,6 +19,7 @@ zstandard = ">=0.23.0"
[tool.poetry.group.dev.dependencies]
black = "^24.10.0"
+fakeredis = { version = "^2.32.0", extras = ["lua"] }
isort = "^5.13.2"
pyright = "^1.1.407"
pytest = "^8.4.1"
diff --git a/autogpt_platform/swap_proxy/swap_proxy/__main__.py b/autogpt_platform/swap_proxy/swap_proxy/__main__.py
index 206e45d774..565b8c4043 100644
--- a/autogpt_platform/swap_proxy/swap_proxy/__main__.py
+++ b/autogpt_platform/swap_proxy/swap_proxy/__main__.py
@@ -11,6 +11,7 @@ from redis.asyncio.cluster import ClusterNode, RedisCluster
from swap_proxy.addon import MAX_BODY_BYTES, SwapProxyAddon
from swap_proxy.egress import EgressGuard
from swap_proxy.owners import OwnerDirectory
+from swap_proxy.quota import RequestQuota
from swap_proxy.settings import Settings
from swap_proxy.source import BackendCredentialSource
@@ -110,8 +111,15 @@ async def run() -> None:
redis = connect_redis(settings)
await redis.ping()
source = BackendCredentialSource(settings.backend_url)
+ # Raises on a setting that would quietly weaken the quota: no start.
+ quota = RequestQuota(
+ redis,
+ per_box=settings.quota_per_box,
+ per_user=settings.quota_per_user,
+ window=settings.quota_window_seconds,
+ )
addon = SwapProxyAddon(
- OwnerDirectory(redis), source, EgressGuard(settings.egress_allow)
+ OwnerDirectory(redis), source, EgressGuard(settings.egress_allow), quota=quota
)
master = build_master(
addon, settings.listen_host, settings.listen_port, settings.confdir
diff --git a/autogpt_platform/swap_proxy/swap_proxy/addon.py b/autogpt_platform/swap_proxy/swap_proxy/addon.py
index b2f8ad1f31..d2028ecc6c 100644
--- a/autogpt_platform/swap_proxy/swap_proxy/addon.py
+++ b/autogpt_platform/swap_proxy/swap_proxy/addon.py
@@ -16,7 +16,10 @@ One connection's life:
4. ``request``: placeholders are swapped for the owner's values (``swap.py``),
fetched from the backend for this user and this host (``source.py``). By
default only in the ``Authorization`` header; anywhere else only for a
- credential with ``swap_anywhere``, which nothing sets yet.
+ credential with ``swap_anywhere``, which nothing sets yet. A request that
+ gets a value counts against the box's and the user's quota
+ (``quota.py``); past it, the request is not sent and the box is answered
+ with why.
5. ``response``: known values are scrubbed back into placeholders. Websocket
messages get the same two steps, one per direction (the swap into a
message only with ``swap_anywhere``).
@@ -87,6 +90,7 @@ from OpenSSL import SSL
from swap_proxy.decode import DecodedTooLarge, Undecodable, bounded_decode
from swap_proxy.egress import EgressGuard
from swap_proxy.owners import Owner, OwnerDirectory
+from swap_proxy.quota import QuotaVerdict, RequestQuota
from swap_proxy.source import CredentialSource, SourceUnavailable
from swap_proxy.swap import (
PLACEHOLDER_RE,
@@ -118,6 +122,12 @@ MAX_BODY_BYTES = 5 * 1024 * 1024
MAX_DECODED_BYTES = 4 * MAX_BODY_BYTES
_HEAD_SWAPPED = "swap_proxy_head_swapped"
+# Set by ``requestheaders`` when its swap was over quota: the verdict, which
+# ``request`` answers with. And whether this flow's request was counted.
+_OVER_QUOTA = "swap_proxy_over_quota"
+_COUNTED = "swap_proxy_counted"
+# The response is the proxy's own answer, not a provider's: nothing to scrub.
+_OWN_ANSWER = "swap_proxy_own_answer"
# The credentials whose values went into this flow, by name, and the swapped
# Basic pairs (``RequestSwap.encoded``).
_SWAPPED = "swap_proxy_swapped"
@@ -285,11 +295,14 @@ class SwapProxyAddon:
source: CredentialSource,
guard: EgressGuard,
*,
+ quota: Optional[RequestQuota] = None,
allow_insecure_swap: bool = False,
):
self._directory = owners
self._source = source
self._guard = guard
+ # None: no quota (tests only; ``__main__`` always passes one).
+ self._quota = quota
# Tests only: lets a swap happen over plain http to a local upstream.
self._allow_insecure_swap = allow_insecure_swap
self._owners: weakref.WeakKeyDictionary[connection.Client, Owner] = (
@@ -430,7 +443,18 @@ class SwapProxyAddon:
if names:
lookup = await self._lookup(flow, owner, host, names)
swap = RequestSwap(lookup.credentials, host, request.method, request.path)
+ headers, path = request.headers.copy(), request.path
swap.head(request)
+ verdict = await self._quota_verdict(flow, owner, host, swap)
+ if verdict is not None:
+ # Put the placeholders back and let the request carry on. If
+ # its body turns out not to stream (an HTTP/2 GET, most of all,
+ # has no length and no body), ``request`` answers it with why;
+ # if it streams, it goes out with no value in it and fails at
+ # the provider. Either way nothing is sent with a credential.
+ request.headers, request.path = headers, path
+ flow.metadata[_OVER_QUOTA] = verdict
+ return
self._record_swap(flow, owner, host, lookup, names, swap)
if not is_scrubbable(request.headers.get("content-type", "")):
return # binary: streams through as it is
@@ -480,6 +504,9 @@ class SwapProxyAddon:
# Streamed: its bytes have left. ``requestheaders`` swapped what
# could be swapped; a swap now would reach no wire, only the audit.
return
+ if (over := flow.metadata.get(_OVER_QUOTA)) is not None:
+ self._answer_over_quota(flow, over)
+ return
head = not flow.metadata.get(_HEAD_SWAPPED)
host = request.pretty_host
names = placeholder_names(request, head=head, body=False)
@@ -506,6 +533,10 @@ class SwapProxyAddon:
swap.body(plain)
if plain.raw_content != before:
put_back(request, plain)
+ verdict = await self._quota_verdict(flow, owner, host, swap)
+ if verdict is not None:
+ self._answer_over_quota(flow, verdict)
+ return
self._record_swap(flow, owner, host, lookup, names, swap)
async def websocket_message(self, flow: http.HTTPFlow) -> None:
@@ -593,7 +624,7 @@ class SwapProxyAddon:
async def response(self, flow: http.HTTPFlow) -> None:
owner = self._owners.get(flow.client_conn)
response = flow.response
- if owner is None or response is None:
+ if owner is None or response is None or flow.metadata.get(_OWN_ANSWER):
return
if (
response.status_code == 101
@@ -675,6 +706,42 @@ class SwapProxyAddon:
swapped: dict[str, Credential] = flow.metadata.get(_SWAPPED, {})
return [*credentials, *swapped.values()]
+ async def _quota_verdict(
+ self, flow: http.HTTPFlow, owner: Owner, host: str, swap: RequestSwap
+ ) -> Optional[QuotaVerdict]:
+ """Count a request that got a value, once per flow; the refusal
+ (audited) if it must not be sent with it, else ``None``.
+
+ Called after the swap and before anything leaves. A body swapped
+ later, while it streams (``swap_anywhere`` only), is not counted.
+ """
+ if (
+ self._quota is None
+ or flow.metadata.get(_COUNTED)
+ or not any(e.kind == "swapped" for e in swap.events)
+ ):
+ return None
+ verdict = await self._quota.take(owner.label, owner.user_id)
+ if verdict.allowed:
+ flow.metadata[_COUNTED] = True
+ return None
+ self._audit(
+ owner, host, "refused-request", reason=verdict.reason, scope=verdict.scope
+ )
+ return verdict
+
+ @staticmethod
+ def _answer_over_quota(flow: http.HTTPFlow, verdict: QuotaVerdict) -> None:
+ """Answer instead of the provider, so the box reads why: set in
+ ``request``, the answer is what mitmproxy sends and the request is
+ not."""
+ headers = {"content-type": "text/plain; charset=utf-8"}
+ if verdict.retry_after:
+ headers["retry-after"] = str(verdict.retry_after)
+ status = 429 if verdict.reason == "quota-exceeded" else 503
+ flow.response = http.Response.make(status, verdict.message(), headers)
+ flow.metadata[_OWN_ANSWER] = True
+
def _refuse_response(self, flow: http.HTTPFlow, owner: Owner, reason: str) -> None:
self._audit(owner, flow.request.pretty_host, "refused-response", reason=reason)
if flow.killable:
diff --git a/autogpt_platform/swap_proxy/swap_proxy/addon_test.py b/autogpt_platform/swap_proxy/swap_proxy/addon_test.py
index 6d3ef82ef5..384c5d0029 100644
--- a/autogpt_platform/swap_proxy/swap_proxy/addon_test.py
+++ b/autogpt_platform/swap_proxy/swap_proxy/addon_test.py
@@ -28,6 +28,7 @@ from swap_proxy.addon import (
)
from swap_proxy.egress import EgressGuard
from swap_proxy.owners import Owner, OwnerDirectory
+from swap_proxy.quota import QuotaVerdict
from swap_proxy.source import SourceUnavailable
from swap_proxy.swap import Credential
@@ -71,10 +72,13 @@ def addon_for(
swaps: bool = True,
source: Source | None = None,
anywhere: bool = False,
+ quota: Any = None,
) -> SwapProxyAddon:
source = source or Source()
source.anywhere = anywhere
- addon = SwapProxyAddon(OwnerDirectory(NoRedis()), source, EgressGuard())
+ addon = SwapProxyAddon(
+ OwnerDirectory(NoRedis()), source, EgressGuard(), quota=quota
+ )
addon._owners[flow.client_conn] = Owner(
"session:s-a", "user-a", "sb-1", swaps, box="box-0123456789abcdef"
)
@@ -923,3 +927,149 @@ async def test_a_content_host_is_scrubbed_and_never_sent_the_value(caplog):
assert flow.request.headers["authorization"] == "token hsurr:github"
assert flow.response.text == "stored earlier: hsurr:github"
assert audit(caplog) == [("refused", "hsurr:github"), ("scrubbed", None)]
+
+
+# ------------------------------------------------------------ quotas
+
+
+class Quota:
+ """Answers every count with *verdict* and remembers who was counted."""
+
+ def __init__(self, verdict: QuotaVerdict):
+ self.verdict = verdict
+ self.taken: list[tuple[str, str | None]] = []
+
+ async def take(self, owner_label, user_id):
+ self.taken.append((owner_label, user_id))
+ return self.verdict
+
+
+OVER = QuotaVerdict(
+ reason="quota-exceeded", scope="box", limit=1000, window=3600, retry_after=42
+)
+
+
+async def test_a_credentialed_request_within_quota_is_counted_and_sent(caplog):
+ flow = tflow.tflow()
+ quota = Quota(QuotaVerdict())
+ addon = addon_for(flow, quota=quota)
+ flow.request.headers["authorization"] = "Bearer hsurr:github"
+ with caplog.at_level(logging.INFO, logger="swap_proxy.audit"):
+ await addon.requestheaders(flow)
+ await addon.request(flow)
+ assert flow.request.headers["authorization"] == f"Bearer {TOKEN}"
+ assert flow.response is None
+ assert quota.taken == [("session:s-a", "user-a")]
+ assert audit(caplog) == [("swapped", "hsurr:github")]
+
+
+async def test_a_request_with_nothing_swapped_is_not_counted():
+ flow = tflow.tflow()
+ quota = Quota(OVER)
+ addon = addon_for(flow, quota=quota)
+ await addon.requestheaders(flow)
+ await addon.request(flow)
+ assert quota.taken == [] and flow.response is None
+
+
+ "verdict, status",
+ [(OVER, 429), (QuotaVerdict(reason="quota-unavailable"), 503)],
+ ids=["exceeded", "unavailable"],
+)
+async def test_past_the_quota_the_box_is_answered_and_nothing_is_sent(
+ caplog, verdict, status
+):
+ flow = tflow.tflow()
+ addon = addon_for(flow, quota=Quota(verdict))
+ flow.request.headers["authorization"] = "Bearer hsurr:github"
+ with caplog.at_level(logging.INFO, logger="swap_proxy.audit"):
+ await addon.requestheaders(flow)
+ await addon.request(flow)
+ # An answer set in ``request`` is what mitmproxy sends instead of the
+ # request itself.
+ assert flow.response is not None
+ assert flow.response.status_code == status
+ assert "request not sent" in (flow.response.text or "")
+ if status == 429:
+ assert flow.response.headers["retry-after"] == "42"
+ # Audited once, as refused, and never as swapped.
+ assert audit(caplog) == [("refused-request", verdict.reason)]
+ assert TOKEN not in caplog.text
+
+
[email protected]("which", ["down", "bindings_down"])
+async def test_the_quota_answer_reaches_the_box_when_the_backend_is_down(caplog, which):
+ """The answer is the proxy's own text and carries no value, so a backend
+ that cannot vouch for a scrub must not turn it into a dropped connection."""
+ source = Source()
+ flow = tflow.tflow()
+ addon = addon_for(flow, source=source, quota=Quota(OVER))
+ flow.request.headers["authorization"] = "Bearer hsurr:github"
+ with caplog.at_level(logging.INFO, logger="swap_proxy.audit"):
+ await addon.requestheaders(flow)
+ await addon.request(flow)
+ setattr(source, which, True)
+ # mitmproxy fires both hooks for an answer the addon set itself.
+ await addon.responseheaders(flow)
+ await addon.response(flow)
+ assert flow.error is None
+ assert flow.response is not None and flow.response.status_code == 429
+ assert audit(caplog) == [("refused-request", "quota-exceeded")]
+
+
+def h2_get(flow: http.HTTPFlow) -> None:
+ """An HTTP/2 GET as ``gh`` or curl sends it: no length, no body."""
+ flow.request.http_version = "HTTP/2.0"
+ flow.request.headers.pop("content-length", None)
+ flow.request.headers["authorization"] = "Bearer hsurr:github"
+ flow.request.raw_content = None
+
+
+async def test_past_the_quota_an_http2_get_is_answered_not_reset(caplog):
+ """No length, so ``requestheaders`` swaps its head; it has no body to
+ stream, so ``request`` still fires and answers it with the 429."""
+ flow = tflow.tflow()
+ flow.live = True
+ quota = Quota(OVER)
+ addon = addon_for(flow, quota=quota)
+ h2_get(flow)
+ with caplog.at_level(logging.INFO, logger="swap_proxy.audit"):
+ await addon.requestheaders(flow)
+ # Nothing with a value in it, whatever happens next.
+ assert flow.request.headers["authorization"] == "Bearer hsurr:github"
+ flow.request.raw_content = b"" # the end of the (empty) body
+ await addon.request(flow)
+ assert flow.error is None
+ assert flow.response is not None and flow.response.status_code == 429
+ assert flow.response.headers["retry-after"] == "42"
+ assert "Do not retry in a loop" in (flow.response.text or "")
+ assert quota.taken == [("session:s-a", "user-a")] # counted once
+ assert audit(caplog) == [("refused-request", "quota-exceeded")]
+
+
+async def test_past_the_quota_a_streaming_body_goes_out_with_no_value(caplog):
+ """A body that does stream cannot be answered; its head goes out with the
+ placeholder it came with, and the provider refuses it."""
+ flow = tflow.tflow()
+ addon = addon_for(flow, quota=Quota(OVER))
+ h2_get(flow)
+ with caplog.at_level(logging.INFO, logger="swap_proxy.audit"):
+ await addon.requestheaders(flow)
+ flow.request.stream = True # mitmproxy found the body too large to hold
+ await addon.request(flow)
+ assert flow.request.headers["authorization"] == "Bearer hsurr:github"
+ assert flow.response is None and flow.error is None
+ assert audit(caplog) == [("refused-request", "quota-exceeded")]
+
+
+async def test_a_request_counted_in_requestheaders_is_not_counted_again():
+ flow = tflow.tflow()
+ quota = Quota(QuotaVerdict())
+ addon = addon_for(flow, quota=quota)
+ h2_get(flow)
+ await addon.requestheaders(flow)
+ flow.request.raw_content = b""
+ await addon.request(flow)
+ assert flow.request.headers["authorization"] == f"Bearer {TOKEN}"
+ assert quota.taken == [("session:s-a", "user-a")]
diff --git a/autogpt_platform/swap_proxy/swap_proxy/e2e_test.py b/autogpt_platform/swap_proxy/swap_proxy/e2e_test.py
index 3d9b53af9d..efc42db08b 100644
--- a/autogpt_platform/swap_proxy/swap_proxy/e2e_test.py
+++ b/autogpt_platform/swap_proxy/swap_proxy/e2e_test.py
@@ -12,12 +12,14 @@ import json
import logging
from typing import Optional
+import fakeredis
import pytest
from swap_proxy.__main__ import build_master
from swap_proxy.addon import MAX_BODY_BYTES, SwapProxyAddon
from swap_proxy.egress import EgressGuard
from swap_proxy.owners import CREDENTIAL_KEY_PREFIX, OwnerDirectory
+from swap_proxy.quota import RequestQuota
from swap_proxy.source import SourceUnavailable
from swap_proxy.swap import Credential, host_in_list
@@ -29,10 +31,15 @@ UPSTREAM_HOST = "localhost"
class FakeRedis:
def __init__(self):
self.store: dict[str, str] = {}
+ self.lua = fakeredis.FakeAsyncRedis()
async def get(self, name):
return self.store.get(name)
+ async def eval(self, script, numkeys, *keys_and_args):
+ # The quota's script, run as Lua.
+ return await self.lua.eval(script, numkeys, *keys_and_args)
+
def add_box(
self,
username: str,
@@ -161,12 +168,13 @@ class Upstream:
class Proxy:
- def __init__(self, redis, source, allow, allow_insecure_swap=True):
+ def __init__(self, redis, source, allow, allow_insecure_swap=True, quota=None):
self.redis, self.source = redis, source
self._addon = SwapProxyAddon(
OwnerDirectory(redis),
source,
EgressGuard(allow),
+ quota=quota,
allow_insecure_swap=allow_insecure_swap,
)
@@ -596,3 +604,31 @@ async def test_gh_style_token_authorization_is_swapped(stack):
)
assert upstream.seen[0]["headers"]["authorization"] == f"token {TOKEN_A}"
assert body_of(raw)["headers"]["authorization"] == "token hsurr:github"
+
+
+async def test_past_its_quota_a_box_reads_why_and_the_provider_sees_nothing(tmp_path):
+ redis, source, upstream = FakeRedis(), FakeSource(), Upstream()
+ await upstream.start()
+ quota = RequestQuota(redis, per_box=1, per_user=0, window=3600)
+ proxy = Proxy(redis, source, allow=["127.0.0.0/8", "::1"], quota=quota)
+ await proxy.start(tmp_path)
+ redis.add_box(*BOX_A, "user-a", "session:s-a")
+ try:
+ replies = [
+ await socks5_request(
+ proxy.port,
+ *BOX_A,
+ UPSTREAM_HOST,
+ upstream.port,
+ http_get(headers=BEARER),
+ )
+ for _ in range(2)
+ ]
+ finally:
+ await proxy.stop()
+ await upstream.stop()
+ assert len(upstream.seen) == 1 # the second never left
+ assert replies[1] is not None
+ assert replies[1].startswith(b"HTTP/1.1 429")
+ assert b"request not sent" in replies[1]
+ assert TOKEN_A.encode() not in replies[1]
diff --git a/autogpt_platform/swap_proxy/swap_proxy/quota.py b/autogpt_platform/swap_proxy/swap_proxy/quota.py
new file mode 100644
index 0000000000..e2970d8238
--- /dev/null
+++ b/autogpt_platform/swap_proxy/swap_proxy/quota.py
@@ -0,0 +1,149 @@
+"""How many credentialed requests a box, and a user, may make.
+
+The swap stops a credential leaving the box; it does not stop a box using it.
+A compromised session could walk an entire account through the provider's
+API, so every request that gets a value swapped in is counted: per box (its
+owner, a session or an expert) and per user, in a fixed window. Past either
+limit the request is not sent, and the box gets an answer it can read instead
+of a failure it would retry.
+
+Counted in Redis, which every replica shares, and failing closed: if the
+count cannot be taken, the request goes out with no value in it. Only
+credentialed requests count; everything else a box does is untouched.
+"""
+
+import asyncio
+import logging
+import time
+from dataclasses import dataclass
+from typing import Any, Awaitable, Optional, Protocol
+
+logger = logging.getLogger(__name__)
+
+KEY_PREFIX = "swap:quota:"
+# A count that takes longer is refused rather than held: while Redis is down
+# the client's own timeouts and retries would stall every credentialed request.
+_TAKE_TIMEOUT = 2.0
+# Past the window's edge by this replica's clock: replicas' clocks differ by a
+# little, and a key gone before another replica's window ends would restart its
+# count. The window's index is in the key, so this never carries into the next.
+_EXPIRY_SLACK = 60
+
+
+# One round trip, atomic, over every scope at once: each counter is checked
+# first, and only if all are under their limit are all incremented (a new key
+# gets its expiry in the same step). A request refused on one scope spends
+# nothing on another. Returns 0, or the 1-based index of the scope that
+# refused. KEYS[i] pairs with ARGV[i + 1] (its limit); ARGV[1] is a new key's
+# expiry: the seconds left in its window, plus ``_EXPIRY_SLACK``.
+_TAKE_SCRIPT = """
+for i, key in ipairs(KEYS) do
+ local count = tonumber(redis.call('GET', key) or '0')
+ if count >= tonumber(ARGV[i + 1]) then return i end
+end
+for _, key in ipairs(KEYS) do
+ if redis.call('INCR', key) == 1 then redis.call('EXPIRE', key, ARGV[1]) end
+end
+return 0
+"""
+
+
+class RedisCounter(Protocol):
+ def eval(
+ self, script: str, numkeys: int, *keys_and_args: Any
+ ) -> Awaitable[Any]: ...
+
+
+@dataclass(frozen=True)
+class QuotaVerdict:
+ """*reason* is empty when the request may go; otherwise ``quota-exceeded``
+ (with the *scope* whose *limit* it passed) or ``quota-unavailable``."""
+
+ reason: str = ""
+ scope: str = ""
+ limit: int = 0
+ window: int = 0
+ retry_after: int = 0
+
+ @property
+ def allowed(self) -> bool:
+ return not self.reason
+
+ def message(self) -> str:
+ """What the box reads instead of the provider's answer."""
+ if self.reason == "quota-unavailable":
+ return (
+ "swap-proxy: request not sent. The request quota could not be "
+ "checked, so no credential was attached. Try again shortly.\n"
+ )
+ who = "this sandbox" if self.scope == "box" else "this user's sandboxes"
+ return (
+ f"swap-proxy: request not sent. {who.capitalize()} made "
+ f"{self.limit} requests with the user's connected accounts in "
+ f"{self.window} s, the limit. It resets in {self.retry_after} s. "
+ "Do not retry in a loop: wait, or tell the user the task needs "
+ "more requests than it is allowed.\n"
+ )
+
+
+class RequestQuota:
+ """*per_box* and *per_user* requests per *window* seconds; 0 turns one
+ off. Anything else that would weaken the quota without saying so (a
+ negative limit, a window under a second) is refused here."""
+
+ def __init__(
+ self, redis: RedisCounter, *, per_box: int, per_user: int, window: int
+ ):
+ if per_box < 0 or per_user < 0:
+ raise ValueError("Quota limits must be 0 (off) or more")
+ if window <= 0:
+ raise ValueError("The quota window must be at least a second")
+ self._redis = redis
+ self._per_box = per_box
+ self._per_user = per_user
+ self._window = window
+
+ async def take(
+ self, owner_label: str, user_id: Optional[str], *, now: Optional[float] = None
+ ) -> QuotaVerdict:
+ """Count one credentialed request for this box and user."""
+ now = time.time() if now is None else now
+ index = int(now) // self._window
+ retry_after = self._window - int(now) % self._window
+ # The user id is the hash tag of both keys: a script may only touch
+ # keys of one cluster slot, and it is what both scopes share.
+ tag = "{" + (user_id or owner_label) + "}"
+ scopes = [("box", f"{tag}:box:{owner_label}", self._per_box)]
+ if user_id:
+ scopes.append(("user", f"{tag}:user", self._per_user))
+ scopes = [scope for scope in scopes if scope[2] > 0]
+ if not scopes:
+ return QuotaVerdict()
+ keys = [f"{KEY_PREFIX}{key}:{index}" for _, key, _ in scopes]
+ limits = [limit for _, _, limit in scopes]
+ try:
+ refused = int(
+ await asyncio.wait_for(
+ self._redis.eval(
+ _TAKE_SCRIPT,
+ len(keys),
+ *keys,
+ retry_after + _EXPIRY_SLACK,
+ *limits,
+ ),
+ timeout=_TAKE_TIMEOUT,
+ )
+ )
+ except Exception:
+ logger.warning("Quota count failed; refusing the swap", exc_info=True)
+ return QuotaVerdict(reason="quota-unavailable")
+ if refused:
+ scope, _, limit = scopes[refused - 1]
+ return QuotaVerdict(
+ reason="quota-exceeded",
+ scope=scope,
+ limit=limit,
+ window=self._window,
+ retry_after=retry_after,
+ )
+ return QuotaVerdict()
diff --git a/autogpt_platform/swap_proxy/swap_proxy/quota_test.py b/autogpt_platform/swap_proxy/swap_proxy/quota_test.py
new file mode 100644
index 0000000000..9c6350a14c
--- /dev/null
+++ b/autogpt_platform/swap_proxy/swap_proxy/quota_test.py
@@ -0,0 +1,146 @@
+"""The request quota: counted per box and per user, failing closed."""
+
+import asyncio
+
+import fakeredis
+import pytest
+
+from swap_proxy.quota import RequestQuota
+
+
+class Counter:
+ """A Redis that runs the real ``_TAKE_SCRIPT`` (fakeredis evaluates Lua),
+ recording the scripts it is sent; *fail* makes every call fail."""
+
+ def __init__(self, fail: bool = False):
+ self.redis = fakeredis.FakeAsyncRedis(decode_responses=True)
+ self.fail = fail
+ self.scripts: list[str] = []
+
+ async def eval(self, script, numkeys, *keys_and_args):
+ if self.fail:
+ raise ConnectionError("redis down")
+ self.scripts.append(script)
+ return await self.redis.eval(script, numkeys, *keys_and_args)
+
+ async def counts(self) -> dict[str, int]:
+ keys = await self.redis.keys("*")
+ return {str(k): int(str(await self.redis.get(k))) for k in keys}
+
+ async def ttls(self) -> dict[str, int]:
+ keys = await self.redis.keys("*")
+ return {str(k): await self.redis.ttl(k) for k in keys}
+
+
+NOW = 7200.0 + 600 # 600 s into a window of an hour
+
+
+def quota(redis, per_box=2, per_user=3, window=3600) -> RequestQuota:
+ return RequestQuota(redis, per_box=per_box, per_user=per_user, window=window)
+
+
+async def test_a_box_is_refused_past_its_limit_and_told_when_it_resets():
+ q = quota(Counter())
+ assert (await q.take("session:s-1", "user-a", now=NOW)).allowed
+ assert (await q.take("session:s-1", "user-a", now=NOW)).allowed
+ verdict = await q.take("session:s-1", "user-a", now=NOW)
+ assert not verdict.allowed
+ assert (verdict.reason, verdict.scope, verdict.limit) == (
+ "quota-exceeded",
+ "box",
+ 2,
+ )
+ assert verdict.retry_after == 3000
+ assert "resets in 3000 s" in verdict.message()
+ assert "Do not retry in a loop" in verdict.message()
+
+
+async def test_a_user_is_limited_across_their_boxes():
+ q = quota(Counter())
+ for box in ("session:s-1", "session:s-2", "expert:e-1"):
+ assert (await q.take(box, "user-a", now=NOW)).allowed
+ verdict = await q.take("session:s-3", "user-a", now=NOW)
+ assert (verdict.reason, verdict.scope) == ("quota-exceeded", "user")
+ # Another user's boxes are not touched.
+ assert (await q.take("session:s-9", "user-b", now=NOW)).allowed
+
+
+async def test_the_window_is_fixed_and_the_count_expires_with_it():
+ redis = Counter()
+ q = quota(redis, per_box=1)
+ assert (await q.take("session:s-1", "user-a", now=NOW)).allowed
+ assert not (await q.take("session:s-1", "user-a", now=NOW)).allowed
+ assert (await q.take("session:s-1", "user-a", now=NOW + 3600)).allowed
+ # Set by the script on first increment, and only then: to the window's
+ # edge, so a key made late in a window does not outlive it.
+ assert set((await redis.ttls()).values()) == {3600 - 600 + 60}
+
+
+async def test_a_count_that_hangs_is_refused_not_held(monkeypatch):
+ monkeypatch.setattr("swap_proxy.quota._TAKE_TIMEOUT", 0.01)
+
+ class Hanging:
+ async def eval(self, *args):
+ await asyncio.sleep(10)
+
+ verdict = await quota(Hanging()).take("session:s-1", "user-a", now=NOW)
+ assert verdict.reason == "quota-unavailable"
+
+
+async def test_a_count_that_cannot_be_taken_refuses():
+ verdict = await quota(Counter(fail=True)).take("session:s-1", "user-a", now=NOW)
+ assert verdict.reason == "quota-unavailable"
+ assert "no credential was attached" in verdict.message()
+
+
+ "per_box, per_user, window",
+ [(-1, 3, 3600), (2, -1, 3600), (2, 3, 0), (2, 3, -60)],
+)
+def test_a_value_that_would_quietly_weaken_the_quota_is_refused(
+ per_box, per_user, window
+):
+ with pytest.raises(ValueError):
+ quota(Counter(), per_box=per_box, per_user=per_user, window=window)
+
+
[email protected]("per_box, per_user", [(0, 0)])
+async def test_zero_turns_a_limit_off(per_box, per_user):
+ redis = Counter(fail=True) # never asked
+ q = quota(redis, per_box=per_box, per_user=per_user)
+ assert (await q.take("session:s-1", "user-a", now=NOW)).allowed
+
+
+async def test_a_request_refused_on_the_user_quota_spends_nothing_on_the_box():
+ redis = Counter()
+ q = quota(redis, per_box=5, per_user=1)
+ assert (await q.take("session:s-1", "user-a", now=NOW)).allowed
+ before = await redis.counts()
+ verdict = await q.take("session:s-2", "user-a", now=NOW)
+ assert (verdict.reason, verdict.scope) == ("quota-exceeded", "user")
+ # Neither counter moved: not the refused user's, not the new box's.
+ after = await redis.counts()
+ assert after == before
+ assert not any("session:s-2" in key for key in after)
+
+
+async def test_both_scopes_are_counted_in_one_script_on_one_cluster_slot():
+ redis = Counter()
+ await quota(redis).take("session:s-1", "user-a", now=NOW)
+ assert len(redis.scripts) == 1
+ keys = list(await redis.counts())
+ assert len(keys) == 2
+ # One hash tag, so Redis Cluster runs the script on one slot.
+ assert all("{user-a}" in key for key in keys)
+
+
+async def test_the_script_runs_as_lua_and_refuses_at_the_limit_not_past_it():
+ """The real script, no Python stand-in: with a limit of 2 the third
+ request is refused and the counters stop at 2."""
+ redis = Counter()
+ q = quota(redis, per_box=2, per_user=0)
+ results = [
+ (await q.take("session:s-1", "user-a", now=NOW)).allowed for _ in range(3)
+ ]
+ assert results == [True, True, False]
+ assert list((await redis.counts()).values()) == [2]
diff --git a/autogpt_platform/swap_proxy/swap_proxy/settings.py b/autogpt_platform/swap_proxy/swap_proxy/settings.py
index 940beae9dd..d7c2931391 100644
--- a/autogpt_platform/swap_proxy/swap_proxy/settings.py
+++ b/autogpt_platform/swap_proxy/swap_proxy/settings.py
@@ -43,6 +43,20 @@ class Settings:
egress_allow: list[str] = field(
default_factory=lambda: _list("SWAP_PROXY_EGRESS_ALLOW")
)
+ # Credentialed requests (ones that get a value swapped in) per box and per
+ # user in each window; 0 turns a limit off. Past either, the request is
+ # not sent and the box is told why (``quota.py``).
+ quota_per_box: int = field(
+ default_factory=lambda: int(os.getenv("SWAP_PROXY_QUOTA_PER_BOX", "1000"))
+ )
+ quota_per_user: int = field(
+ default_factory=lambda: int(os.getenv("SWAP_PROXY_QUOTA_PER_USER", "3000"))
+ )
+ quota_window_seconds: int = field(
+ default_factory=lambda: int(
+ os.getenv("SWAP_PROXY_QUOTA_WINDOW_SECONDS", "3600")
+ )
+ )
redis_host: str = field(
default_factory=lambda: os.getenv("REDIS_CLUSTER_HOST")
or os.getenv("REDIS_HOST", "localhost")