Skip to content

Speakr is a personal, self-hosted web application designed for transcribing audio recordings

Python

Count the account rate limit per endpoint, like the token limit, so a single-token client polling several endpoints is never held by it (#484)

Murtaza Nasirce5ce6a10ba4
diff --git a/config/env.asr.example b/config/env.asr.example
index 88b8da77..13905010 100644
--- a/config/env.asr.example
+++ b/config/env.asr.example
@@ -399,8 +399,9 @@ MAX_CONCURRENT_UPLOADS=3
# API_TOKEN_RATE_LIMIT_WRITE=30 per minute
# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute
#
-# Account limits: all tokens of one user share one budget per kind, so another
-# token adds no throughput. Default: twice the token limit of the same kind.
+# Account limits: on each endpoint, all tokens of one user share one budget per
+# kind, so another token adds no throughput. Counted per endpoint, like the
+# token limits. Default: twice the token limit of the same kind.
# API_USER_RATE_LIMIT_READ=240 per minute
# API_USER_RATE_LIMIT_WRITE=60 per minute
# API_USER_RATE_LIMIT_PROCESS=20 per minute
diff --git a/config/env.transcription.example b/config/env.transcription.example
index 031bffe7..2919e3fd 100644
--- a/config/env.transcription.example
+++ b/config/env.transcription.example
@@ -567,8 +567,9 @@ MAX_CONCURRENT_UPLOADS=3
# API_TOKEN_RATE_LIMIT_WRITE=30 per minute
# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute
#
-# Account limits: all tokens of one user share one budget per kind, so another
-# token adds no throughput. Default: twice the token limit of the same kind.
+# Account limits: on each endpoint, all tokens of one user share one budget per
+# kind, so another token adds no throughput. Counted per endpoint, like the
+# token limits. Default: twice the token limit of the same kind.
# API_USER_RATE_LIMIT_READ=240 per minute
# API_USER_RATE_LIMIT_WRITE=60 per minute
# API_USER_RATE_LIMIT_PROCESS=20 per minute
diff --git a/config/env.whisper.example b/config/env.whisper.example
index ea1a6f39..3b08adc2 100644
--- a/config/env.whisper.example
+++ b/config/env.whisper.example
@@ -395,8 +395,9 @@ MAX_CONCURRENT_UPLOADS=3
# API_TOKEN_RATE_LIMIT_WRITE=30 per minute
# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute
#
-# Account limits: all tokens of one user share one budget per kind, so another
-# token adds no throughput. Default: twice the token limit of the same kind.
+# Account limits: on each endpoint, all tokens of one user share one budget per
+# kind, so another token adds no throughput. Counted per endpoint, like the
+# token limits. Default: twice the token limit of the same kind.
# API_USER_RATE_LIMIT_READ=240 per minute
# API_USER_RATE_LIMIT_WRITE=60 per minute
# API_USER_RATE_LIMIT_PROCESS=20 per minute
diff --git a/config/env.whisperx.example b/config/env.whisperx.example
index 90728e01..65c345d2 100644
--- a/config/env.whisperx.example
+++ b/config/env.whisperx.example
@@ -310,8 +310,9 @@ MAX_CONCURRENT_UPLOADS=3
# API_TOKEN_RATE_LIMIT_WRITE=30 per minute
# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute
#
-# Account limits: all tokens of one user share one budget per kind, so another
-# token adds no throughput. Default: twice the token limit of the same kind.
+# Account limits: on each endpoint, all tokens of one user share one budget per
+# kind, so another token adds no throughput. Counted per endpoint, like the
+# token limits. Default: twice the token limit of the same kind.
# API_USER_RATE_LIMIT_READ=240 per minute
# API_USER_RATE_LIMIT_WRITE=60 per minute
# API_USER_RATE_LIMIT_PROCESS=20 per minute
diff --git a/docs/user-guide/api-reference.md b/docs/user-guide/api-reference.md
index b8603382..4720e533 100644
--- a/docs/user-guide/api-reference.md
+++ b/docs/user-guide/api-reference.md
@@ -2042,10 +2042,10 @@ All endpoints return consistent error responses:
## Rate Limits
-Requests made with an API token are held by two limits for each kind of request:
+Requests made with an API token are held by two limits for each kind of request. Both are counted separately for each endpoint:
-- **Token limit.** Each token has its own budget, counted separately for each endpoint, so two integrations with their own tokens do not share it.
-- **Account limit.** All tokens of one account draw on one budget across all endpoints, so a further token adds no throughput.
+- **Token limit.** Each token has its own budget, so two integrations with their own tokens do not share it.
+- **Account limit.** On each endpoint, all tokens of one account draw on one budget, so a further token adds no throughput there.
| Kind | Endpoints | Token limit | Account limit |
|------|-----------|-------------|---------------|
@@ -2053,13 +2053,13 @@ Requests made with an API token are held by two limits for each kind of request:
| Write | Endpoints that need `write`, `upload`, `share`, `delete`, `webhooks` or `account` | 30 requests/minute (`API_TOKEN_RATE_LIMIT_WRITE`) | 60 requests/minute (`API_USER_RATE_LIMIT_WRITE`) |
| Process | Endpoints that need `process` | 10 requests/minute (`API_TOKEN_RATE_LIMIT_PROCESS`) | 20 requests/minute (`API_USER_RATE_LIMIT_PROCESS`) |
-An account limit that is not set is twice the token limit of the same kind, so a client with a single token on one endpoint is held by its token limit. The limits apply to full-access and scoped tokens alike. A request that the token limit refuses does not count against the account limit.
+An account limit that is not set is twice the token limit of the same kind, so a client with a single token is held by its token limit. Because both limits are per endpoint, a client that polls several endpoints, such as a mail client during a sync, is never refused by the account limit while each endpoint stays within its token limit. The limits apply to full-access and scoped tokens alike. A request that the token limit refuses does not count against the account limit.
A request over a limit gets `429` with a `Retry-After` header in seconds. The body names the limit that ran out:
```json
{
- "error": "Rate limit reached for your account: 240 per minute (read requests), shared by all your API tokens.",
+ "error": "Rate limit reached for your account: 240 per minute (read requests on this endpoint), shared by all your API tokens.",
"code": "account_rate_limit_exceeded",
"limit": "240 per 1 minute"
}
diff --git a/docs/user-guide/api-tokens.md b/docs/user-guide/api-tokens.md
index 9a91eaf5..2baa95bb 100644
--- a/docs/user-guide/api-tokens.md
+++ b/docs/user-guide/api-tokens.md
@@ -207,7 +207,7 @@ Click the trash icon next to any token to revoke it immediately. Revoked tokens:
By default an account can hold up to 10 active tokens. Revoked and expired tokens do not count, and revoking a token frees its slot at once. Creating a token beyond the limit gives `409` with `"code": "token_limit_reached"`.
-Requests made with a token are held by a limit for the token and by a limit for the whole account, so spreading work over more tokens adds no throughput. [Rate Limits](api-reference.md#rate-limits) lists the values.
+Requests made with a token are held by a limit for the token and by a limit for the whole account, both counted per endpoint, so spreading work over more tokens adds no throughput on an endpoint. [Rate Limits](api-reference.md#rate-limits) lists the values.
Administrators set these limits with environment variables:
@@ -215,7 +215,7 @@ Administrators set these limits with environment variables:
|----------|---------|---------|
| `API_MAX_TOKENS_PER_USER` | `10` | Active tokens per account; `0` means no limit. Tokens above a lowered limit keep working. |
| `API_TOKEN_RATE_LIMIT_READ`, `API_TOKEN_RATE_LIMIT_WRITE`, `API_TOKEN_RATE_LIMIT_PROCESS` | `120 per minute`, `30 per minute`, `10 per minute` | Budget of each token, per endpoint. |
-| `API_USER_RATE_LIMIT_READ`, `API_USER_RATE_LIMIT_WRITE`, `API_USER_RATE_LIMIT_PROCESS` | Twice the token limit of the same kind | Budget that all tokens of an account share across endpoints. |
+| `API_USER_RATE_LIMIT_READ`, `API_USER_RATE_LIMIT_WRITE`, `API_USER_RATE_LIMIT_PROCESS` | Twice the token limit of the same kind | Budget that all tokens of an account share, per endpoint. |
| `RATELIMIT_STORAGE_URI` | Memory of each worker process | Shared storage for the counters, such as `redis://redis:6379/0`. Without it, each of the three worker processes of the standard image counts separately, so a limit allows up to about three times its value. |
## Security Best Practices
@@ -303,7 +303,7 @@ curl -s -H "Authorization: Bearer $TOKEN" \
### 429 Too Many Requests
-- With `"code": "token_rate_limit_exceeded"`, the token sent more requests than its own limit allows. With `"code": "account_rate_limit_exceeded"`, all tokens of the account together reached the account limit, and another token does not help (see [Rate Limits](api-reference.md#rate-limits)). Wait the number of seconds in the `Retry-After` header.
+- With `"code": "token_rate_limit_exceeded"`, the token sent more requests than its own limit allows. With `"code": "account_rate_limit_exceeded"`, all tokens of the account together reached the account limit on that endpoint, and another token does not help (see [Rate Limits](api-reference.md#rate-limits)). Wait the number of seconds in the `Retry-After` header.
---
diff --git a/src/utils/token_auth.py b/src/utils/token_auth.py
index ab89f1d0..857ba021 100644
--- a/src/utils/token_auth.py
+++ b/src/utils/token_auth.py
@@ -179,9 +179,11 @@ def _doubled(limit_string):
for item in items)
-# Account-wide limits (#484): all tokens of one user draw on one budget per
-# kind, so another token adds no throughput. The default is twice the
-# per-token limit, so a client with a single token is held by its token limit.
+# Account limits (#484): on each endpoint, all tokens of one user draw on one
+# budget per kind, so another token adds no throughput there. Like the token
+# budget it is counted per endpoint, so a client that polls several endpoints
+# is not held by it. The default is twice the per-token limit, so a client
+# with a single token is held by its token limit.
_USER_RATE_LIMITS = {
kind: os.environ.get(f'API_USER_RATE_LIMIT_{kind.upper()}') or _doubled(limit) or limit
for kind, limit in _RATE_LIMITS.items()
@@ -247,9 +249,12 @@ def require_scope(*scopes):
def token_message():
return f"Rate limit reached for this token: {_RATE_LIMITS[category]} ({category} requests)."
+ def user_scope(endpoint):
+ return f"user-{category}"
+
def user_message():
return (f"Rate limit reached for your account: {_USER_RATE_LIMITS[category]} "
- f"({category} requests), shared by all your API tokens.")
+ f"({category} requests on this endpoint), shared by all your API tokens.")
def decorator(f):
state = {'limiter': None, 'limited': None}
@@ -279,8 +284,8 @@ def require_scope(*scopes):
# Both limits register under the view's name, so one check
# evaluates the two. With equal windows the token limit is checked
# first, and a request it refuses is not counted against the
- # account. The token budget is counted per route, as before; the
- # account budget is one shared bucket per kind across all routes.
+ # account. Both are counted per route: the bucket is the endpoint
+ # plus the token or the user.
target = f
if route_view is not None:
target = limiter.limit(route_limit, **getattr(f, '_rate_limit_options', {}))(route_view)
@@ -288,9 +293,9 @@ def require_scope(*scopes):
token_limit, key_func=token_rate_key, exempt_when=is_exempt,
scope=token_scope, error_message=token_message,
)(target)
- return limiter.shared_limit(
- user_limit, scope=f"user-{category}", key_func=user_rate_key,
- exempt_when=is_exempt, override_defaults=False, error_message=user_message,
+ return limiter.limit(
+ user_limit, key_func=user_rate_key, exempt_when=is_exempt,
+ scope=user_scope, error_message=user_message,
)(per_token)
wrapper._required_scopes = required
diff --git a/tests/test_api_token_scopes.py b/tests/test_api_token_scopes.py
index e5087c87..6d6dfb56 100644
--- a/tests/test_api_token_scopes.py
+++ b/tests/test_api_token_scopes.py
@@ -313,16 +313,19 @@ def test_a_single_token_meets_its_own_budget_first(world, live_limiter):
assert refused.get_json()["code"] == "token_rate_limit_exceeded"
-def test_the_account_budget_spans_routes_and_kinds_stay_apart(world, live_limiter):
+def test_one_token_polling_several_endpoints_is_never_held_by_the_account(world, live_limiter):
+ """Both budgets are per endpoint: a sync client at its token limit on each
+ of three endpoints (9 reads, account budget 5) meets only its token limit."""
plain = world["token"](None)
+ paths = ["/api/v1/recordings", "/api/v1/tags", "/api/v1/users/me"]
with app.app_context():
c = _Client(app, app.response_class)
- # The token budget is counted per route, the account budget across them.
- codes = [_read(c, plain).status_code for _ in range(3)]
- codes += [_read(c, plain, "/api/v1/tags").status_code for _ in range(3)]
+ codes = [_read(c, plain, path).status_code for path in paths for _ in range(3)]
+ refused = _read(c, plain, "/api/v1/tags")
write = c.patch(f"/api/v1/recordings/{world['rec']}", json={"title": "Still allowed"},
headers=_h(plain))
- assert codes == [200, 200, 200, 200, 200, 429]
+ assert codes == [200] * 9
+ assert refused.status_code == 429 and refused.get_json()["code"] == "token_rate_limit_exceeded"
assert write.status_code == 200