Speakr is a personal, self-hosted web application designed for transcribing audio recordings
Python
Count the account rate limit per endpoint, like the token limit, so a single-token client polling several endpoints is never held by it (#484)
Murtaza Nasirce5ce6a10ba4
diff --git a/config/env.asr.example b/config/env.asr.exampleindex 88b8da77..13905010 100644--- a/config/env.asr.example+++ b/config/env.asr.example@@ -399,8 +399,9 @@ MAX_CONCURRENT_UPLOADS=3# API_TOKEN_RATE_LIMIT_WRITE=30 per minute# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute#-# Account limits: all tokens of one user share one budget per kind, so another-# token adds no throughput. Default: twice the token limit of the same kind.+# Account limits: on each endpoint, all tokens of one user share one budget per+# kind, so another token adds no throughput. Counted per endpoint, like the+# token limits. Default: twice the token limit of the same kind.# API_USER_RATE_LIMIT_READ=240 per minute# API_USER_RATE_LIMIT_WRITE=60 per minute# API_USER_RATE_LIMIT_PROCESS=20 per minutediff --git a/config/env.transcription.example b/config/env.transcription.exampleindex 031bffe7..2919e3fd 100644--- a/config/env.transcription.example+++ b/config/env.transcription.example@@ -567,8 +567,9 @@ MAX_CONCURRENT_UPLOADS=3# API_TOKEN_RATE_LIMIT_WRITE=30 per minute# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute#-# Account limits: all tokens of one user share one budget per kind, so another-# token adds no throughput. Default: twice the token limit of the same kind.+# Account limits: on each endpoint, all tokens of one user share one budget per+# kind, so another token adds no throughput. Counted per endpoint, like the+# token limits. Default: twice the token limit of the same kind.# API_USER_RATE_LIMIT_READ=240 per minute# API_USER_RATE_LIMIT_WRITE=60 per minute# API_USER_RATE_LIMIT_PROCESS=20 per minutediff --git a/config/env.whisper.example b/config/env.whisper.exampleindex ea1a6f39..3b08adc2 100644--- a/config/env.whisper.example+++ b/config/env.whisper.example@@ -395,8 +395,9 @@ MAX_CONCURRENT_UPLOADS=3# API_TOKEN_RATE_LIMIT_WRITE=30 per minute# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute#-# Account limits: all tokens of one user share one budget per kind, so another-# token adds no throughput. Default: twice the token limit of the same kind.+# Account limits: on each endpoint, all tokens of one user share one budget per+# kind, so another token adds no throughput. Counted per endpoint, like the+# token limits. Default: twice the token limit of the same kind.# API_USER_RATE_LIMIT_READ=240 per minute# API_USER_RATE_LIMIT_WRITE=60 per minute# API_USER_RATE_LIMIT_PROCESS=20 per minutediff --git a/config/env.whisperx.example b/config/env.whisperx.exampleindex 90728e01..65c345d2 100644--- a/config/env.whisperx.example+++ b/config/env.whisperx.example@@ -310,8 +310,9 @@ MAX_CONCURRENT_UPLOADS=3# API_TOKEN_RATE_LIMIT_WRITE=30 per minute# API_TOKEN_RATE_LIMIT_PROCESS=10 per minute#-# Account limits: all tokens of one user share one budget per kind, so another-# token adds no throughput. Default: twice the token limit of the same kind.+# Account limits: on each endpoint, all tokens of one user share one budget per+# kind, so another token adds no throughput. Counted per endpoint, like the+# token limits. Default: twice the token limit of the same kind.# API_USER_RATE_LIMIT_READ=240 per minute# API_USER_RATE_LIMIT_WRITE=60 per minute# API_USER_RATE_LIMIT_PROCESS=20 per minutediff --git a/docs/user-guide/api-reference.md b/docs/user-guide/api-reference.mdindex b8603382..4720e533 100644--- a/docs/user-guide/api-reference.md+++ b/docs/user-guide/api-reference.md@@ -2042,10 +2042,10 @@ All endpoints return consistent error responses:## Rate Limits-Requests made with an API token are held by two limits for each kind of request:+Requests made with an API token are held by two limits for each kind of request. Both are counted separately for each endpoint:-- **Token limit.** Each token has its own budget, counted separately for each endpoint, so two integrations with their own tokens do not share it.-- **Account limit.** All tokens of one account draw on one budget across all endpoints, so a further token adds no throughput.+- **Token limit.** Each token has its own budget, so two integrations with their own tokens do not share it.+- **Account limit.** On each endpoint, all tokens of one account draw on one budget, so a further token adds no throughput there.| Kind | Endpoints | Token limit | Account limit ||------|-----------|-------------|---------------|@@ -2053,13 +2053,13 @@ Requests made with an API token are held by two limits for each kind of request:| Write | Endpoints that need `write`, `upload`, `share`, `delete`, `webhooks` or `account` | 30 requests/minute (`API_TOKEN_RATE_LIMIT_WRITE`) | 60 requests/minute (`API_USER_RATE_LIMIT_WRITE`) || Process | Endpoints that need `process` | 10 requests/minute (`API_TOKEN_RATE_LIMIT_PROCESS`) | 20 requests/minute (`API_USER_RATE_LIMIT_PROCESS`) |-An account limit that is not set is twice the token limit of the same kind, so a client with a single token on one endpoint is held by its token limit. The limits apply to full-access and scoped tokens alike. A request that the token limit refuses does not count against the account limit.+An account limit that is not set is twice the token limit of the same kind, so a client with a single token is held by its token limit. Because both limits are per endpoint, a client that polls several endpoints, such as a mail client during a sync, is never refused by the account limit while each endpoint stays within its token limit. The limits apply to full-access and scoped tokens alike. A request that the token limit refuses does not count against the account limit.A request over a limit gets `429` with a `Retry-After` header in seconds. The body names the limit that ran out:```json{- "error": "Rate limit reached for your account: 240 per minute (read requests), shared by all your API tokens.",+ "error": "Rate limit reached for your account: 240 per minute (read requests on this endpoint), shared by all your API tokens.","code": "account_rate_limit_exceeded","limit": "240 per 1 minute"}diff --git a/docs/user-guide/api-tokens.md b/docs/user-guide/api-tokens.mdindex 9a91eaf5..2baa95bb 100644--- a/docs/user-guide/api-tokens.md+++ b/docs/user-guide/api-tokens.md@@ -207,7 +207,7 @@ Click the trash icon next to any token to revoke it immediately. Revoked tokens:By default an account can hold up to 10 active tokens. Revoked and expired tokens do not count, and revoking a token frees its slot at once. Creating a token beyond the limit gives `409` with `"code": "token_limit_reached"`.-Requests made with a token are held by a limit for the token and by a limit for the whole account, so spreading work over more tokens adds no throughput. [Rate Limits](api-reference.md#rate-limits) lists the values.+Requests made with a token are held by a limit for the token and by a limit for the whole account, both counted per endpoint, so spreading work over more tokens adds no throughput on an endpoint. [Rate Limits](api-reference.md#rate-limits) lists the values.Administrators set these limits with environment variables:@@ -215,7 +215,7 @@ Administrators set these limits with environment variables:|----------|---------|---------|| `API_MAX_TOKENS_PER_USER` | `10` | Active tokens per account; `0` means no limit. Tokens above a lowered limit keep working. || `API_TOKEN_RATE_LIMIT_READ`, `API_TOKEN_RATE_LIMIT_WRITE`, `API_TOKEN_RATE_LIMIT_PROCESS` | `120 per minute`, `30 per minute`, `10 per minute` | Budget of each token, per endpoint. |-| `API_USER_RATE_LIMIT_READ`, `API_USER_RATE_LIMIT_WRITE`, `API_USER_RATE_LIMIT_PROCESS` | Twice the token limit of the same kind | Budget that all tokens of an account share across endpoints. |+| `API_USER_RATE_LIMIT_READ`, `API_USER_RATE_LIMIT_WRITE`, `API_USER_RATE_LIMIT_PROCESS` | Twice the token limit of the same kind | Budget that all tokens of an account share, per endpoint. || `RATELIMIT_STORAGE_URI` | Memory of each worker process | Shared storage for the counters, such as `redis://redis:6379/0`. Without it, each of the three worker processes of the standard image counts separately, so a limit allows up to about three times its value. |## Security Best Practices@@ -303,7 +303,7 @@ curl -s -H "Authorization: Bearer $TOKEN" \### 429 Too Many Requests-- With `"code": "token_rate_limit_exceeded"`, the token sent more requests than its own limit allows. With `"code": "account_rate_limit_exceeded"`, all tokens of the account together reached the account limit, and another token does not help (see [Rate Limits](api-reference.md#rate-limits)). Wait the number of seconds in the `Retry-After` header.+- With `"code": "token_rate_limit_exceeded"`, the token sent more requests than its own limit allows. With `"code": "account_rate_limit_exceeded"`, all tokens of the account together reached the account limit on that endpoint, and another token does not help (see [Rate Limits](api-reference.md#rate-limits)). Wait the number of seconds in the `Retry-After` header.---diff --git a/src/utils/token_auth.py b/src/utils/token_auth.pyindex ab89f1d0..857ba021 100644--- a/src/utils/token_auth.py+++ b/src/utils/token_auth.py@@ -179,9 +179,11 @@ def _doubled(limit_string):for item in items)-# Account-wide limits (#484): all tokens of one user draw on one budget per-# kind, so another token adds no throughput. The default is twice the-# per-token limit, so a client with a single token is held by its token limit.+# Account limits (#484): on each endpoint, all tokens of one user draw on one+# budget per kind, so another token adds no throughput there. Like the token+# budget it is counted per endpoint, so a client that polls several endpoints+# is not held by it. The default is twice the per-token limit, so a client+# with a single token is held by its token limit._USER_RATE_LIMITS = {kind: os.environ.get(f'API_USER_RATE_LIMIT_{kind.upper()}') or _doubled(limit) or limitfor kind, limit in _RATE_LIMITS.items()@@ -247,9 +249,12 @@ def require_scope(*scopes):def token_message():return f"Rate limit reached for this token: {_RATE_LIMITS[category]} ({category} requests)."+ def user_scope(endpoint):+ return f"user-{category}"+def user_message():return (f"Rate limit reached for your account: {_USER_RATE_LIMITS[category]} "- f"({category} requests), shared by all your API tokens.")+ f"({category} requests on this endpoint), shared by all your API tokens.")def decorator(f):state = {'limiter': None, 'limited': None}@@ -279,8 +284,8 @@ def require_scope(*scopes):# Both limits register under the view's name, so one check# evaluates the two. With equal windows the token limit is checked# first, and a request it refuses is not counted against the- # account. The token budget is counted per route, as before; the- # account budget is one shared bucket per kind across all routes.+ # account. Both are counted per route: the bucket is the endpoint+ # plus the token or the user.target = fif route_view is not None:target = limiter.limit(route_limit, **getattr(f, '_rate_limit_options', {}))(route_view)@@ -288,9 +293,9 @@ def require_scope(*scopes):token_limit, key_func=token_rate_key, exempt_when=is_exempt,scope=token_scope, error_message=token_message,)(target)- return limiter.shared_limit(- user_limit, scope=f"user-{category}", key_func=user_rate_key,- exempt_when=is_exempt, override_defaults=False, error_message=user_message,+ return limiter.limit(+ user_limit, key_func=user_rate_key, exempt_when=is_exempt,+ scope=user_scope, error_message=user_message,)(per_token)wrapper._required_scopes = requireddiff --git a/tests/test_api_token_scopes.py b/tests/test_api_token_scopes.pyindex e5087c87..6d6dfb56 100644--- a/tests/test_api_token_scopes.py+++ b/tests/test_api_token_scopes.py@@ -313,16 +313,19 @@ def test_a_single_token_meets_its_own_budget_first(world, live_limiter):assert refused.get_json()["code"] == "token_rate_limit_exceeded"-def test_the_account_budget_spans_routes_and_kinds_stay_apart(world, live_limiter):+def test_one_token_polling_several_endpoints_is_never_held_by_the_account(world, live_limiter):+ """Both budgets are per endpoint: a sync client at its token limit on each+ of three endpoints (9 reads, account budget 5) meets only its token limit."""plain = world["token"](None)+ paths = ["/api/v1/recordings", "/api/v1/tags", "/api/v1/users/me"]with app.app_context():c = _Client(app, app.response_class)- # The token budget is counted per route, the account budget across them.- codes = [_read(c, plain).status_code for _ in range(3)]- codes += [_read(c, plain, "/api/v1/tags").status_code for _ in range(3)]+ codes = [_read(c, plain, path).status_code for path in paths for _ in range(3)]+ refused = _read(c, plain, "/api/v1/tags")write = c.patch(f"/api/v1/recordings/{world['rec']}", json={"title": "Still allowed"},headers=_h(plain))- assert codes == [200, 200, 200, 200, 200, 429]+ assert codes == [200] * 9+ assert refused.status_code == 429 and refused.get_json()["code"] == "token_rate_limit_exceeded"assert write.status_code == 200