~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
Python academiaanthropicarxivbravedeep-researchencryptionhome-automationhomeserverlocallocal-deep-researchlocal-llmmistralollamaopenaipubmedresearchresearch-toolretrieval-augmented-generationsearxngself-hosted
fix(api): honour custom app.api_rate_limit values on /api/v1 (#6400)
Sebastian Cao1115cdba3f16
diff --git a/changelog.d/5988.bugfix.md b/changelog.d/5988.bugfix.mdnew file mode 100644index 0000000000..615b7d9918--- /dev/null+++ b/changelog.d/5988.bugfix.md@@ -0,0 +1,10 @@+Custom `app.api_rate_limit` values are honoured again on `/api/v1/*`. The+FastAPI migration registered the per-user API limit with a static value, so+every user was silently held to the 60/minute default - a lower configured+limit was not enforced and a higher one was throttled at 60. Custom nonzero+values now cap each user at their own value with independent per-user+buckets; the default stays 60/minute and `0` still disables the limit. A+numeric string (what an env override or a text-typed settings row reads back+as, since the setting has no schema entry) is accepted as its integer value,+so `"0"` disables the limit too; anything else that is neither `0` nor a+positive integer is logged and falls back to the default.diff --git a/src/local_deep_research/web/dependencies/rate_limit.py b/src/local_deep_research/web/dependencies/rate_limit.pyindex 7337b24a85..797b4bbe23 100644--- a/src/local_deep_research/web/dependencies/rate_limit.py+++ b/src/local_deep_research/web/dependencies/rate_limit.py@@ -434,34 +434,91 @@ upload_rate_limit_ip = limiter.shared_limit(# ---------------------------------------------------------------------------# /api/v1 per-user rate limiting. Port of main's api_rate_limit shared limit.#-# The limit VALUE is static on purpose. slowapi exempts only routes with-# *static* limits from SlowAPIMiddleware (_should_exempt checks-# _route_limits, not _dynamic_route_limits); a callable limit value makes-# the route dynamic, so the middleware — which runs OUTSIDE SessionMiddleware-# and before route dependencies — would evaluate it with no session (the-# per-user key collapses to per-IP) and before require_api_access caches the-# user's setting. A static value keeps the route exempt so the decorator-# checks it at call time, after the dependency has run, where both the-# session (key) and the cached setting (exempt_when) are available.+# Each decorated route registers TWO limits, of which exactly one is live+# per request:#-# Consequence vs main: the per-user CUSTOM rate value (app.api_rate_limit)-# is not honored — every user gets API_RATE_LIMIT_DEFAULT. Per-user keying-# and the 0-disables-it switch (via exempt_when, below) are preserved.+# * a STATIC default (API_RATE_LIMIT_DEFAULT per minute), enforced while+# the user runs at the default value, and+# * a DYNAMIC (callable) limit, enforced at the user's custom nonzero+# app.api_rate_limit when one is set - this is what restores main's+# per-user custom values, which an earlier revision of this module+# dropped by registering only the static limit (#5988).+#+# The static registration is load-bearing: slowapi's SlowAPIMiddleware+# exempts a route from middleware-time checking only when its name is in+# _route_limits (static limits - see _should_exempt in slowapi/middleware+# .py); _dynamic_route_limits does not count. A dynamic limit registered+# ALONE therefore puts the route under the middleware - which runs OUTSIDE+# SessionMiddleware and before route dependencies, where the per-user key+# collapses to per-IP and the middleware additionally applies the global+# default limits. With the static limit alongside it, the route stays+# middleware-exempt and BOTH limits are checked by the decorator at call+# time (in_middleware=False reads _route_limits AND _dynamic_route_limits),+# after require_api_access has cached the user's setting.+#+# The exempt_when hooks make the two mutually exclusive, so a request is+# never counted against both: value == default → static enforces; custom+# nonzero → dynamic enforces at that value; 0 → both step aside (main's+# kill switch).+#+# The hooks and _api_custom_limit_value trust the cache to hold only 0 or+# a positive int, which set_request_api_rate_limit guarantees (anything+# else is logged and replaced by the default; see its docstring for why+# that matters).# ---------------------------------------------------------------------------API_RATE_LIMIT_DEFAULT = 60 # requests per minute+# How much of an invalid app.api_rate_limit value the warning echoes.+_INVALID_VALUE_LOG_CHARS = 60+# Cached at call time by the api_v1 router's require_api_access dependency# (which already reads the user's settings for the app.enable_api gate).-# ContextVar keeps it request-scoped under asyncio. Consumed by-# _api_exempt at the decorator's call-time check.+# ContextVar keeps it request-scoped under asyncio. Consumed by the+# exempt hooks and _api_custom_limit_value at the decorator's call-time+# check._api_rate_limit_ctx: ContextVar[int] = ContextVar("ldr_api_rate_limit", default=API_RATE_LIMIT_DEFAULT)-def set_request_api_rate_limit(value: int) -> None:- """Cache the authenticated user's app.api_rate_limit for this request."""+def set_request_api_rate_limit(value: object) -> None:+ """Cache the authenticated user's app.api_rate_limit for this request.++ Only 0 (the kill switch) and positive ints are cached as given. A+ string of decimal digits (surrounding whitespace allowed) is cached as+ the int it spells: app.api_rate_limit has no defaults-JSON entry, so+ an env override (LDR_APP_API_RATE_LIMIT=120) or a text-typed settings+ row reaches here as the raw string "120", which main's Flask code+ honoured ("0" therefore disables the limit just as 0 does). The+ default is cached in place of anything else, with a+ warning naming the value, because nothing validates the setting+ upstream and a negative, fractional, boolean or non-numeric value can+ arrive here. Cached as-is, such a value would count as "custom": the+ static limit steps aside, and slowapi, unable to parse the limit+ string built from it, logs an error and skips the dynamic limit -+ leaving the user with no limit at all.+ """+ if isinstance(value, str) and value.strip().isdecimal():+ try:+ value = int(value.strip())+ except ValueError:+ # Past the interpreter's int() digit limit (4300 by default).+ # A string that long is not a rate limit, and the ValueError+ # must not escape as a 500; left as-is for the check below.+ pass+ if isinstance(value, bool) or not isinstance(value, int) or value < 0:+ # The app.* namespace is client-writable, so the echoed value is+ # capped rather than logged whole.+ shown = repr(value)+ if len(shown) > _INVALID_VALUE_LOG_CHARS:+ shown = shown[:_INVALID_VALUE_LOG_CHARS] + "..."+ logger.warning(+ f"Ignoring invalid app.api_rate_limit {shown}: expected 0 "+ "(disabled) or a positive integer; applying the default of "+ f"{API_RATE_LIMIT_DEFAULT} per minute"+ )+ value = API_RATE_LIMIT_DEFAULT_api_rate_limit_ctx.set(value)@@ -473,13 +530,72 @@ def _api_user_key(request: Request) -> str:def _api_exempt() -> bool:- """app.api_rate_limit = 0 disables the limit (parity with main)."""+ """app.api_rate_limit = 0 disables the limit (parity with main).++ Kill switch only - deliberately unaware of custom values, because the+ log-export limiter (routers/research.py) reuses it: a custom API rate+ must not turn off the 10/min export cap.+ """return not _api_rate_limit_ctx.get()-api_rate_limit = limiter.shared_limit(+def _api_custom_limit_active() -> bool:+ """Whether the cached app.api_rate_limit is a custom nonzero value."""+ value = _api_rate_limit_ctx.get()+ return bool(value) and value != API_RATE_LIMIT_DEFAULT+++def _api_default_exempt() -> bool:+ """Skip the static default limit when the user disabled the limit+ (0, via _api_exempt) or set a custom value (the dynamic limit below+ enforces that instead)."""+ return _api_exempt() or _api_custom_limit_active()+++def _api_custom_exempt() -> bool:+ """Skip the dynamic limit unless a custom nonzero value is active."""+ return not _api_custom_limit_active()+++def _api_custom_limit_value() -> str:+ """The user's custom limit in slowapi string form.++ Returns the default string while the custom limit is inactive:+ slowapi parses the provider's return value on every request BEFORE+ consulting exempt_when, so it must always be a valid limit string.+ 0 means "disabled" and is handled by the exempt path (_api_custom_exempt+ skips this limit) rather than by handing slowapi a zero-amount rule, so+ the value returned here is never enforced in that case.+ """+ if _api_custom_limit_active():+ return f"{_api_rate_limit_ctx.get()} per minute"+ return f"{API_RATE_LIMIT_DEFAULT} per minute"+++_api_default_rate_limit = limiter.shared_limit(f"{API_RATE_LIMIT_DEFAULT} per minute",scope="api_v1",key_func=_api_user_key,- exempt_when=_api_exempt,+ exempt_when=_api_default_exempt,)++_api_custom_rate_limit = limiter.shared_limit(+ _api_custom_limit_value,+ scope="api_v1",+ key_func=_api_user_key,+ exempt_when=_api_custom_exempt,+)+++def api_rate_limit(func):+ """Apply the static-default + dynamic-custom limit pair to a route.++ The second decorator call is for its registration side effect only and+ its wrapper is discarded: one slowapi wrapper already checks every+ limit registered under the endpoint's name, and keeping a single+ wrapper preserves ``route.__wrapped__`` as the authored body, which+ tests unwrap through in one hop.+ """+ wrapped = _api_default_rate_limit(func)+ _api_custom_rate_limit(wrapped)+ return wrappeddiff --git a/tests/web/routers/test_api_v1_external_contract.py b/tests/web/routers/test_api_v1_external_contract.pyindex a6d7b3284c..42bd02339f 100644--- a/tests/web/routers/test_api_v1_external_contract.py+++ b/tests/web/routers/test_api_v1_external_contract.py@@ -94,7 +94,7 @@ from limits import parse as parse_rate_limitfrom local_deep_research.web.dependencies.rate_limit import (API_RATE_LIMIT_DEFAULT,- _api_exempt,+ _api_default_exempt,limiter,)from local_deep_research.web.routers import api_v1@@ -1040,8 +1040,8 @@ def api_limit_of_two():Only the configured VALUE is swapped on the live ``Limit`` objects —``key_func`` (per-user bucket) and ``exempt_when``- (``_api_exempt``) stay the real ones, so what the test measures is- the production limiter, not a stand-in. Also forces+ (``_api_default_exempt``) stay the real ones, so what the test measures+ is the production limiter, not a stand-in. Also forces``limiter.enabled`` on, because ``_RATE_LIMITING_ENABLED`` isresolved at import time and CI may have disabled it."""@@ -1058,8 +1058,9 @@ def api_limit_of_two():assert lim.scope == "api_v1", (f"{key} must share the 'api_v1' bucket, got {lim.scope!r}")- assert lim.exempt_when is _api_exempt, (- f"{key} lost the 0-disables-it exemption hook"+ assert lim.exempt_when is _api_default_exempt, (+ f"{key} lost the 0-disables-it / custom-value-handoff "+ "exemption hook")saved[key] = [lim.limit for lim in registered]for lim in registered:@@ -1164,29 +1165,22 @@ class TestApiRateLimitOverHttp:assert 429 in statuses, statuses- @pytest.mark.xfail(- strict=True,- reason=(- "Known, deliberate divergence from main: the @api_rate_limit "- "shared limit is registered with a STATIC value, so a user's "- "custom app.api_rate_limit is never honoured — everyone gets "- "API_RATE_LIMIT_DEFAULT (60/min). rate_limit.py documents why "- "(a callable limit makes the route 'dynamic', which un-exempts "- "it from SlowAPIMiddleware, and the middleware runs before the "- "session and before require_api_access caches the value). This "- "xfail is the tripwire: if per-user values are ever restored, "- "it XPASSes and fails the build so the deviation note can be "- "removed."- ),- )+ @pytest.mark.parametrize("value", [2, "2"], ids=["int", "numeric-string"])def test_a_custom_per_user_rate_limit_value_is_honoured(- self, authenticated_client+ self, authenticated_client, value):+ """A custom nonzero ``app.api_rate_limit`` caps the user at THAT+ value (#5988). Runs against the real registered limits - no+ ``api_limit_of_two`` - so what is measured is the dynamic custom+ limit ``rate_limit.py`` registers alongside the static default,+ not a swapped-in stand-in. The string form is what an env override+ or a text-typed settings row reads back as (the key has no+ defaults entry, so nothing converts it); main honoured it too."""original_enabled = limiter.enabledlimiter.enabled = Truelimiter.reset()try:- with _api_settings(api_rate_limit=2):+ with _api_settings(api_rate_limit=value):statuses = [authenticated_client.get(DOCS).status_code for _ in range(4)]@@ -1195,5 +1189,30 @@ class TestApiRateLimitOverHttp:limiter.reset()assert 429 in statuses, (- f"app.api_rate_limit=2 should cap this user at 2/min; {statuses}"+ f"app.api_rate_limit={value!r} should cap this user at 2/min; "+ f"{statuses}")++ @pytest.mark.parametrize(+ "bad_value",+ [-5, "abc", "12a", "9" * 5000],+ ids=["negative", "non-numeric", "mixed-string", "over-int-digit-limit"],+ )+ def test_an_invalid_setting_falls_back_to_the_default_limit(+ self, authenticated_client, api_limit_of_two, bad_value+ ):+ """``app.api_rate_limit`` has no settings-schema entry, so a+ negative or non-numeric value can be stored and read back. It must+ NOT count as a custom value: that would exempt the static default+ and hand the dynamic limit a string slowapi cannot parse, which it+ logs and skips - no limit at all. (``"12a"`` is the near miss for+ the digit-string coercion above; the 5000-digit string is past+ ``int()``'s conversion limit and must not 500 the request.)+ ``api_limit_of_two`` pins the static default at 2/min, so the 429+ proves the user stayed on it."""+ with _api_settings(api_rate_limit=bad_value):+ statuses = [+ authenticated_client.get(DOCS).status_code for _ in range(6)+ ]++ assert 429 in statuses, statusesdiff --git a/tests/web/test_rate_limit_census.py b/tests/web/test_rate_limit_census.pyindex d134da3aa6..64def76dcc 100644--- a/tests/web/test_rate_limit_census.py+++ b/tests/web/test_rate_limit_census.py@@ -56,10 +56,14 @@ from starlette.requests import Request# Importing the app registers every router's limits with the limiter.from local_deep_research.web.fastapi_app import appfrom local_deep_research.web.dependencies.rate_limit import (+ API_RATE_LIMIT_DEFAULT,DEFAULT_RATE_LIMIT,+ _api_rate_limit_ctx,limiter,+ set_request_api_rate_limit,)from local_deep_research.web.routers import (+ api_v1,auth,benchmark,chat,@@ -78,7 +82,13 @@ Bucket = namedtuple("Bucket", "scope key exempt limits")PER_URL = "" # sentinel: no shared scope -> bucket is the request path# --- shared buckets (limiter.shared_limit(scope=...)) -----------------------API_V1 = Bucket("api_v1", "_api_user_key", "_api_exempt", ("60 per 1 minute",))+# The static default half of the api_v1 pair; the dynamic custom-value+# limit registered alongside it (#5988) lives in _dynamic_route_limits,+# which _observed() deliberately does not read - the census pins the+# static registrations that keep these routes middleware-exempt.+API_V1 = Bucket(+ "api_v1", "_api_user_key", "_api_default_exempt", ("60 per 1 minute",)+)LOG_EXPORT = Bucket("log_export", "_api_user_key", "_log_export_exempt", ("10 per 1 minute",))@@ -560,6 +570,184 @@ class TestSharedBucketsAreEnforced:ip=ip,)+ def test_api_v1_custom_values_are_enforced_per_user(self, check):+ """#5988: a custom nonzero ``app.api_rate_limit`` caps each user+ at THEIR value. Alice at 2/min and Bob at 5/min (same IP) drain+ independent counters - Alice's exhaustion does not touch Bob, and+ Bob's cap is his five, not Alice's two or the static default's+ sixty. The ContextVar stands in for ``require_api_access``, which+ caches the setting there before the decorator's check runs."""+ alice = _unique_user("apiv1-two")+ bob = _unique_user("apiv1-five")+ ip = _unique_ip()++ token = _api_rate_limit_ctx.set(2)+ try:+ for _ in range(2):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=alice,+ ip=ip,+ method="GET",+ )+ with pytest.raises(RateLimitExceeded):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=alice,+ ip=ip,+ method="GET",+ )++ _api_rate_limit_ctx.set(5)+ for _ in range(5):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=bob,+ ip=ip,+ method="GET",+ )+ with pytest.raises(RateLimitExceeded):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=bob,+ ip=ip,+ method="GET",+ )+ finally:+ _api_rate_limit_ctx.reset(token)++ @pytest.mark.parametrize(+ "raw,limit",+ [("120", 120), (" 2 ", 2)],+ ids=["env-override-string", "padded-string"],+ )+ def test_api_v1_numeric_strings_are_honoured(+ self, check, loguru_caplog, raw, limit+ ):+ """``app.api_rate_limit`` has no defaults-JSON entry, so an env+ override (``LDR_APP_API_RATE_LIMIT=120``) or a text-typed settings+ row reaches ``set_request_api_rate_limit`` as the raw string. Main+ honoured it (``f"{value} per minute"``), so the string is cached as+ the int it spells and enforced at THAT value, with no warning."""+ user = _unique_user("apiv1-str")+ ip = _unique_ip()++ token = _api_rate_limit_ctx.set(API_RATE_LIMIT_DEFAULT)+ try:+ set_request_api_rate_limit(raw)+ assert _api_rate_limit_ctx.get() == limit+ for _ in range(limit):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=user,+ ip=ip,+ method="GET",+ )+ with pytest.raises(RateLimitExceeded):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=user,+ ip=ip,+ method="GET",+ )+ finally:+ _api_rate_limit_ctx.reset(token)++ assert "invalid app.api_rate_limit" not in loguru_caplog.text++ @pytest.mark.parametrize(+ "bad_value",+ [-5, 2.5, "abc", "12a", None],+ ids=["negative", "fractional", "non-numeric", "mixed-string", "none"],+ )+ def test_api_v1_invalid_values_fall_back_to_the_default(+ self, check, loguru_caplog, bad_value+ ):+ """``app.api_rate_limit`` has no settings-schema entry, so a+ negative, fractional, non-numeric or null value can reach+ ``set_request_api_rate_limit``. It must be logged and the user held+ at the static default, NOT treated as a custom value: that would+ exempt the static limit and hand the dynamic one a limit string+ slowapi cannot parse, which it logs and skips - no limit at all."""+ user = _unique_user("apiv1-bad")+ ip = _unique_ip()++ token = _api_rate_limit_ctx.set(API_RATE_LIMIT_DEFAULT)+ try:+ set_request_api_rate_limit(bad_value)+ for _ in range(API_RATE_LIMIT_DEFAULT):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=user,+ ip=ip,+ method="GET",+ )+ with pytest.raises(RateLimitExceeded):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=user,+ ip=ip,+ method="GET",+ )+ assert _api_rate_limit_ctx.get() == API_RATE_LIMIT_DEFAULT+ finally:+ _api_rate_limit_ctx.reset(token)++ assert f"invalid app.api_rate_limit {bad_value!r}" in loguru_caplog.text++ @pytest.mark.parametrize(+ "huge", ["x" * 4096, "9" * 5000], ids=["text", "digits"]+ )+ def test_api_v1_invalid_value_warning_is_bounded(self, loguru_caplog, huge):+ """The ``app.*`` namespace is client-writable, so the warning must+ not echo a multi-KB value whole: ``repr`` is cut at 60 characters+ and the default is still what gets cached. The digit string is+ also past ``int()``'s 4300-digit conversion limit, so it must be+ refused like any other garbage rather than raise out of the+ request."""+ token = _api_rate_limit_ctx.set(API_RATE_LIMIT_DEFAULT)+ try:+ set_request_api_rate_limit(huge)+ assert _api_rate_limit_ctx.get() == API_RATE_LIMIT_DEFAULT+ finally:+ _api_rate_limit_ctx.reset(token)++ assert "invalid app.api_rate_limit '" + huge[:59] + "..." in (+ loguru_caplog.text+ )+ assert huge not in loguru_caplog.text++ @pytest.mark.parametrize("zero", [0, "0"], ids=["int", "numeric-string"])+ def test_api_v1_zero_still_disables_the_limit(self, check, zero):+ """The kill switch survives the validation above: ``0`` is cached+ as-is (and ``"0"`` as 0, like any digit string) and exempts both+ halves of the api_v1 pair, so the user is never counted, let alone+ cut off past the static default."""+ user = _unique_user("apiv1-zero")+ ip = _unique_ip()++ token = _api_rate_limit_ctx.set(API_RATE_LIMIT_DEFAULT)+ try:+ set_request_api_rate_limit(zero)+ for _ in range(API_RATE_LIMIT_DEFAULT + 1):+ check(+ api_v1.api_documentation,+ "/api/v1/",+ username=user,+ ip=ip,+ method="GET",+ )+ finally:+ _api_rate_limit_ctx.reset(token)+def test_journal_read_endpoints_drain_one_bucket(self, check):"""Regression fence for the fix documented in metrics.py."""user = _unique_user("journals")diff --git a/tests/web/test_rate_limit_coverage.py b/tests/web/test_rate_limit_coverage.pyindex 3fc00e46e2..15be8b5fb5 100644--- a/tests/web/test_rate_limit_coverage.py+++ b/tests/web/test_rate_limit_coverage.py@@ -70,14 +70,22 @@ class TestLimitRegistry:assert any(m.cls is SlowAPIMiddleware for m in app.user_middleware)- def test_no_route_uses_a_dynamic_limit(self):- """Regression fence: a callable (dynamic) limit value makes a route- ineligible for SlowAPIMiddleware exemption, so it gets checked at- middleware time — OUTSIDE SessionMiddleware and before route- dependencies, where the per-user key collapses to per-IP and any- dependency-cached state is missing. Every limited route must use a- static limit value so the decorator checks it at call time."""- assert limiter._dynamic_route_limits == {}+ def test_every_dynamic_limit_rides_on_a_static_registration(self):+ """Regression fence: SlowAPIMiddleware's exemption looks only at+ _route_limits (static), so a route whose ONLY limit is a callable+ (dynamic) one gets checked at middleware time - OUTSIDE+ SessionMiddleware and before route dependencies, where the+ per-user key collapses to per-IP and any dependency-cached state+ is missing. A dynamic limit is allowed only alongside a static+ limit on the same endpoint (the api_v1 custom-value pattern,+ #5988): the static registration keeps the route middleware-exempt,+ and the decorator then checks BOTH at call time."""+ unpaired = {+ name+ for name in limiter._dynamic_route_limits+ if name not in limiter._route_limits+ }+ assert unpaired == set()@pytest.mark.parametrize("name",