🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
fix(ai-gateway): keep control-plane connections to web alive (#17894)
Hassieb Pakzadfb60e50df051
diff --git a/ai-gateway/src/resolution/mod.rs b/ai-gateway/src/resolution/mod.rsindex 60a98d271d..23bae11f23 100644--- a/ai-gateway/src/resolution/mod.rs+++ b/ai-gateway/src/resolution/mod.rs@@ -18,6 +18,11 @@ use std::{};const RESOLVE_PATH: &str = "/api/internal/ai-gateway/v1/resolve";+// Web sits behind load balancers that drop idle connections after 60s by default+// (AWS ALB). Evicting pooled connections first keeps the resolver from sending on a+// socket the balancer is closing, which would fail without a retry.+const POOL_IDLE_TIMEOUT: Duration = Duration::from_secs(50);+const TCP_KEEPALIVE: Duration = Duration::from_secs(30);struct ResolutionLimits {timeout: Duration,@@ -98,6 +103,8 @@ impl ControlPlaneClient {.no_brotli().no_zstd().no_deflate()+ .pool_idle_timeout(POOL_IDLE_TIMEOUT)+ .tcp_keepalive(TCP_KEEPALIVE).build().map_err(|_| ResolutionError::Configuration)?;Ok(Self {