Skip to content

armgit / letta

Public Imported from GitHub
0

Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.

fix: Bound async queue during streaming (#5976)

Matthew Zhoudbad510a6e25
diff --git a/letta/server/rest_api/streaming_response.py b/letta/server/rest_api/streaming_response.py
index 5e400d632..385caca9f 100644
--- a/letta/server/rest_api/streaming_response.py
+++ b/letta/server/rest_api/streaming_response.py
@@ -53,8 +53,9 @@ async def add_keepalive_to_stream(
Yields:
Original stream chunks interspersed with keepalive messages
"""
- # Use a queue to decouple the stream reading from keepalive timing
- queue = asyncio.Queue()
+ # Use a bounded queue to decouple reading from keepalive while preserving backpressure
+ # A small maxsize prevents unbounded memory growth if the client is slow
+ queue = asyncio.Queue(maxsize=1)
stream_exhausted = False
async def stream_reader():