Skip to content

armgit / dify

Public Imported from GitHub
0

Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.

fix(agent): bound Redis run event streams (#41566)

zyssyz123fe39211aac11
diff --git a/dify-agent/.example.env b/dify-agent/.example.env
index 50294c9bf85..24d3390f7ec 100644
--- a/dify-agent/.example.env
+++ b/dify-agent/.example.env
@@ -11,8 +11,17 @@ DIFY_AGENT_REDIS_PREFIX=dify-agent
# Shutdown and retention
# Seconds to wait for active local runs during graceful shutdown before cancellation.
DIFY_AGENT_SHUTDOWN_GRACE_SECONDS=30
-# Seconds to retain Redis run records and per-run event streams (default: 3 days).
-DIFY_AGENT_RUN_RETENTION_SECONDS=259200
+# Seconds to retain Redis run records and per-run event streams after the last write (default: 2 hours).
+DIFY_AGENT_RUN_RETENTION_SECONDS=7200
+# Approximate target maximum for replayable events retained in each per-run Redis Stream.
+DIFY_AGENT_RUN_EVENT_STREAM_MAX_LENGTH=5000
+# Set false to publish every text delta without coalescing.
+DIFY_AGENT_STREAM_TEXT_DELTA_COALESCING_ENABLED=true
+# Soft debounce interval for compatible text deltas. Already-ready events may continue
+# to merge past this interval; a waiting source triggers the timed flush.
+DIFY_AGENT_STREAM_TEXT_DELTA_FLUSH_INTERVAL_MS=100
+# Flush a text-delta batch immediately once it reaches this many characters.
+DIFY_AGENT_STREAM_TEXT_DELTA_MAX_CHARS=4096
# Plugin daemon
# Base URL for the Dify plugin daemon used by local runs.
diff --git a/dify-agent/docs/dify-agent/guide/index.md b/dify-agent/docs/dify-agent/guide/index.md
index de72c0ba286..99ff7b0ac28 100644
--- a/dify-agent/docs/dify-agent/guide/index.md
+++ b/dify-agent/docs/dify-agent/guide/index.md
@@ -34,7 +34,11 @@ also reads `.env` and `dify-agent/.env` when present.
| `DIFY_AGENT_REDIS_URL` | `redis://localhost:6379/0` | Redis connection URL. |
| `DIFY_AGENT_REDIS_PREFIX` | `dify-agent` | Prefix for Redis record and event keys. |
| `DIFY_AGENT_SHUTDOWN_GRACE_SECONDS` | `30` | Seconds to wait for active local runs during graceful shutdown before cancellation. |
-| `DIFY_AGENT_RUN_RETENTION_SECONDS` | `259200` | Seconds to retain Redis run records and per-run event streams; defaults to 3 days. |
+| `DIFY_AGENT_RUN_RETENTION_SECONDS` | `7200` | Seconds to retain Redis run records and per-run event streams after their last write; defaults to 2 hours. |
+| `DIFY_AGENT_RUN_EVENT_STREAM_MAX_LENGTH` | `5000` | Approximate target maximum for replayable events retained in each per-run Redis Stream. |
+| `DIFY_AGENT_STREAM_TEXT_DELTA_COALESCING_ENABLED` | `true` | Set `false` to publish each text delta without coalescing. |
+| `DIFY_AGENT_STREAM_TEXT_DELTA_FLUSH_INTERVAL_MS` | `100` | Soft debounce interval for compatible text deltas. Already-ready source events may continue to merge after this interval; a waiting source triggers the timed flush. Must be greater than zero; use `DIFY_AGENT_STREAM_TEXT_DELTA_COALESCING_ENABLED=false` to disable coalescing. |
+| `DIFY_AGENT_STREAM_TEXT_DELTA_MAX_CHARS` | `4096` | Character threshold that flushes a buffered text-delta event immediately. |
| `DIFY_AGENT_RUN_TIMEOUT_SECONDS` | `3600` | Wall-clock deadline in seconds for the Pydantic AI `agent.run(...)` model/tool loop. Deadline failures use `agent_run_limit_exceeded`. Its default intentionally matches `DIFY_AGENT_E2B_ACTIVE_TIMEOUT_SECONDS`, but the settings are independently configurable. |
| `DIFY_AGENT_BINDING_FILE_DOWNLOAD_COMMAND_TIMEOUT_SECONDS` | `210` | Shell command deadline for running the sandbox `dify-agent file upload --no-download-link` conversion. Keep it above the CLI's 180-second upload deadline. |
| `DIFY_AGENT_API_TOKEN` | empty | Optional Bearer token required by private run, Execution Binding, Home Snapshot, and Binding file control-plane routes. Must match Dify API `AGENT_BACKEND_API_TOKEN`. |
@@ -75,7 +79,10 @@ Example `.env`:
DIFY_AGENT_REDIS_URL=redis://localhost:6379/0
DIFY_AGENT_REDIS_PREFIX=dify-agent-dev
DIFY_AGENT_SHUTDOWN_GRACE_SECONDS=30
-DIFY_AGENT_RUN_RETENTION_SECONDS=259200
+DIFY_AGENT_RUN_RETENTION_SECONDS=7200
+DIFY_AGENT_RUN_EVENT_STREAM_MAX_LENGTH=5000
+DIFY_AGENT_STREAM_TEXT_DELTA_FLUSH_INTERVAL_MS=100
+DIFY_AGENT_STREAM_TEXT_DELTA_MAX_CHARS=4096
DIFY_AGENT_RUN_TIMEOUT_SECONDS=3600
DIFY_AGENT_API_TOKEN=replace-with-agent-backend-token
DIFY_AGENT_PLUGIN_DAEMON_URL=http://localhost:5002
@@ -138,7 +145,15 @@ and lifecycle contract.
Run records and event streams use the same retention. Status writes refresh the
record TTL, and event writes refresh both the stream TTL and the corresponding
-record TTL so active runs that keep producing events remain observable.
+record TTL so active runs that keep producing events remain observable. Text
+deltas for the same response part and provider metadata are coalesced within a
+soft debounce/size window before being published. The debounce timer flushes
+when reading the next source event would block; already-ready events may keep
+merging until the character threshold or a non-compatible event is reached.
+Every Redis Stream write also applies the configured approximate maximum length. A reconnect cursor older than the
+retained window resumes from the oldest remaining event, so callers must treat
+the terminal snapshot and application-owned history as authoritative rather
+than relying on the Agent event stream as long-term history.
## Validate the E2B Compose deployment
diff --git a/dify-agent/src/dify_agent/runtime/event_coalescer.py b/dify-agent/src/dify_agent/runtime/event_coalescer.py
new file mode 100644
index 00000000000..a35d2c52fe3
--- /dev/null
+++ b/dify-agent/src/dify_agent/runtime/event_coalescer.py
@@ -0,0 +1,143 @@
+"""Bound high-frequency text deltas before they reach the shared event sink."""
+
+import asyncio
+from collections.abc import AsyncIterable, AsyncIterator
+from contextlib import suppress
+from dataclasses import replace
+
+from pydantic_ai.messages import AgentStreamEvent, PartDeltaEvent, TextPartDelta
+
+
+DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS = 0.1
+DEFAULT_TEXT_DELTA_MAX_CHARS = 4096
+
+
+async def coalesce_agent_stream_events(
+ events: AsyncIterable[AgentStreamEvent],
+ *,
+ enabled: bool = True,
+ flush_interval_seconds: float = DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS,
+ max_chars: int = DEFAULT_TEXT_DELTA_MAX_CHARS,
+) -> AsyncIterator[AgentStreamEvent]:
+ """Merge compatible text deltas with a soft debounce interval.
+
+ One source event is read ahead while a text delta is buffered. Non-text events
+ flush the buffer first, preserving the public Pydantic AI event ordering. The
+ timer starts with the first buffered delta instead of sliding on every token.
+ Already-ready source events may continue to merge after the interval; the
+ interval triggers a flush once reading the next source event would block.
+ """
+ if flush_interval_seconds <= 0:
+ raise ValueError("flush_interval_seconds must be positive")
+ if max_chars <= 0:
+ raise ValueError("max_chars must be positive")
+ if not enabled:
+ async for event in events:
+ yield event
+ return
+
+ iterator = aiter(events)
+ next_event_task: asyncio.Future[AgentStreamEvent] | None = asyncio.ensure_future(anext(iterator))
+ buffered: PartDeltaEvent | None = None
+ flush_deadline: float | None = None
+ loop = asyncio.get_running_loop()
+
+ try:
+ while next_event_task is not None:
+ timeout = None if flush_deadline is None else max(0.0, flush_deadline - loop.time())
+ try:
+ done, _pending = await asyncio.wait((next_event_task,), timeout=timeout)
+ except asyncio.CancelledError:
+ if buffered is not None:
+ yield buffered
+ raise
+
+ if not done:
+ assert buffered is not None
+ yield buffered
+ buffered = None
+ flush_deadline = None
+ continue
+
+ try:
+ event = next_event_task.result()
+ except StopAsyncIteration:
+ next_event_task = None
+ if buffered is not None:
+ yield buffered
+ return
+ except BaseException:
+ next_event_task = None
+ if buffered is not None:
+ yield buffered
+ raise
+
+ next_event_task = asyncio.ensure_future(anext(iterator))
+ text_delta = _text_delta(event)
+ if text_delta is None:
+ if buffered is not None:
+ yield buffered
+ buffered = None
+ flush_deadline = None
+ yield event
+ continue
+
+ if buffered is not None and _can_merge(buffered, event):
+ buffered = _merge_text_deltas(buffered, event)
+ else:
+ if buffered is not None:
+ yield buffered
+ assert isinstance(event, PartDeltaEvent)
+ buffered = event
+ flush_deadline = loop.time() + flush_interval_seconds
+
+ buffered_delta = _text_delta(buffered)
+ assert buffered_delta is not None
+ if len(buffered_delta.content_delta) >= max_chars:
+ yield buffered
+ buffered = None
+ flush_deadline = None
+ finally:
+ if next_event_task is not None and not next_event_task.done():
+ next_event_task.cancel()
+ with suppress(asyncio.CancelledError, StopAsyncIteration):
+ _ = await next_event_task
+
+
+def _text_delta(event: AgentStreamEvent) -> TextPartDelta | None:
+ if isinstance(event, PartDeltaEvent) and isinstance(event.delta, TextPartDelta):
+ return event.delta
+ return None
+
+
+def _can_merge(left: PartDeltaEvent, right: AgentStreamEvent) -> bool:
+ right_delta = _text_delta(right)
+ if right_delta is None:
+ return False
+ left_delta = left.delta
+ assert isinstance(left_delta, TextPartDelta)
+ assert isinstance(right, PartDeltaEvent)
+ return (
+ left.index == right.index
+ and left_delta.provider_name == right_delta.provider_name
+ and left_delta.provider_details == right_delta.provider_details
+ )
+
+
+def _merge_text_deltas(left: PartDeltaEvent, right: AgentStreamEvent) -> PartDeltaEvent:
+ left_delta = left.delta
+ right_delta = _text_delta(right)
+ assert isinstance(left_delta, TextPartDelta)
+ assert right_delta is not None
+ merged_delta = replace(
+ left_delta,
+ content_delta=left_delta.content_delta + right_delta.content_delta,
+ )
+ return replace(left, delta=merged_delta)
+
+
+__all__ = [
+ "DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS",
+ "DEFAULT_TEXT_DELTA_MAX_CHARS",
+ "coalesce_agent_stream_events",
+]
diff --git a/dify-agent/src/dify_agent/runtime/run_scheduler.py b/dify-agent/src/dify_agent/runtime/run_scheduler.py
index a0bf260bd25..5a57999547f 100644
--- a/dify-agent/src/dify_agent/runtime/run_scheduler.py
+++ b/dify-agent/src/dify_agent/runtime/run_scheduler.py
@@ -24,6 +24,10 @@ from agenton.compositor import CompositorSessionSnapshot, LayerProviderInput
from dify_agent.protocol.schemas import AgentRunUsage, CancelRunRequest, CancelRunResponse, CreateRunRequest, RunStatus
from dify_agent.runtime.cancellation import RunCancellationIntent
from dify_agent.runtime.compositor_factory import create_default_layer_providers
+from dify_agent.runtime.event_coalescer import (
+ DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS,
+ DEFAULT_TEXT_DELTA_MAX_CHARS,
+)
from dify_agent.runtime.event_sink import RunEventSink, RunFinalizationResult, emit_run_failed
from dify_agent.runtime.runner import DEFAULT_AGENT_RUN_TIMEOUT_SECONDS, AgentRunRunner
from dify_agent.server.schemas import RunRecord
@@ -106,6 +110,9 @@ class RunScheduler:
store: RunStore
shutdown_grace_seconds: float
run_timeout_seconds: float
+ stream_text_delta_coalescing_enabled: bool
+ stream_text_delta_flush_interval_seconds: float
+ stream_text_delta_max_chars: int
active_tasks: dict[str, asyncio.Task[None]]
stopping: bool
runner_factory: RunRunnerFactory | None
@@ -122,12 +129,18 @@ class RunScheduler:
dify_api_http_client: httpx.AsyncClient,
shutdown_grace_seconds: float = 30,
run_timeout_seconds: float = DEFAULT_AGENT_RUN_TIMEOUT_SECONDS,
+ stream_text_delta_coalescing_enabled: bool = True,
+ stream_text_delta_flush_interval_seconds: float = DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS,
+ stream_text_delta_max_chars: int = DEFAULT_TEXT_DELTA_MAX_CHARS,
layer_providers: tuple[LayerProviderInput, ...] | None = None,
runner_factory: RunRunnerFactory | None = None,
) -> None:
self.store = store
self.shutdown_grace_seconds = shutdown_grace_seconds
self.run_timeout_seconds = run_timeout_seconds
+ self.stream_text_delta_coalescing_enabled = stream_text_delta_coalescing_enabled
+ self.stream_text_delta_flush_interval_seconds = stream_text_delta_flush_interval_seconds
+ self.stream_text_delta_max_chars = stream_text_delta_max_chars
self.active_tasks = {}
self.stopping = False
self.plugin_daemon_http_client = plugin_daemon_http_client
@@ -304,6 +317,9 @@ class RunScheduler:
layer_providers=self.layer_providers,
is_cancelled=is_cancelled,
run_timeout_seconds=self.run_timeout_seconds,
+ stream_text_delta_coalescing_enabled=self.stream_text_delta_coalescing_enabled,
+ stream_text_delta_flush_interval_seconds=self.stream_text_delta_flush_interval_seconds,
+ stream_text_delta_max_chars=self.stream_text_delta_max_chars,
)
def _discard_active_run(self, run_id: str) -> None:
diff --git a/dify-agent/src/dify_agent/runtime/runner.py b/dify-agent/src/dify_agent/runtime/runner.py
index b7dcd127df1..b20edcbb74f 100644
--- a/dify-agent/src/dify_agent/runtime/runner.py
+++ b/dify-agent/src/dify_agent/runtime/runner.py
@@ -7,8 +7,9 @@ policy is validated:
- model runs: enter a fresh ``CompositorRun`` (or resume one from a snapshot),
pass the current Dify system prompts as run-level instructions, run
pydantic-ai with either the current ``run.user_prompts`` or deferred external
- tool results, emit raw stream events with agent-message delta annotations, apply
- request-level ``on_exit`` signals, and publish a terminal success or failure event;
+ tool results, emit stream events with bounded text-delta coalescing and
+ agent-message annotations, apply request-level ``on_exit`` signals, and publish
+ a terminal success or failure event;
The Pydantic AI model is resolved from the active Agenton layer named by
``DIFY_AGENT_MODEL_LAYER_ID``. An optional history layer contributes stored
message history only through session state. Once pydantic-ai binds and builds
@@ -67,6 +68,11 @@ from dify_agent.runtime.agenton_validation import is_agenton_enter_validation_ru
from dify_agent.runtime.compositor_factory import build_pydantic_ai_compositor, create_default_layer_providers
from dify_agent.runtime.compaction import build_compaction_capability
from dify_agent.runtime_backend import BindingLostError
+from dify_agent.runtime.event_coalescer import (
+ DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS,
+ DEFAULT_TEXT_DELTA_MAX_CHARS,
+ coalesce_agent_stream_events,
+)
from dify_agent.runtime.event_sink import (
RunEventSink,
emit_pydantic_ai_event,
@@ -187,6 +193,9 @@ class AgentRunRunner:
dify_api_http_client: httpx.AsyncClient
is_cancelled: Callable[[], bool]
run_timeout_seconds: float
+ stream_text_delta_coalescing_enabled: bool
+ stream_text_delta_flush_interval_seconds: float
+ stream_text_delta_max_chars: int
_terminal_session_snapshot: CompositorSessionSnapshot | None
_terminal_usage: AgentRunUsage | None
@@ -201,7 +210,14 @@ class AgentRunRunner:
layer_providers: tuple[LayerProviderInput, ...] | None = None,
is_cancelled: Callable[[], bool] | None = None,
run_timeout_seconds: float = DEFAULT_AGENT_RUN_TIMEOUT_SECONDS,
+ stream_text_delta_coalescing_enabled: bool = True,
+ stream_text_delta_flush_interval_seconds: float = DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS,
+ stream_text_delta_max_chars: int = DEFAULT_TEXT_DELTA_MAX_CHARS,
) -> None:
+ if stream_text_delta_flush_interval_seconds <= 0:
+ raise ValueError("stream_text_delta_flush_interval_seconds must be positive")
+ if stream_text_delta_max_chars <= 0:
+ raise ValueError("stream_text_delta_max_chars must be positive")
self.sink = sink
self.request = request
self.run_id = run_id
@@ -210,6 +226,9 @@ class AgentRunRunner:
self.layer_providers = layer_providers if layer_providers is not None else create_default_layer_providers()
self.is_cancelled = is_cancelled or (lambda: False)
self.run_timeout_seconds = run_timeout_seconds
+ self.stream_text_delta_coalescing_enabled = stream_text_delta_coalescing_enabled
+ self.stream_text_delta_flush_interval_seconds = stream_text_delta_flush_interval_seconds
+ self.stream_text_delta_max_chars = stream_text_delta_max_chars
self._terminal_session_snapshot = None
self._terminal_usage = None
@@ -320,7 +339,13 @@ class AgentRunRunner:
raise AgentRunValidationError(EMPTY_USER_PROMPTS_ERROR)
async def handle_events(_ctx: object, events: AsyncIterable[AgentStreamEvent]) -> None:
- async for event in events:
+ published_events = coalesce_agent_stream_events(
+ events,
+ enabled=self.stream_text_delta_coalescing_enabled,
+ flush_interval_seconds=self.stream_text_delta_flush_interval_seconds,
+ max_chars=self.stream_text_delta_max_chars,
+ )
+ async for event in published_events:
if self.is_cancelled():
raise asyncio.CancelledError
text_delta = _extract_agent_message_delta(event)
diff --git a/dify-agent/src/dify_agent/server/app.py b/dify-agent/src/dify_agent/server/app.py
index 8144c66ae20..4b1d4ce3782 100644
--- a/dify-agent/src/dify_agent/server/app.py
+++ b/dify-agent/src/dify_agent/server/app.py
@@ -105,6 +105,7 @@ def create_app(settings: ServerSettings | None = None) -> FastAPI:
redis,
prefix=resolved_settings.redis_prefix,
run_retention_seconds=resolved_settings.run_retention_seconds,
+ run_event_stream_max_length=resolved_settings.run_event_stream_max_length,
)
scheduler = RunScheduler(
store=store,
@@ -112,6 +113,9 @@ def create_app(settings: ServerSettings | None = None) -> FastAPI:
dify_api_http_client=dify_api_inner_http_client,
shutdown_grace_seconds=resolved_settings.shutdown_grace_seconds,
run_timeout_seconds=resolved_settings.run_timeout_seconds,
+ stream_text_delta_coalescing_enabled=resolved_settings.stream_text_delta_coalescing_enabled,
+ stream_text_delta_flush_interval_seconds=(resolved_settings.stream_text_delta_flush_interval_ms / 1000),
+ stream_text_delta_max_chars=resolved_settings.stream_text_delta_max_chars,
layer_providers=layer_providers,
)
state["store"] = store
diff --git a/dify-agent/src/dify_agent/server/settings.py b/dify-agent/src/dify_agent/server/settings.py
index 3d27e4439b7..80669d6bf19 100644
--- a/dify-agent/src/dify_agent/server/settings.py
+++ b/dify-agent/src/dify_agent/server/settings.py
@@ -21,6 +21,10 @@ from dify_agent.agent_stub.protocol.agent_stub import normalize_agent_stub_api_b
from dify_agent.agent_stub.server.agent_stub_config import DifyApiAgentStubConfigRequestHandler
from dify_agent.agent_stub.server.agent_stub_files import DifyApiAgentStubFileRequestHandler
from dify_agent.agent_stub.server.tokens.agent_stub import AgentStubTokenCodec, decode_server_secret_key
+from dify_agent.runtime.event_coalescer import (
+ DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS,
+ DEFAULT_TEXT_DELTA_MAX_CHARS,
+)
from dify_agent.runtime.runner import DEFAULT_AGENT_RUN_TIMEOUT_SECONDS
from dify_agent.runtime_backend import RuntimeBackendProfile
from dify_agent.runtime_backend.e2b import E2B_MAX_ACTIVE_TIMEOUT_SECONDS
@@ -32,7 +36,8 @@ from dify_agent.runtime_backend.profile import (
create_runtime_backend_profile,
)
-DEFAULT_RUN_RETENTION_SECONDS = 3 * 24 * 60 * 60
+DEFAULT_RUN_RETENTION_SECONDS = 2 * 60 * 60
+DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH = 5000
class ServerSettings(BaseSettings):
@@ -42,6 +47,13 @@ class ServerSettings(BaseSettings):
redis_prefix: str = "dify-agent"
shutdown_grace_seconds: float = 30
run_retention_seconds: int = Field(default=DEFAULT_RUN_RETENTION_SECONDS, ge=1)
+ run_event_stream_max_length: int = Field(default=DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH, ge=1)
+ stream_text_delta_coalescing_enabled: bool = True
+ stream_text_delta_flush_interval_ms: int = Field(
+ default=int(DEFAULT_TEXT_DELTA_FLUSH_INTERVAL_SECONDS * 1000),
+ ge=1,
+ )
+ stream_text_delta_max_chars: int = Field(default=DEFAULT_TEXT_DELTA_MAX_CHARS, ge=1)
run_timeout_seconds: float = Field(default=DEFAULT_AGENT_RUN_TIMEOUT_SECONDS, gt=0)
plugin_daemon_url: str = "http://localhost:5002"
plugin_daemon_api_key: str = ""
@@ -255,4 +267,4 @@ class ServerSettings(BaseSettings):
)
-__all__ = ["DEFAULT_RUN_RETENTION_SECONDS", "ServerSettings"]
+__all__ = ["DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH", "DEFAULT_RUN_RETENTION_SECONDS", "ServerSettings"]
diff --git a/dify-agent/src/dify_agent/storage/redis_run_store.py b/dify-agent/src/dify_agent/storage/redis_run_store.py
index e9f708e22de..a2a61cd9231 100644
--- a/dify-agent/src/dify_agent/storage/redis_run_store.py
+++ b/dify-agent/src/dify_agent/storage/redis_run_store.py
@@ -35,7 +35,7 @@ from dify_agent.runtime.event_sink import (
terminal_event_status_fields,
)
from dify_agent.server.schemas import RunRecord, new_run_id
-from dify_agent.server.settings import DEFAULT_RUN_RETENTION_SECONDS
+from dify_agent.server.settings import DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH, DEFAULT_RUN_RETENTION_SECONDS
from dify_agent.storage.redis_keys import run_cancel_intent_key, run_events_key, run_record_key
_TERMINAL_RUN_EVENT_TYPES = {"run_succeeded", "run_failed", "run_cancelled"}
@@ -75,7 +75,7 @@ end
local ttl = tonumber(ARGV[8])
local updated_record_json = cjson.encode(record)
-local event_id = redis.call("XADD", KEYS[2], "*", "payload", ARGV[7])
+local event_id = redis.call("XADD", KEYS[2], "MAXLEN", "~", ARGV[9], "*", "payload", ARGV[7])
redis.call("EXPIRE", KEYS[2], ttl)
redis.call("SET", KEYS[1], updated_record_json, "EX", ttl)
return {1, ARGV[1], event_id}
@@ -100,7 +100,7 @@ if redis.call("EXISTS", KEYS[2]) == 1 then
end
local ttl = tonumber(ARGV[2])
-redis.call("XADD", KEYS[2], "*", "payload", ARGV[1])
+redis.call("XADD", KEYS[2], "MAXLEN", "1", "*", "payload", ARGV[1])
redis.call("EXPIRE", KEYS[2], ttl)
redis.call("EXPIRE", KEYS[1], ttl)
redis.call("EXPIRE", KEYS[3], ttl)
@@ -135,7 +135,7 @@ end
record.error_type = cjson.null
local ttl = tonumber(ARGV[5])
-local event_id = redis.call("XADD", KEYS[3], "*", "payload", ARGV[4])
+local event_id = redis.call("XADD", KEYS[3], "MAXLEN", "~", ARGV[6], "*", "payload", ARGV[4])
redis.call("DEL", KEYS[2])
redis.call("EXPIRE", KEYS[3], ttl)
redis.call("SET", KEYS[1], cjson.encode(record), "EX", ttl)
@@ -147,16 +147,19 @@ class RedisRunStore(RunEventSink):
"""Async Redis implementation for run records and event logs.
``run_retention_seconds`` is applied to both the run record key and the
- per-run Redis stream. Event writes run ``XADD`` and both TTL refreshes in one
- Redis transaction so a newly created stream is not left without expiration if
- the client is interrupted between commands. Event writes also refresh the
- record TTL so long-running runs that keep producing events do not lose their
- status record mid-run.
+ per-run Redis stream. Every public stream write applies an approximate
+ maximum length before refreshing both TTLs, so active runs cannot grow one
+ Redis key without bound without paying the CPU cost of exact per-entry
+ trimming. The transaction also ensures a newly created stream is not left
+ without expiration if the client is interrupted between commands.
+ Event writes refresh the record TTL so long-running runs that keep producing
+ events do not lose their status record mid-run.
"""
redis: Redis
prefix: str
run_retention_seconds: int
+ run_event_stream_max_length: int
def __init__(
self,
@@ -164,12 +167,16 @@ class RedisRunStore(RunEventSink):
*,
prefix: str = "dify-agent",
run_retention_seconds: int = DEFAULT_RUN_RETENTION_SECONDS,
+ run_event_stream_max_length: int = DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH,
) -> None:
if run_retention_seconds <= 0:
raise ValueError("run_retention_seconds must be positive")
+ if run_event_stream_max_length <= 0:
+ raise ValueError("run_event_stream_max_length must be positive")
self.redis = redis
self.prefix = prefix
self.run_retention_seconds = run_retention_seconds
+ self.run_event_stream_max_length = run_event_stream_max_length
async def create_run(self) -> RunRecord:
"""Persist a running run record without storing the create request."""
@@ -199,6 +206,8 @@ class RedisRunStore(RunEventSink):
_ = pipeline.xadd(
events_key,
{"payload": payload},
+ maxlen=self.run_event_stream_max_length,
+ approximate=True,
)
_ = pipeline.expire(events_key, self.run_retention_seconds)
_ = pipeline.expire(run_record_key(self.prefix, event.run_id), self.run_retention_seconds)
@@ -226,6 +235,7 @@ class RedisRunStore(RunEventSink):
error_type.value if error_type is not None else "",
payload,
str(self.run_retention_seconds),
+ str(self.run_event_stream_max_length),
),
)
raw_result = await evaluation
@@ -316,6 +326,7 @@ class RedisRunStore(RunEventSink):
error or "",
payload,
str(self.run_retention_seconds),
+ str(self.run_event_stream_max_length),
),
)
result = cast(list[object], await evaluation)
@@ -385,4 +396,9 @@ def _decode_redis_text(value: object) -> str:
return value.decode() if isinstance(value, bytes) else str(value)
-__all__ = ["DEFAULT_RUN_RETENTION_SECONDS", "RedisRunStore", "RunNotFoundError"]
+__all__ = [
+ "DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH",
+ "DEFAULT_RUN_RETENTION_SECONDS",
+ "RedisRunStore",
+ "RunNotFoundError",
+]
diff --git a/dify-agent/tests/integration/dify_agent/runtime_backend/test_working_environment.py b/dify-agent/tests/integration/dify_agent/runtime_backend/test_working_environment.py
index 93a67c2e816..41deb4dfa3d 100644
--- a/dify-agent/tests/integration/dify_agent/runtime_backend/test_working_environment.py
+++ b/dify-agent/tests/integration/dify_agent/runtime_backend/test_working_environment.py
@@ -34,6 +34,7 @@ async def _run(lease, script: str, *, cwd: str) -> str:
env={"HOME": lease.layout.home_dir},
timeout=30.0,
max_output_bytes=4096,
+ mode="stdio",
)
assert result.exit_code == 0
assert result.output_complete
diff --git a/dify-agent/tests/integration/dify_agent/storage/test_terminal_finalization.py b/dify-agent/tests/integration/dify_agent/storage/test_terminal_finalization.py
index af3a77c687d..5bf579bee03 100644
--- a/dify-agent/tests/integration/dify_agent/storage/test_terminal_finalization.py
+++ b/dify-agent/tests/integration/dify_agent/storage/test_terminal_finalization.py
@@ -153,6 +153,50 @@ def test_success_and_cancel_intent_commit_exactly_one_matching_terminal(redis_ur
asyncio.run(scenario())
+def test_event_stream_is_trimmed_and_keeps_the_terminal_event(redis_url: str) -> None:
+ async def scenario() -> None:
+ client = Redis.from_url(redis_url)
+ prefix = f"bounded-events-{uuid4().hex}"
+ max_length = 100
+ store = RedisRunStore(
+ client,
+ prefix=prefix,
+ run_retention_seconds=60,
+ run_event_stream_max_length=max_length,
+ )
+ run_id: str | None = None
+ try:
+ record = await store.create_run()
+ run_id = record.run_id
+ for _ in range(1000):
+ _ = await store.append_event(RunStartedEvent(run_id=record.run_id))
+ result = await store.finalize_run(
+ RunSucceededEvent(
+ run_id=record.run_id,
+ data=RunSucceededEventData(
+ output="done",
+ session_snapshot=CompositorSessionSnapshot(layers=[]),
+ ),
+ )
+ )
+
+ assert result.applied is True
+ stream_length = await client.xlen(run_events_key(prefix, record.run_id))
+ assert max_length <= stream_length <= max_length * 2
+ events = await store.get_events(record.run_id, limit=max_length * 2)
+ assert events.events[-1].type == "run_succeeded"
+ finally:
+ if run_id is not None:
+ await client.delete(
+ run_record_key(prefix, run_id),
+ run_events_key(prefix, run_id),
+ run_cancel_intent_key(prefix, run_id),
+ )
+ await client.aclose()
+
+ asyncio.run(scenario())
+
+
@pytest.mark.parametrize("terminal_status", ["succeeded", "failed"])
def test_terminal_first_rejects_late_cancellation(redis_url: str, terminal_status: str) -> None:
async def scenario() -> None:
@@ -303,6 +347,10 @@ def test_non_owner_scheduler_cancellation_stops_owner_runner(redis_url: str) ->
def terminal_session_snapshot(self) -> None:
return None
+ @property
+ def terminal_usage(self) -> None:
+ return None
+
async def run(self) -> None:
self.started.set()
try:
diff --git a/dify-agent/tests/local/dify_agent/runtime/test_event_coalescer.py b/dify-agent/tests/local/dify_agent/runtime/test_event_coalescer.py
new file mode 100644
index 00000000000..2ebc475e0de
--- /dev/null
+++ b/dify-agent/tests/local/dify_agent/runtime/test_event_coalescer.py
@@ -0,0 +1,227 @@
+import asyncio
+from collections.abc import AsyncIterator
+from typing import Any
+
+import pytest
+from pydantic_ai.messages import AgentStreamEvent, PartDeltaEvent, PartEndEvent, TextPart, TextPartDelta
+
+from dify_agent.runtime.event_coalescer import coalesce_agent_stream_events
+
+
+def _delta(content: str, *, index: int = 0, provider_details: dict[str, Any] | None = None) -> PartDeltaEvent:
+ return PartDeltaEvent(
+ index=index,
+ delta=TextPartDelta(content, provider_name="test", provider_details=provider_details),
+ )
+
+
+def _part_end(content: str = "done") -> PartEndEvent:
+ return PartEndEvent(index=0, part=TextPart(content))
+
+
+async def _events(*events: PartDeltaEvent | PartEndEvent) -> AsyncIterator[PartDeltaEvent | PartEndEvent]:
+ for event in events:
+ yield event
+
+
+async def _collect(events: AsyncIterator[AgentStreamEvent]) -> list[AgentStreamEvent]:
+ return [event async for event in events]
+
+
+def _delta_content(event: AgentStreamEvent) -> str:
+ assert isinstance(event, PartDeltaEvent)
+ assert isinstance(event.delta, TextPartDelta)
+ return event.delta.content_delta
+
+
+def test_coalesces_compatible_text_deltas_and_preserves_non_text_order() -> None:
+ async def scenario() -> list[AgentStreamEvent]:
+ return await _collect(
+ coalesce_agent_stream_events(
+ _events(_delta("hel"), _delta("lo"), _part_end()),
+ flush_interval_seconds=10,
+ max_chars=100,
+ )
+ )
+
+ result = asyncio.run(scenario())
+
+ assert len(result) == 2
+ assert isinstance(result[0], PartDeltaEvent)
+ assert isinstance(result[0].delta, TextPartDelta)
+ assert result[0].delta.content_delta == "hello"
+ assert isinstance(result[1], PartEndEvent)
+
+
+def test_disabled_coalescing_preserves_each_source_event() -> None:
+ async def scenario() -> list[AgentStreamEvent]:
+ return await _collect(
+ coalesce_agent_stream_events(
+ _events(_delta("hel"), _delta("lo"), _part_end()),
+ enabled=False,
+ flush_interval_seconds=10,
+ max_chars=100,
+ )
+ )
+
+ result = asyncio.run(scenario())
+
+ assert len(result) == 3
+ assert [_delta_content(event) for event in result[:2]] == ["hel", "lo"]
+ assert isinstance(result[2], PartEndEvent)
+
+
+def test_does_not_merge_different_parts_or_provider_details() -> None:
+ async def scenario() -> list[AgentStreamEvent]:
+ return await _collect(
+ coalesce_agent_stream_events(
+ _events(
+ _delta("a", index=0),
+ _delta("b", index=1),
+ _delta("c", index=1, provider_details={"token": 1}),
+ ),
+ flush_interval_seconds=10,
+ max_chars=100,
+ )
+ )
+
+ result = asyncio.run(scenario())
+
+ assert [_delta_content(event) for event in result] == ["a", "b", "c"]
+
+
+def test_flushes_when_character_limit_is_reached() -> None:
+ async def scenario() -> list[AgentStreamEvent]:
+ return await _collect(
+ coalesce_agent_stream_events(
+ _events(_delta("ab"), _delta("cd"), _delta("ef")),
+ flush_interval_seconds=10,
+ max_chars=4,
+ )
+ )
+
+ result = asyncio.run(scenario())
+
+ assert [_delta_content(event) for event in result] == ["abcd", "ef"]
+
+
+def test_high_volume_text_deltas_are_reduced_to_size_bounded_batches() -> None:
+ async def scenario() -> list[AgentStreamEvent]:
+ async def many_deltas() -> AsyncIterator[AgentStreamEvent]:
+ for _ in range(20_000):
+ yield _delta("x")
+
+ return await _collect(
+ coalesce_agent_stream_events(
+ many_deltas(),
+ flush_interval_seconds=10,
+ max_chars=4096,
+ )
+ )
+
+ result = asyncio.run(scenario())
+
+ assert len(result) == 5
+ assert sum(len(_delta_content(event)) for event in result) == 20_000
+
+
+def test_flushes_on_deadline_while_source_is_idle() -> None:
+ async def scenario() -> tuple[PartDeltaEvent, PartEndEvent]:
+ release_source = asyncio.Event()
+
+ async def delayed_events() -> AsyncIterator[PartDeltaEvent | PartEndEvent]:
+ yield _delta("ready")
+ await release_source.wait()
+ yield _part_end()
+
+ events = coalesce_agent_stream_events(
+ delayed_events(),
+ flush_interval_seconds=0.01,
+ max_chars=100,
+ )
+ first = await asyncio.wait_for(anext(events), timeout=0.2)
+ release_source.set()
+ second = await asyncio.wait_for(anext(events), timeout=0.2)
+ with pytest.raises(StopAsyncIteration):
+ _ = await anext(events)
+ assert isinstance(first, PartDeltaEvent)
+ assert isinstance(second, PartEndEvent)
+ return first, second
+
+ first, second = asyncio.run(scenario())
+
+ assert isinstance(first.delta, TextPartDelta)
+ assert first.delta.content_delta == "ready"
+ assert isinstance(second.part, TextPart)
+ assert second.part.content == "done"
+
+
+def test_flushes_buffer_before_propagating_source_failure() -> None:
+ async def scenario() -> PartDeltaEvent:
+ async def failing_events() -> AsyncIterator[AgentStreamEvent]:
+ yield _delta("partial")
+ raise RuntimeError("stream failed")
+
+ events = coalesce_agent_stream_events(
+ failing_events(),
+ flush_interval_seconds=10,
+ max_chars=100,
+ )
+ first = await anext(events)
+ with pytest.raises(RuntimeError, match="stream failed"):
+ _ = await anext(events)
+ assert isinstance(first, PartDeltaEvent)
+ return first
+
+ first = asyncio.run(scenario())
+
+ assert isinstance(first.delta, TextPartDelta)
+ assert first.delta.content_delta == "partial"
+
+
+def test_flushes_buffer_before_consumer_cancellation() -> None:
+ async def scenario() -> list[AgentStreamEvent]:
+ source_waiting = asyncio.Event()
+ emitted: list[AgentStreamEvent] = []
+
+ async def blocked_events() -> AsyncIterator[AgentStreamEvent]:
+ yield _delta("partial")
+ source_waiting.set()
+ await asyncio.Event().wait()
+
+ async def consume() -> None:
+ async for event in coalesce_agent_stream_events(
+ blocked_events(),
+ flush_interval_seconds=10,
+ max_chars=100,
+ ):
+ emitted.append(event)
+
+ task = asyncio.create_task(consume())
+ await asyncio.wait_for(source_waiting.wait(), timeout=0.2)
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+ return emitted
+
+ emitted = asyncio.run(scenario())
+
+ assert len(emitted) == 1
+ assert _delta_content(emitted[0]) == "partial"
+
+
+ ("flush_interval_seconds", "max_chars", "message"),
+ [(-0.1, 100, "positive"), (0, 100, "positive"), (0.1, 0, "positive")],
+)
+def test_rejects_invalid_bounds(flush_interval_seconds: float, max_chars: int, message: str) -> None:
+ async def scenario() -> None:
+ events = coalesce_agent_stream_events(
+ _events(_delta("a")),
+ flush_interval_seconds=flush_interval_seconds,
+ max_chars=max_chars,
+ )
+ with pytest.raises(ValueError, match=message):
+ _ = await anext(events)
+
+ asyncio.run(scenario())
diff --git a/dify-agent/tests/local/dify_agent/runtime/test_run_scheduler.py b/dify-agent/tests/local/dify_agent/runtime/test_run_scheduler.py
index 45e788d0c7e..7c00424f243 100644
--- a/dify-agent/tests/local/dify_agent/runtime/test_run_scheduler.py
+++ b/dify-agent/tests/local/dify_agent/runtime/test_run_scheduler.py
@@ -461,7 +461,7 @@ class FinalizeSuccessOnCancellationRunner(SnapshotlessRunner):
assert result.applied is True
-def test_default_runner_factory_passes_run_timeout_to_runner() -> None:
+def test_default_runner_factory_passes_runtime_limits_to_runner() -> None:
async def scenario() -> None:
store = FakeStore()
record = await store.create_run()
@@ -471,12 +471,18 @@ def test_default_runner_factory_passes_run_timeout_to_runner() -> None:
plugin_daemon_http_client=client,
dify_api_http_client=client,
run_timeout_seconds=17,
+ stream_text_delta_coalescing_enabled=False,
+ stream_text_delta_flush_interval_seconds=0.25,
+ stream_text_delta_max_chars=2048,
)
runner = scheduler._default_runner_factory(record, _request(), is_cancelled=lambda: False)
assert isinstance(runner, AgentRunRunner)
assert runner.run_timeout_seconds == 17
+ assert runner.stream_text_delta_coalescing_enabled is False
+ assert runner.stream_text_delta_flush_interval_seconds == 0.25
+ assert runner.stream_text_delta_max_chars == 2048
asyncio.run(scenario())
diff --git a/dify-agent/tests/local/dify_agent/server/test_app.py b/dify-agent/tests/local/dify_agent/server/test_app.py
index 14d49f9f3be..7fa209c8bd1 100644
--- a/dify-agent/tests/local/dify_agent/server/test_app.py
+++ b/dify-agent/tests/local/dify_agent/server/test_app.py
@@ -69,6 +69,9 @@ class FakeRunScheduler:
store: object
shutdown_grace_seconds: float
run_timeout_seconds: float
+ stream_text_delta_coalescing_enabled: bool
+ stream_text_delta_flush_interval_seconds: float
+ stream_text_delta_max_chars: int
layer_providers: tuple[DifyAgentLayerProvider, ...]
plugin_daemon_http_client: FakePluginDaemonHttpClient
dify_api_http_client: FakePluginDaemonHttpClient
@@ -82,11 +85,17 @@ class FakeRunScheduler:
dify_api_http_client: FakePluginDaemonHttpClient,
shutdown_grace_seconds: float,
run_timeout_seconds: float,
+ stream_text_delta_coalescing_enabled: bool,
+ stream_text_delta_flush_interval_seconds: float,
+ stream_text_delta_max_chars: int,
layer_providers: tuple[DifyAgentLayerProvider, ...],
) -> None:
self.store = store
self.shutdown_grace_seconds = shutdown_grace_seconds
self.run_timeout_seconds = run_timeout_seconds
+ self.stream_text_delta_coalescing_enabled = stream_text_delta_coalescing_enabled
+ self.stream_text_delta_flush_interval_seconds = stream_text_delta_flush_interval_seconds
+ self.stream_text_delta_max_chars = stream_text_delta_max_chars
self.layer_providers = layer_providers
self.plugin_daemon_http_client = plugin_daemon_http_client
self.dify_api_http_client = dify_api_http_client
@@ -203,6 +212,10 @@ def test_create_app_creates_scheduler_and_closes_after_shutdown(monkeypatch: pyt
shutdown_grace_seconds=5,
run_timeout_seconds=17,
run_retention_seconds=7,
+ run_event_stream_max_length=23,
+ stream_text_delta_coalescing_enabled=False,
+ stream_text_delta_flush_interval_ms=250,
+ stream_text_delta_max_chars=2048,
plugin_daemon_url="http://plugin-daemon",
plugin_daemon_api_key="daemon-secret",
inner_api_url="http://dify-api",
@@ -226,6 +239,9 @@ def test_create_app_creates_scheduler_and_closes_after_shutdown(monkeypatch: pyt
scheduler = FakeRunScheduler.created[0]
assert scheduler.shutdown_grace_seconds == 5
assert scheduler.run_timeout_seconds == 17
+ assert scheduler.stream_text_delta_coalescing_enabled is False
+ assert scheduler.stream_text_delta_flush_interval_seconds == 0.25
+ assert scheduler.stream_text_delta_max_chars == 2048
layer_providers = scheduler.layer_providers
assert isinstance(layer_providers, tuple)
execution_context_provider = next(
@@ -284,6 +300,7 @@ def test_create_app_creates_scheduler_and_closes_after_shutdown(monkeypatch: pyt
store = scheduler.store
assert isinstance(store, RedisRunStore)
assert store.run_retention_seconds == 7
+ assert store.run_event_stream_max_length == 23
assert any(getattr(route, "path", None) == "/agent-stub/connections" for route in create_app(settings).routes)
assert any(
getattr(route, "path", None) == "/agent-stub/files/upload-request" for route in create_app(settings).routes
diff --git a/dify-agent/tests/local/dify_agent/server/test_settings.py b/dify-agent/tests/local/dify_agent/server/test_settings.py
index b94dc033122..465c6a3b304 100644
--- a/dify-agent/tests/local/dify_agent/server/test_settings.py
+++ b/dify-agent/tests/local/dify_agent/server/test_settings.py
@@ -8,7 +8,11 @@ from pydantic import ValidationError
from dify_agent.agent_stub.server.agent_stub_files import DifyApiAgentStubFileRequestHandler
from dify_agent.agent_stub.server.tokens.agent_stub import AgentStubTokenCodec
-from dify_agent.server.settings import ServerSettings
+from dify_agent.server.settings import (
+ DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH,
+ DEFAULT_RUN_RETENTION_SECONDS,
+ ServerSettings,
+)
from dify_agent.runtime.runner import DEFAULT_AGENT_RUN_TIMEOUT_SECONDS
from dify_agent.runtime_backend.e2b import E2B_MAX_ACTIVE_TIMEOUT_SECONDS, E2BExecutionBindingBackend
from dify_agent.runtime_backend.enterprise import EnterpriseExecutionBindingBackend, EnterpriseHomeSnapshotBackend
@@ -79,6 +83,53 @@ def test_server_settings_rejects_non_positive_run_timeout() -> None:
_ = ServerSettings(run_timeout_seconds=0)
+def test_server_settings_defaults_to_bounded_short_lived_run_events(
+ monkeypatch: pytest.MonkeyPatch,
+ tmp_path: Path,
+) -> None:
+ monkeypatch.delenv("DIFY_AGENT_RUN_RETENTION_SECONDS", raising=False)
+ monkeypatch.delenv("DIFY_AGENT_RUN_EVENT_STREAM_MAX_LENGTH", raising=False)
+ monkeypatch.delenv("DIFY_AGENT_STREAM_TEXT_DELTA_COALESCING_ENABLED", raising=False)
+ monkeypatch.delenv("DIFY_AGENT_STREAM_TEXT_DELTA_FLUSH_INTERVAL_MS", raising=False)
+ monkeypatch.delenv("DIFY_AGENT_STREAM_TEXT_DELTA_MAX_CHARS", raising=False)
+ monkeypatch.chdir(tmp_path)
+
+ settings = ServerSettings()
+
+ assert settings.run_retention_seconds == DEFAULT_RUN_RETENTION_SECONDS == 7200
+ assert settings.run_event_stream_max_length == DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH == 5000
+ assert settings.stream_text_delta_coalescing_enabled is True
+ assert settings.stream_text_delta_flush_interval_ms == 100
+ assert settings.stream_text_delta_max_chars == 4096
+
+
+def test_server_settings_reads_run_event_bounds_from_env(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setenv("DIFY_AGENT_RUN_EVENT_STREAM_MAX_LENGTH", "1234")
+ monkeypatch.setenv("DIFY_AGENT_STREAM_TEXT_DELTA_COALESCING_ENABLED", "false")
+ monkeypatch.setenv("DIFY_AGENT_STREAM_TEXT_DELTA_FLUSH_INTERVAL_MS", "250")
+ monkeypatch.setenv("DIFY_AGENT_STREAM_TEXT_DELTA_MAX_CHARS", "2048")
+
+ settings = ServerSettings()
+
+ assert settings.run_event_stream_max_length == 1234
+ assert settings.stream_text_delta_coalescing_enabled is False
+ assert settings.stream_text_delta_flush_interval_ms == 250
+ assert settings.stream_text_delta_max_chars == 2048
+
+
+ ("field", "value"),
+ [
+ ("run_event_stream_max_length", 0),
+ ("stream_text_delta_flush_interval_ms", 0),
+ ("stream_text_delta_max_chars", 0),
+ ],
+)
+def test_server_settings_rejects_invalid_run_event_bounds(field: str, value: int) -> None:
+ with pytest.raises(ValidationError):
+ _ = ServerSettings(**{field: value}) # pyright: ignore[reportArgumentType]
+
+
def test_server_settings_reads_binding_file_download_command_timeout_from_env(
monkeypatch: pytest.MonkeyPatch,
) -> None:
diff --git a/dify-agent/tests/local/dify_agent/storage/test_redis_run_store.py b/dify-agent/tests/local/dify_agent/storage/test_redis_run_store.py
index a39b960c7cc..68641aa7a10 100644
--- a/dify-agent/tests/local/dify_agent/storage/test_redis_run_store.py
+++ b/dify-agent/tests/local/dify_agent/storage/test_redis_run_store.py
@@ -24,7 +24,12 @@ from dify_agent.protocol.schemas import (
)
from dify_agent.runtime.cancellation import RunCancellationIntent
from dify_agent.runtime.event_sink import RunFinalizationResult
-from dify_agent.storage.redis_run_store import DEFAULT_RUN_RETENTION_SECONDS, RedisRunStore, RunNotFoundError
+from dify_agent.storage.redis_run_store import (
+ DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH,
+ DEFAULT_RUN_RETENTION_SECONDS,
+ RedisRunStore,
+ RunNotFoundError,
+)
class FakeRedis:
@@ -48,18 +53,34 @@ class FakeRedis:
self.commands.append(("get", key))
return self.values.get(key)
- async def xadd(self, key: str, fields: Mapping[str, object]) -> str:
- self.commands.append(("xadd", key, dict(fields)))
- return self._append_stream_entry(key, fields)
+ async def xadd(
+ self,
+ key: str,
+ fields: Mapping[str, object],
+ *,
+ maxlen: int | None = None,
+ approximate: bool = True,
+ ) -> str:
+ self.commands.append(("xadd", key, dict(fields), maxlen, approximate))
+ return self._append_stream_entry(key, fields, maxlen=maxlen)
def pipeline(self, transaction: bool = True, shard_hint: str | None = None) -> "FakeRedisPipeline":
self.commands.append(("pipeline", transaction, shard_hint))
return FakeRedisPipeline(self)
- def _append_stream_entry(self, key: str, fields: Mapping[str, object]) -> str:
+ def _append_stream_entry(
+ self,
+ key: str,
+ fields: Mapping[str, object],
+ *,
+ maxlen: int | None = None,
+ ) -> str:
entries = self.streams.setdefault(key, [])
- event_id = f"{len(entries) + 1}-0"
+ next_sequence = self._stream_id_value(entries[-1][0])[0] + 1 if entries else 1
+ event_id = f"{next_sequence}-0"
entries.append((event_id, dict(fields)))
+ if maxlen is not None and len(entries) > maxlen:
+ del entries[: len(entries) - maxlen]
self.stream_changed.set()
return event_id
@@ -137,9 +158,16 @@ class FakeRedisPipeline:
async def __aexit__(self, exc_type: object, exc: object, traceback: object) -> None:
del exc_type, exc, traceback
- def xadd(self, key: str, fields: Mapping[str, object]) -> "FakeRedisPipeline":
- self.redis.commands.append(("xadd", key, dict(fields)))
- self.results.append(self.redis._append_stream_entry(key, fields))
+ def xadd(
+ self,
+ key: str,
+ fields: Mapping[str, object],
+ *,
+ maxlen: int | None = None,
+ approximate: bool = True,
+ ) -> "FakeRedisPipeline":
+ self.redis.commands.append(("xadd", key, dict(fields), maxlen, approximate))
+ self.results.append(self.redis._append_stream_entry(key, fields, maxlen=maxlen))
return self
def expire(self, key: str, seconds: int) -> "FakeRedisPipeline":
@@ -184,6 +212,15 @@ def test_create_run_writes_running_record_without_job_queue_and_with_retention()
assert "request" not in str(redis.commands[0][2])
+ "kwargs",
+ [{"run_retention_seconds": 0}, {"run_event_stream_max_length": 0}],
+)
+def test_rejects_non_positive_run_storage_bounds(kwargs: dict[str, int]) -> None:
+ with pytest.raises(ValueError, match="must be positive"):
+ _ = RedisRunStore(FakeRedis(), **kwargs) # pyright: ignore[reportArgumentType]
+
+
def test_get_run_accepts_legacy_record_without_error_type() -> None:
redis = FakeRedis()
store = RedisRunStore(redis, prefix="test") # pyright: ignore[reportArgumentType]
@@ -223,6 +260,7 @@ def test_request_cancellation_maps_eval_result_and_arguments() -> None:
assert intent_payload["reason"] == "workflow_aborted"
assert intent_payload["message"] == "workflow stopped"
assert eval_command[7] == "60"
+ assert '"MAXLEN", "1"' in cast(str, eval_command[1])
def test_finalize_cancellation_maps_eval_result_and_arguments() -> None:
@@ -262,6 +300,8 @@ def test_finalize_cancellation_maps_eval_result_and_arguments() -> None:
assert payload["data"]["usage"]["completion_tokens"] == 8
assert payload["data"]["usage"]["total_tokens"] == 21
assert eval_command[10] == "60"
+ assert eval_command[11] == str(DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH)
+ assert '"MAXLEN", "~", ARGV[6]' in cast(str, eval_command[1])
def test_finalize_failed_run_maps_eval_result_and_arguments() -> None:
@@ -290,6 +330,9 @@ def test_finalize_failed_run_maps_eval_result_and_arguments() -> None:
assert eval_command[8:12] == ("1", "model failed", "1", "agent_run_limit_exceeded")
payload = json.loads(cast(str, eval_command[12]))
assert payload["data"]["error_type"] == "agent_run_limit_exceeded"
+ assert eval_command[13] == str(DEFAULT_RUN_RETENTION_SECONDS)
+ assert eval_command[14] == str(DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH)
+ assert '"MAXLEN", "~", ARGV[9]' in cast(str, eval_command[1])
def test_request_cancellation_raises_when_record_is_missing() -> None:
@@ -368,6 +411,7 @@ def test_append_event_serializes_typed_event_without_id_and_expires_run_keys() -
assert isinstance(fields, dict)
assert '"id"' not in str(fields["payload"])
assert '"type":"run_started"' in str(fields["payload"])
+ assert xadd_commands[0][3:] == (DEFAULT_RUN_EVENT_STREAM_MAX_LENGTH, True)
expire_commands = {command for command in redis.commands if command[0] == "expire"}
assert expire_commands == {
("expire", "test:runs:run-1:events", 60),
@@ -376,6 +420,26 @@ def test_append_event_serializes_typed_event_without_id_and_expires_run_keys() -
assert ("execute",) in redis.commands
+def test_append_event_keeps_only_the_configured_number_of_entries() -> None:
+ redis = FakeRedis()
+ store = RedisRunStore(
+ redis, # pyright: ignore[reportArgumentType]
+ prefix="test",
+ run_retention_seconds=60,
+ run_event_stream_max_length=2,
+ )
+
+ async def scenario() -> None:
+ for _ in range(3):
+ _ = await store.append_event(RunStartedEvent(run_id="run-1"))
+
+ asyncio.run(scenario())
+
+ assert len(redis.streams["test:runs:run-1:events"]) == 2
+ xadd_commands = [command for command in redis.commands if command[0] == "xadd"]
+ assert all(command[3:] == (2, True) for command in xadd_commands)
+
+
def test_get_events_round_trips_run_succeeded_output_and_session_snapshot() -> None:
redis = FakeRedis()
store = RedisRunStore(redis, prefix="test", run_retention_seconds=60) # pyright: ignore[reportArgumentType]
diff --git a/docker/envs/core-services/dify-agent.env.example b/docker/envs/core-services/dify-agent.env.example
index e4ab19342a3..c7ac2458c4e 100644
--- a/docker/envs/core-services/dify-agent.env.example
+++ b/docker/envs/core-services/dify-agent.env.example
@@ -8,7 +8,11 @@ AGENT_BACKEND_BASE_URL=http://agent_backend:5050
DIFY_AGENT_REDIS_URL=
DIFY_AGENT_REDIS_PREFIX=dify-agent
DIFY_AGENT_SHUTDOWN_GRACE_SECONDS=30
-DIFY_AGENT_RUN_RETENTION_SECONDS=259200
+DIFY_AGENT_RUN_RETENTION_SECONDS=7200
+DIFY_AGENT_RUN_EVENT_STREAM_MAX_LENGTH=5000
+DIFY_AGENT_STREAM_TEXT_DELTA_COALESCING_ENABLED=true
+DIFY_AGENT_STREAM_TEXT_DELTA_FLUSH_INTERVAL_MS=100
+DIFY_AGENT_STREAM_TEXT_DELTA_MAX_CHARS=4096
# Internal deadline for the Agent Backend model/tool loop. The effective run
# limit is whichever expires first: this value or the relevant API/Workflow
# outer execution limit. To allow every path to run for one hour, also set