Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
Python agentagentic-aiagentic-frameworkagentic-workflowaiautomationclaudedeepseekgenaigptllmlow-codemcpnextjsno-codeopenaipythonskillsworkflow
refactor(docker): name API and worker configuration explicitly (#45376)
-LAN-1c8aac74b8ff
diff --git a/docker/README.md b/docker/README.mdindex 6123e4c37e1..0ba7dad7033 100644--- a/docker/README.md+++ b/docker/README.md@@ -85,13 +85,13 @@ in the settings before use.Find settings in:- [Core services](envs/core-services/): API, workers, web, plugin daemon, and- sandboxes. [shared.defaults](envs/core-services/shared.defaults) contains the+ sandboxes. [api-worker.defaults](envs/core-services/api-worker.defaults) contains thecommon API, worker, and Beat settings, grouped by function. API process optionslive in `api.defaults`; worker autoscaling lives in `worker.defaults`.- [Databases](envs/databases/): PostgreSQL, MySQL, and Redis.- [Vector stores](envs/vectorstores/): bundled and external vector databases,including their API/worker client settings. Providers without a dedicated- template are documented in `shared.defaults`.+ template are documented in `api-worker.defaults`.- [Infrastructure](envs/infrastructure/): nginx, Certbot, proxies, and supporting services.### Change a service setting@@ -226,11 +226,13 @@ Bundled service templates now use the `.defaults` extension. Existing service`.env` paths still work. Root `.env.example` remains the only required startingtemplate, and the Compose generator creates only a missing root `.env`.-The shared application reference is now consolidated in-[shared.defaults](envs/core-services/shared.defaults). Existing `shared.env`-and other override paths still work; do not replace or rename your local files.-New customizations use the matching template listed above. Root `.env` retains-precedence, including for settings that have moved to a provider template.+Common API and worker application settings are documented in+[api-worker.defaults](envs/core-services/api-worker.defaults). New customizations+use `envs/core-services/api-worker.env`. Existing `shared.env` files and other+service override paths remain supported; keep local files and edit them in place.+If both `shared.env` and `api-worker.env` exist, `api-worker.env` takes precedence+for matching settings. Root `.env` retains precedence over both files, including+for settings documented in a provider template.Review accidental empty assignments left by older template copies. For example,`CELERY_WORKER_AMOUNT=` overrides the Docker default of `4` and lets the worker@@ -275,7 +277,7 @@ Weaviate. Update that value if you need different profiles.For HTTPS certificates, follow the [Certbot guide](certbot/README.md).-For OpenTelemetry, create `envs/core-services/shared.env` from `shared.defaults`+For OpenTelemetry, create `envs/core-services/api-worker.env` from `api-worker.defaults`if it does not exist. Uncomment `ENABLE_OTEL` and set it to `true`; uncomment`OTLP_BASE_ENDPOINT` and set it to your collector's endpoint, reachable fromthe API and worker containers. The same file documents optional `OTEL_*`@@ -298,7 +300,7 @@ provider's interaction configuration after changing the public origin.### Ops tracing rolling upgradesNew tracing tasks use `OPS_TRACE_QUEUE=ops_trace_v2` by default. Set it consistently on the API,-workers, and Beat in `docker/envs/core-services/shared.env` (or `docker/.env`). Default worker+workers, and Beat in `docker/envs/core-services/api-worker.env` (or `docker/.env`). Default workersubscriptions follow this value; explicit `CELERY_QUEUES` or `CELERY_WORKER_QUEUES` lists mustinclude it and exclude legacy `ops_trace` while old workers are draining.@@ -445,10 +447,10 @@ For a directly exposed API, leave `RESPECT_XFORWARD_HEADERS_ENABLED=false`.The Python configuration and `api/.env.example` retain that default. To overridethe Compose default, set the variable in `docker/.env`, the shell used by Compose,or a Compose override. The API services' explicit `environment` entry takes-precedence over `envs/core-services/shared.env`.+precedence over `envs/core-services/api-worker.env` and legacy `shared.env` files.Existing `false` overrides in `docker/.env`, the invoking shell, or a Composeoverride remain in effect during upgrades. Move settings previously supplied only-by `shared.env` to one of those override locations if you need to preserve them.+by `api-worker.env` or `shared.env` to one of those override locations if you need to preserve them.After establishing the trusted ingress boundary, enable the setting to retainvisitor IPs.diff --git a/docker/docker-compose-template.yaml b/docker/docker-compose-template.yamlindex fd6f5e33799..c7bdd785021 100644--- a/docker/docker-compose-template.yaml+++ b/docker/docker-compose-template.yaml@@ -3,6 +3,8 @@ x-shared-api-config: &shared-api-configenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -60,7 +62,7 @@ x-shared-api-config: &shared-api-config- path: ./envs/core-services/api.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/api.defaultsnetworks:- ssrf_proxy_network@@ -71,6 +73,8 @@ x-shared-worker-config: &shared-worker-configenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -128,7 +132,7 @@ x-shared-worker-config: &shared-worker-config- path: ./envs/core-services/worker.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/worker.defaultsnetworks:- ssrf_proxy_network@@ -147,6 +151,8 @@ x-knowledge-fs-celery-worker: &knowledge-fs-celery-workerenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -162,7 +168,7 @@ x-knowledge-fs-celery-worker: &knowledge-fs-celery-worker- path: ./envs/core-services/worker.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/worker.defaultsenvironment: &knowledge-fs-celery-environmentMODE: worker@@ -186,6 +192,8 @@ x-shared-worker-beat-config: &shared-worker-beat-configenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -243,7 +251,7 @@ x-shared-worker-beat-config: &shared-worker-beat-config- path: ./envs/core-services/worker-beat.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/worker-beat.defaultsnetworks:- ssrf_proxy_network@@ -600,6 +608,8 @@ services:env_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/core-services/plugin-daemon.envrequired: false- path: ./envs/security.env@@ -611,7 +621,7 @@ services:- path: ./envs/databases/redis.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/plugin-daemon.defaultsnetworks:- ssrf_proxy_networkdiff --git a/docker/docker-compose.yaml b/docker/docker-compose.yamlindex 347d6fd4017..1ccf4b38632 100644--- a/docker/docker-compose.yaml+++ b/docker/docker-compose.yaml@@ -9,6 +9,8 @@ x-shared-api-config: &shared-api-configenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -66,7 +68,7 @@ x-shared-api-config: &shared-api-config- path: ./envs/core-services/api.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/api.defaultsnetworks:- ssrf_proxy_network@@ -77,6 +79,8 @@ x-shared-worker-config: &shared-worker-configenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -134,7 +138,7 @@ x-shared-worker-config: &shared-worker-config- path: ./envs/core-services/worker.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/worker.defaultsnetworks:- ssrf_proxy_network@@ -153,6 +157,8 @@ x-knowledge-fs-celery-worker: &knowledge-fs-celery-workerenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -168,7 +174,7 @@ x-knowledge-fs-celery-worker: &knowledge-fs-celery-worker- path: ./envs/core-services/worker.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/worker.defaultsenvironment: &knowledge-fs-celery-environmentMODE: worker@@ -192,6 +198,8 @@ x-shared-worker-beat-config: &shared-worker-beat-configenv_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/security.envrequired: false- path: ./envs/databases/db-postgres.env@@ -249,7 +257,7 @@ x-shared-worker-beat-config: &shared-worker-beat-config- path: ./envs/core-services/worker-beat.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/worker-beat.defaultsnetworks:- ssrf_proxy_network@@ -606,6 +614,8 @@ services:env_file:- path: ./envs/core-services/shared.envrequired: false+ - path: ./envs/core-services/api-worker.env+ required: false- path: ./envs/core-services/plugin-daemon.envrequired: false- path: ./envs/security.env@@ -617,7 +627,7 @@ services:- path: ./envs/databases/redis.envrequired: false- ./.env- - ./envs/core-services/shared.defaults+ - ./envs/core-services/api-worker.defaults- ./envs/core-services/plugin-daemon.defaultsnetworks:- ssrf_proxy_networkdiff --git a/docker/envs/core-services/shared.defaults b/docker/envs/core-services/api-worker.defaultssimilarity index 98%rename from docker/envs/core-services/shared.defaultsrename to docker/envs/core-services/api-worker.defaultsindex d8d4dafa9e2..b149783833e 100644--- a/docker/envs/core-services/shared.defaults+++ b/docker/envs/core-services/api-worker.defaults@@ -1,5 +1,6 @@-# Shared API, worker and Beat configuration. Copy this file to shared.env to customize it.+# Common API and worker configuration, including Beat. Copy this file to api-worker.env to customize it.# Edit the copy; docker/.env takes precedence. Existing shared.env files remain supported.+# If both override files exist, api-worker.env takes precedence over shared.env.# Active assignments preserve Docker defaults and explicit overrides, including empty values.# Optional application/provider settings are commented examples: uncomment only what you need.# Do not uncomment empty values or placeholders unless you intend to set them.@@ -77,7 +78,7 @@ CELERY_WORKER_AMOUNT=${CELERY_WORKER_AMOUNT-4}# Celery broker and KnowledgeFS dispatch# ------------------------------# Broker URL defaults/mappings live in api.defaults, worker.defaults and worker-beat.defaults.-# Set a common CELERY_BROKER_URL override in shared.env or docker/.env for every producer/consumer.+# Set a common CELERY_BROKER_URL override in api-worker.env or docker/.env for every producer/consumer.# Optional long-running KnowledgeFS Celery rollout: set identically in every API, worker and Beat# process sharing this Redis broker. It must exceed the KnowledgeFS hard task limit (7230s by# default). Keep unset to preserve Celery's existing default when this rollout is disabled.diff --git a/docker/tests/test_compose_environment.py b/docker/tests/test_compose_environment.pyindex 8de74992ea6..672bf2566c8 100644--- a/docker/tests/test_compose_environment.py+++ b/docker/tests/test_compose_environment.py@@ -116,9 +116,17 @@ class ComposeEnvironmentTests(unittest.TestCase):def test_api_and_weaviate_defaults_and_isolated_overrides(self):cases = [- ("core-services/shared", "worker", "CELERY_WORKER_AMOUNT", "CELERY_WORKER_AMOUNT", "4", "2", "3"),- ("core-services/shared", "api", "SERVER_WORKER_CLASS", "SERVER_WORKER_CLASS", "gevent", "sync", "gthread"),- ("core-services/shared", "api", "GUNICORN_TIMEOUT", "GUNICORN_TIMEOUT", "360", "120", "240"),+ ("core-services/api-worker", "worker", "CELERY_WORKER_AMOUNT", "CELERY_WORKER_AMOUNT", "4", "2", "3"),+ (+ "core-services/api-worker",+ "api",+ "SERVER_WORKER_CLASS",+ "SERVER_WORKER_CLASS",+ "gevent",+ "sync",+ "gthread",+ ),+ ("core-services/api-worker", "api", "GUNICORN_TIMEOUT", "GUNICORN_TIMEOUT", "360", "120", "240"),("vectorstores/weaviate","weaviate",@@ -156,12 +164,54 @@ class ComposeEnvironmentTests(unittest.TestCase):(self.project / target).unlink(missing_ok=True)root.write_text(initial_root)- def test_copying_shared_defaults_does_not_activate_optional_settings(self):+ def test_copying_api_worker_defaults_does_not_activate_optional_settings(self):baseline = self.render()- self.copy_template("envs/core-services/shared.defaults", "envs/core-services/shared.env")+ self.copy_template("envs/core-services/api-worker.defaults", "envs/core-services/api-worker.env")# Compare the entire configuration, including extra variables introduced by an unsafe copy.self.assertEqual(self.render(), baseline)+ def test_api_worker_overrides_preserve_legacy_settings_and_precedence(self):+ consumers = (+ "api",+ "api_websocket",+ "worker",+ "worker_knowledge_fs_upgrade",+ "worker_beat",+ "knowledge_fs_document_worker",+ "knowledge_fs_source_worker",+ "knowledge_fs_research_worker",+ "knowledge_fs_maintenance_worker",+ "knowledge_fs_dispatch_worker",+ "plugin_daemon",+ )+ baseline = self.render()+ legacy = self.project / "envs/core-services/shared.env"+ legacy.write_text("DB_HOST=legacy-db\n")++ services = self.render()+ for name in consumers:+ self.assertEqual(services[name]["environment"]["DB_HOST"], "legacy-db", name)++ source = "envs/core-services/api-worker.defaults"+ target = "envs/core-services/api-worker.env"+ # An unchanged copy must not mask existing shared.env customizations.+ self.copy_template(source, target)+ self.assertEqual(self.render(), services)++ self.copy_template(source, target, {"DB_HOST": "api-worker-db"})+ services = self.render()+ for name in consumers:+ self.assertEqual(services[name]["environment"]["DB_HOST"], "api-worker-db", name)+ for name in baseline.keys() - set(consumers):+ self.assertEqual(services[name], baseline[name], name)++ # Change only the root file while retaining both local override files.+ self.override_root({"DB_HOST": "root-db"})+ services = self.render()+ for name in consumers:+ self.assertEqual(services[name]["environment"]["DB_HOST"], "root-db", name)+ self.assertEqual(legacy.read_text(), "DB_HOST=legacy-db\n")+def test_consolidated_settings_reach_consumers_and_respect_root_precedence(self):api_services = ("api", "api_websocket")worker_services = (@@ -176,7 +226,7 @@ class ComposeEnvironmentTests(unittest.TestCase):vector_clients = (*api_services, "worker", "worker_knowledge_fs_upgrade", "worker_beat")cases = [(- "core-services/shared.defaults",+ "core-services/api-worker.defaults",(*api_services, *worker_services, "worker_beat"),{"CODE_MAX_STRING_LENGTH": ("500000", "600000"),@@ -369,7 +419,7 @@ class ComposeEnvironmentTests(unittest.TestCase):def test_database_redis_overrides_reach_clients_and_commands(self):for name in (- "shared",+ "api-worker","api","worker","worker-beat",@@ -451,7 +501,7 @@ class ComposeEnvironmentTests(unittest.TestCase):self.assertFalse((self.project / "should-not-exist").exists())def test_root_plugin_credentials_reach_every_consumer(self):- for name in ("shared", "api", "worker", "worker-beat", "plugin-daemon", "dify-agent"):+ for name in ("api-worker", "api", "worker", "worker-beat", "plugin-daemon", "dify-agent"):self.copy_template(f"envs/core-services/{name}.defaults", f"envs/core-services/{name}.env")api_services = ("api",