Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
Python agentagentic-aiagentic-frameworkagentic-workflowaiautomationclaudedeepseekgenaigptllmlow-codemcpnextjsno-codeopenaipythonskillsworkflow
fix(docker): load bundled vector store connection defaults (#45407)
-LAN-b7c7b2951708
diff --git a/docker/.env.example b/docker/.env.exampleindex f3edbfed600..60bc1889ad1 100644--- a/docker/.env.example+++ b/docker/.env.example@@ -3,7 +3,7 @@# Application and Docker defaults are loaded automatically; do not repeat them here.# Copy this file to .env, then select the services for your deployment.# Optional overrides and provider settings are documented in envs/.-# Values in .env take precedence over the matching envs/*.env files.+# See README.md for precedence; each service loads only its assigned inputs.# Compose requires an explicit selection of the bundled database, vector store,# and websocket service. Match these profiles to your DB_TYPE and VECTOR_STOREdiff --git a/docker/README.md b/docker/README.mdindex 0ba7dad7033..dd0a06df878 100644--- a/docker/README.md+++ b/docker/README.md@@ -72,7 +72,8 @@ and configure [HTTPS](#configure-https-and-telemetry) and| `.env` | Deployment inputs and explicit overrides, including Compose settings | Yes || `envs/**/*.defaults` | Tracked defaults for the matching services | Yes, automatically || `envs/**/*.env` | Optional service or provider overrides | Yes, when the matching service references the file |-| `envs/**/*.env.example` | Templates for optional application and provider settings | No; remove only `.example` when copying |+| `envs/**/*.env.example` | Templates for optional application and provider settings | No; remove `.example`, except for the middleware template below |+| `envs/middleware.env.example` | Template for the separate source-code development stack | No; copy to `middleware.env` in `docker/` |Keep settings with usable defaults out of the root `.env.example`. To customize aservice, copy its `.defaults` file to the override path named in its comments,@@ -165,24 +166,102 @@ These optional overrides do not need to be added to `.env.example`.### Select a database or vector store-Set `DB_TYPE` or `VECTOR_STORE` and the provider's connection settings in the-appropriate override file. In root `.env`, update `COMPOSE_PROFILES` to select the-matching bundled services. Omit a service's profile when that service runs-externally. Starting a different database does not migrate existing data.+In root `.env`, set `VECTOR_STORE` and update `COMPOSE_PROFILES` to select the+matching bundled vector store. Compose supplies its connection defaults to the+API, workers and Beat automatically. For example, a new deployment with Qdrant+and PostgreSQL uses:++```dotenv+VECTOR_STORE=qdrant+COMPOSE_PROFILES=qdrant,postgresql,collaboration+```++Changing `VECTOR_STORE` selects the default for new indexes. Existing knowledge+bases keep using their stored backend, so preserve its connection settings and+keep its service running. If any use bundled Weaviate, retain its profile with+`COMPOSE_PROFILES=weaviate,qdrant,postgresql,collaboration` for this example.++Bundled Weaviate connection defaults remain available when another provider is+selected. An unchanged bundled service needs no additional connection settings;+existing overrides in `weaviate.env`, `api-worker.env`, and `shared.env` still+apply, with root `.env` taking precedence.++To customize the connection, copy the provider's `.defaults` file under+`envs/vectorstores/` to its `.env` path if that file does not already exist. Edit+the copy or the existing `.env` file. Elasticsearch's two selectors use+`elasticsearch.env`. Existing provider `.env` and `shared.env`+overrides remain supported. Keep `VECTOR_STORE` in root `.env`, since Compose+selects the defaults file before reading service files.++For an external vector store, omit its bundled profile and provide its connection+settings and credentials. For databases with stored user passwords, change the+password in the database before updating the corresponding client setting.+Starting a different database or vector store does not migrate existing data.++For OpenSearch HTTPS connections, `OPENSEARCH_VERIFY_CERTS` defaults to `true`+whenever `VECTOR_STORE=opensearch`, even without the bundled `opensearch` profile.+Keep verification enabled for production and external HTTPS servers, and use+certificates trusted by the client.++The bundled OpenSearch image uses demo certificates. For local development with+those certificates, explicitly set `OPENSEARCH_VERIFY_CERTS=false` in+`envs/vectorstores/opensearch.env` or root `.env`. To keep verification enabled,+configure certificates trusted by the client.For example, the default deployment selects PostgreSQL and Weaviate with`COMPOSE_PROFILES=weaviate,postgresql,collaboration`. The `collaboration` profile-starts the WebSocket API. See the [database](envs/databases/) and+starts the WebSocket API. Set `DB_TYPE` and the SQL database's connection settings+when changing that provider. See the [database](envs/databases/) and[vector store](envs/vectorstores/) files for provider-specific settings.+For a new deployment using bundled MySQL and Weaviate, set the complete connection+in root `.env`:++```dotenv+DB_TYPE=mysql+DB_HOST=db_mysql+DB_PORT=3306+DB_USERNAME=root+COMPOSE_PROFILES=weaviate,mysql,collaboration+```++Keep your selected vector-store profile if it differs from this example. The+MySQL profile starts the database; it does not replace the PostgreSQL connection+defaults. Set `DB_PASSWORD` to the same value used to initialize the MySQL server.+Changing these settings on an existing PostgreSQL deployment does not migrate its data.++### Configure sandbox credentials++Set `DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN` only in root `.env`, using the secret+generated during setup. Compose supplies it to both the Agent backend and its+local sandbox. Move any existing copy of this token from `dify-agent.env` to root+`.env` and remove the duplicate; keep the same secret during the move.++The separate Code sandbox uses two settings that must match: `SANDBOX_API_KEY`+in `envs/core-services/sandbox.env` and `CODE_EXECUTION_API_KEY` in+`envs/security.env` (copied from `security.env.example`). Use the same generated+secret in both files, then run `docker compose up -d` to apply both changes.+You can also set both variables to the same value in root `.env`.+### Understand which value takes effect-For settings loaded through these env files, precedence is:+For services that load root `.env`, precedence is:1. Root `.env` overrides.2. Optional service `.env` overrides.3. Tracked `.defaults` values, followed by application defaults for unset settings.+Other services, including databases, vector stores, and nginx, receive root values+only through `${...}` expressions in their defaults or explicit Compose mappings.+For those inputs, root `.env` takes precedence over the service file. Arbitrary+native container variables do not inherit this precedence: for example, put+Weaviate's `ASYNC_INDEXING` in `envs/vectorstores/weaviate.env`; adding it to root+`.env` alone does not configure the Weaviate container.++The `knowledge_fs_unstructured` parser loads its defaults followed by+`envs/core-services/knowledge-fs-unstructured.env`, without loading root `.env`.+Set its parsing options in that service file.+Shell variables take precedence when Compose interpolates `${...}` expressions.An explicit Compose `environment` entry also takes precedence over service envfiles; [trusted-proxy settings](#client-ip-and-trusted-proxies) are one example.@@ -214,7 +293,7 @@ not application readiness.| A service still uses its old value | Run `docker compose up -d`. Check for a higher-priority value in root `.env`, the shell, or an explicit Compose `environment` entry. || A port or profile change has no effect | Put the setting in root `.env`; these settings cannot be supplied through a service env file. || Service overrides resolve unexpectedly | Check that Docker Compose is v2.35.0 or later and that container-variable mapping expressions are intact. |-| Python sandbox code returns `operation not permitted` | Check the network-setting note in [sandbox.defaults](envs/core-services/sandbox.defaults). On sandbox 0.2.15/ARM64, requests with networking disabled can trigger a runtime syscall restriction. |+| Python sandbox code returns `operation not permitted` | Check the network-setting note in [sandbox.defaults](envs/core-services/sandbox.defaults). Sandbox 0.2.15 restricts the `epoll_pwait` syscall when networking is disabled on both ARM64 and AMD64. The runtime failure has been reproduced on ARM64. |## Review configuration after an upgrade@@ -222,6 +301,10 @@ Review the new root `.env.example` for required deployment inputs and the matchi`.defaults` files for changed service defaults. Preserve your existing `.env`files; a copied service file can pin values from an older release.+Review any saved `OPENSEARCH_VERIFY_CERTS=false`: it stays in effect after an+upgrade. Remove the assignment or set it to `true` to enable certificate+verification.+Bundled service templates now use the `.defaults` extension. Existing service`.env` paths still work. Root `.env.example` remains the only required startingtemplate, and the Compose generator creates only a missing root `.env`.diff --git a/docker/docker-compose-template.yaml b/docker/docker-compose-template.yamlindex b1ef5d85dff..d589f221295 100644--- a/docker/docker-compose-template.yaml+++ b/docker/docker-compose-template.yaml@@ -62,6 +62,8 @@ x-shared-api-config: &shared-api-config- path: ./envs/core-services/api.envrequired: false- ./.env+ - path: ./envs/vectorstores/${VECTOR_STORE:-weaviate}.defaults+ required: false- ./envs/core-services/api-worker.defaults- ./envs/core-services/api.defaultsnetworks:@@ -132,6 +134,8 @@ x-shared-worker-config: &shared-worker-config- path: ./envs/core-services/worker.envrequired: false- ./.env+ - path: ./envs/vectorstores/${VECTOR_STORE:-weaviate}.defaults+ required: false- ./envs/core-services/api-worker.defaults- ./envs/core-services/worker.defaultsnetworks:@@ -251,6 +255,8 @@ x-shared-worker-beat-config: &shared-worker-beat-config- path: ./envs/core-services/worker-beat.envrequired: false- ./.env+ - path: ./envs/vectorstores/${VECTOR_STORE:-weaviate}.defaults+ required: false- ./envs/core-services/api-worker.defaults- ./envs/core-services/worker-beat.defaultsnetworks:@@ -744,6 +750,8 @@ services:required: false- ./.env- ./envs/core-services/dify-agent.defaults+ environment:+ DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN: ${DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN:-${DIFY_AGENT_SHELLCTL_AUTH_TOKEN:-}}depends_on:redis:condition: service_started@@ -1176,6 +1184,7 @@ services:- 9091:9091networks:- milvus+ - default# Opensearch vector databaseopensearch:@@ -1198,6 +1207,7 @@ services:- ./volumes/opensearch/data:/usr/share/opensearch/datanetworks:- opensearch-net+ - defaultopensearch-dashboards:container_name: opensearch-dashboardsdiff --git a/docker/docker-compose.yaml b/docker/docker-compose.yamlindex 71ddde39ff0..141b63fa1ac 100644--- a/docker/docker-compose.yaml+++ b/docker/docker-compose.yaml@@ -68,6 +68,8 @@ x-shared-api-config: &shared-api-config- path: ./envs/core-services/api.envrequired: false- ./.env+ - path: ./envs/vectorstores/${VECTOR_STORE:-weaviate}.defaults+ required: false- ./envs/core-services/api-worker.defaults- ./envs/core-services/api.defaultsnetworks:@@ -138,6 +140,8 @@ x-shared-worker-config: &shared-worker-config- path: ./envs/core-services/worker.envrequired: false- ./.env+ - path: ./envs/vectorstores/${VECTOR_STORE:-weaviate}.defaults+ required: false- ./envs/core-services/api-worker.defaults- ./envs/core-services/worker.defaultsnetworks:@@ -257,6 +261,8 @@ x-shared-worker-beat-config: &shared-worker-beat-config- path: ./envs/core-services/worker-beat.envrequired: false- ./.env+ - path: ./envs/vectorstores/${VECTOR_STORE:-weaviate}.defaults+ required: false- ./envs/core-services/api-worker.defaults- ./envs/core-services/worker-beat.defaultsnetworks:@@ -750,6 +756,8 @@ services:required: false- ./.env- ./envs/core-services/dify-agent.defaults+ environment:+ DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN: ${DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN:-${DIFY_AGENT_SHELLCTL_AUTH_TOKEN:-}}depends_on:redis:condition: service_started@@ -1182,6 +1190,7 @@ services:- 9091:9091networks:- milvus+ - default# Opensearch vector databaseopensearch:@@ -1204,6 +1213,7 @@ services:- ./volumes/opensearch/data:/usr/share/opensearch/datanetworks:- opensearch-net+ - defaultopensearch-dashboards:container_name: opensearch-dashboardsdiff --git a/docker/envs/core-services/api-worker.defaults b/docker/envs/core-services/api-worker.defaultsindex b149783833e..db972b0d301 100644--- a/docker/envs/core-services/api-worker.defaults+++ b/docker/envs/core-services/api-worker.defaults@@ -460,12 +460,14 @@ OPENDAL_FS_ROOT=${OPENDAL_FS_ROOT-storage}# ------------------------------# Vector store selection and Docker connection defaults# -------------------------------# Provider client options live in ../vectorstores/{weaviate,qdrant,oceanbase,pgvector}.defaults-# and ../vectorstores/milvus.env.example. Their .env overrides reach API, workers and Beat.+# Set VECTOR_STORE and COMPOSE_PROFILES in docker/.env to select a bundled provider.+# Compose loads that provider's ../vectorstores/<VECTOR_STORE>.defaults automatically.+# Provider .env overrides reach API, workers and Beat; existing shared.env overrides still work.# KnowledgeFS uses the API vector bridge, not its own vector provider connection.VECTOR_STORE=${VECTOR_STORE-weaviate}# Prefix used to create collection names in the vector database.# VECTOR_INDEX_NAME_PREFIX=Vector_index+# Keep legacy Weaviate connections for existing datasets after changing VECTOR_STORE.WEAVIATE_ENDPOINT=${WEAVIATE_ENDPOINT-http://weaviate:8080}WEAVIATE_API_KEY=${WEAVIATE_API_KEY-WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih}WEAVIATE_GRPC_ENDPOINT=${WEAVIATE_GRPC_ENDPOINT-grpc://weaviate:50051}diff --git a/docker/envs/core-services/dify-agent.defaults b/docker/envs/core-services/dify-agent.defaultsindex 82e7183dcbe..b5f43f38870 100644--- a/docker/envs/core-services/dify-agent.defaults+++ b/docker/envs/core-services/dify-agent.defaults@@ -62,8 +62,7 @@ DIFY_AGENT_INNER_API_KEY=${DIFY_AGENT_INNER_API_KEY:-${PLUGIN_DIFY_INNER_API_KEY# Select exactly one coherent Home Snapshot + Sandbox backend.DIFY_AGENT_RUNTIME_BACKEND=${DIFY_AGENT_RUNTIME_BACKEND:-local}DIFY_AGENT_LOCAL_SANDBOX_ENDPOINT=${DIFY_AGENT_LOCAL_SANDBOX_ENDPOINT:-${DIFY_AGENT_SHELLCTL_ENTRYPOINT:-http://local_sandbox:5004}}-# Required nonblank secret for Local. Set in docker/.env to also configure shellctl.-DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN=${DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN:-${DIFY_AGENT_SHELLCTL_AUTH_TOKEN:-}}+# Set DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN only in docker/.env; Compose supplies it to both services.# E2B_API_KEY and E2B_API_TOKEN remain accepted as deployment-level fallbacks.DIFY_AGENT_E2B_API_KEY=${DIFY_AGENT_E2B_API_KEY:-${E2B_API_KEY:-${E2B_API_TOKEN:-}}}DIFY_AGENT_E2B_TEMPLATE=${DIFY_AGENT_E2B_TEMPLATE:-difys-default-team/dify-agent-local-sandbox}diff --git a/docker/envs/core-services/sandbox.defaults b/docker/envs/core-services/sandbox.defaultsindex 45b1d9e312c..db0a1074cd1 100644--- a/docker/envs/core-services/sandbox.defaults+++ b/docker/envs/core-services/sandbox.defaults@@ -11,10 +11,10 @@ PIP_MIRROR_URL=${PIP_MIRROR_URL:-}SANDBOX_API_KEY=${SANDBOX_API_KEY:-dify-sandbox}SANDBOX_GIN_MODE=${SANDBOX_GIN_MODE:-release}SANDBOX_WORKER_TIMEOUT=${SANDBOX_WORKER_TIMEOUT:-15}-# Dify's code executor requests networking; false rejects those requests.-# Sandbox 0.2.15 on ARM64 can also fail with "operation not permitted" for-# requests with enable_network=false, even without network operations:-# the runtime needs epoll_pwait, which is only allowed with networking enabled.+# Dify's code executor requests networking for Code, Template, Jinja2 prompts and the Code tool.+# Setting false rejects those requests on every architecture.+# Sandbox 0.2.15 only allows epoll_pwait with networking enabled on both ARM64 and AMD64;+# network-disabled Python requests can fail with "operation not permitted" (observed on ARM64).SANDBOX_ENABLE_NETWORK=${SANDBOX_ENABLE_NETWORK:-true}# Container variable mappings; keep these expressions unchanged in the copy.diff --git a/docker/envs/databases/db-mysql.defaults b/docker/envs/databases/db-mysql.defaultsindex 54c7db22388..8d095b46743 100644--- a/docker/envs/databases/db-mysql.defaults+++ b/docker/envs/databases/db-mysql.defaults@@ -1,6 +1,8 @@# Defaults for db_mysql. Copy this file to db-mysql.env to customize it.# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.+# For bundled MySQL, set DB_TYPE=mysql, DB_HOST=db_mysql, DB_PORT=3306 and DB_USERNAME=root+# in docker/.env, and replace postgresql with mysql in COMPOSE_PROFILES.MYSQL_INNODB_LOG_FILE_SIZE=${MYSQL_INNODB_LOG_FILE_SIZE:-128M}MYSQL_INNODB_FLUSH_LOG_AT_TRX_COMMIT=${MYSQL_INNODB_FLUSH_LOG_AT_TRX_COMMIT:-2}diff --git a/docker/envs/middleware.env.example b/docker/envs/middleware.env.exampleindex fcd58cef5a9..bd5ad4ecace 100644--- a/docker/envs/middleware.env.example+++ b/docker/envs/middleware.env.example@@ -1,3 +1,5 @@+# Copy to docker/middleware.env for the source-code development stack.+# From docker/, run: docker compose --env-file middleware.env -f docker-compose.middleware.yaml -p dify up -d# ------------------------------# Environment Variables for db Service# ------------------------------diff --git a/docker/envs/vectorstores/chroma.defaults b/docker/envs/vectorstores/chroma.defaultsindex e28ce786cff..e967b096799 100644--- a/docker/envs/vectorstores/chroma.defaults+++ b/docker/envs/vectorstores/chroma.defaults@@ -2,15 +2,16 @@# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.-# CHROMA_DATABASE=default_database-# CHROMA_AUTH_PROVIDER=chromadb.auth.token_authn.TokenAuthClientProvider-# CHROMA_AUTH_CREDENTIALS=-# CHROMA_HOST=127.0.0.1-# CHROMA_PORT=8000-# CHROMA_TENANT=default_tenantCHROMA_SERVER_AUTHN_CREDENTIALS=${CHROMA_SERVER_AUTHN_CREDENTIALS:-difyai123456}CHROMA_SERVER_AUTHN_PROVIDER=${CHROMA_SERVER_AUTHN_PROVIDER:-chromadb.auth.token_authn.TokenAuthenticationServerProvider}CHROMA_IS_PERSISTENT=${CHROMA_IS_PERSISTENT:-TRUE}+# CHROMA_DATABASE=default_database+CHROMA_AUTH_PROVIDER=${CHROMA_AUTH_PROVIDER-chromadb.auth.token_authn.TokenAuthClientProvider}+CHROMA_AUTH_CREDENTIALS=${CHROMA_AUTH_CREDENTIALS-${CHROMA_SERVER_AUTHN_CREDENTIALS:-difyai123456}}+CHROMA_HOST=${CHROMA_HOST-chroma}+# CHROMA_PORT=8000+# CHROMA_TENANT=default_tenant+# Container variable mappings; keep these expressions unchanged in the copy.IS_PERSISTENT=${CHROMA_IS_PERSISTENT:-TRUE}diff --git a/docker/envs/vectorstores/couchbase.defaults b/docker/envs/vectorstores/couchbase.defaultsindex a5137e6f6a0..fe12b0080c1 100644--- a/docker/envs/vectorstores/couchbase.defaults+++ b/docker/envs/vectorstores/couchbase.defaults@@ -2,11 +2,11 @@# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.-COUCHBASE_PASSWORD=${COUCHBASE_PASSWORD:-password}-COUCHBASE_BUCKET_NAME=${COUCHBASE_BUCKET_NAME:-Embeddings}-# COUCHBASE_SCOPE_NAME=_default-# COUCHBASE_CONNECTION_STRING=couchbase://couchbase-server-COUCHBASE_USER=${COUCHBASE_USER:-Administrator}+COUCHBASE_PASSWORD=${COUCHBASE_PASSWORD-password}+COUCHBASE_BUCKET_NAME=${COUCHBASE_BUCKET_NAME-Embeddings}+COUCHBASE_SCOPE_NAME=${COUCHBASE_SCOPE_NAME-_default}+COUCHBASE_CONNECTION_STRING=${COUCHBASE_CONNECTION_STRING-couchbase://couchbase-server}+COUCHBASE_USER=${COUCHBASE_USER-Administrator}# Container variable mappings; keep these expressions unchanged in the copy.COUCHBASE_ADMINISTRATOR_USERNAME=${COUCHBASE_USER:-Administrator}diff --git a/docker/envs/vectorstores/elasticsearch-ja.defaults b/docker/envs/vectorstores/elasticsearch-ja.defaultsnew file mode 100644index 00000000000..9117d34e292--- /dev/null+++ b/docker/envs/vectorstores/elasticsearch-ja.defaults@@ -0,0 +1,5 @@+# Client defaults for the elasticsearch-ja selector, which uses the elasticsearch service.+# Customize envs/vectorstores/elasticsearch.env (copied from elasticsearch.defaults).+# docker/.env takes precedence; the same overrides apply to both Elasticsearch variants.+ELASTICSEARCH_HOST=${ELASTICSEARCH_HOST-elasticsearch}+ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD-elastic}diff --git a/docker/envs/vectorstores/elasticsearch.defaults b/docker/envs/vectorstores/elasticsearch.defaultsindex 39314377f35..d2f699f2a08 100644--- a/docker/envs/vectorstores/elasticsearch.defaults+++ b/docker/envs/vectorstores/elasticsearch.defaults@@ -3,7 +3,7 @@# Optional application settings are commented to retain their application defaults.# ELASTICSEARCH_CLOUD_URL=YOUR-ELASTICSEARCH_CLOUD_URL-ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD:-elastic}+ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD-elastic}# Set KIBANA_PORT in docker/.env; Compose needs it before loading service files.# ELASTICSEARCH_USE_CLOUD=false# ELASTICSEARCH_API_KEY=YOUR-ELASTICSEARCH_API_KEY@@ -12,7 +12,7 @@ ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD:-elastic}# ELASTICSEARCH_REQUEST_TIMEOUT=100000# ELASTICSEARCH_RETRY_ON_TIMEOUT=True# ELASTICSEARCH_MAX_RETRIES=10-# ELASTICSEARCH_HOST=0.0.0.0+ELASTICSEARCH_HOST=${ELASTICSEARCH_HOST-elasticsearch}# Set ELASTICSEARCH_PORT in docker/.env; Compose needs it before loading service files.# ELASTICSEARCH_USERNAME=elastic# Set VECTOR_STORE in docker/.env to select the backend and its Compose profile together.diff --git a/docker/envs/vectorstores/iris.defaults b/docker/envs/vectorstores/iris.defaultsindex c2dfeacb0a3..daf8f2dcd94 100644--- a/docker/envs/vectorstores/iris.defaults+++ b/docker/envs/vectorstores/iris.defaults@@ -11,7 +11,7 @@ IRIS_TIMEZONE=${IRIS_TIMEZONE:-UTC}# IRIS_PASSWORD=Dify@1234# IRIS_DATABASE=USER# IRIS_SCHEMA=dify-# IRIS_HOST=iris+IRIS_HOST=${IRIS_HOST-iris}# Set IRIS_SUPER_SERVER_PORT in docker/.env; Compose needs it before loading service files.# Set IRIS_WEB_SERVER_PORT in docker/.env; Compose needs it before loading service files.# IRIS_USER=_SYSTEMdiff --git a/docker/envs/vectorstores/matrixone.defaults b/docker/envs/vectorstores/matrixone.defaultsnew file mode 100644index 00000000000..d484fcade67--- /dev/null+++ b/docker/envs/vectorstores/matrixone.defaults@@ -0,0 +1,8 @@+# Defaults for MatrixOne. Copy this file to matrixone.env to customize its client connection.+# docker/.env takes precedence. Change credentials in the database before updating this client.++MATRIXONE_PASSWORD=${MATRIXONE_PASSWORD-111}+MATRIXONE_HOST=${MATRIXONE_HOST-matrixone}+MATRIXONE_PORT=${MATRIXONE_PORT-6001}+MATRIXONE_USER=${MATRIXONE_USER-dump}+MATRIXONE_DATABASE=${MATRIXONE_DATABASE-dify}diff --git a/docker/envs/vectorstores/matrixone.env.example b/docker/envs/vectorstores/matrixone.env.exampledeleted file mode 100644index 931375f8b48..00000000000--- a/docker/envs/vectorstores/matrixone.env.example+++ /dev/null@@ -1,9 +0,0 @@-# -------------------------------# Matrixone Configuration-# --------------------------------MATRIXONE_PASSWORD=111-MATRIXONE_HOST=matrixone-MATRIXONE_PORT=6001-MATRIXONE_USER=dump-MATRIXONE_DATABASE=difydiff --git a/docker/envs/vectorstores/milvus.defaults b/docker/envs/vectorstores/milvus.defaultsnew file mode 100644index 00000000000..a9bc14773d6--- /dev/null+++ b/docker/envs/vectorstores/milvus.defaults@@ -0,0 +1,25 @@+# Defaults for Milvus. Copy this file to milvus.env to customize it.+# docker/.env takes precedence. Credentials authenticate to an existing Milvus user;+# change that user's password in Milvus before updating MILVUS_PASSWORD here.++MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY-minioadmin}+MINIO_SECRET_KEY=${MINIO_SECRET_KEY-minioadmin}+ETCD_ENDPOINTS=${ETCD_ENDPOINTS-etcd:2379}+MINIO_ADDRESS=${MINIO_ADDRESS-minio:9000}+ETCD_AUTO_COMPACTION_MODE=${ETCD_AUTO_COMPACTION_MODE-revision}+ETCD_AUTO_COMPACTION_RETENTION=${ETCD_AUTO_COMPACTION_RETENTION-1000}+ETCD_QUOTA_BACKEND_BYTES=${ETCD_QUOTA_BACKEND_BYTES-4294967296}+ETCD_SNAPSHOT_COUNT=${ETCD_SNAPSHOT_COUNT-50000}+MILVUS_AUTHORIZATION_ENABLED=${MILVUS_AUTHORIZATION_ENABLED-true}+# MILVUS_SECURE=false+# MILVUS_SERVER_PEM_PATH=+# MILVUS_SERVER_NAME=++# Dify vector client settings (API, workers and Beat)+MILVUS_URI=${MILVUS_URI-http://milvus-standalone:19530}+# MILVUS_TOKEN=+MILVUS_USER=${MILVUS_USER-root}+MILVUS_PASSWORD=${MILVUS_PASSWORD-Milvus}+# MILVUS_ANALYZER_PARAMS=+MILVUS_DATABASE=${MILVUS_DATABASE-default}+# MILVUS_ENABLE_HYBRID_SEARCH=Falsediff --git a/docker/envs/vectorstores/milvus.env.example b/docker/envs/vectorstores/milvus.env.exampledeleted file mode 100644index e7cc77919ab..00000000000--- a/docker/envs/vectorstores/milvus.env.example+++ /dev/null@@ -1,25 +0,0 @@-# -------------------------------# Milvus Configuration-# --------------------------------MINIO_ACCESS_KEY=minioadmin-MINIO_SECRET_KEY=minioadmin-ETCD_ENDPOINTS=etcd:2379-MINIO_ADDRESS=minio:9000-ETCD_AUTO_COMPACTION_MODE=revision-ETCD_AUTO_COMPACTION_RETENTION=1000-ETCD_QUOTA_BACKEND_BYTES=4294967296-ETCD_SNAPSHOT_COUNT=50000-MILVUS_AUTHORIZATION_ENABLED=true-MILVUS_SECURE=false-MILVUS_SERVER_PEM_PATH=-MILVUS_SERVER_NAME=--# Dify vector client settings (API, workers and Beat)-# MILVUS_URI=http://host.docker.internal:19530-# MILVUS_TOKEN=-# MILVUS_USER=-# MILVUS_PASSWORD=-# MILVUS_ANALYZER_PARAMS=-# MILVUS_DATABASE=-# MILVUS_ENABLE_HYBRID_SEARCH=Falsediff --git a/docker/envs/vectorstores/myscale.defaults b/docker/envs/vectorstores/myscale.defaultsnew file mode 100644index 00000000000..64707e21d6d--- /dev/null+++ b/docker/envs/vectorstores/myscale.defaults@@ -0,0 +1,9 @@+# Defaults for MyScale. Copy this file to myscale.env to customize its client connection.+# docker/.env takes precedence. Server users are configured in volumes/myscale/config/users.d/.++MYSCALE_PASSWORD=${MYSCALE_PASSWORD-}+MYSCALE_DATABASE=${MYSCALE_DATABASE-dify}+# MYSCALE_FTS_PARAMS=+MYSCALE_HOST=${MYSCALE_HOST-myscale}+MYSCALE_PORT=${MYSCALE_PORT-8123}+MYSCALE_USER=${MYSCALE_USER-default}diff --git a/docker/envs/vectorstores/myscale.env.example b/docker/envs/vectorstores/myscale.env.exampledeleted file mode 100644index eaa9e88cc02..00000000000--- a/docker/envs/vectorstores/myscale.env.example+++ /dev/null@@ -1,10 +0,0 @@-# -------------------------------# Myscale Configuration-# --------------------------------MYSCALE_PASSWORD=-MYSCALE_DATABASE=dify-MYSCALE_FTS_PARAMS=-MYSCALE_HOST=myscale-MYSCALE_PORT=8123-MYSCALE_USER=defaultdiff --git a/docker/envs/vectorstores/oceanbase.defaults b/docker/envs/vectorstores/oceanbase.defaultsindex f3ab862bbac..7fa4dffb772 100644--- a/docker/envs/vectorstores/oceanbase.defaults+++ b/docker/envs/vectorstores/oceanbase.defaults@@ -4,13 +4,13 @@OCEANBASE_CLUSTER_NAME=${OCEANBASE_CLUSTER_NAME:-difyai}OCEANBASE_MEMORY_LIMIT=${OCEANBASE_MEMORY_LIMIT:-6G}-OCEANBASE_VECTOR_PASSWORD=${OCEANBASE_VECTOR_PASSWORD:-difyai123456}+OCEANBASE_VECTOR_PASSWORD=${OCEANBASE_VECTOR_PASSWORD-difyai123456}# Dify vector client settings (API, workers and Beat)# Set OCEANBASE_VECTOR_PORT in docker/.env to keep the client and published port aligned.-# OCEANBASE_VECTOR_HOST=oceanbase-# OCEANBASE_VECTOR_USER=root@test-# OCEANBASE_VECTOR_DATABASE=test+OCEANBASE_VECTOR_HOST=${OCEANBASE_VECTOR_HOST-oceanbase}+OCEANBASE_VECTOR_USER=${OCEANBASE_VECTOR_USER-root@test}+OCEANBASE_VECTOR_DATABASE=${OCEANBASE_VECTOR_DATABASE-test}# OCEANBASE_ENABLE_HYBRID_SEARCH=false# OCEANBASE_FULLTEXT_PARSER=ikdiff --git a/docker/envs/vectorstores/opengauss.defaults b/docker/envs/vectorstores/opengauss.defaultsindex b2049164b29..38ee158819d 100644--- a/docker/envs/vectorstores/opengauss.defaults+++ b/docker/envs/vectorstores/opengauss.defaults@@ -2,14 +2,14 @@# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.-OPENGAUSS_PASSWORD=${OPENGAUSS_PASSWORD:-Dify@123}-OPENGAUSS_DATABASE=${OPENGAUSS_DATABASE:-dify}+OPENGAUSS_PASSWORD=${OPENGAUSS_PASSWORD-Dify@123}+OPENGAUSS_DATABASE=${OPENGAUSS_DATABASE-dify}# OPENGAUSS_MIN_CONNECTION=1# OPENGAUSS_MAX_CONNECTION=5# OPENGAUSS_ENABLE_PQ=false-# OPENGAUSS_HOST=opengauss+OPENGAUSS_HOST=${OPENGAUSS_HOST-opengauss}# Set OPENGAUSS_PORT in docker/.env; Compose needs it before loading service files.-OPENGAUSS_USER=${OPENGAUSS_USER:-postgres}+OPENGAUSS_USER=${OPENGAUSS_USER-postgres}# Container variable mappings; keep these expressions unchanged in the copy.GS_DB=${OPENGAUSS_DATABASE:-dify}diff --git a/docker/envs/vectorstores/opensearch.defaults b/docker/envs/vectorstores/opensearch.defaultsindex 1c73b004724..4d6a3dd3979 100644--- a/docker/envs/vectorstores/opensearch.defaults+++ b/docker/envs/vectorstores/opensearch.defaults@@ -2,20 +2,21 @@# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.-# OPENSEARCH_PASSWORD=admin+OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD:-Qazwsxedc!@#123}+OPENSEARCH_PASSWORD=${OPENSEARCH_PASSWORD-${OPENSEARCH_INITIAL_ADMIN_PASSWORD:-Qazwsxedc!@#123}}# OPENSEARCH_AWS_REGION=ap-southeast-1# OPENSEARCH_AWS_SERVICE=aoss-OPENSEARCH_INITIAL_ADMIN_PASSWORD=${OPENSEARCH_INITIAL_ADMIN_PASSWORD:-Qazwsxedc!@#123}# Set OPENSEARCH_MEMLOCK_SOFT in docker/.env; Compose needs it before loading service files.# Set OPENSEARCH_MEMLOCK_HARD in docker/.env; Compose needs it before loading service files.# Set OPENSEARCH_NOFILE_SOFT in docker/.env; Compose needs it before loading service files.# Set OPENSEARCH_NOFILE_HARD in docker/.env; Compose needs it before loading service files.-# OPENSEARCH_HOST=opensearch+OPENSEARCH_HOST=${OPENSEARCH_HOST-opensearch}# OPENSEARCH_PORT=9200-# OPENSEARCH_SECURE=true-# OPENSEARCH_VERIFY_CERTS=true+OPENSEARCH_SECURE=${OPENSEARCH_SECURE-true}+# Bundled demo certificates require an explicit false override; use trusted certificates in production.+OPENSEARCH_VERIFY_CERTS=${OPENSEARCH_VERIFY_CERTS-true}# OPENSEARCH_AUTH_METHOD=basic-# OPENSEARCH_USER=admin+OPENSEARCH_USER=${OPENSEARCH_USER-admin}OPENSEARCH_DISCOVERY_TYPE=${OPENSEARCH_DISCOVERY_TYPE:-single-node}OPENSEARCH_BOOTSTRAP_MEMORY_LOCK=${OPENSEARCH_BOOTSTRAP_MEMORY_LOCK:-true}OPENSEARCH_JAVA_OPTS_MIN=${OPENSEARCH_JAVA_OPTS_MIN:-512m}diff --git a/docker/envs/vectorstores/oracle.defaults b/docker/envs/vectorstores/oracle.defaultsindex 7055e1e73e0..c4dbade5131 100644--- a/docker/envs/vectorstores/oracle.defaults+++ b/docker/envs/vectorstores/oracle.defaults@@ -2,12 +2,12 @@# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.-# ORACLE_PASSWORD=dify-# ORACLE_DSN=oracle:1521/FREEPDB1+ORACLE_PASSWORD=${ORACLE_PASSWORD-dify}+ORACLE_DSN=${ORACLE_DSN-oracle:1521/FREEPDB1}# ORACLE_CONFIG_DIR=/app/api/storage/wallet# ORACLE_WALLET_LOCATION=/app/api/storage/wallet# ORACLE_WALLET_PASSWORD=dify# ORACLE_IS_AUTONOMOUS=false-# ORACLE_USER=dify+ORACLE_USER=${ORACLE_USER-dify}ORACLE_PWD=${ORACLE_PWD:-Dify123456}ORACLE_CHARACTERSET=${ORACLE_CHARACTERSET:-AL32UTF8}diff --git a/docker/envs/vectorstores/pgvecto-rs.defaults b/docker/envs/vectorstores/pgvecto-rs.defaultsindex 11c6b7c0479..7c9e3e20c89 100644--- a/docker/envs/vectorstores/pgvecto-rs.defaults+++ b/docker/envs/vectorstores/pgvecto-rs.defaults@@ -2,18 +2,21 @@# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.-# PGVECTO_RS_HOST=pgvecto-rs-# PGVECTO_RS_PORT=5432-# PGVECTO_RS_USER=postgres-# PGVECTO_RS_PASSWORD=difyai123456-# PGVECTO_RS_DATABASE=difyPGVECTOR_PGDATA=${PGVECTOR_PGDATA:-/var/lib/postgresql/data/pgdata}PGVECTOR_PGUSER=${PGVECTOR_PGUSER:-postgres}PGVECTOR_POSTGRES_DB=${PGVECTOR_POSTGRES_DB:-dify}PGVECTOR_POSTGRES_PASSWORD=${PGVECTOR_POSTGRES_PASSWORD:-difyai123456}+# Dify vector client settings (API, workers and Beat)+PGVECTO_RS_HOST=${PGVECTO_RS_HOST-pgvecto-rs}+PGVECTO_RS_PORT=${PGVECTO_RS_PORT-5432}+PGVECTO_RS_USER=${PGVECTO_RS_USER-${PGVECTOR_PGUSER:-postgres}}+PGVECTO_RS_PASSWORD=${PGVECTO_RS_PASSWORD-${PGVECTOR_POSTGRES_PASSWORD:-difyai123456}}+PGVECTO_RS_DATABASE=${PGVECTO_RS_DATABASE-${PGVECTOR_POSTGRES_DB:-dify}}+# Container variable mappings; keep these expressions unchanged in the copy.PGDATA=${PGVECTOR_PGDATA:-/var/lib/postgresql/data/pgdata}PGUSER=${PGVECTOR_PGUSER:-postgres}+POSTGRES_USER=${PGVECTOR_PGUSER:-postgres}POSTGRES_DB=${PGVECTOR_POSTGRES_DB:-dify}POSTGRES_PASSWORD=${PGVECTOR_POSTGRES_PASSWORD:-difyai123456}diff --git a/docker/envs/vectorstores/pgvector.defaults b/docker/envs/vectorstores/pgvector.defaultsindex da0e4dea62f..72704c6ac32 100644--- a/docker/envs/vectorstores/pgvector.defaults+++ b/docker/envs/vectorstores/pgvector.defaults@@ -10,17 +10,18 @@ PGVECTOR_PG_BIGM=${PGVECTOR_PG_BIGM:-false}PGVECTOR_PG_BIGM_VERSION=${PGVECTOR_PG_BIGM_VERSION:-1.2-20240606}# Dify vector client settings (API, workers and Beat)-# PGVECTOR_HOST=pgvector-# PGVECTOR_PORT=5432-# PGVECTOR_USER=postgres-# PGVECTOR_PASSWORD=difyai123456-# PGVECTOR_DATABASE=dify+PGVECTOR_HOST=${PGVECTOR_HOST-pgvector}+PGVECTOR_PORT=${PGVECTOR_PORT-5432}+PGVECTOR_USER=${PGVECTOR_USER-${PGVECTOR_PGUSER:-postgres}}+PGVECTOR_PASSWORD=${PGVECTOR_PASSWORD-${PGVECTOR_POSTGRES_PASSWORD:-difyai123456}}+PGVECTOR_DATABASE=${PGVECTOR_DATABASE-${PGVECTOR_POSTGRES_DB:-dify}}# PGVECTOR_MIN_CONNECTION=1# PGVECTOR_MAX_CONNECTION=5# Container variable mappings; keep these expressions unchanged in the copy.PGDATA=${PGVECTOR_PGDATA:-/var/lib/postgresql/data/pgdata}PGUSER=${PGVECTOR_PGUSER:-postgres}+POSTGRES_USER=${PGVECTOR_PGUSER:-postgres}PG_BIGM=${PGVECTOR_PG_BIGM:-false}PG_BIGM_VERSION=${PGVECTOR_PG_BIGM_VERSION:-1.2-20240606}POSTGRES_DB=${PGVECTOR_POSTGRES_DB:-dify}diff --git a/docker/envs/vectorstores/qdrant.defaults b/docker/envs/vectorstores/qdrant.defaultsindex 7bc128267ca..72b94f6f226 100644--- a/docker/envs/vectorstores/qdrant.defaults+++ b/docker/envs/vectorstores/qdrant.defaults@@ -2,15 +2,18 @@# Edit the settings below in the copy; docker/.env takes precedence.# Optional application settings are commented to retain their application defaults.-QDRANT_API_KEY=${QDRANT_API_KEY:-difyai123456}+QDRANT_API_KEY=${QDRANT_API_KEY-difyai123456}# Container file descriptor limits default to 524288 (soft and hard).# Set QDRANT_NOFILE_SOFT and QDRANT_NOFILE_HARD in docker/.env to override them;# Compose needs these settings before loading service env files.# Dify vector client settings (API, workers and Beat)-# QDRANT_URL=http://qdrant:6333+QDRANT_URL=${QDRANT_URL-http://qdrant:6333}# QDRANT_CLIENT_TIMEOUT=20# QDRANT_GRPC_ENABLED=false# QDRANT_GRPC_PORT=6334# QDRANT_REPLICATION_FACTOR=1++# Container variable mapping; keep this expression unchanged in the copy.+QDRANT__SERVICE__API_KEY=${QDRANT_API_KEY-difyai123456}diff --git a/docker/envs/vectorstores/seekdb.defaults b/docker/envs/vectorstores/seekdb.defaultsindex 56faf95284f..1d882e0700a 100644--- a/docker/envs/vectorstores/seekdb.defaults+++ b/docker/envs/vectorstores/seekdb.defaults@@ -3,7 +3,12 @@# Optional application settings are commented to retain their application defaults.SEEKDB_MEMORY_LIMIT=${SEEKDB_MEMORY_LIMIT:-2G}-OCEANBASE_VECTOR_PASSWORD=${OCEANBASE_VECTOR_PASSWORD:-difyai123456}+OCEANBASE_VECTOR_PASSWORD=${OCEANBASE_VECTOR_PASSWORD-difyai123456}++# Dify uses the OceanBase client for SeekDB.+OCEANBASE_VECTOR_HOST=${OCEANBASE_VECTOR_HOST-seekdb}+OCEANBASE_VECTOR_USER=${OCEANBASE_VECTOR_USER-root}+OCEANBASE_VECTOR_DATABASE=${OCEANBASE_VECTOR_DATABASE-test}# Container variable mappings; keep these expressions unchanged in the copy.MEMORY_LIMIT=${SEEKDB_MEMORY_LIMIT:-2G}diff --git a/docker/envs/vectorstores/vastbase.defaults b/docker/envs/vectorstores/vastbase.defaultsnew file mode 100644index 00000000000..4aa90dd047c--- /dev/null+++ b/docker/envs/vectorstores/vastbase.defaults@@ -0,0 +1,10 @@+# Defaults for Vastbase. Copy this file to vastbase.env to customize its client connection.+# docker/.env takes precedence. Keep credentials aligned with the server's VB_* settings.++VASTBASE_PASSWORD=${VASTBASE_PASSWORD-Difyai123456}+VASTBASE_DATABASE=${VASTBASE_DATABASE-dify}+# VASTBASE_MIN_CONNECTION=1+# VASTBASE_MAX_CONNECTION=5+VASTBASE_HOST=${VASTBASE_HOST-vastbase}+VASTBASE_PORT=${VASTBASE_PORT-5432}+VASTBASE_USER=${VASTBASE_USER-dify}diff --git a/docker/envs/vectorstores/vastbase.env.example b/docker/envs/vectorstores/vastbase.env.exampledeleted file mode 100644index 2c9db50fbe6..00000000000--- a/docker/envs/vectorstores/vastbase.env.example+++ /dev/null@@ -1,11 +0,0 @@-# -------------------------------# Vastbase Configuration-# --------------------------------VASTBASE_PASSWORD=Difyai123456-VASTBASE_DATABASE=dify-VASTBASE_MIN_CONNECTION=1-VASTBASE_MAX_CONNECTION=5-VASTBASE_HOST=vastbase-VASTBASE_PORT=5432-VASTBASE_USER=difydiff --git a/docker/envs/vectorstores/weaviate.defaults b/docker/envs/vectorstores/weaviate.defaultsindex fceab9a881b..1304950ce68 100644--- a/docker/envs/vectorstores/weaviate.defaults+++ b/docker/envs/vectorstores/weaviate.defaults@@ -8,6 +8,7 @@ WEAVIATE_AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED=${WEAVIATE_AUTHENTICATION_ANONYWEAVIATE_DEFAULT_VECTORIZER_MODULE=${WEAVIATE_DEFAULT_VECTORIZER_MODULE:-none}WEAVIATE_CLUSTER_HOSTNAME=${WEAVIATE_CLUSTER_HOSTNAME:-node1}WEAVIATE_AUTHENTICATION_APIKEY_ENABLED=${WEAVIATE_AUTHENTICATION_APIKEY_ENABLED:-true}+# To require authentication, disable anonymous access and set WEAVIATE_API_KEY to an allowed key.WEAVIATE_AUTHENTICATION_APIKEY_ALLOWED_KEYS=${WEAVIATE_AUTHENTICATION_APIKEY_ALLOWED_KEYS:-WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih}WEAVIATE_AUTHENTICATION_APIKEY_USERS=${WEAVIATE_AUTHENTICATION_APIKEY_USERS:[email protected]}WEAVIATE_AUTHORIZATION_ADMINLIST_ENABLED=${WEAVIATE_AUTHORIZATION_ADMINLIST_ENABLED:-true}@@ -18,6 +19,9 @@ WEAVIATE_ENABLE_TOKENIZER_KAGOME_JA=${WEAVIATE_ENABLE_TOKENIZER_KAGOME_JA:-falseWEAVIATE_ENABLE_TOKENIZER_KAGOME_KR=${WEAVIATE_ENABLE_TOKENIZER_KAGOME_KR:-false}# Dify vector client settings (API, workers and Beat)+WEAVIATE_ENDPOINT=${WEAVIATE_ENDPOINT-http://weaviate:8080}+WEAVIATE_API_KEY=${WEAVIATE_API_KEY-WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih}+WEAVIATE_GRPC_ENDPOINT=${WEAVIATE_GRPC_ENDPOINT-grpc://weaviate:50051}# WEAVIATE_TOKENIZATION=word# Container variable mappings; keep these expressions unchanged in the copy.diff --git a/docker/tests/test_compose_environment.py b/docker/tests/test_compose_environment.pyindex 9b0da0e904c..7a7e1f299dd 100644--- a/docker/tests/test_compose_environment.py+++ b/docker/tests/test_compose_environment.py@@ -46,7 +46,7 @@ class ComposeEnvironmentTests(unittest.TestCase):with (self.project / ".env").open("a") as target:target.write("\n" + "\n".join(f"{key}={value}" for key, value in values.items()) + "\n")- def render(self, *compose_args):+ def render(self, *compose_args, all_profiles=True):inputs = set(re.findall(r"\$\{(\w+)", (DOCKER_DIR / "docker-compose.yaml").read_text()))for source in (DOCKER_DIR / "envs").rglob("*.defaults"):inputs.update(re.findall(r"\$\{(\w+)", source.read_text()))@@ -56,8 +56,7 @@ class ComposeEnvironmentTests(unittest.TestCase):"docker","compose",*compose_args,- "--profile",- "*",+ *(["--profile", "*"] if all_profiles else []),"config","--format","json",@@ -123,6 +122,496 @@ class ComposeEnvironmentTests(unittest.TestCase):self.assertNotIn("QDRANT_NOFILE_SOFT", services["qdrant"]["environment"])self.assertNotIn("QDRANT_NOFILE_HARD", services["qdrant"]["environment"])+ def test_documented_mysql_root_configuration_reaches_database_clients(self):+ values = {+ "DB_TYPE": "mysql",+ "DB_HOST": "db_mysql",+ "DB_PORT": "3306",+ "DB_USERNAME": "root",+ "DB_PASSWORD": "mysql-root-override",+ }+ self.override_root({**values, "COMPOSE_PROFILES": "weaviate,mysql,collaboration"})+ services = self.render()+ for name in ("api", "api_websocket", "worker", "worker_knowledge_fs_upgrade", "worker_beat", "plugin_daemon"):+ for key, expected in values.items():+ self.assertEqual(services[name]["environment"][key], expected, f"{name}.{key}")+ self.assertEqual(services["db_mysql"]["environment"]["MYSQL_ROOT_PASSWORD"], values["DB_PASSWORD"])+ self.assertEqual(services["db_mysql"]["environment"]["MYSQL_DATABASE"], "dify")++ def test_code_sandbox_credentials_must_be_configured_as_a_pair(self):+ clients = ("api", "api_websocket", "worker", "worker_knowledge_fs_upgrade", "worker_beat")+ for source, target, key in (+ ("core-services/sandbox.defaults", "core-services/sandbox.env", "SANDBOX_API_KEY"),+ ("security.env.example", "security.env", "CODE_EXECUTION_API_KEY"),+ ):+ with self.subTest(source=source):+ self.copy_template(f"envs/{source}", f"envs/{target}", {key: "service-sandbox-key"})+ services = self.render()+ self.assertEqual(+ services["sandbox"]["environment"]["API_KEY"],+ "service-sandbox-key" if key == "SANDBOX_API_KEY" else "dify-sandbox",+ )+ for name in clients:+ self.assertEqual(+ services[name]["environment"].get("CODE_EXECUTION_API_KEY", "dify-sandbox"),+ "service-sandbox-key" if key == "CODE_EXECUTION_API_KEY" else "dify-sandbox",+ name,+ )+ (self.project / "envs" / target).unlink()++ self.override_root({"SANDBOX_API_KEY": "root-sandbox-key", "CODE_EXECUTION_API_KEY": "root-sandbox-key"})+ services = self.render()+ self.assertEqual(services["sandbox"]["environment"]["API_KEY"], "root-sandbox-key")+ for name in clients:+ self.assertEqual(services[name]["environment"]["CODE_EXECUTION_API_KEY"], "root-sandbox-key", name)++ def test_root_overrides_only_reach_mapped_inputs_on_services_without_root_env_file(self):+ root = self.project / ".env"+ original = root.read_text()+ for template, target, service, local_values, root_values, expected in (+ (+ "vectorstores/weaviate.defaults",+ "vectorstores/weaviate.env",+ "weaviate",+ {"WEAVIATE_QUERY_DEFAULTS_LIMIT": "66"},+ {"WEAVIATE_QUERY_DEFAULTS_LIMIT": "67", "ASYNC_INDEXING": "true"},+ {"QUERY_DEFAULTS_LIMIT": "67", "ASYNC_INDEXING": "false"},+ ),+ (+ "core-services/knowledge-fs-unstructured.env.example",+ "core-services/knowledge-fs-unstructured.env",+ "knowledge_fs_unstructured",+ {"PDF_RENDER_DPI": "200"},+ {"PDF_RENDER_DPI": "72"},+ {"PDF_RENDER_DPI": "200"},+ ),+ ):+ with self.subTest(service=service):+ self.copy_template(f"envs/{template}", f"envs/{target}", local_values)+ local = self.project / "envs" / target+ if service == "weaviate":+ local.write_text(local.read_text() + "\nASYNC_INDEXING=false\n")+ before = self.render()[service]["environment"]+ self.assertEqual(+ before.get("QUERY_DEFAULTS_LIMIT", before.get("PDF_RENDER_DPI")),+ "66" if service == "weaviate" else "200",+ )+ self.override_root(root_values)+ values = self.render()[service]["environment"]+ for key, value in expected.items():+ self.assertEqual(values[key], value, f"{service}.{key}")+ local.unlink()+ root.write_text(original)++ def test_middleware_template_is_loaded_from_docker_directory(self):+ shutil.copyfile(DOCKER_DIR / "docker-compose.middleware.yaml", self.project / "docker-compose.middleware.yaml")+ self.copy_template(+ "envs/middleware.env.example",+ "middleware.env",+ {"DB_PASSWORD": "middleware-db-key", "EXPOSE_POSTGRES_PORT": "15432", "POSTGRES_MAX_CONNECTIONS": "121"},+ )+ services = self.render("--env-file", "middleware.env", "-f", "docker-compose.middleware.yaml", "-p", "dify")+ postgres = services["db_postgres"]+ self.assertEqual(postgres["environment"]["POSTGRES_PASSWORD"], "middleware-db-key")+ self.assertEqual(postgres["ports"][0]["published"], "15432")+ self.assertIn("max_connections=121", postgres["command"])++ def test_agent_sandbox_token_is_a_root_input_for_both_consumers(self):+ template = (DOCKER_DIR / "envs/core-services/dify-agent.defaults").read_text()+ self.assertNotRegex(template, r"(?m)^DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN=")+ self.copy_template(+ "envs/core-services/dify-agent.defaults",+ "envs/core-services/dify-agent.env",+ )+ local = self.project / "envs/core-services/dify-agent.env"+ local.write_text(local.read_text() + "\nDIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN=service-only-token\n")+ root = self.project / ".env"+ original = root.read_text()+ for setting, expected in (+ ("DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN=root-token", "root-token"),+ (+ "DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN=\nDIFY_AGENT_SHELLCTL_AUTH_TOKEN=legacy-token",+ "legacy-token",+ ),+ ("DIFY_AGENT_SHELLCTL_AUTH_TOKEN=legacy-token", "legacy-token"),+ ("DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN=", ""),+ ):+ with self.subTest(setting=setting):+ root.write_text(+ re.sub(+ r"(?m)^DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN=.*$",+ setting,+ original,+ )+ )+ services = self.render()+ self.assertEqual(+ services["agent_backend"]["environment"]["DIFY_AGENT_LOCAL_SANDBOX_AUTH_TOKEN"],+ expected,+ )+ self.assertEqual(+ services["local_sandbox"]["environment"]["SHELLCTL_AUTH_TOKEN"],+ expected,+ )++ def test_legacy_agent_names_and_canonical_overrides(self):+ self.override_root({"DIFY_AGENT_SHELLCTL_ENTRYPOINT": "http://legacy-sandbox:5004", "ENABLE_AGENT_V2": "false"})+ services = self.render()+ self.assertEqual(+ services["agent_backend"]["environment"]["DIFY_AGENT_LOCAL_SANDBOX_ENDPOINT"], "http://legacy-sandbox:5004"+ )+ self.assertEqual(services["web"]["environment"]["NEXT_PUBLIC_ENABLE_AGENT_V2"], "true")+ self.override_root(+ {"DIFY_AGENT_LOCAL_SANDBOX_ENDPOINT": "http://current-sandbox:5004", "NEXT_PUBLIC_ENABLE_AGENT_V2": "false"}+ )+ services = self.render()+ self.assertEqual(+ services["agent_backend"]["environment"]["DIFY_AGENT_LOCAL_SANDBOX_ENDPOINT"], "http://current-sandbox:5004"+ )+ self.assertEqual(services["web"]["environment"]["NEXT_PUBLIC_ENABLE_AGENT_V2"], "false")+ self.override_root({"NEXT_PUBLIC_ENABLE_AGENT_V2": ""})+ self.assertEqual(self.render()["web"]["environment"]["NEXT_PUBLIC_ENABLE_AGENT_V2"], "true")++ def test_bundled_vector_defaults_and_isolated_overrides(self):+ cases = {+ "weaviate": {"WEAVIATE_ENDPOINT": "http://weaviate:8080"},+ "qdrant": {+ "QDRANT_URL": "http://qdrant:6333",+ "QDRANT_API_KEY": "difyai123456",+ },+ "oceanbase": {+ "OCEANBASE_VECTOR_HOST": "oceanbase",+ "OCEANBASE_VECTOR_USER": "root@test",+ "OCEANBASE_VECTOR_PASSWORD": "difyai123456",+ "OCEANBASE_VECTOR_DATABASE": "test",+ },+ "seekdb": {+ "OCEANBASE_VECTOR_HOST": "seekdb",+ "OCEANBASE_VECTOR_USER": "root",+ "OCEANBASE_VECTOR_PASSWORD": "difyai123456",+ "OCEANBASE_VECTOR_DATABASE": "test",+ },+ "couchbase": {+ "COUCHBASE_CONNECTION_STRING": "couchbase://couchbase-server",+ "COUCHBASE_USER": "Administrator",+ "COUCHBASE_PASSWORD": "password",+ "COUCHBASE_BUCKET_NAME": "Embeddings",+ "COUCHBASE_SCOPE_NAME": "_default",+ },+ "pgvector": {+ "PGVECTOR_HOST": "pgvector",+ "PGVECTOR_PORT": "5432",+ "PGVECTOR_USER": "postgres",+ "PGVECTOR_PASSWORD": "difyai123456",+ "PGVECTOR_DATABASE": "dify",+ },+ "pgvecto-rs": {+ "PGVECTO_RS_HOST": "pgvecto-rs",+ "PGVECTO_RS_PORT": "5432",+ "PGVECTO_RS_USER": "postgres",+ "PGVECTO_RS_PASSWORD": "difyai123456",+ "PGVECTO_RS_DATABASE": "dify",+ },+ "vastbase": {+ "VASTBASE_HOST": "vastbase",+ "VASTBASE_PORT": "5432",+ "VASTBASE_USER": "dify",+ "VASTBASE_PASSWORD": "Difyai123456",+ "VASTBASE_DATABASE": "dify",+ },+ "chroma": {+ "CHROMA_HOST": "chroma",+ "CHROMA_AUTH_CREDENTIALS": "difyai123456",+ "CHROMA_AUTH_PROVIDER": "chromadb.auth.token_authn.TokenAuthClientProvider",+ },+ "iris": {"IRIS_HOST": "iris"},+ "oracle": {+ "ORACLE_DSN": "oracle:1521/FREEPDB1",+ "ORACLE_USER": "dify",+ "ORACLE_PASSWORD": "dify",+ },+ "milvus": {+ "MILVUS_URI": "http://milvus-standalone:19530",+ "MILVUS_USER": "root",+ "MILVUS_PASSWORD": "Milvus",+ "MILVUS_DATABASE": "default",+ },+ "opensearch": {+ "OPENSEARCH_HOST": "opensearch",+ "OPENSEARCH_USER": "admin",+ "OPENSEARCH_PASSWORD": "Qazwsxedc!@#123",+ "OPENSEARCH_SECURE": "true",+ "OPENSEARCH_VERIFY_CERTS": "true",+ },+ "opengauss": {+ "OPENGAUSS_HOST": "opengauss",+ "OPENGAUSS_USER": "postgres",+ "OPENGAUSS_PASSWORD": "Dify@123",+ "OPENGAUSS_DATABASE": "dify",+ },+ "myscale": {+ "MYSCALE_HOST": "myscale",+ "MYSCALE_PORT": "8123",+ "MYSCALE_USER": "default",+ "MYSCALE_PASSWORD": "",+ "MYSCALE_DATABASE": "dify",+ },+ "matrixone": {+ "MATRIXONE_HOST": "matrixone",+ "MATRIXONE_PORT": "6001",+ "MATRIXONE_USER": "dump",+ "MATRIXONE_PASSWORD": "111",+ "MATRIXONE_DATABASE": "dify",+ },+ "elasticsearch": {+ "ELASTICSEARCH_HOST": "elasticsearch",+ "ELASTICSEARCH_PASSWORD": "elastic",+ },+ "elasticsearch-ja": {+ "ELASTICSEARCH_HOST": "elasticsearch",+ "ELASTICSEARCH_PASSWORD": "elastic",+ },+ }+ consumers = (+ "api",+ "api_websocket",+ "worker",+ "worker_knowledge_fs_upgrade",+ "worker_beat",+ )+ root = self.project / ".env"+ original = root.read_text()+ for provider, expected in cases.items():+ with self.subTest(provider=provider):+ root.write_text(original)+ self.override_root(+ {+ "VECTOR_STORE": provider,+ "COMPOSE_PROFILES": f"{provider},postgresql,collaboration",+ }+ )+ selected = root.read_text()+ template = "elasticsearch" if provider == "elasticsearch-ja" else provider+ source = f"envs/vectorstores/{template}.defaults"+ target = f"envs/vectorstores/{template}.env"+ key = next(iter(expected))+ try:+ baseline = self.render()+ for name in consumers:+ for variable, value in expected.items():+ self.assertEqual(+ baseline[name]["environment"].get(variable),+ value,+ f"{name}.{variable}",+ )+ server = {+ "milvus": "milvus-standalone",+ "couchbase": "couchbase-server",+ "elasticsearch-ja": "elasticsearch",+ }.get(provider, provider)+ self.assertTrue(+ set(baseline[name]["networks"]) & set(baseline[server]["networks"]),+ name,+ )++ # Change only the root file, then restore it before exercising the provider file.+ self.override_root({key: "root-only"})+ root_only = self.render()+ for name in consumers:+ self.assertEqual(root_only[name]["environment"][key], "root-only", name)+ root.write_text(selected)++ self.copy_template(source, target)+ copied = self.render()+ for name in consumers:+ for variable, value in expected.items():+ self.assertEqual(+ copied[name]["environment"][variable],+ value,+ f"copy: {name}.{variable}",+ )+ self.copy_template(source, target, {key: "provider-only"})+ customized = self.render()+ for name in consumers:+ self.assertEqual(customized[name]["environment"][key], "provider-only", name)++ self.override_root({key: "root-wins"})+ overridden = self.render()+ for name in consumers:+ self.assertEqual(overridden[name]["environment"][key], "root-wins", name)+ finally:+ (self.project / target).unlink(missing_ok=True)+ root.write_text(original)++ def test_external_opensearch_tls_verification_and_overrides(self):+ self.override_root(+ {+ "VECTOR_STORE": "opensearch",+ "COMPOSE_PROFILES": "postgresql,collaboration",+ "OPENSEARCH_HOST": "search.example.test",+ "OPENSEARCH_PORT": "443",+ "OPENSEARCH_SECURE": "true",+ }+ )+ root = self.project / ".env"+ original = root.read_text()+ source = "envs/vectorstores/opensearch.defaults"+ target = "envs/vectorstores/opensearch.env"++ def assert_verification(expected):+ services = self.render(all_profiles=False)+ self.assertNotIn("opensearch", services)+ for name in ("api", "api_websocket", "worker", "worker_beat"):+ values = services[name]["environment"]+ self.assertEqual(values["OPENSEARCH_HOST"], "search.example.test", name)+ self.assertEqual(values["OPENSEARCH_SECURE"], "true", name)+ self.assertEqual(values["OPENSEARCH_VERIFY_CERTS"], expected, name)++ assert_verification("true")+ self.copy_template(source, target)+ assert_verification("true")+ self.copy_template(source, target, {"OPENSEARCH_VERIFY_CERTS": "false"})+ assert_verification("false")+ self.override_root({"OPENSEARCH_VERIFY_CERTS": "true"})+ assert_verification("true")+ root.write_text(original)+ assert_verification("false")+ (self.project / target).unlink()+ assert_verification("true")+ self.override_root({"OPENSEARCH_VERIFY_CERTS": "false"})+ assert_verification("false")++ def test_weaviate_connections_survive_selecting_another_vector_store(self):+ self.override_root({"VECTOR_STORE": "qdrant", "COMPOSE_PROFILES": "weaviate,qdrant,postgresql,collaboration"})+ root = self.project / ".env"+ original = root.read_text()+ defaults = {+ "WEAVIATE_ENDPOINT": "http://weaviate:8080",+ "WEAVIATE_API_KEY": "WVF5YThaHlkYwhGUSmCRgsX3tD5ngdN8pkih",+ "WEAVIATE_GRPC_ENDPOINT": "grpc://weaviate:50051",+ }+ custom = {+ "WEAVIATE_ENDPOINT": "https://existing-weaviate:443",+ "WEAVIATE_API_KEY": "",+ "WEAVIATE_GRPC_ENDPOINT": "",+ }+ root_values = {+ "WEAVIATE_ENDPOINT": "https://root-weaviate:443",+ "WEAVIATE_API_KEY": "root-weaviate-key",+ "WEAVIATE_GRPC_ENDPOINT": "grpcs://root-weaviate:443",+ }++ def assert_connections(expected):+ services = self.render()+ for name in ("api", "api_websocket", "worker", "worker_knowledge_fs_upgrade", "worker_beat"):+ values = services[name]["environment"]+ self.assertEqual(values["VECTOR_STORE"], "qdrant", name)+ self.assertEqual(values["QDRANT_URL"], "http://qdrant:6333", name)+ for key, value in expected.items():+ self.assertEqual(values.get(key), value, f"{name}.{key}")++ active = self.render(all_profiles=False)+ self.assertIn("weaviate", active)+ self.assertIn("qdrant", active)+ assert_connections(defaults)+ for source, target in (+ ("core-services/api-worker.defaults", "core-services/api-worker.env"),+ ("core-services/api-worker.defaults", "core-services/shared.env"),+ ("vectorstores/weaviate.defaults", "vectorstores/weaviate.env"),+ ):+ with self.subTest(source=target):+ try:+ self.copy_template(f"envs/{source}", f"envs/{target}")+ assert_connections(defaults)+ self.copy_template(f"envs/{source}", f"envs/{target}", custom)+ assert_connections(custom)+ self.override_root(root_values)+ assert_connections(root_values)+ root.write_text(original)+ assert_connections(custom)+ finally:+ (self.project / "envs" / target).unlink(missing_ok=True)+ root.write_text(original)+ assert_connections(defaults)+ self.override_root(root_values)+ assert_connections(root_values)++ def test_selected_vector_defaults_preserve_credentials_and_existing_overrides(self):+ root = self.project / ".env"+ original = root.read_text()+ cases = (+ ("qdrant", "QDRANT_API_KEY", "QDRANT_API_KEY", "qdrant", "QDRANT__SERVICE__API_KEY"),+ ("pgvector", "PGVECTOR_PGUSER", "PGVECTOR_USER", "pgvector", "POSTGRES_USER"),+ ("pgvector", "PGVECTOR_POSTGRES_DB", "PGVECTOR_DATABASE", "pgvector", "POSTGRES_DB"),+ ("pgvecto-rs", "PGVECTOR_PGUSER", "PGVECTO_RS_USER", "pgvecto-rs", "POSTGRES_USER"),+ ("pgvecto-rs", "PGVECTOR_POSTGRES_DB", "PGVECTO_RS_DATABASE", "pgvecto-rs", "POSTGRES_DB"),+ (+ "pgvector",+ "PGVECTOR_POSTGRES_PASSWORD",+ "PGVECTOR_PASSWORD",+ "pgvector",+ "POSTGRES_PASSWORD",+ ),+ (+ "pgvecto-rs",+ "PGVECTOR_POSTGRES_PASSWORD",+ "PGVECTO_RS_PASSWORD",+ "pgvecto-rs",+ "POSTGRES_PASSWORD",+ ),+ (+ "chroma",+ "CHROMA_SERVER_AUTHN_CREDENTIALS",+ "CHROMA_AUTH_CREDENTIALS",+ "chroma",+ "CHROMA_SERVER_AUTHN_CREDENTIALS",+ ),+ (+ "opensearch",+ "OPENSEARCH_INITIAL_ADMIN_PASSWORD",+ "OPENSEARCH_PASSWORD",+ "opensearch",+ "OPENSEARCH_INITIAL_ADMIN_PASSWORD",+ ),+ )+ for provider, key, client_key, server, server_key in cases:+ with self.subTest(provider=provider):+ root.write_text(original)+ self.override_root({"VECTOR_STORE": provider})+ target = f"envs/vectorstores/{provider}.env"+ try:+ self.copy_template(+ f"envs/vectorstores/{provider}.defaults",+ target,+ {key: "provider-password"},+ )+ for value in ("provider-password", "root-password"):+ if value == "root-password":+ self.override_root({key: value})+ services = self.render()+ for name in ("api", "worker", "worker_beat"):+ self.assertEqual(services[name]["environment"][client_key], value, name)+ self.assertEqual(services[server]["environment"][server_key], value)+ finally:+ (self.project / target).unlink(missing_ok=True)+ root.write_text(original)++ self.override_root({"VECTOR_STORE": "qdrant"})+ legacy = self.project / "envs/core-services/shared.env"+ legacy.write_text("QDRANT_URL=http://existing-qdrant:6333\nQDRANT_API_KEY=\n")+ services = self.render()+ for name in ("api", "worker", "worker_beat"):+ values = services[name]["environment"]+ self.assertEqual(values["QDRANT_URL"], "http://existing-qdrant:6333")+ self.assertEqual(values["QDRANT_API_KEY"], "")+ self.assertEqual(values["WEAVIATE_ENDPOINT"], "http://weaviate:8080")+ self.assertNotIn("PGVECTOR_HOST", values)++ # External providers have no bundled defaults file and keep their explicit connection settings.+ root.write_text(original)+ self.override_root({"VECTOR_STORE": "relyt", "RELYT_HOST": "external-relyt"})+ self.assertEqual(self.render()["api"]["environment"]["RELYT_HOST"], "external-relyt")+def test_copying_defaults_without_changes_preserves_service_values(self):baseline = self.render()for source in (DOCKER_DIR / "envs").rglob("*.defaults"):@@ -283,7 +772,7 @@ class ComposeEnvironmentTests(unittest.TestCase):{"OCEANBASE_VECTOR_HOST": ("custom-oceanbase", "root-oceanbase")},),("vectorstores/pgvector.defaults", vector_clients, {"PGVECTOR_MIN_CONNECTION": ("2", "3")}),- ("vectorstores/milvus.env.example", vector_clients, {"MILVUS_DATABASE": ("custom_dify", "root_dify")}),+ ("vectorstores/milvus.defaults", vector_clients, {"MILVUS_DATABASE": ("custom_dify", "root_dify")}),]root = self.project / ".env"initial_root = root.read_text()