Skip to content

armgit / dify

Public Imported from GitHub
0

Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.

fix(e2e): skip hidden Agent knowledge E2E scenarios (#40691)

Joel190780e12545
diff --git a/e2e/AGENTS.md b/e2e/AGENTS.md
index fd10b188b1a..240d77c1be8 100644
--- a/e2e/AGENTS.md
+++ b/e2e/AGENTS.md
@@ -44,6 +44,7 @@ An uninitialized instance is installed and authenticated lazily; an initialized
- `@external-model` and `@external-tool` identify scenarios that call real external runtimes. Deterministic commands exclude these tags; external commands are opt-in.
- `@microphone` uses the checked-in fake audio fixture and an isolated Chromium context.
- `@browser-smoke` runs focused keyboard and navigation coverage in Chromium and WebKit CI lanes.
+- `@skip` temporarily excludes a scenario from every runner profile. Remove it as soon as the covered product behavior is available again; do not use it for permanent or environment-dependent suppression.
- Feature-owned services use their own tags. Agent v2 runtime scenarios use `@agent-backend-runtime` and require the explicit runtime-availability step. Set `E2E_START_AGENT_BACKEND=1` to start it locally, or provide `E2E_AGENT_BACKEND_URL` / `AGENT_BACKEND_BASE_URL`.
Seed and Cucumber must share one runtime lifecycle. Combined commands own reset, middleware, services, seed, Cucumber, and teardown; CI must not reproduce that lifecycle in workflow YAML. `E2E_START_AGENT_BACKEND=1` starts a managed local backend before the API; it is mutually exclusive with an explicit Agent backend URL.
diff --git a/e2e/cucumber.config.ts b/e2e/cucumber.config.ts
index 81fbcacb7e2..b7768c36d7b 100644
--- a/e2e/cucumber.config.ts
+++ b/e2e/cucumber.config.ts
@@ -4,8 +4,9 @@ import './scripts/env-register'
const hasCliTags = process.argv.some((arg) => arg === '--tags' || arg.startsWith('--tags='))
const defaultNonExternalTags =
'not @axe and not @prepared and not @external-model and not @external-tool'
-const defaultTags =
+const selectedTags =
process.env.E2E_CUCUMBER_TAGS || (hasCliTags ? undefined : defaultNonExternalTags)
+const tags = selectedTags ? `(${selectedTags}) and not @skip` : 'not @skip'
const config = {
format: [
@@ -16,7 +17,7 @@ const config = {
],
import: ['./tsx-register.js', 'features/**/*.ts'],
paths: ['features/**/*.feature'],
- ...(defaultTags ? { tags: defaultTags } : {}),
+ tags,
timeout: 60_000,
} satisfies Partial<IConfiguration> & {
timeout: number
diff --git a/e2e/features/agent-v2/agent-edit.feature b/e2e/features/agent-v2/agent-edit.feature
index ca13b4fc845..179a772a4c7 100644
--- a/e2e/features/agent-v2/agent-edit.feature
+++ b/e2e/features/agent-v2/agent-edit.feature
@@ -1,6 +1,6 @@
@agent-v2 @authenticated @agent-edit
Feature: Agent v2 Agent Edit page
- @core @prepared @stable-model @full-config-agent
+ @core @prepared @stable-model @full-config-agent @skip
Scenario: Saved orchestration sections are visible on the Agent Edit page
Given I am signed in as the default E2E admin
And the Agent Builder stable chat model is available
@@ -32,7 +32,7 @@ Feature: Agent v2 Agent Edit page
When I open the preseeded Agent v2 configure page for "E2E New Agent Builder Tool States" from the Agent Roster
Then I should see the Agent v2 tool state fixture tools
- @core @prepared @dual-retrieval-fixture
+ @core @prepared @dual-retrieval-fixture @skip
Scenario: Dual Knowledge Retrieval settings are visible on the Agent Edit page
Given I am signed in as the default E2E admin
And the Agent Builder preseeded Agent "E2E Agent With Dual Retrieval" is available
diff --git a/e2e/features/agent-v2/knowledge.feature b/e2e/features/agent-v2/knowledge.feature
index 20184bf697c..bb235815945 100644
--- a/e2e/features/agent-v2/knowledge.feature
+++ b/e2e/features/agent-v2/knowledge.feature
@@ -1,6 +1,6 @@
@agent-v2 @authenticated @knowledge @knowledge-fixture
Feature: Agent v2 Knowledge Retrieval
- @core @prepared
+ @core @prepared @skip
Scenario: Agent decide Knowledge Retrieval settings are saved and restored
Given I am signed in as the default E2E admin
And the Agent Builder preseeded dataset "E2E Agent Knowledge Base" is indexed and ready
@@ -12,7 +12,7 @@ Feature: Agent v2 Knowledge Retrieval
When I refresh the current page
Then I should see the Agent v2 Agent decide Knowledge Retrieval settings
- @core @prepared
+ @core @prepared @skip
Scenario: Custom query Knowledge Retrieval settings are saved and restored
Given I am signed in as the default E2E admin
And the Agent Builder preseeded dataset "E2E Agent Knowledge Base" is indexed and ready
@@ -24,7 +24,7 @@ Feature: Agent v2 Knowledge Retrieval
When I refresh the current page
Then I should see the Agent v2 Custom query Knowledge Retrieval settings
- @service-api-runtime @external-model @agent-backend-runtime @agent-decision-model @backend-api-access
+ @service-api-runtime @external-model @agent-backend-runtime @agent-decision-model @backend-api-access @skip
Scenario: Agent decide Knowledge Retrieval answers through Backend service API
Given I am signed in as the default E2E admin
And the Agent Builder agent-decision chat model is available
@@ -41,7 +41,7 @@ Feature: Agent v2 Knowledge Retrieval
When I send the Agent v2 Backend service API knowledge request
Then the Agent v2 Backend service API response should include the knowledge E2E marker
- @service-api-runtime @external-model @agent-backend-runtime @stable-model @backend-api-access
+ @service-api-runtime @external-model @agent-backend-runtime @stable-model @backend-api-access @skip
Scenario: Custom query Knowledge Retrieval answers through Backend service API
Given I am signed in as the default E2E admin
And the Agent Builder stable chat model is available
@@ -58,7 +58,7 @@ Feature: Agent v2 Knowledge Retrieval
When I send the Agent v2 Backend service API knowledge request
Then the Agent v2 Backend service API response should include the knowledge E2E marker
- @core @prepared
+ @core @prepared @skip
Scenario: Removing Knowledge Retrieval clears the saved dataset reference
Given I am signed in as the default E2E admin
And the Agent Builder preseeded dataset "E2E Agent Knowledge Base" is indexed and ready
diff --git a/e2e/features/step-definitions/agent-v2/advanced-settings.steps.ts b/e2e/features/step-definitions/agent-v2/advanced-settings.steps.ts
index 9d9d33044d0..4434e58b2da 100644
--- a/e2e/features/step-definitions/agent-v2/advanced-settings.steps.ts
+++ b/e2e/features/step-definitions/agent-v2/advanced-settings.steps.ts
@@ -36,7 +36,7 @@ Then(
await expect(advancedSettings).toBeVisible()
await expect(
- advancedSettings.getByText('For power users. Env vars, sandbox & memory.'),
+ advancedSettings.getByText('For power users, such as environment variables.'),
).toBeVisible()
await expect(advancedSettings.getByRole('heading', { name: 'Env Editor' })).not.toBeVisible()
},
diff --git a/e2e/features/step-definitions/agent-v2/agent-edit.steps.ts b/e2e/features/step-definitions/agent-v2/agent-edit.steps.ts
index 698784b11f1..29cbc883992 100644
--- a/e2e/features/step-definitions/agent-v2/agent-edit.steps.ts
+++ b/e2e/features/step-definitions/agent-v2/agent-edit.steps.ts
@@ -174,7 +174,7 @@ Then('I should see the Agent v2 full-config fixture sections', async function (t
const advancedSettings = page.getByRole('region', { name: 'Advanced Settings' })
await expect(advancedSettings).toBeVisible()
await expect(
- advancedSettings.getByText('For power users. Env vars, sandbox & memory.'),
+ advancedSettings.getByText('For power users, such as environment variables.'),
).toBeVisible()
})