Build Agentic workflows, RAG pipelines, with rich AI model and tool support on one collaborative workspace. Deploy on cloud, VPC, or self-hosted, so teams move from prototype to production without rebuilding the stack.
Python agentagentic-aiagentic-frameworkagentic-workflowaiautomationclaudedeepseekgenaigptllmlow-codemcpnextjsno-codeopenaipythonskillsworkflow
test(e2e): align Agent v2 publish scenarios with guidance (#42787)
yyh1aafa481f32c
diff --git a/e2e/features/agent-v2/access-point.feature b/e2e/features/agent-v2/access-point.featureindex 2870e5270f9..cd784aa0589 100644--- a/e2e/features/agent-v2/access-point.feature+++ b/e2e/features/agent-v2/access-point.feature@@ -151,7 +151,7 @@ Feature: Agent v2 Access PointAnd a runnable Agent v2 test agent has been created via APIWhen I open the Agent v2 configure pageAnd I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date+ Then the Agent v2 first publication should succeedGiven an Agent v2 Backend service API key has been created via APIWhen I send the Agent v2 Backend service API minimal requestThen the Agent v2 Backend service API request should succeed with the normal E2E markerdiff --git a/e2e/features/agent-v2/knowledge.feature b/e2e/features/agent-v2/knowledge.featureindex bb235815945..4d7162cb8b3 100644--- a/e2e/features/agent-v2/knowledge.feature+++ b/e2e/features/agent-v2/knowledge.feature@@ -36,7 +36,7 @@ Feature: Agent v2 Knowledge RetrievalThen the Agent v2 Agent decide Knowledge Retrieval should be saved in the Agent v2 draftAnd the Agent v2 configuration should be saved automaticallyWhen I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date+ Then the Agent v2 first publication should succeedGiven an Agent v2 Backend service API key has been created via APIWhen I send the Agent v2 Backend service API knowledge requestThen the Agent v2 Backend service API response should include the knowledge E2E marker@@ -53,7 +53,7 @@ Feature: Agent v2 Knowledge RetrievalThen the Agent v2 Custom query Knowledge Retrieval should be saved in the Agent v2 draftAnd the Agent v2 configuration should be saved automaticallyWhen I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date+ Then the Agent v2 first publication should succeedGiven an Agent v2 Backend service API key has been created via APIWhen I send the Agent v2 Backend service API knowledge requestThen the Agent v2 Backend service API response should include the knowledge E2E markerdiff --git a/e2e/features/agent-v2/publish.feature b/e2e/features/agent-v2/publish.featureindex 30a8eabb799..975d6cce08d 100644--- a/e2e/features/agent-v2/publish.feature+++ b/e2e/features/agent-v2/publish.feature@@ -1,110 +1,95 @@@agent-v2 @authenticated @publishFeature: Agent v2 publish- @core- Scenario: Publish is blocked until a model is configured+ Background:Given I am signed in as the default E2E admin- And an Agent v2 test agent has been created via API- And the Agent v2 composer draft uses the normal E2E prompt- When I open the Agent v2 configure page- And I try to publish the Agent v2 draft without a model- Then Agent v2 publish should be blocked until a model is configured- And the Agent v2 draft should remain unpublished@core @prepared @stable-modelScenario: Publish a configured Agent v2 draft- Given I am signed in as the default E2E admin- And the Agent Builder stable chat model is available+ Given the Agent Builder stable chat model is availableAnd a runnable Agent v2 test agent has been created via APIWhen I open the Agent v2 configure pageAnd I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date- When I switch to the Agent v2 Access Point section- Then Agent v2 Web app access should be in service- And Agent v2 Backend service API access should be in service+ Then the Agent v2 first publication should succeed@core @prepared @stable-model- Scenario: Publish action follows unpublished changes- Given I am signed in as the default E2E admin- And the Agent Builder stable chat model is available+ Scenario: Publish guidance opens Agent v2 access methods+ Given the Agent Builder stable chat model is available+ And a runnable Agent v2 test agent has been created via API+ When I open the Agent v2 configure page+ And I publish the Agent v2 draft+ And I follow the access methods link in the Agent v2 publish guidance+ Then the Agent v2 Access Point should open++ @core @prepared @stable-model @published-web-app+ Scenario: Publish guidance opens the Agent v2 Web app+ Given the Agent Builder stable chat model is availableAnd a runnable Agent v2 test agent has been created via APIWhen I open the Agent v2 configure page- Then the Agent v2 publish action should be available for unpublished changes- When I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date- And the Agent v2 publish action should be unavailable while up to date- When I fill the Agent v2 prompt editor with the updated E2E prompt- Then the Agent v2 configuration should be saved automatically- And the Agent v2 publish action should be available for unpublished changes+ And I publish the Agent v2 draft+ And I follow the Web app link in the Agent v2 publish guidance+ Then the Agent v2 Web app should open in a new tab@core @prepared @stable-model- Scenario: Restoring a published Agent v2 version shows the restored configuration in Builder- Given I am signed in as the default E2E admin- And the Agent Builder stable chat model is available+ Scenario: Editing a published Agent v2 enables Publish update+ Given the Agent Builder stable chat model is availableAnd a runnable Agent v2 test agent has been created via API+ And the Agent v2 draft has been published via APIWhen I open the Agent v2 configure page+ And I fill the Agent v2 prompt editor with the updated E2E prompt+ Then the Agent v2 Publish update action should be available++ @core @prepared @stable-model+ Scenario: Publishing Agent v2 draft changes shows update guidance+ Given the Agent Builder stable chat model is available+ And a runnable Agent v2 test agent has been created via API+ And the Agent v2 draft has been published via API+ When I open the Agent v2 configure page+ And I fill the Agent v2 prompt editor with the updated E2E promptAnd I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date- When I fill the Agent v2 prompt editor with the updated E2E prompt- Then the Agent v2 configuration should be saved automatically- And the normal Agent v2 draft should use the updated E2E prompt- When I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date- When I open the Agent v2 version history- And I select Agent v2 published version 1- Then the selected Agent v2 version should be displayed in view-only mode- And I should see the normal E2E prompt in the Agent v2 prompt editor- When I restore the selected Agent v2 version- Then I should see the normal E2E prompt in the Agent v2 prompt editor- And the normal Agent v2 draft should use the normal E2E prompt- And the Agent v2 publish action should be available for unpublished changes+ Then the Agent v2 update publication should succeed++ @core @prepared @stable-model+ Scenario: Viewing a published Agent v2 version is read-only+ Given the Agent Builder stable chat model is available+ And an Agent v2 has original and updated published prompts+ When I view the original Agent v2 published version in version history+ Then the original Agent v2 version should be view-only++ @core @prepared @stable-model+ Scenario: Restoring a published Agent v2 version shows the restored configuration in Builder+ Given the Agent Builder stable chat model is available+ And an Agent v2 has original and updated published prompts+ When I view the original Agent v2 published version in version history+ And I restore the selected Agent v2 version+ Then the original Agent v2 configuration should be an unpublished draft@web-app-runtime @external-model @agent-backend-runtime @published-web-app @stable-modelScenario: Published Agent v2 answers through Web app- Given I am signed in as the default E2E admin- And the Agent Builder stable chat model is available+ Given the Agent Builder stable chat model is availableAnd the Agent v2 runtime backend is availableAnd a runnable Agent v2 test agent has been created via API- When I open the Agent v2 configure page- And I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date+ And the Agent v2 draft has been published via APIWhen I open the Agent v2 Web app URLAnd I send an E2E message in the Agent v2 Web appThen the Agent v2 Web app response should include the normal E2E marker- When I close the Agent v2 Web app@web-app-runtime @external-model @agent-backend-runtime @published-web-app @stable-modelScenario: Published Web app remains isolated from unpublished Agent v2 draft edits- Given I am signed in as the default E2E admin- And the Agent Builder stable chat model is available+ Given the Agent Builder stable chat model is availableAnd the Agent v2 runtime backend is availableAnd a runnable Agent v2 test agent has been created via API- When I open the Agent v2 configure page- And I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date- When I fill the Agent v2 prompt editor with the updated E2E prompt- Then the Agent v2 configuration should be saved automatically- And the normal Agent v2 draft should use the updated E2E prompt+ And the Agent v2 draft has been published via API+ And the Agent v2 draft has unpublished updated prompt changesWhen I open the Agent v2 Web app URLAnd I send an E2E message in the Agent v2 Web appThen the Agent v2 Web app response should include the normal E2E markerAnd the Agent v2 Web app response should not include the updated E2E marker- When I close the Agent v2 Web app@web-app-runtime @external-model @agent-backend-runtime @published-web-app @stable-modelScenario: Published Web app uses the latest Agent v2 published configuration- Given I am signed in as the default E2E admin- And the Agent Builder stable chat model is available+ Given the Agent Builder stable chat model is availableAnd the Agent v2 runtime backend is available- And a runnable Agent v2 test agent has been created via API- When I open the Agent v2 configure page- And I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date- When I fill the Agent v2 prompt editor with the updated E2E prompt- Then the Agent v2 configuration should be saved automatically- And the normal Agent v2 draft should use the updated E2E prompt- When I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date+ And an Agent v2 has original and updated published promptsWhen I open the Agent v2 Web app URLAnd I send an E2E message in the Agent v2 Web appThen the Agent v2 Web app response should include the updated E2E marker- When I close the Agent v2 Web appdiff --git a/e2e/features/agent-v2/tools.feature b/e2e/features/agent-v2/tools.featureindex c84f4274148..d05fd65c2de 100644--- a/e2e/features/agent-v2/tools.feature+++ b/e2e/features/agent-v2/tools.feature@@ -22,7 +22,7 @@ Feature: Agent v2 toolsWhen I open the Agent v2 configure pageThen the Agent v2 JSON Replace tool should be saved in the Agent v2 draftWhen I publish the Agent v2 draft- Then the Agent v2 draft should be published and up to date+ Then the Agent v2 first publication should succeedGiven an Agent v2 Backend service API key has been created via APIWhen I send the Agent v2 Backend service API JSON Replace requestThen the Agent v2 Backend service API response should include the JSON Replace E2E markerdiff --git a/e2e/features/step-definitions/agent-v2/publish.steps.ts b/e2e/features/step-definitions/agent-v2/publish.steps.tsindex 42d08e58c65..510271e3919 100644--- a/e2e/features/step-definitions/agent-v2/publish.steps.ts+++ b/e2e/features/step-definitions/agent-v2/publish.steps.ts@@ -1,18 +1,58 @@import type { DifyWorld } from '../../support/world.ts'-import { Then, When } from '@cucumber/cucumber'+import { Given, Then, When } from '@cucumber/cucumber'import { expect } from '@playwright/test'import { waitForAgentConfigureAutosaved } from '../../../support/agent-configure.ts'-import { expectAgentModelRequiredFeedback, getCurrentAgentId } from './configure-helpers.ts'+import {+ createAgentSoulConfigWithModel,+ normalAgentPrompt,+ normalAgentSoulConfig,+ updatedAgentSoulConfig,+} from '../../agent-v2/support/agent-soul.ts'+import {+ createTestAgent,+ getAgentConfigurePath,+ publishAgentWithPublishableDraft,+ saveAgentComposerDraft,+} from '../../agent-v2/support/agent.ts'+import { getCurrentAgentId } from './configure-helpers.ts'++Given('an Agent v2 has original and updated published prompts', async function (this: DifyWorld) {+ const stableModel = this.agentBuilder.fixtures.stableModel+ if (!stableModel)+ throw new Error('Create published Agent v2 versions after stable model fixture setup.')++ const client = this.getConsoleClient()+ const agent = await createTestAgent(client)+ this.createdAgentIds.push(agent.id)+ await saveAgentComposerDraft(+ client,+ agent.id,+ createAgentSoulConfigWithModel(normalAgentSoulConfig, stableModel),+ )+ await publishAgentWithPublishableDraft(client, agent.id, 'E2E original prompt')+ await saveAgentComposerDraft(+ client,+ agent.id,+ createAgentSoulConfigWithModel(updatedAgentSoulConfig, stableModel),+ )+ await publishAgentWithPublishableDraft(client, agent.id, 'E2E updated prompt')+})-When('I publish the Agent v2 draft', async function (this: DifyWorld) {- const page = this.getPage()- const publishButton = page.getByRole('button', { name: /^Publish(?: update)?$/ })+Given(+ 'the Agent v2 draft has unpublished updated prompt changes',+ async function (this: DifyWorld) {+ const stableModel = this.agentBuilder.fixtures.stableModel+ if (!stableModel) throw new Error('Update the Agent v2 draft after stable model fixture setup.')- await expect(publishButton).toBeEnabled({ timeout: 30_000 })- await publishButton.click()-})+ await saveAgentComposerDraft(+ this.getConsoleClient(),+ getCurrentAgentId(this),+ createAgentSoulConfigWithModel(updatedAgentSoulConfig, stableModel),+ )+ },+)-When('I try to publish the Agent v2 draft without a model', async function (this: DifyWorld) {+When('I publish the Agent v2 draft', async function (this: DifyWorld) {const page = this.getPage()const publishButton = page.getByRole('button', { name: /^Publish(?: update)?$/ })@@ -20,102 +60,94 @@ When('I try to publish the Agent v2 draft without a model', async function (thisawait publishButton.click()})-Then(- 'Agent v2 publish should be blocked until a model is configured',- async function (this: DifyWorld) {- await expectAgentModelRequiredFeedback(this.getPage())- },-)--Then('the Agent v2 draft should remain unpublished', async function (this: DifyWorld) {- await expect- .poll(- async () => {- const agentId = getCurrentAgentId(this)- const composer = await this.getConsoleClient().agent.byAgentId.composer.get({- params: { agent_id: agentId },- })- return composer.active_config_is_published- },- { timeout: 30_000 },- )- .toBe(false)-})-Then('the Agent v2 configuration should be saved automatically', async function (this: DifyWorld) {await waitForAgentConfigureAutosaved(this.getPage())})-Then('the Agent v2 draft should be published and up to date', async function (this: DifyWorld) {+Then('the Agent v2 first publication should succeed', async function (this: DifyWorld) {const page = this.getPage()- const agentId = getCurrentAgentId(this)- await expect(page.getByRole('button', { name: 'Published' })).toBeVisible({ timeout: 30_000 })- await expect(page.getByRole('status', { name: /^Up to date\./ })).toBeVisible()- await expect(page.getByText('Up to date')).toBeVisible()- await expect- .poll(async () => {- const composer = await this.getConsoleClient().agent.byAgentId.composer.get({- params: { agent_id: agentId },- })- return composer.active_config_is_published- })- .toBe(true)+ await expect(+ page.getByRole('paragraph').filter({ hasText: /^Published successfully$/ }),+ ).toBeVisible({ timeout: 30_000 })})-Then(- 'the Agent v2 publish action should be available for unpublished changes',+Then('the Agent v2 update publication should succeed', async function (this: DifyWorld) {+ await expect(+ this.getPage()+ .getByRole('paragraph')+ .filter({ hasText: /^Changes published$/ }),+ ).toBeVisible({ timeout: 30_000 })+})++When(+ 'I follow the access methods link in the Agent v2 publish guidance',async function (this: DifyWorld) {- const page = this.getPage()+ const link = this.getPage().getByRole('link', { name: 'View all access methods' })- await expect(- page.getByRole('status', { name: /^(?:Draft|Unpublished changes)\./ }),- ).toBeVisible({ timeout: 30_000 })- await expect(page.getByRole('button', { name: /^Publish(?: update)?$/ })).toBeEnabled()+ await expect(link).toBeVisible({ timeout: 30_000 })+ await link.click()},)-Then(- 'the Agent v2 publish action should be unavailable while up to date',+Then('the Agent v2 Access Point should open', async function (this: DifyWorld) {+ const page = this.getPage()++ await expect(page).toHaveURL(new RegExp(`/agents/${getCurrentAgentId(this)}/access(?:\\?.*)?$`))+ await expect(page.getByRole('region', { name: 'Access Point' })).toBeVisible()+})++When(+ 'I follow the Web app link in the Agent v2 publish guidance',async function (this: DifyWorld) {const page = this.getPage()+ const link = page.getByRole('link', { name: 'Open Web App' })++ await expect(link).toBeVisible({ timeout: 30_000 })+ const href = await link.getAttribute('href')+ if (!href) throw new Error('Agent v2 publish guidance Web app link has no destination.')- await expect(page.getByRole('status', { name: /^Up to date\./ })).toBeVisible({- timeout: 30_000,- })- await expect(page.getByRole('button', { name: 'Published' })).toBeDisabled()+ const [webAppPage] = await Promise.all([page.waitForEvent('popup'), link.click()])+ this.agentBuilder.accessPoint.webAppURL = href+ this.agentBuilder.accessPoint.webAppPage = webAppPage},)-When('I open the Agent v2 version history', async function (this: DifyWorld) {+Then('the Agent v2 Publish update action should be available', async function (this: DifyWorld) {const page = this.getPage()- await page.getByRole('button', { name: 'Open version history' }).click()- await expect(page.getByRole('heading', { name: 'Versions' })).toBeVisible({ timeout: 30_000 })+ await expect(page.getByRole('status', { name: /^Unpublished changes\./ })).toBeVisible({+ timeout: 30_000,+ })+ await expect(page.getByRole('button', { name: 'Publish update' })).toBeEnabled()})When(- 'I select Agent v2 published version {int}',- async function (this: DifyWorld, versionNumber: number) {+ 'I view the original Agent v2 published version in version history',+ async function (this: DifyWorld) {const page = this.getPage()- const versionButton = page.getByRole('button', {- name: new RegExp(`^Version ${versionNumber}\\b`),- })++ await page.goto(getAgentConfigurePath(getCurrentAgentId(this)))+ await page.getByRole('button', { name: 'Open version history' }).click()+ await expect(page.getByRole('heading', { name: 'Versions' })).toBeVisible({ timeout: 30_000 })+ const versionButton = page.getByRole('button', { name: /^E2E original prompt\b/ })await expect(versionButton).toBeVisible({ timeout: 30_000 })await versionButton.click()},)-Then(- 'the selected Agent v2 version should be displayed in view-only mode',- async function (this: DifyWorld) {- const page = this.getPage()-- await expect(page.getByText('View Only')).toBeVisible({ timeout: 30_000 })- await expect(page.getByRole('button', { name: 'Restore' })).toBeEnabled()- },-)+Then('the original Agent v2 version should be view-only', async function (this: DifyWorld) {+ const page = this.getPage()+ const promptEditor = page+ .getByRole('region', { name: 'Prompt' })+ .getByRole('textbox', { name: 'Prompt' })++ await expect(page.getByText('View Only')).toBeVisible({ timeout: 30_000 })+ await expect(promptEditor).toContainText(normalAgentPrompt)+ await expect(promptEditor).not.toBeEditable()+ await expect(page.getByRole('button', { name: 'Restore' })).toBeEnabled()+})When('I restore the selected Agent v2 version', async function (this: DifyWorld) {const page = this.getPage()@@ -136,3 +168,18 @@ When('I restore the selected Agent v2 version', async function (this: DifyWorld)const response = await restoreResponseexpect(response.ok()).toBe(true)})++Then(+ 'the original Agent v2 configuration should be an unpublished draft',+ async function (this: DifyWorld) {+ const page = this.getPage()+ const promptEditor = page+ .getByRole('region', { name: 'Prompt' })+ .getByRole('textbox', { name: 'Prompt' })++ await expect(promptEditor).toContainText(normalAgentPrompt, { timeout: 30_000 })+ await expect(promptEditor).toBeEditable()+ await expect(page.getByRole('status', { name: /^Unpublished changes\./ })).toBeVisible()+ await expect(page.getByRole('button', { name: 'Publish update' })).toBeEnabled()+ },+)