Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
E2E: run every throwaway-stack lane on each release tag
Petr Bauch02fe9c683fdc
diff --git a/.github/workflows/e2e-scheduled.yml b/.github/workflows/e2e-scheduled.ymlindex 55a1dcf92..2bd491e05 100644--- a/.github/workflows/e2e-scheduled.yml+++ b/.github/workflows/e2e-scheduled.yml@@ -6,6 +6,13 @@ name: E2E scheduled# after a merge to main the provider lane on a throwaway stack. Real model# calls, so not on every PR, and never a required check# nightly the provider lane again, then a notification+# a release tag (switch-v*, candidates included) every lane that runs+# on a throwaway stack: fast, provider and agentic, on+# the tagged commit, always reported. A candidate may+# be cut from a release branch, whose commits no main+# run has seen, and it is the code about to be+# promoted; the deployed environments test what they+# run, not what is about to reach them## A failed run after a merge is reported too, since the lane is not a required# check and nothing else would surface it before the night; a passing one is@@ -33,6 +40,9 @@ on:- '!docs/**'- '!**/*.md'- 'console/packages/plugins/src/switch-skill/SKILL.md'+ # Path filters do not apply to tags: every release tag runs.+ tags:+ - 'switch-v*'workflow_dispatch:permissions:@@ -46,10 +56,25 @@ jobs:lane: provideragent_type: claude-code+ fast:+ name: Fast lane+ if: ${{ startsWith(github.ref, 'refs/tags/') }}+ uses: ./.github/workflows/e2e.yml+ with:+ lane: fast+ agent_type: claude-code++ agentic:+ name: Agentic lane+ if: ${{ startsWith(github.ref, 'refs/tags/') }}+ uses: ./.github/workflows/e2e.yml+ with:+ lane: agentic+ agent_type: claude-codenotify:name: Report- needs: [provider]+ needs: [provider, fast, agentic]if: ${{ always() && github.event_name != 'workflow_dispatch' }}runs-on: ubuntu-lateststeps:@@ -59,8 +84,14 @@ jobs:RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}PROVIDER_RESULT: ${{ needs.provider.result }}PROVIDER_SUMMARY: ${{ needs.provider.outputs.summary }}+ FAST_RESULT: ${{ needs.fast.result }}+ FAST_SUMMARY: ${{ needs.fast.outputs.summary }}+ AGENTIC_RESULT: ${{ needs.agentic.result }}+ AGENTIC_SUMMARY: ${{ needs.agentic.outputs.summary }}run: |- if [ "$GITHUB_EVENT_NAME" = push ] && [ "$PROVIDER_RESULT" = success ]; then+ tagged=false+ [ "$GITHUB_REF_TYPE" = tag ] && tagged=true+ if [ "$GITHUB_EVENT_NAME" = push ] && [ "$tagged" = false ] && [ "$PROVIDER_RESULT" = success ]; thenecho "The provider lane passed after the merge; nothing to report."exit 0fi@@ -69,14 +100,21 @@ jobs:[ "$2" = success ] && mark=✅printf '%s %s — %s\n' "$mark" "$1" "${3:-$2, no scenario summary}"}- if [ "$GITHUB_EVENT_NAME" = push ]; then+ if [ "$tagged" = true ]; then+ title="Switch E2E on $GITHUB_REF_NAME"+ elif [ "$GITHUB_EVENT_NAME" = push ]; thentitle="Switch E2E after merging ${GITHUB_SHA:0:9}"elsetitle='Switch E2E nightly'fi- text="$(printf '%s\n%s\n%s' "$title" \- "$(line 'Provider lane (ephemeral)' "$PROVIDER_RESULT" "$PROVIDER_SUMMARY")" \- "$RUN_URL")"+ lines="$(line 'Provider lane (ephemeral)' "$PROVIDER_RESULT" "$PROVIDER_SUMMARY")"+ if [ "$tagged" = true ]; then+ lines="$(printf '%s\n%s\n%s' \+ "$(line 'Fast lane (ephemeral)' "$FAST_RESULT" "$FAST_SUMMARY")" \+ "$lines" \+ "$(line 'Agentic lane (ephemeral)' "$AGENTIC_RESULT" "$AGENTIC_SUMMARY")")"+ fi+ text="$(printf '%s\n%s\n%s' "$title" "$lines" "$RUN_URL")"echo "$text"if [ -z "$WEBHOOK" ]; thenecho "::warning::E2E_NOTIFY_WEBHOOK_URL is not set; this report was not sent anywhere."diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.ymlindex 6326f05ff..7e64267eb 100644--- a/.github/workflows/e2e.yml+++ b/.github/workflows/e2e.yml@@ -22,6 +22,8 @@ name: E2E# GitHub Environment, whose protection rules decide which refs may run it:## e2e-provider secret ANTHROPIC_API_KEY, for the controller's Claude Code+# (and the agentic lane's driver and judge); open to main and+# to the release tags, switch-v*## Runs are strict (SWITCH_E2E_STRICT=1): a scenario that cannot run fails# rather than skips.diff --git a/console/tooling-e2e/README.md b/console/tooling-e2e/README.mdindex 0e6c34751..79c8adf94 100644--- a/console/tooling-e2e/README.md+++ b/console/tooling-e2e/README.md@@ -417,12 +417,14 @@ hand.| Every PR, and every push to main (`pr-ci.yml`) | fast | ephemeral | yes || After a merge to main (`e2e-scheduled.yml`) | provider | ephemeral | no || Nightly (`e2e-scheduled.yml`) | provider | ephemeral | no |-| By hand only, for now (`e2e.yml`, dispatch) | agentic | ephemeral | no |+| A release tag, candidates included (`e2e-scheduled.yml`) | fast; provider; agentic | ephemeral | no |+| By hand (`e2e.yml`, dispatch) | any | ephemeral | no |- **Secrets.** An ephemeral fast run generates every secret it uses on therunner, so it works the same on a fork PR. Every other run reads its secretsfrom the GitHub Environment `e2e-provider`, which holds `ANTHROPIC_API_KEY`- and whose protection rules decide which branches may use it.+ and whose protection rules decide which refs may use it: `main`, and the+ release tags (`switch-v*`).- **Artifacts.** The output directory is uploaded only after it passes twoscans. The first (`scan-artifacts.sh`) looks for the exact secret values therun used: the ones it was configured with, plus every credential the server@@ -434,7 +436,8 @@ hand.- **Reporting.** Each run writes a scenario table to the job summary. Reportsgo to the webhook in the repository secret `E2E_NOTIFY_WEBHOOK_URL`:- one summary each night;- - a report when the provider lane fails after a merge.+ - a report when the provider lane fails after a merge;+ - a report for every release tag, pass or fail.## Against a deployed environment