Skip to content

πŸš€ The fast, Pythonic way to build MCP servers and clients.

Add a workflow that deploys agentic triage to Prefect Cloud (#6380)

nate nowack55a3cd232472
diff --git a/.github/agentic/README.md b/.github/agentic/README.md
index a6fc21456..623e727a5 100644
--- a/.github/agentic/README.md
+++ b/.github/agentic/README.md
@@ -88,23 +88,29 @@ remain separate steps. No live workflow changes follow from this patch.
## Deploy
Merging does not deploy. Live automations run the `source_sha` pinned in their
-runtime parameters. To deploy a new main commit, from a clean checkout of it:
+runtime parameters. Run the [Deploy agentic triage](../workflows/deploy-agentic.yml)
+workflow from main to deploy main's tip, or pass a commit on main as `ref`. It uses the
+`PREFECT_API_KEY` repo secret against `prefect-technologies/marvin-bot` and runs:
```sh
-uv run python -m flows.agentic_stack_plans # only if plans or models changed
-bash scripts/agentic_stack.sh validate
-bash scripts/agentic_stack.sh register # registers deployments from prefect.yaml
+bash scripts/agentic_stack.sh deploy
```
-Then publish any changed role plans with the Agentic Workflows tooling and update
-`source_sha` (and any new plan versions) in both triage automations right away.
-Flows refuse to run when their deployment version differs from the pinned
-`source_sha`, so between `register` and that update, new intakes and any in-flight
-run that starts a child fail. Check for running intakes first. If the alert template
-changed, re-run `agentic_stack.sh alerts`.
+`deploy` validates the role plans, waits for no triage runs to be scheduled or in
+flight, registers the deployments from `prefect.yaml`, and immediately repins
+`source_sha` in both triage automations. Flows refuse to run when their deployment
+version differs from the pinned `source_sha`, so those two steps stay back to back.
+It is a no-op when the commit is already live.
+
+`deploy` refuses when the execution plans or `stack-models.json` changed since the
+live commit. Then, from a clean checkout of the new commit, rebuild the plans
+(`uv run python -m flows.agentic_stack_plans`), publish them with the Agentic
+Workflows tooling, run `agentic_stack.sh register`, and update `source_sha` and the
+plan versions in both automations right away. If the alert template changed, re-run
+`agentic_stack.sh alerts`.
## Roll back
-Set `source_sha` in both automations back to the previous commit and re-register
-from that commit. If a plan changed, also point the runtime back at the previous
-plan versions.
+Run the deploy workflow with the previous commit as `ref`. If the plans differ between
+the two commits, roll back by hand: re-register from the previous commit and point
+`source_sha` and the plan versions in both automations back.
diff --git a/.github/agentic/scripts/agentic_stack.sh b/.github/agentic/scripts/agentic_stack.sh
index fb3211a99..fa353b64a 100644
--- a/.github/agentic/scripts/agentic_stack.sh
+++ b/.github/agentic/scripts/agentic_stack.sh
@@ -1,6 +1,7 @@
#!/usr/bin/env bash
set -euo pipefail
-cd "$(dirname "$0")/.."
+# FASTMCP_AGENTIC_DIR lets a newer driver operate on an older checkout (rollback).
+cd "${FASTMCP_AGENTIC_DIR:-$(dirname "$0")/..}"
: "${PREFECT_API_URL:?Set the Prefect Cloud workspace API URL}"
[[ $PREFECT_API_URL == https://api.prefect.cloud/api/accounts/*/workspaces/* ]] || {
echo "Expected a Prefect Cloud workspace API URL" >&2; exit 2;
@@ -16,28 +17,100 @@ upsert() {
else api POST "/$kind/" --data "@$body" | jq -er .id; fi
}
+DEPLOYMENTS=(cloud-stack-batch cloud-contribution-verification cloud-triage-intake cloud-triage-verify cloud-triage-finalize)
+TRIGGERS=(fastmcp-triage-on-github-issue fastmcp-triage-on-github-pr)
+json_list() { printf '%s\n' "$@" | jq -R . | jq -sc .; }
+
+validate() {
+ local scratch role
+ scratch=$(mktemp -d)
+ for role in assess reproduce triage_review; do
+ jq '{plan: .}' "workflows/fastmcp-stack-$role.plan.json" > "$scratch/$role.json"
+ api POST /execution-plans/validate --data "@$scratch/$role.json" | jq -e '.valid == true'
+ done
+ rm -rf "$scratch"
+}
+
+register() {
+ local revision published name args=()
+ [[ -z $(git status --porcelain) ]] || {
+ echo 'Commit local changes before registration' >&2; exit 1;
+ }
+ revision=$(git rev-parse HEAD)
+ published=$(gh api "repos/PrefectHQ/fastmcp/commits/$revision" --jq .sha)
+ [[ $published == "$revision" ]] || {
+ echo 'Source commit must be available in PrefectHQ/fastmcp' >&2; exit 1;
+ }
+ for name in "${DEPLOYMENTS[@]}"; do args+=(--name "$name"); done
+ FASTMCP_AGENTIC_REF=$revision uv run --locked prefect --no-prompt deploy \
+ --prefect-file prefect.yaml "${args[@]}"
+}
+
+# Distinct source_sha values pinned by the two triage automations, as a JSON list.
+live_sources() {
+ jq -ec --argjson names "$(json_list "${TRIGGERS[@]}")" '
+ [.[] | select(.name | IN($names[]))] as $found
+ | if ([$found[].name] | sort) != ($names | sort)
+ then error("Expected exactly one of each triage automation") else . end
+ | [$found[].actions[0].parameters.runtime.source_sha] | unique
+ | if all(test("^[a-f0-9]{40}$")) then . else error("Invalid source_sha") end' "$1"
+}
+
case "${1:-}" in
validate)
- scratch=$(mktemp -d)
- trap 'rm -rf "$scratch"' EXIT
- for role in assess reproduce triage_review; do
- jq '{plan: .}' "workflows/fastmcp-stack-$role.plan.json" > "$scratch/$role.json"
- api POST /execution-plans/validate --data "@$scratch/$role.json" | jq -e '.valid == true'
- done
+ validate
;;
register)
- [[ -z $(git status --porcelain) ]] || {
- echo 'Commit local changes before registration' >&2; exit 1;
- }
+ register
+ ;;
+ deploy)
+ scratch=$(mktemp -d)
+ trap 'rm -rf "$scratch"' EXIT
revision=$(git rev-parse HEAD)
- published=$(gh api "repos/PrefectHQ/fastmcp/commits/$revision" --jq .sha)
- [[ $published == "$revision" ]] || {
- echo 'Source commit must be available in PrefectHQ/fastmcp' >&2; exit 1;
+ jq -n '{limit:200}' > "$scratch/filter.json"
+ api POST /automations/filter --data "@$scratch/filter.json" > "$scratch/automations.json"
+ # An interrupted deploy can leave one automation on the target already; treat
+ # the other commit as live so a retry finishes the repin.
+ current=$(live_sources "$scratch/automations.json" | jq -er --arg revision "$revision" '
+ map(select(. != $revision))
+ | if length == 0 then $revision elif length == 1 then .[0]
+ else error("Triage automations are pinned to several commits; repin them by hand") end')
+ if [[ $current == "$revision" ]]; then
+ echo "Already deployed $revision"; exit 0
+ fi
+ git cat-file -e "$current^{commit}" 2>/dev/null || {
+ echo "Live source $current is not in this checkout; fetch full history" >&2; exit 1;
+ }
+ git diff --quiet "$current" "$revision" -- 'workflows/*.plan.json' workflows/stack-models.json || {
+ echo "Execution plans changed since $current; publish them and update the automations' plan versions by hand" >&2; exit 1;
+ }
+ validate
+ jq -n --argjson names "$(json_list "${DEPLOYMENTS[@]}")" \
+ '{deployments:{name:{any_:$names}},flow_runs:{state:{type:{any_:["SCHEDULED","PENDING","RUNNING","PAUSED","CANCELLING"]}}},limit:10}' \
+ > "$scratch/active.json"
+ api POST /flow_runs/filter --data "@$scratch/active.json" > "$scratch/runs.json"
+ jq -e 'length == 0' "$scratch/runs.json" >/dev/null || {
+ echo 'Triage runs are in flight; deploy once they finish:' >&2
+ jq -r '.[] | " \(.name) \(.state_type)"' "$scratch/runs.json" >&2
+ exit 1
+ }
+ register
+ # Flows refuse to run while their deployment version differs from the pinned
+ # source_sha, so repin both automations immediately after registration.
+ for name in "${TRIGGERS[@]}"; do
+ jq --arg name "$name" --arg sha "$revision" '
+ .[] | select(.name == $name)
+ | .actions[0].parameters.runtime.source_sha = $sha
+ | with_entries(select(.key | IN("name","description","enabled","tags","trigger","actions","actions_on_trigger","actions_on_resolve")))' \
+ "$scratch/automations.json" > "$scratch/$name.json"
+ id=$(jq -er --arg name "$name" '.[] | select(.name == $name) | .id' "$scratch/automations.json")
+ api PUT "/automations/$id" --data "@$scratch/$name.json" >/dev/null
+ done
+ api POST /automations/filter --data "@$scratch/filter.json" > "$scratch/automations.json"
+ [[ $(live_sources "$scratch/automations.json") == "[\"$revision\"]" ]] || {
+ echo 'Automations did not take the new source_sha; roll back by deploying the previous commit' >&2; exit 1;
}
- FASTMCP_AGENTIC_REF=$revision uv run --locked prefect --no-prompt deploy \
- --prefect-file prefect.yaml \
- --name cloud-stack-batch --name cloud-contribution-verification --name cloud-triage-intake \
- --name cloud-triage-verify --name cloud-triage-finalize
+ echo "Deployed $revision (was $current)"
;;
intake)
config=${2:?intake configuration JSON file required}
@@ -152,7 +225,7 @@ case "${1:-}" in
echo "automation alert $(upsert automations "$scratch/automation-alert.json")"
;;
*)
- echo 'Usage: agentic_stack.sh validate | register | intake CONFIG RUNTIME CONTROLLER DEPLOYMENT OPERATION | run BATCH RUNTIME DEPLOYMENT | results RUN | events RUNTIME CONTROLLER DEPLOYMENT [NOTIFY_BLOCK] | alerts RUNTIME NOTIFY_BLOCK' >&2
+ echo 'Usage: agentic_stack.sh validate | register | deploy | intake CONFIG RUNTIME CONTROLLER DEPLOYMENT OPERATION | run BATCH RUNTIME DEPLOYMENT | results RUN | events RUNTIME CONTROLLER DEPLOYMENT [NOTIFY_BLOCK] | alerts RUNTIME NOTIFY_BLOCK' >&2
exit 2
;;
esac
diff --git a/.github/agentic/tests/test_agentic_deploy.py b/.github/agentic/tests/test_agentic_deploy.py
new file mode 100644
index 000000000..54defa7b8
--- /dev/null
+++ b/.github/agentic/tests/test_agentic_deploy.py
@@ -0,0 +1,229 @@
+import json
+import os
+import shutil
+import subprocess
+import sys
+from pathlib import Path
+
+import pytest
+from prefect.server.events.schemas.automations import AutomationCore
+
+ROOT = Path(__file__).resolve().parents[1]
+TRIGGERS = ("fastmcp-triage-on-github-issue", "fastmcp-triage-on-github-pr")
+
+FAKE_UV = """\
+import json, os, pathlib, sys
+state = pathlib.Path(os.environ["STATE"])
+args = sys.argv[sys.argv.index("--no-prompt") + 1 :]
+automations = json.loads((state / "automations.json").read_text())
+if args[0] == "deploy":
+ (state / "deploy.json").write_text(
+ json.dumps({"ref": os.environ["FASTMCP_AGENTIC_REF"], "args": args[1:]})
+ )
+ sys.exit(0)
+method, path = args[1], args[2]
+body = json.loads(pathlib.Path(args[4][1:]).read_text()) if len(args) > 4 else None
+if path == "/execution-plans/validate":
+ print(json.dumps({"valid": True}))
+elif path == "/automations/filter":
+ print(json.dumps(automations))
+elif path == "/flow_runs/filter":
+ (state / "active-filter.json").write_text(json.dumps(body))
+ print(os.environ.get("ACTIVE_RUNS", "[]"))
+elif method == "PUT" and path.startswith("/automations/"):
+ automation_id = path.rsplit("/", 1)[1]
+ (state / f"put-{automation_id}.json").write_text(json.dumps(body))
+ for automation in automations:
+ if automation["id"] == automation_id:
+ automation.update(body)
+ (state / "automations.json").write_text(json.dumps(automations))
+else:
+ sys.exit(f"unexpected call {args}")
+"""
+
+FAKE_GH = """\
+import sys
+print(sys.argv[sys.argv.index("api") + 1].rsplit("/", 1)[1])
+"""
+
+
+def git(repo: Path, *args: str) -> str:
+ return subprocess.run(
+ ["git", "-C", str(repo), *args], check=True, capture_output=True, text=True
+ ).stdout.strip()
+
+
+def automation(name: str, number: int, source_sha: str) -> dict:
+ return {
+ "id": f"{number}1111111-1111-4111-8111-111111111111",
+ "created": "2026-11-01T00:00:00Z",
+ "workspace": "w",
+ "name": name,
+ "description": "",
+ "enabled": True,
+ "trigger": {
+ "type": "event",
+ "posture": "Reactive",
+ "threshold": 1,
+ "within": 60,
+ "expect": ["fastmcp.github.issues.opened"],
+ "match": {},
+ "match_related": {},
+ "after": [],
+ "for_each": ["prefect.resource.id"],
+ },
+ "actions": [
+ {
+ "type": "run-deployment",
+ "source": "selected",
+ "deployment_id": "22222222-2222-4222-8222-222222222222",
+ "job_variables": None,
+ "parameters": {
+ "controller_id": "c",
+ "runtime": {"source_sha": source_sha, "plans": {}},
+ },
+ }
+ ],
+ "actions_on_trigger": [],
+ "actions_on_resolve": [],
+ }
+
+
+def stack(tmp_path):
+ repo = tmp_path / "repo"
+ agentic = repo / ".github/agentic"
+ agentic.mkdir(parents=True)
+ # The checkout under deploy has no copy of the driver, like a rollback target
+ # that predates it.
+ shutil.copytree(ROOT / "workflows", agentic / "workflows")
+ shutil.copy(ROOT / "prefect.yaml", agentic)
+ git(repo, "init", "-q")
+ git(repo, "-c", "user.name=t", "-c", "user.email=t@t", "add", ".")
+ git(repo, "-c", "user.name=t", "-c", "user.email=t@t", "commit", "-qm", "live")
+ live = git(repo, "rev-parse", "HEAD")
+
+ state = tmp_path / "state"
+ state.mkdir()
+ automations = [automation(name, i, live) for i, name in enumerate(TRIGGERS, 1)]
+ (state / "automations.json").write_text(json.dumps(automations))
+ bin_dir = tmp_path / "bin"
+ bin_dir.mkdir()
+ for name, source in (("uv", FAKE_UV), ("gh", FAKE_GH)):
+ executable = bin_dir / name
+ executable.write_text(f"#!{sys.executable}\n{source}")
+ executable.chmod(0o755)
+
+ def commit(path: str, content: str) -> str:
+ (agentic / path).write_text(content)
+ git(repo, "-c", "user.name=t", "-c", "user.email=t@t", "add", ".")
+ git(repo, "-c", "user.name=t", "-c", "user.email=t@t", "commit", "-qm", path)
+ return git(repo, "rev-parse", "HEAD")
+
+ def pin(name: str, source_sha: str) -> None:
+ current = json.loads((state / "automations.json").read_text())
+ for item in current:
+ if item["name"] == name:
+ item["actions"][0]["parameters"]["runtime"]["source_sha"] = source_sha
+ (state / "automations.json").write_text(json.dumps(current))
+
+ def deploy(active_runs: list | None = None) -> subprocess.CompletedProcess:
+ return subprocess.run(
+ ["bash", str(ROOT / "scripts/agentic_stack.sh"), "deploy"],
+ env={
+ **os.environ,
+ "FASTMCP_AGENTIC_DIR": str(agentic),
+ "PATH": str(bin_dir) + os.pathsep + os.environ["PATH"],
+ "STATE": str(state),
+ "ACTIVE_RUNS": json.dumps(active_runs or []),
+ "PREFECT_API_URL": "https://api.prefect.cloud/api/accounts/a/workspaces/w",
+ },
+ capture_output=True,
+ text=True,
+ )
+
+ deploy.pin = pin
+ return commit, deploy, state, live
+
+
+def test_deploy_registers_and_repins_both_automations(stack):
+ commit, deploy, state, live = stack
+ revision = commit("notes.txt", "code change\n")
+
+ result = deploy()
+
+ assert result.returncode == 0, result.stderr
+ assert f"Deployed {revision} (was {live})" in result.stdout
+ registered = json.loads((state / "deploy.json").read_text())
+ assert registered["ref"] == revision
+ names = registered["args"][registered["args"].index("prefect.yaml") + 1 :][1::2]
+ assert set(names) == set(
+ json.loads((state / "active-filter.json").read_text())["deployments"]["name"][
+ "any_"
+ ]
+ )
+ puts = sorted(state.glob("put-*.json"))
+ assert len(puts) == 2
+ for put in puts:
+ body = json.loads(put.read_text())
+ assert body["actions"][0]["parameters"]["runtime"]["source_sha"] == revision
+ assert "id" not in body and "workspace" not in body
+ AutomationCore.model_validate(body)
+
+
+def test_deploy_retry_finishes_a_partial_repin(stack):
+ commit, deploy, state, live = stack
+ revision = commit("notes.txt", "code change\n")
+ deploy.pin(TRIGGERS[0], revision)
+
+ result = deploy()
+
+ assert result.returncode == 0, result.stderr
+ assert f"Deployed {revision} (was {live})" in result.stdout
+ assert len(list(state.glob("put-*.json"))) == 2
+
+
+def test_deploy_refuses_automations_on_several_other_commits(stack):
+ commit, deploy, state, _ = stack
+ other = commit("other.txt", "x\n")
+ commit("notes.txt", "code change\n")
+ deploy.pin(TRIGGERS[0], other)
+
+ result = deploy()
+
+ assert result.returncode != 0
+ assert "pinned to several commits" in result.stderr
+ assert not (state / "deploy.json").exists()
+
+
+def test_deploy_is_a_no_op_for_the_live_commit(stack):
+ _, deploy, state, live = stack
+
+ result = deploy()
+
+ assert result.returncode == 0, result.stderr
+ assert f"Already deployed {live}" in result.stdout
+ assert not (state / "deploy.json").exists()
+
+
+def test_deploy_waits_for_in_flight_runs(stack):
+ commit, deploy, state, _ = stack
+ commit("notes.txt", "code change\n")
+
+ result = deploy([{"name": "fastmcp-stack-intake-1", "state_type": "RUNNING"}])
+
+ assert result.returncode == 1
+ assert "fastmcp-stack-intake-1 RUNNING" in result.stderr
+ assert not (state / "deploy.json").exists()
+ assert not list(state.glob("put-*.json"))
+
+
+def test_deploy_refuses_changed_execution_plans(stack):
+ commit, deploy, state, _ = stack
+ commit("workflows/fastmcp-stack-assess.plan.json", "{}\n")
+
+ result = deploy()
+
+ assert result.returncode == 1
+ assert "Execution plans changed" in result.stderr
+ assert not (state / "deploy.json").exists()
diff --git a/.github/workflows/deploy-agentic.yml b/.github/workflows/deploy-agentic.yml
new file mode 100644
index 000000000..d82606eb8
--- /dev/null
+++ b/.github/workflows/deploy-agentic.yml
@@ -0,0 +1,71 @@
+name: Deploy agentic triage
+
+# Registers the triage deployments in Prefect Cloud at a commit on main and
+# repins both triage automations to it. Dispatching an older commit rolls back.
+
+on:
+ workflow_dispatch:
+ inputs:
+ ref:
+ description: "Commit on main to deploy (blank for main's tip)"
+ required: false
+ type: string
+
+permissions:
+ contents: read
+
+concurrency:
+ group: deploy-agentic
+ cancel-in-progress: false
+
+jobs:
+ deploy:
+ if: github.ref == 'refs/heads/main'
+ runs-on: ubuntu-latest
+ timeout-minutes: 20
+ defaults:
+ run:
+ working-directory: source/.github/agentic
+ env:
+ # prefect-technologies/marvin-bot
+ PREFECT_WORKSPACE_ID: cafa2ffa-f6cc-4ed6-ab76-eaa4ba1ad40e
+ PREFECT_WORKSPACE_HANDLE: prefect-technologies/marvin-bot
+ steps:
+ # The driver always comes from main's tip so older targets can be deployed
+ # (rollback) even when they predate it.
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+ with:
+ path: driver
+ persist-credentials: false
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+ with:
+ path: source
+ ref: ${{ inputs.ref || github.sha }}
+ fetch-depth: 0
+ persist-credentials: false
+ - name: Require a commit on main
+ run: git merge-base --is-ancestor HEAD origin/main
+ - uses: astral-sh/setup-uv@07a047e608db84b3d8c06cd9a7c7dcc9c355c18e # v10.3.2
+ with:
+ python-version: "3.12"
+ enable-cache: true
+ cache-dependency-glob: "source/.github/agentic/uv.lock"
+ - run: uv sync --locked
+ - name: Resolve workspace API URL
+ env:
+ PREFECT_API_KEY: ${{ secrets.PREFECT_API_KEY }}
+ run: |
+ url=$(curl -fsS -H "Authorization: Bearer $PREFECT_API_KEY" \
+ https://api.prefect.cloud/api/me/workspaces | jq -er \
+ --arg id "$PREFECT_WORKSPACE_ID" --arg handle "$PREFECT_WORKSPACE_HANDLE" '
+ [.[] | select(.workspace_id == $id)]
+ | if length != 1 then error("Workspace not visible to PREFECT_API_KEY")
+ elif .[0].account_handle + "/" + .[0].workspace_handle != $handle
+ then error("Workspace handle mismatch")
+ else "https://api.prefect.cloud/api/accounts/\(.[0].account_id)/workspaces/\(.[0].workspace_id)" end')
+ echo "PREFECT_API_URL=$url" >> "$GITHUB_ENV"
+ - name: Deploy
+ env:
+ PREFECT_API_KEY: ${{ secrets.PREFECT_API_KEY }}
+ GH_TOKEN: ${{ github.token }}
+ run: FASTMCP_AGENTIC_DIR="$PWD" bash "$GITHUB_WORKSPACE/driver/.github/agentic/scripts/agentic_stack.sh" deploy