Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
TypeScript agentic-aiagentsaiai-agentsaiagentsdeveloper-toolsfunction-callinggpt-4javascriptjsllmllmopsmcppythonremote-mcp-serverssetypescript
Add full-cli-test agent skill with Slack integration tests (#3081)
Rahul Tarakd268e4ab5852
diff --git a/.claude/skills/cli-test-with-bundling/SKILL.md b/.claude/skills/cli-test-with-bundling/SKILL.mdindex 9faf1bb9a..68c9a49f3 100644--- a/.claude/skills/cli-test-with-bundling/SKILL.md+++ b/.claude/skills/cli-test-with-bundling/SKILL.md@@ -13,21 +13,28 @@ Trigger the `Build CLI Binaries` workflow on GitHub Actions, wait for it to comp## Step 1: Determine the version-Read the version from `ts/packages/cli/package.json`:+Read the version from `ts/packages/cli/package.json` and append a beta prerelease suffix so the build is always treated as a beta release. This prevents accidental production releases and marks the GitHub release as a prerelease.```bash-jq -r .version ts/packages/cli/package.json+BASE_VERSION=$(jq -r .version ts/packages/cli/package.json)+BETA_VERSION="${BASE_VERSION}-beta.$(date +%Y%m%d%H%M%S)"```+For example, if `package.json` has `1.2.3`, the version becomes `1.2.3-beta.20260331143022`.+## Step 2: Trigger the workflow+**Always use the beta version** — never pass the bare version from `package.json`:+```bashgh workflow run build-cli-binaries.yml \--repo ComposioHQ/composio \--ref "$(git rev-parse --abbrev-ref HEAD)" \- --field version="<version-from-step-1>"+ --field version="${BETA_VERSION}"```+This ensures the CI workflow creates a **prerelease** GitHub release (the workflow auto-detects `beta` in the version string).+## Step 3: Find and monitor the runAfter triggering, wait a few seconds then find the run:@@ -71,10 +78,10 @@ esacARTIFACT="composio-${OS}-${ARCH}"```-Then download from the release. The workflow creates a GitHub release tagged `@composio/cli@<version>`:+Then download from the release. The workflow creates a GitHub release tagged `@composio/cli@<beta-version>`:```bash-VERSION="<version-from-step-1>"+VERSION="${BETA_VERSION}" # the beta version from Step 1TAG="@composio/cli@${VERSION}"ENCODED_TAG=$(python3 -c "import urllib.parse; print(urllib.parse.quote('${TAG}', safe=''))")diff --git a/.claude/skills/full-cli-test/SKILL.md b/.claude/skills/full-cli-test/SKILL.mdnew file mode 100644index 000000000..be65b0295--- /dev/null+++ b/.claude/skills/full-cli-test/SKILL.md@@ -0,0 +1,171 @@+---+description: Full CLI test pipeline — monitor CI for types/lint, then run local binary test, then run bundled binary test via CI.+---++# Full CLI Test++End-to-end CLI validation pipeline that ensures types/lint pass in CI, then tests the CLI binary built from source, and finally tests the CI-bundled binary.++## Overview++This skill runs three phases sequentially:++1. **Phase 1: CI Lint/Type Check** — Poll CI until the CLI type-check and lint jobs pass.+2. **Phase 2: Local Binary Test** — Build and test the CLI binary from source (`/cli-test`).+3. **Phase 3: Bundled Binary Test** — Trigger CI binary build, download, and test (`/cli-test-with-bundling`).++## Phase 1: Monitor CI for Types/Lint++Use `/loop 5m` to poll the CI status for the current branch every 5 minutes. Check that **all type-check and lint jobs pass** for the CLI package before proceeding.++### How to check++```bash+# Get the latest CI run for the current branch+gh run list \+ --repo ComposioHQ/composio \+ --branch "$(git rev-parse --abbrev-ref HEAD)" \+ --limit 5 \+ --json databaseId,status,conclusion,name,headBranch+```++Look for workflow runs that cover type-checking and linting (e.g., the main CI workflow). If none are running or completed yet, wait for one to appear.++### What to verify++- All type-check jobs: **pass**+- All lint jobs: **pass**+- If any fail, report the failure to the user and stop — do NOT proceed to Phase 2.++Use `/loop 5m` to poll: run the `gh run list` / `gh run view` command every 5 minutes until the relevant jobs succeed. Once all type/lint checks are green, move on.++## Phase 2: Local Binary Test (`/cli-test`)++Once CI lint/types pass, run the `/cli-test` skill:++1. Install dependencies: `pnpm install`+2. Build all packages: `pnpm turbo build`+3. Build the standalone binary: `pnpm --dir ts/packages/cli build:binary`+4. Test the binary:+ ```bash+ ./ts/packages/cli/dist/composio version+ ./ts/packages/cli/dist/composio whoami+ ./ts/packages/cli/dist/composio --help+ ```+5. Run the Slack integration test (see [Slack Integration Test](#slack-integration-test) below):+ ```bash+ ./ts/packages/cli/dist/composio run '<SLACK_TEST_SCRIPT>'+ ```++If any command fails, report to the user and stop — do NOT proceed to Phase 3.++## Phase 3: Bundled Binary Test (`/cli-test-with-bundling`)++Once the local binary test passes, run the `/cli-test-with-bundling` skill:++1. Read the version from `ts/packages/cli/package.json` and append a `-beta.<timestamp>` suffix (e.g. `1.2.3-beta.20260331143022`) — **always trigger as a beta release**+2. Trigger the `build-cli-binaries.yml` workflow via `gh workflow run` with the beta version+3. Monitor the workflow run (use `/loop 5m` to poll)+4. Download the platform-appropriate binary artifact+5. Test the binary:+ ```bash+ $BINARY version+ $BINARY whoami+ $BINARY --help+ $BINARY run 'console.log("hello from composio run")'+ $BINARY run 'const result = await experimental_subAgent({ goal: "What is 2+2?", toolNames: [] }); console.log(result)'+ ```+6. Run the Slack integration test (see [Slack Integration Test](#slack-integration-test) below):+ ```bash+ $BINARY run '<SLACK_TEST_SCRIPT>'+ ```+7. If on a PR, post results as a PR comment++## Slack Integration Test++This test validates `execute()`, `experimental_subAgent()`, and end-to-end Slack connectivity. It must be run in **both Phase 2 and Phase 3** to ensure it works with both the locally-built and CI-bundled binaries.++### Channel++**`#buzz-skill-based-cli-testing`** — dedicated Slack channel for automated CLI test runs.++### Test Script++Use `composio run` (or `$BINARY run` in Phase 3) with the following script. Replace `$BINARY` with the appropriate binary path for the phase.++```bash+$BINARY run '+ // Step 1: Find the #buzz-skill-based-cli-testing channel+ const channels = await execute("SLACK_LIST_CHANNELS", {+ types: "public_channel",+ limit: 200,+ });+ const channel = channels.data?.channels?.find(+ (c) => c.name === "buzz-skill-based-cli-testing"+ );+ if (!channel) throw new Error("Channel #buzz-skill-based-cli-testing not found");++ // Step 2: Send an initial message tagging @cryogenicplanet+ const buildType = "local"; // use "bundled" in Phase 3+ await execute("SLACK_SEND_A_MESSAGE_TO_A_SLACK_CHANNEL", {+ channel: channel.id,+ text: `<@cryogenicplanet> CLI test run started (${buildType} build) at ${new Date().toISOString()}`,+ });++ // Step 3: Fetch recent channel history for the subAgent to summarize+ const history = await execute("SLACK_GET_CHANNEL_HISTORY", {+ channel: channel.id,+ limit: 20,+ });++ // Step 4: Use experimental_subAgent to summarize what happened in the channel+ const summary = await experimental_subAgent(+ `Summarize the recent activity in this Slack channel in 2-3 sentences. Focus on what tests were run and their outcomes.\n\n${history.prompt()}`,+ {+ schema: z.object({+ summary: z.string(),+ messageCount: z.number(),+ }),+ }+ );++ // Step 5: Post the summary back to the channel+ await execute("SLACK_SEND_A_MESSAGE_TO_A_SLACK_CHANNEL", {+ channel: channel.id,+ text: `CLI Test Summary (${buildType} build):\n${summary.structuredOutput.summary}\n(${summary.structuredOutput.messageCount} messages analyzed)`,+ });++ console.log("Slack integration test passed:", summary.structuredOutput);+'+```++### What this validates++| Capability | How it's tested |+|---|---|+| `execute()` with Slack tools | List channels, send messages, fetch history |+| `experimental_subAgent()` | Summarizes channel history with structured output via `z` schema |+| `z` (Zod) global | Used in the subAgent schema definition |+| `result.prompt()` | Feeds channel history into the subAgent |+| End-to-end Slack connectivity | Reads from and writes to a real Slack channel |++### Important Notes++- Change `buildType` to `"bundled"` when running in Phase 3.+- The Slack user tag `<@cryogenicplanet>` should resolve to the correct user in your workspace. If the mention doesn't resolve, find the Slack user ID first and use `<@U_XXXXX>` format.+- This test requires an active Slack connection in Composio. If not connected, run `composio connect slack` first.++## Failure Handling++- **Phase 1 failure (lint/types):** Report which jobs failed with links. Do not continue.+- **Phase 2 failure (local build/test):** Report the error output. Do not continue.+- **Phase 3 failure (bundled build/test):** Report which commands failed. Post results to PR if applicable.++## Reference Files++| File | Purpose |+|---|---|+| `.github/workflows/build-cli-binaries.yml` | CI workflow that builds binaries |+| `ts/packages/cli/package.json` | Source of CLI version |+| `ts/packages/cli/scripts/build-binary.ts` | Local binary build script |+| `ts/packages/cli/dist/composio` | Built binary output |