Skip to content

An agentic skills framework & software development methodology that works.

tests: remove skill-triggering bash prompts (covered by drill triggering-* scenarios)

Jesse Vincent8611a4ea978f
diff --git a/tests/skill-triggering/prompts/dispatching-parallel-agents.txt b/tests/skill-triggering/prompts/dispatching-parallel-agents.txt
deleted file mode 100644
index fb5423f29..000000000
--- a/tests/skill-triggering/prompts/dispatching-parallel-agents.txt
+++ /dev/null
@@ -1,8 +0,0 @@
-I have 4 independent test failures happening in different modules:
-
-1. tests/auth/login.test.ts - "should redirect after login" is failing
-2. tests/api/users.test.ts - "should return user list" returns 500
-3. tests/components/Button.test.tsx - snapshot mismatch
-4. tests/utils/date.test.ts - timezone handling broken
-
-These are unrelated issues in different parts of the codebase. Can you investigate all of them?
\ No newline at end of file
diff --git a/tests/skill-triggering/prompts/executing-plans.txt b/tests/skill-triggering/prompts/executing-plans.txt
deleted file mode 100644
index 86ed26257..000000000
--- a/tests/skill-triggering/prompts/executing-plans.txt
+++ /dev/null
@@ -1 +0,0 @@
-I have a plan document at docs/superpowers/plans/2024-01-15-auth-system.md that needs to be executed. Please implement it.
\ No newline at end of file
diff --git a/tests/skill-triggering/prompts/requesting-code-review.txt b/tests/skill-triggering/prompts/requesting-code-review.txt
deleted file mode 100644
index f1be2672a..000000000
--- a/tests/skill-triggering/prompts/requesting-code-review.txt
+++ /dev/null
@@ -1,3 +0,0 @@
-I just finished implementing the user authentication feature. All the code is committed. Can you review the changes before I merge to main?
-
-The commits are between abc123 and def456.
\ No newline at end of file
diff --git a/tests/skill-triggering/prompts/systematic-debugging.txt b/tests/skill-triggering/prompts/systematic-debugging.txt
deleted file mode 100644
index d3806b9c2..000000000
--- a/tests/skill-triggering/prompts/systematic-debugging.txt
+++ /dev/null
@@ -1,11 +0,0 @@
-The tests are failing with this error:
-
-```
-FAIL src/utils/parser.test.ts
- ● Parser › should handle nested objects
- TypeError: Cannot read property 'value' of undefined
- at parse (src/utils/parser.ts:42:18)
- at Object.<anonymous> (src/utils/parser.test.ts:28:20)
-```
-
-Can you figure out what's going wrong and fix it?
\ No newline at end of file
diff --git a/tests/skill-triggering/prompts/test-driven-development.txt b/tests/skill-triggering/prompts/test-driven-development.txt
deleted file mode 100644
index f386eeab0..000000000
--- a/tests/skill-triggering/prompts/test-driven-development.txt
+++ /dev/null
@@ -1,7 +0,0 @@
-I need to add a new feature to validate email addresses. It should:
-- Check that there's an @ symbol
-- Check that there's at least one character before the @
-- Check that there's a dot in the domain part
-- Return true/false
-
-Can you implement this?
\ No newline at end of file
diff --git a/tests/skill-triggering/prompts/writing-plans.txt b/tests/skill-triggering/prompts/writing-plans.txt
deleted file mode 100644
index 74803133a..000000000
--- a/tests/skill-triggering/prompts/writing-plans.txt
+++ /dev/null
@@ -1,10 +0,0 @@
-Here's the spec for our new authentication system:
-
-Requirements:
-- Users can register with email/password
-- Users can log in and receive a JWT token
-- Protected routes require valid JWT
-- Tokens expire after 24 hours
-- Support password reset via email
-
-We need to implement this. There are multiple steps involved - user model, auth routes, middleware, email service integration.
\ No newline at end of file
diff --git a/tests/skill-triggering/run-all.sh b/tests/skill-triggering/run-all.sh
deleted file mode 100755
index 1a35dd932..000000000
--- a/tests/skill-triggering/run-all.sh
+++ /dev/null
@@ -1,60 +0,0 @@
-#!/usr/bin/env bash
-# Run all skill triggering tests
-# Usage: ./run-all.sh
-
-set -e
-
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-PROMPTS_DIR="$SCRIPT_DIR/prompts"
-
-SKILLS=(
- "systematic-debugging"
- "test-driven-development"
- "writing-plans"
- "dispatching-parallel-agents"
- "executing-plans"
- "requesting-code-review"
-)
-
-echo "=== Running Skill Triggering Tests ==="
-echo ""
-
-PASSED=0
-FAILED=0
-RESULTS=()
-
-for skill in "${SKILLS[@]}"; do
- prompt_file="$PROMPTS_DIR/${skill}.txt"
-
- if [ ! -f "$prompt_file" ]; then
- echo "⚠️ SKIP: No prompt file for $skill"
- continue
- fi
-
- echo "Testing: $skill"
-
- if "$SCRIPT_DIR/run-test.sh" "$skill" "$prompt_file" 3 2>&1 | tee /tmp/skill-test-$skill.log; then
- PASSED=$((PASSED + 1))
- RESULTS+=("✅ $skill")
- else
- FAILED=$((FAILED + 1))
- RESULTS+=("❌ $skill")
- fi
-
- echo ""
- echo "---"
- echo ""
-done
-
-echo ""
-echo "=== Summary ==="
-for result in "${RESULTS[@]}"; do
- echo " $result"
-done
-echo ""
-echo "Passed: $PASSED"
-echo "Failed: $FAILED"
-
-if [ $FAILED -gt 0 ]; then
- exit 1
-fi
diff --git a/tests/skill-triggering/run-test.sh b/tests/skill-triggering/run-test.sh
deleted file mode 100755
index ba9199583..000000000
--- a/tests/skill-triggering/run-test.sh
+++ /dev/null
@@ -1,88 +0,0 @@
-#!/usr/bin/env bash
-# Test skill triggering with naive prompts
-# Usage: ./run-test.sh <skill-name> <prompt-file>
-#
-# Tests whether Claude triggers a skill based on a natural prompt
-# (without explicitly mentioning the skill)
-
-set -e
-
-SKILL_NAME="$1"
-PROMPT_FILE="$2"
-MAX_TURNS="${3:-3}"
-
-if [ -z "$SKILL_NAME" ] || [ -z "$PROMPT_FILE" ]; then
- echo "Usage: $0 <skill-name> <prompt-file> [max-turns]"
- echo "Example: $0 systematic-debugging ./test-prompts/debugging.txt"
- exit 1
-fi
-
-# Get the directory where this script lives (should be tests/skill-triggering)
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-# Get the superpowers plugin root (two levels up from tests/skill-triggering)
-PLUGIN_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
-
-TIMESTAMP=$(date +%s)
-OUTPUT_DIR="/tmp/superpowers-tests/${TIMESTAMP}/skill-triggering/${SKILL_NAME}"
-mkdir -p "$OUTPUT_DIR"
-
-# Read prompt from file
-PROMPT=$(cat "$PROMPT_FILE")
-
-echo "=== Skill Triggering Test ==="
-echo "Skill: $SKILL_NAME"
-echo "Prompt file: $PROMPT_FILE"
-echo "Max turns: $MAX_TURNS"
-echo "Output dir: $OUTPUT_DIR"
-echo ""
-
-# Copy prompt for reference
-cp "$PROMPT_FILE" "$OUTPUT_DIR/prompt.txt"
-
-# Run Claude
-LOG_FILE="$OUTPUT_DIR/claude-output.json"
-cd "$OUTPUT_DIR"
-
-echo "Plugin dir: $PLUGIN_DIR"
-echo "Running claude -p with naive prompt..."
-timeout 300 claude -p "$PROMPT" \
- --plugin-dir "$PLUGIN_DIR" \
- --dangerously-skip-permissions \
- --max-turns "$MAX_TURNS" \
- --output-format stream-json \
- > "$LOG_FILE" 2>&1 || true
-
-echo ""
-echo "=== Results ==="
-
-# Check if skill was triggered (look for Skill tool invocation)
-# In stream-json, tool invocations have "name":"Skill" (not "tool":"Skill")
-# Match either "skill":"skillname" or "skill":"namespace:skillname"
-SKILL_PATTERN='"skill":"([^"]*:)?'"${SKILL_NAME}"'"'
-if grep -q '"name":"Skill"' "$LOG_FILE" && grep -qE "$SKILL_PATTERN" "$LOG_FILE"; then
- echo "✅ PASS: Skill '$SKILL_NAME' was triggered"
- TRIGGERED=true
-else
- echo "❌ FAIL: Skill '$SKILL_NAME' was NOT triggered"
- TRIGGERED=false
-fi
-
-# Show what skills WERE triggered
-echo ""
-echo "Skills triggered in this run:"
-grep -o '"skill":"[^"]*"' "$LOG_FILE" 2>/dev/null | sort -u || echo " (none)"
-
-# Show first assistant message
-echo ""
-echo "First assistant response (truncated):"
-grep '"type":"assistant"' "$LOG_FILE" | head -1 | jq -r '.message.content[0].text // .message.content' 2>/dev/null | head -c 500 || echo " (could not extract)"
-
-echo ""
-echo "Full log: $LOG_FILE"
-echo "Timestamp: $TIMESTAMP"
-
-if [ "$TRIGGERED" = "true" ]; then
- exit 0
-else
- exit 1
-fi