An agentic skills framework & software development methodology that works.
tests: remove skill-triggering bash prompts (covered by drill triggering-* scenarios)
Jesse Vincent8611a4ea978f
diff --git a/tests/skill-triggering/prompts/dispatching-parallel-agents.txt b/tests/skill-triggering/prompts/dispatching-parallel-agents.txtdeleted file mode 100644index fb5423f29..000000000--- a/tests/skill-triggering/prompts/dispatching-parallel-agents.txt+++ /dev/null@@ -1,8 +0,0 @@-I have 4 independent test failures happening in different modules:--1. tests/auth/login.test.ts - "should redirect after login" is failing-2. tests/api/users.test.ts - "should return user list" returns 500-3. tests/components/Button.test.tsx - snapshot mismatch-4. tests/utils/date.test.ts - timezone handling broken--These are unrelated issues in different parts of the codebase. Can you investigate all of them?\ No newline at end of filediff --git a/tests/skill-triggering/prompts/executing-plans.txt b/tests/skill-triggering/prompts/executing-plans.txtdeleted file mode 100644index 86ed26257..000000000--- a/tests/skill-triggering/prompts/executing-plans.txt+++ /dev/null@@ -1 +0,0 @@-I have a plan document at docs/superpowers/plans/2024-01-15-auth-system.md that needs to be executed. Please implement it.\ No newline at end of filediff --git a/tests/skill-triggering/prompts/requesting-code-review.txt b/tests/skill-triggering/prompts/requesting-code-review.txtdeleted file mode 100644index f1be2672a..000000000--- a/tests/skill-triggering/prompts/requesting-code-review.txt+++ /dev/null@@ -1,3 +0,0 @@-I just finished implementing the user authentication feature. All the code is committed. Can you review the changes before I merge to main?--The commits are between abc123 and def456.\ No newline at end of filediff --git a/tests/skill-triggering/prompts/systematic-debugging.txt b/tests/skill-triggering/prompts/systematic-debugging.txtdeleted file mode 100644index d3806b9c2..000000000--- a/tests/skill-triggering/prompts/systematic-debugging.txt+++ /dev/null@@ -1,11 +0,0 @@-The tests are failing with this error:--```-FAIL src/utils/parser.test.ts- ● Parser › should handle nested objects- TypeError: Cannot read property 'value' of undefined- at parse (src/utils/parser.ts:42:18)- at Object.<anonymous> (src/utils/parser.test.ts:28:20)-```--Can you figure out what's going wrong and fix it?\ No newline at end of filediff --git a/tests/skill-triggering/prompts/test-driven-development.txt b/tests/skill-triggering/prompts/test-driven-development.txtdeleted file mode 100644index f386eeab0..000000000--- a/tests/skill-triggering/prompts/test-driven-development.txt+++ /dev/null@@ -1,7 +0,0 @@-I need to add a new feature to validate email addresses. It should:-- Check that there's an @ symbol-- Check that there's at least one character before the @-- Check that there's a dot in the domain part-- Return true/false--Can you implement this?\ No newline at end of filediff --git a/tests/skill-triggering/prompts/writing-plans.txt b/tests/skill-triggering/prompts/writing-plans.txtdeleted file mode 100644index 74803133a..000000000--- a/tests/skill-triggering/prompts/writing-plans.txt+++ /dev/null@@ -1,10 +0,0 @@-Here's the spec for our new authentication system:--Requirements:-- Users can register with email/password-- Users can log in and receive a JWT token-- Protected routes require valid JWT-- Tokens expire after 24 hours-- Support password reset via email--We need to implement this. There are multiple steps involved - user model, auth routes, middleware, email service integration.\ No newline at end of filediff --git a/tests/skill-triggering/run-all.sh b/tests/skill-triggering/run-all.shdeleted file mode 100755index 1a35dd932..000000000--- a/tests/skill-triggering/run-all.sh+++ /dev/null@@ -1,60 +0,0 @@-#!/usr/bin/env bash-# Run all skill triggering tests-# Usage: ./run-all.sh--set -e--SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"-PROMPTS_DIR="$SCRIPT_DIR/prompts"--SKILLS=(- "systematic-debugging"- "test-driven-development"- "writing-plans"- "dispatching-parallel-agents"- "executing-plans"- "requesting-code-review"-)--echo "=== Running Skill Triggering Tests ==="-echo ""--PASSED=0-FAILED=0-RESULTS=()--for skill in "${SKILLS[@]}"; do- prompt_file="$PROMPTS_DIR/${skill}.txt"-- if [ ! -f "$prompt_file" ]; then- echo "⚠️ SKIP: No prompt file for $skill"- continue- fi-- echo "Testing: $skill"-- if "$SCRIPT_DIR/run-test.sh" "$skill" "$prompt_file" 3 2>&1 | tee /tmp/skill-test-$skill.log; then- PASSED=$((PASSED + 1))- RESULTS+=("✅ $skill")- else- FAILED=$((FAILED + 1))- RESULTS+=("❌ $skill")- fi-- echo ""- echo "---"- echo ""-done--echo ""-echo "=== Summary ==="-for result in "${RESULTS[@]}"; do- echo " $result"-done-echo ""-echo "Passed: $PASSED"-echo "Failed: $FAILED"--if [ $FAILED -gt 0 ]; then- exit 1-fidiff --git a/tests/skill-triggering/run-test.sh b/tests/skill-triggering/run-test.shdeleted file mode 100755index ba9199583..000000000--- a/tests/skill-triggering/run-test.sh+++ /dev/null@@ -1,88 +0,0 @@-#!/usr/bin/env bash-# Test skill triggering with naive prompts-# Usage: ./run-test.sh <skill-name> <prompt-file>-#-# Tests whether Claude triggers a skill based on a natural prompt-# (without explicitly mentioning the skill)--set -e--SKILL_NAME="$1"-PROMPT_FILE="$2"-MAX_TURNS="${3:-3}"--if [ -z "$SKILL_NAME" ] || [ -z "$PROMPT_FILE" ]; then- echo "Usage: $0 <skill-name> <prompt-file> [max-turns]"- echo "Example: $0 systematic-debugging ./test-prompts/debugging.txt"- exit 1-fi--# Get the directory where this script lives (should be tests/skill-triggering)-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"-# Get the superpowers plugin root (two levels up from tests/skill-triggering)-PLUGIN_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"--TIMESTAMP=$(date +%s)-OUTPUT_DIR="/tmp/superpowers-tests/${TIMESTAMP}/skill-triggering/${SKILL_NAME}"-mkdir -p "$OUTPUT_DIR"--# Read prompt from file-PROMPT=$(cat "$PROMPT_FILE")--echo "=== Skill Triggering Test ==="-echo "Skill: $SKILL_NAME"-echo "Prompt file: $PROMPT_FILE"-echo "Max turns: $MAX_TURNS"-echo "Output dir: $OUTPUT_DIR"-echo ""--# Copy prompt for reference-cp "$PROMPT_FILE" "$OUTPUT_DIR/prompt.txt"--# Run Claude-LOG_FILE="$OUTPUT_DIR/claude-output.json"-cd "$OUTPUT_DIR"--echo "Plugin dir: $PLUGIN_DIR"-echo "Running claude -p with naive prompt..."-timeout 300 claude -p "$PROMPT" \- --plugin-dir "$PLUGIN_DIR" \- --dangerously-skip-permissions \- --max-turns "$MAX_TURNS" \- --output-format stream-json \- > "$LOG_FILE" 2>&1 || true--echo ""-echo "=== Results ==="--# Check if skill was triggered (look for Skill tool invocation)-# In stream-json, tool invocations have "name":"Skill" (not "tool":"Skill")-# Match either "skill":"skillname" or "skill":"namespace:skillname"-SKILL_PATTERN='"skill":"([^"]*:)?'"${SKILL_NAME}"'"'-if grep -q '"name":"Skill"' "$LOG_FILE" && grep -qE "$SKILL_PATTERN" "$LOG_FILE"; then- echo "✅ PASS: Skill '$SKILL_NAME' was triggered"- TRIGGERED=true-else- echo "❌ FAIL: Skill '$SKILL_NAME' was NOT triggered"- TRIGGERED=false-fi--# Show what skills WERE triggered-echo ""-echo "Skills triggered in this run:"-grep -o '"skill":"[^"]*"' "$LOG_FILE" 2>/dev/null | sort -u || echo " (none)"--# Show first assistant message-echo ""-echo "First assistant response (truncated):"-grep '"type":"assistant"' "$LOG_FILE" | head -1 | jq -r '.message.content[0].text // .message.content' 2>/dev/null | head -c 500 || echo " (could not extract)"--echo ""-echo "Full log: $LOG_FILE"-echo "Timestamp: $TIMESTAMP"--if [ "$TRIGGERED" = "true" ]; then- exit 0-else- exit 1-fi