100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.
refactor: update backend result handling and README
Shubhamsabood97685ef4f37
diff --git a/awesome_agent_skills/self-improving-agent-skills/README.md b/awesome_agent_skills/self-improving-agent-skills/README.mdindex af2bcb20..b3c36c5b 100644--- a/awesome_agent_skills/self-improving-agent-skills/README.md+++ b/awesome_agent_skills/self-improving-agent-skills/README.md@@ -1,41 +1,50 @@# Self-Improving Agent Skills-Automatically optimize your agent skills using Gemini's reasoning capabilities. Upload a skill, let AI generate test scenarios and evaluation criteria, then watch as your skill improves itself through iterative optimization.+Automatically optimize your agent skills using a multi-agent system built with **Google ADK (Agent Development Kit)** and **Gemini 3**. Upload a skill, let the agents generate test scenarios and evaluation criteria, then watch as three specialized ADK agents collaborate to improve your skill through iterative optimization.## How It Works-This app implements an automated skill improvement loop inspired by Karpathy's autoresearch methodology:+This app implements an automated skill improvement loop inspired by Karpathy's autoresearch methodology, powered by a team of ADK agents:1. **Upload**: Drop in your skill folder (following [agentskills.io](https://agentskills.io) spec)2. **Configure**: AI generates test scenarios and evaluation criteria. Edit, add, or regenerate as needed-3. **Optimize**: The system runs your skill, scores outputs, mutates the prompt one change at a time, and keeps improvements+3. **Optimize**: Three ADK agents collaborate — one executes, one diagnoses failures, one applies fixes4. **Results**: Download your improved skill with a detailed changelog-The optimization loop:-- Runs the skill against test scenarios-- Scores outputs using binary yes/no evaluation criteria-- Analyzes failure patterns-- Proposes ONE targeted change to the skill prompt-- Tests the change-- Keeps it if score improves, reverts if not-- Repeats until 95%+ pass rate or max rounds reached+### The ADK Agent Team++| Agent | Role | What It Does |+|-------|------|-------------|+| **Executor** | Skill Runner | Faithfully executes the skill against test scenarios, producing outputs exactly as the skill instructs |+| **Analyst** | Failure Diagnostician | Examines failed evaluations, identifies root causes, and recommends a specific mutation strategy |+| **Mutator** | Prompt Editor | Makes exactly ONE targeted change to the skill prompt based on the analyst's diagnosis |++### The Optimization Loop++- The **Executor** agent runs the skill against all test scenarios+- Outputs are scored using binary yes/no evaluation criteria+- The **Analyst** agent diagnoses failure patterns and picks a strategy (`add_example`, `add_constraint`, `restructure`, or `add_edge_case`)+- The **Mutator** agent applies ONE surgical fix to the skill prompt+- The modified skill is re-tested+- Changes are kept if the score improves, reverted if not+- Repeats until the target pass rate is reached or max rounds hit## Architecture```-self-improving-skill/-├── backend/ # FastAPI server + optimization engine-│ ├── app.py # REST API endpoints + SSE streaming-│ ├── optimizer.py # Core optimization logic+self-improving-agent-skills/+├── backend/ # FastAPI server + ADK optimization engine+│ ├── app.py # REST API endpoints + SSE streaming+│ ├── adk_optimizer.py # Multi-agent optimizer (Executor, Analyst, Mutator)│ ├── requirements.txt-│ └── .env.example-├── frontend/ # Next.js + React + Tailwind+│ └── optimizer.py # Legacy single-model optimizer (unused)+├── frontend/ # Next.js + React + Tailwind│ ├── src/-│ │ ├── app/ # Main page + layout-│ │ └── components/ # Upload, Config, Running, Results steps+│ │ ├── app/ # Main page + layout+│ │ └── components/ # Upload, Config, Running, Results steps│ ├── package.json│ └── *.config.ts-├── example_skills/ # Sample skills to test+├── example_skills/ # Sample skills to test│ ├── code-reviewer/│ └── content-writer/└── README.md@@ -43,9 +52,9 @@ self-improving-skill/## Tech Stack-- **Backend**: Python 3.10+, FastAPI, google-genai SDK+- **Backend**: Python 3.10+, FastAPI, Google ADK, google-genai SDK- **Frontend**: Next.js 15, React 19, Tailwind CSS v4, Recharts-- **AI**: Gemini 2.5 Flash for analysis, execution, scoring, and mutation+- **AI**: Google ADK multi-agent system with Gemini 3 Flash for execution, analysis, and mutation- **Real-time**: Server-Sent Events (SSE) for live optimization progress## Quick Start@@ -68,7 +77,7 @@ cp .env.example .env# Run serverpython app.py-# Server runs on http://localhost:8000+# Server runs on http://localhost:8891```### Frontend Setup@@ -91,7 +100,7 @@ npm run dev3. Upload a skill folder as a .zip file (or try an example)4. Enter your Gemini API key5. Review and edit the generated test scenarios and evaluation criteria-6. Click "Start Optimization" and watch the score climb+6. Click "Start Optimization" and watch the agents collaborate to improve your skill7. Download your improved skill when complete## Skill Format@@ -139,24 +148,25 @@ zip -r code-reviewer.zip code-reviewer/Then upload the zip in the app.-## How the Optimization Works+## How the Multi-Agent Optimization Works### 1. Analysis PhaseGemini analyzes your skill and generates:-- 3-5 diverse test scenarios-- 3-6 binary evaluation criteria (yes/no questions)+- 3-4 diverse test scenarios+- 4-6 binary evaluation criteria (yes/no questions)### 2. Baseline Run-The skill runs against all scenarios. Each output is scored against all evaluation criteria. This establishes the starting score.+The **Executor** agent runs the skill against all scenarios. Each output is scored against all evaluation criteria. This establishes the starting score.### 3. Optimization Loop-For each round:-1. Analyze which evaluation criteria fail most frequently-2. Use Gemini to propose ONE specific change to improve the skill-3. Run the modified skill against all scenarios-4. Compare scores-5. Keep the change if score improved, revert if not-6. Repeat until 95%+ pass rate for 3 consecutive rounds or max rounds reached+For each round, the three agents collaborate:+1. **Executor** runs the skill against all test scenarios+2. Outputs are scored against evaluation criteria+3. **Analyst** examines failures, identifies root cause, and selects a mutation strategy+4. **Mutator** applies ONE specific change to improve the skill+5. **Executor** re-runs the modified skill+6. Score is compared — keep the change if improved, revert if not+7. Repeat until target pass rate or max rounds reached### 4. Output- Improved SKILL.md with all successful changes applied@@ -166,19 +176,23 @@ For each round:## API Endpoints- `POST /api/upload` - Upload skill zip file-- `POST /api/analyze` - Generate scenarios and evals (requires GEMINI_API_KEY header)+- `POST /api/upload-files` - Upload multiple files (folder upload)+- `POST /api/analyze` - Generate scenarios and evals (requires Gemini API key)- `POST /api/regenerate` - Regenerate scenarios and evals- `POST /api/update-config` - Save user's selected/edited config- `POST /api/start/{session_id}` - Start optimization- `GET /api/stream/{session_id}` - SSE stream of optimization progress- `POST /api/stop/{session_id}` - Stop optimization- `GET /api/download/{session_id}` - Download improved skill+- `GET /api/examples` - List available example skills+- `POST /api/examples/{name}/load` - Load an example skill+- `GET /api/status/{session_id}` - Poll-based status endpoint## Configuration### Backend-Set `GEMINI_API_KEY` in `.env` or pass via request header.+Set `GEMINI_API_KEY` in `.env` or pass via request header. Server runs on port **8891**.### Frontend@@ -194,10 +208,10 @@ body: JSON.stringify({}),```-In `optimizer.py`, adjust model:+In `adk_optimizer.py`, adjust the model:```python-def __init__(self, api_key: str, model: str = "gemini-2.5-flash"):+def __init__(self, api_key: str, model: str = "gemini-3-flash-preview"):```## Development@@ -206,7 +220,7 @@ def __init__(self, api_key: str, model: str = "gemini-2.5-flash"):```bashcd backend-python -c "from optimizer import SkillOptimizer; print('OK')"+python -c "from adk_optimizer import SkillOptimizer; print('OK')"```### Frontend Build@@ -222,6 +236,6 @@ Both servers support hot reload. Edit code and see changes immediately.## Based on Karpathy's Autoresearch-This tool applies Andrej Karpathy's autoresearch methodology (using LLMs to iteratively improve their own prompts) to agent skills. The key insight: rather than manually tweaking prompts, define success criteria and let the AI optimize itself.+This tool applies Andrej Karpathy's autoresearch methodology (using LLMs to iteratively improve their own prompts) to agent skills. The key insight: rather than manually tweaking prompts, define success criteria and let the AI optimize itself — now powered by a team of specialized ADK agents.Original concept: [https://github.com/karpathy/autoresearch](https://github.com/karpathy/autoresearch)\ No newline at end of filediff --git a/awesome_agent_skills/self-improving-agent-skills/backend/app.py b/awesome_agent_skills/self-improving-agent-skills/backend/app.pyindex fe47705c..7b17e83c 100644--- a/awesome_agent_skills/self-improving-agent-skills/backend/app.py+++ b/awesome_agent_skills/self-improving-agent-skills/backend/app.py@@ -265,8 +265,33 @@ async def start_optimization(session_id: str, request: StartRequest):})elif event["type"] == "complete":session["status"] = "complete"- session["final_result"] = event["data"]- session["current_skill_md"] = event["data"]["improved_skill_md"]+ data = event["data"]+ ml = data.get("mutation_log", [])+ # Transform to match frontend ResultsStep expectations+ session["final_result"] = {+ "baseline_score": data.get("baseline_score", 0),+ "final_score": data.get("final_score", 0),+ "improved_skill_md": data.get("improved_skill_md", ""),+ "original_skill_md": session.get("original_skill_md", ""),+ "score_history": data.get("score_history", []),+ "experiments_run": len(ml),+ "kept": sum(1 for m in ml if m.get("kept")),+ "discarded": sum(1 for m in ml if not m.get("kept")),+ "changelog": [+ {+ "description": m.get("description", m.get("diagnosis", "")),+ "reasoning": m.get("diagnosis", ""),+ "status": "keep" if m.get("kept") else "discard",+ "score_before": m.get("score_before", 0),+ "score_after": m.get("score_after", 0),+ "strategy": m.get("strategy_type", ""),+ }+ for m in ml+ ],+ "mutation_log": ml,+ "strategy_stats": data.get("strategy_stats", {}),+ }+ session["current_skill_md"] = data.get("improved_skill_md", "")if "event_queue" in session:await session["event_queue"].put(None)@@ -279,7 +304,31 @@ async def start_optimization(session_id: str, request: StartRequest):callback=callback,)logger.info(f"Optimization complete: {result['baseline_score']}% -> {result['final_score']}%")- session["final_result"] = result+ # Don't overwrite final_result if callback already set it with transformed data+ if not session.get("final_result"):+ ml = result.get("mutation_log", [])+ session["final_result"] = {+ "baseline_score": result.get("baseline_score", 0),+ "final_score": result.get("final_score", 0),+ "improved_skill_md": result.get("improved_skill_md", ""),+ "original_skill_md": session.get("original_skill_md", ""),+ "score_history": result.get("score_history", []),+ "experiments_run": len(ml),+ "kept": sum(1 for m in ml if m.get("kept")),+ "discarded": sum(1 for m in ml if not m.get("kept")),+ "changelog": [+ {+ "description": m.get("description", m.get("diagnosis", "")),+ "reasoning": m.get("diagnosis", ""),+ "status": "keep" if m.get("kept") else "discard",+ "score_before": m.get("score_before", 0),+ "score_after": m.get("score_after", 0),+ "strategy": m.get("strategy_type", ""),+ }+ for m in ml+ ],+ "mutation_log": ml,+ }session["current_skill_md"] = result["improved_skill_md"]session["status"] = "complete"except Exception as e: