Skip to content

100+ AI Agents, Agent Skills and RAG Apps - Free and Open Source.

refactor: update backend result handling and README

Shubhamsabood97685ef4f37
diff --git a/awesome_agent_skills/self-improving-agent-skills/README.md b/awesome_agent_skills/self-improving-agent-skills/README.md
index af2bcb20..b3c36c5b 100644
--- a/awesome_agent_skills/self-improving-agent-skills/README.md
+++ b/awesome_agent_skills/self-improving-agent-skills/README.md
@@ -1,41 +1,50 @@
# Self-Improving Agent Skills
-Automatically optimize your agent skills using Gemini's reasoning capabilities. Upload a skill, let AI generate test scenarios and evaluation criteria, then watch as your skill improves itself through iterative optimization.
+Automatically optimize your agent skills using a multi-agent system built with **Google ADK (Agent Development Kit)** and **Gemini 3**. Upload a skill, let the agents generate test scenarios and evaluation criteria, then watch as three specialized ADK agents collaborate to improve your skill through iterative optimization.
## How It Works
-This app implements an automated skill improvement loop inspired by Karpathy's autoresearch methodology:
+This app implements an automated skill improvement loop inspired by Karpathy's autoresearch methodology, powered by a team of ADK agents:
1. **Upload**: Drop in your skill folder (following [agentskills.io](https://agentskills.io) spec)
2. **Configure**: AI generates test scenarios and evaluation criteria. Edit, add, or regenerate as needed
-3. **Optimize**: The system runs your skill, scores outputs, mutates the prompt one change at a time, and keeps improvements
+3. **Optimize**: Three ADK agents collaborate — one executes, one diagnoses failures, one applies fixes
4. **Results**: Download your improved skill with a detailed changelog
-The optimization loop:
-- Runs the skill against test scenarios
-- Scores outputs using binary yes/no evaluation criteria
-- Analyzes failure patterns
-- Proposes ONE targeted change to the skill prompt
-- Tests the change
-- Keeps it if score improves, reverts if not
-- Repeats until 95%+ pass rate or max rounds reached
+### The ADK Agent Team
+
+| Agent | Role | What It Does |
+|-------|------|-------------|
+| **Executor** | Skill Runner | Faithfully executes the skill against test scenarios, producing outputs exactly as the skill instructs |
+| **Analyst** | Failure Diagnostician | Examines failed evaluations, identifies root causes, and recommends a specific mutation strategy |
+| **Mutator** | Prompt Editor | Makes exactly ONE targeted change to the skill prompt based on the analyst's diagnosis |
+
+### The Optimization Loop
+
+- The **Executor** agent runs the skill against all test scenarios
+- Outputs are scored using binary yes/no evaluation criteria
+- The **Analyst** agent diagnoses failure patterns and picks a strategy (`add_example`, `add_constraint`, `restructure`, or `add_edge_case`)
+- The **Mutator** agent applies ONE surgical fix to the skill prompt
+- The modified skill is re-tested
+- Changes are kept if the score improves, reverted if not
+- Repeats until the target pass rate is reached or max rounds hit
## Architecture
```
-self-improving-skill/
-├── backend/ # FastAPI server + optimization engine
-│ ├── app.py # REST API endpoints + SSE streaming
-│ ├── optimizer.py # Core optimization logic
+self-improving-agent-skills/
+├── backend/ # FastAPI server + ADK optimization engine
+│ ├── app.py # REST API endpoints + SSE streaming
+│ ├── adk_optimizer.py # Multi-agent optimizer (Executor, Analyst, Mutator)
│ ├── requirements.txt
-│ └── .env.example
-├── frontend/ # Next.js + React + Tailwind
+│ └── optimizer.py # Legacy single-model optimizer (unused)
+├── frontend/ # Next.js + React + Tailwind
│ ├── src/
-│ │ ├── app/ # Main page + layout
-│ │ └── components/ # Upload, Config, Running, Results steps
+│ │ ├── app/ # Main page + layout
+│ │ └── components/ # Upload, Config, Running, Results steps
│ ├── package.json
│ └── *.config.ts
-├── example_skills/ # Sample skills to test
+├── example_skills/ # Sample skills to test
│ ├── code-reviewer/
│ └── content-writer/
└── README.md
@@ -43,9 +52,9 @@ self-improving-skill/
## Tech Stack
-- **Backend**: Python 3.10+, FastAPI, google-genai SDK
+- **Backend**: Python 3.10+, FastAPI, Google ADK, google-genai SDK
- **Frontend**: Next.js 15, React 19, Tailwind CSS v4, Recharts
-- **AI**: Gemini 2.5 Flash for analysis, execution, scoring, and mutation
+- **AI**: Google ADK multi-agent system with Gemini 3 Flash for execution, analysis, and mutation
- **Real-time**: Server-Sent Events (SSE) for live optimization progress
## Quick Start
@@ -68,7 +77,7 @@ cp .env.example .env
# Run server
python app.py
-# Server runs on http://localhost:8000
+# Server runs on http://localhost:8891
```
### Frontend Setup
@@ -91,7 +100,7 @@ npm run dev
3. Upload a skill folder as a .zip file (or try an example)
4. Enter your Gemini API key
5. Review and edit the generated test scenarios and evaluation criteria
-6. Click "Start Optimization" and watch the score climb
+6. Click "Start Optimization" and watch the agents collaborate to improve your skill
7. Download your improved skill when complete
## Skill Format
@@ -139,24 +148,25 @@ zip -r code-reviewer.zip code-reviewer/
Then upload the zip in the app.
-## How the Optimization Works
+## How the Multi-Agent Optimization Works
### 1. Analysis Phase
Gemini analyzes your skill and generates:
-- 3-5 diverse test scenarios
-- 3-6 binary evaluation criteria (yes/no questions)
+- 3-4 diverse test scenarios
+- 4-6 binary evaluation criteria (yes/no questions)
### 2. Baseline Run
-The skill runs against all scenarios. Each output is scored against all evaluation criteria. This establishes the starting score.
+The **Executor** agent runs the skill against all scenarios. Each output is scored against all evaluation criteria. This establishes the starting score.
### 3. Optimization Loop
-For each round:
-1. Analyze which evaluation criteria fail most frequently
-2. Use Gemini to propose ONE specific change to improve the skill
-3. Run the modified skill against all scenarios
-4. Compare scores
-5. Keep the change if score improved, revert if not
-6. Repeat until 95%+ pass rate for 3 consecutive rounds or max rounds reached
+For each round, the three agents collaborate:
+1. **Executor** runs the skill against all test scenarios
+2. Outputs are scored against evaluation criteria
+3. **Analyst** examines failures, identifies root cause, and selects a mutation strategy
+4. **Mutator** applies ONE specific change to improve the skill
+5. **Executor** re-runs the modified skill
+6. Score is compared — keep the change if improved, revert if not
+7. Repeat until target pass rate or max rounds reached
### 4. Output
- Improved SKILL.md with all successful changes applied
@@ -166,19 +176,23 @@ For each round:
## API Endpoints
- `POST /api/upload` - Upload skill zip file
-- `POST /api/analyze` - Generate scenarios and evals (requires GEMINI_API_KEY header)
+- `POST /api/upload-files` - Upload multiple files (folder upload)
+- `POST /api/analyze` - Generate scenarios and evals (requires Gemini API key)
- `POST /api/regenerate` - Regenerate scenarios and evals
- `POST /api/update-config` - Save user's selected/edited config
- `POST /api/start/{session_id}` - Start optimization
- `GET /api/stream/{session_id}` - SSE stream of optimization progress
- `POST /api/stop/{session_id}` - Stop optimization
- `GET /api/download/{session_id}` - Download improved skill
+- `GET /api/examples` - List available example skills
+- `POST /api/examples/{name}/load` - Load an example skill
+- `GET /api/status/{session_id}` - Poll-based status endpoint
## Configuration
### Backend
-Set `GEMINI_API_KEY` in `.env` or pass via request header.
+Set `GEMINI_API_KEY` in `.env` or pass via request header. Server runs on port **8891**.
### Frontend
@@ -194,10 +208,10 @@ body: JSON.stringify({
}),
```
-In `optimizer.py`, adjust model:
+In `adk_optimizer.py`, adjust the model:
```python
-def __init__(self, api_key: str, model: str = "gemini-2.5-flash"):
+def __init__(self, api_key: str, model: str = "gemini-3-flash-preview"):
```
## Development
@@ -206,7 +220,7 @@ def __init__(self, api_key: str, model: str = "gemini-2.5-flash"):
```bash
cd backend
-python -c "from optimizer import SkillOptimizer; print('OK')"
+python -c "from adk_optimizer import SkillOptimizer; print('OK')"
```
### Frontend Build
@@ -222,6 +236,6 @@ Both servers support hot reload. Edit code and see changes immediately.
## Based on Karpathy's Autoresearch
-This tool applies Andrej Karpathy's autoresearch methodology (using LLMs to iteratively improve their own prompts) to agent skills. The key insight: rather than manually tweaking prompts, define success criteria and let the AI optimize itself.
+This tool applies Andrej Karpathy's autoresearch methodology (using LLMs to iteratively improve their own prompts) to agent skills. The key insight: rather than manually tweaking prompts, define success criteria and let the AI optimize itself — now powered by a team of specialized ADK agents.
Original concept: [https://github.com/karpathy/autoresearch](https://github.com/karpathy/autoresearch)
\ No newline at end of file
diff --git a/awesome_agent_skills/self-improving-agent-skills/backend/app.py b/awesome_agent_skills/self-improving-agent-skills/backend/app.py
index fe47705c..7b17e83c 100644
--- a/awesome_agent_skills/self-improving-agent-skills/backend/app.py
+++ b/awesome_agent_skills/self-improving-agent-skills/backend/app.py
@@ -265,8 +265,33 @@ async def start_optimization(session_id: str, request: StartRequest):
})
elif event["type"] == "complete":
session["status"] = "complete"
- session["final_result"] = event["data"]
- session["current_skill_md"] = event["data"]["improved_skill_md"]
+ data = event["data"]
+ ml = data.get("mutation_log", [])
+ # Transform to match frontend ResultsStep expectations
+ session["final_result"] = {
+ "baseline_score": data.get("baseline_score", 0),
+ "final_score": data.get("final_score", 0),
+ "improved_skill_md": data.get("improved_skill_md", ""),
+ "original_skill_md": session.get("original_skill_md", ""),
+ "score_history": data.get("score_history", []),
+ "experiments_run": len(ml),
+ "kept": sum(1 for m in ml if m.get("kept")),
+ "discarded": sum(1 for m in ml if not m.get("kept")),
+ "changelog": [
+ {
+ "description": m.get("description", m.get("diagnosis", "")),
+ "reasoning": m.get("diagnosis", ""),
+ "status": "keep" if m.get("kept") else "discard",
+ "score_before": m.get("score_before", 0),
+ "score_after": m.get("score_after", 0),
+ "strategy": m.get("strategy_type", ""),
+ }
+ for m in ml
+ ],
+ "mutation_log": ml,
+ "strategy_stats": data.get("strategy_stats", {}),
+ }
+ session["current_skill_md"] = data.get("improved_skill_md", "")
if "event_queue" in session:
await session["event_queue"].put(None)
@@ -279,7 +304,31 @@ async def start_optimization(session_id: str, request: StartRequest):
callback=callback,
)
logger.info(f"Optimization complete: {result['baseline_score']}% -> {result['final_score']}%")
- session["final_result"] = result
+ # Don't overwrite final_result if callback already set it with transformed data
+ if not session.get("final_result"):
+ ml = result.get("mutation_log", [])
+ session["final_result"] = {
+ "baseline_score": result.get("baseline_score", 0),
+ "final_score": result.get("final_score", 0),
+ "improved_skill_md": result.get("improved_skill_md", ""),
+ "original_skill_md": session.get("original_skill_md", ""),
+ "score_history": result.get("score_history", []),
+ "experiments_run": len(ml),
+ "kept": sum(1 for m in ml if m.get("kept")),
+ "discarded": sum(1 for m in ml if not m.get("kept")),
+ "changelog": [
+ {
+ "description": m.get("description", m.get("diagnosis", "")),
+ "reasoning": m.get("diagnosis", ""),
+ "status": "keep" if m.get("kept") else "discard",
+ "score_before": m.get("score_before", 0),
+ "score_after": m.get("score_after", 0),
+ "strategy": m.get("strategy_type", ""),
+ }
+ for m in ml
+ ],
+ "mutation_log": ml,
+ }
session["current_skill_md"] = result["improved_skill_md"]
session["status"] = "complete"
except Exception as e: