Skip to content

Create, Evaluate, and Connect AI Skills

upload pip package

LiamLYYYb8bedf106350
diff --git a/skillnet-ai/README.md b/skillnet-ai/README.md
new file mode 100644
index 0000000..52a2d36
--- /dev/null
+++ b/skillnet-ai/README.md
@@ -0,0 +1,247 @@
+# SkillNet AI SDK
+
+[![PyPI version](https://badge.fury.io/py/skillnet-ai.svg)](https://badge.fury.io/py/skillnet-ai)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+
+**SkillNet AI** is the official Python SDK for interacting with the SkillNet platform. It allows AI Agents to **Search**, **Install**, **Evaluate** and **Create** reusable skills dynamically.
+
+SkillNet enables agents to evolve by learning from execution trajectories and sharing capabilities via a centralized registry.
+
+---
+
+## πŸš€ Features
+
+- **πŸ” Search**: Find skills using keywords or natural language queries.
+- **πŸ“¦ One-Line Installation**: Download and install skill packages directly from GitHub repositories.
+- **✨ Skill Creation**: Automatically convert trajectories (agent execution logs, conversation logs ...) into structured, reusable `skill` packages using LLMs.
+- **πŸ“Š Evaluation**: Evaluate and score skills for quality assurance (Safety, Completeness, Excutability, Modifiability, Cost-Aware).
+
+---
+
+## πŸ“₯ Installation
+
+```bash
+pip install skillnet-ai
+```
+
+---
+
+## πŸ›  Usage (Python SDK)
+
+### 1. Search for Skills
+
+You can search for skills using keywords or natural language (AI Search).
+
+```python
+from skillnet_ai import SkillNetSearcher
+
+searcher = SkillNetSearcher()
+
+# 1. Standard Keyword Search
+results = searcher.search(q="pdf")
+
+# 2. Semantic Search
+results = searcher.search(q="Help me analyze financial PDF reports", mode="vector")
+
+top_skill = results[0]
+print(f"Top Skill: {top_skill.skill_name} (Stars: {top_skill.stars})")
+print(f"Description: {top_skill.skill_description}")
+print(f"URL: {skill.skill_url}")
+
+```
+
+### 2. Install a Skill
+
+Once you have a skill's URL (usually from the search results), you can download and install it into your agent's workspace.
+
+```python
+from skillnet_ai import SkillDownloader
+
+skill_url = "https://github.com/anthropics/skills/tree/main/skills/skill-creator"
+
+# Downloads and extracts the skill to target_dir
+skilldownloader = SkillDownloader()
+local_path = skilldownloader.download(skill_url, target_dir="./my_agent_skills")
+
+if local_path:
+ print(f"Skill successfully installed at: {local_path}")
+```
+
+### 3. Create a Skill from Trajectory
+
+Turn raw conversation logs or execution traces into a polished Skill Package (`SKILL.md`, scripts, etc.).
+
+> **Note:** This feature requires an LLM API Key.
+
+```python
+import os
+from skillnet_ai import SkillCreator
+
+# 1. Prepare your trajectory (e.g., a conversation log)
+trajectory_log = """
+User: I need to rename all .jpg files in this folder to .png.
+Agent: I will write a python script to iterate through the folder...
+Agent: Script executed. Renamed 5 files.
+"""
+
+# Set your API key and url
+api_key="sk-...", # Your API Key
+base_url="xxxxxx", # Optional. Useful if using a proxy or compatible API
+
+# 2. Initialize Creator
+creator = SkillCreator(api_key=api_key, base_url=base_url, model="gpt-4o")
+
+# 3. Create Skills
+output_dir = "./created_skills"
+created_paths = creator.create_from_trajectory(trajectory_log, output_dir=output_dir)
+
+print(f"Created {len(created_paths)} new skills in {output_dir}")
+```
+
+### 4. Skill Evaluation
+
+Assess the Safety, Completeness, Excutability, Modifiability and Cost-Aware of a skill before using it. This tool generates a complete evaluation based on the skill's code and documentation.
+
+> **Note:** This feature requires an LLM API Key.
+
+```python
+import os
+from skillnet_ai import SkillEvaluator, EvaluatorConfig
+
+# 1. Configure the Evaluator
+config = EvaluatorConfig(
+ api_key="sk-...", # Your API Key
+ base_url="xxxxxx", # Optional. Useful if using a proxy or compatible API
+ model="gpt-4o" # Model to use for evaluation
+)
+evaluator = SkillEvaluator(config)
+
+# 2. Evaluate a Skill (from URL or Local Path)
+skill_url = "https://github.com/microsoft/autogen/tree/main/samples/tools/web_search"
+
+# You can also use evaluator.evaluate_from_path("./my_skills/web_search")
+result = evaluator.evaluate_from_url(skill_url)
+
+print(f"Evaluation: {result}")
+```
+
+---
+
+## πŸ’» CLI Usage
+
+SkillNet AI provides a robust Command Line Interface (CLI) powered by `Typer` and `Rich`. It allows you to search, download, create, and evaluate skills directly from the terminal with visual feedback.
+
+**Tip:** You can view the full list of options for any command using `--help` (e.g., `skillnet search --help`).
+
+### 1. Search Skills (`search`)
+
+Search the registry using keywords or AI-powered semantic search.
+
+```bash
+# Basic keyword search
+skillnet search "pdf extraction"
+
+# Semantic/Vector search (finds skills by meaning)
+skillnet search "tools for reading financial documents" --mode vector --threshold 0.8
+
+# Filter by category and sort results
+skillnet search "visualization" --category "Data" --sort-by stars --limit 10
+```
+
+#### Key Options:
+
+--mode: Search mode, either keyword (default) or vector.
+
+--threshold: Similarity threshold for vector search (0.0 - 1.0).
+
+--min-stars: Filter results by minimum star rating.
+
+### 2. Download Skills (download)
+
+Download and install a skill directly from a GitHub repository subdirectory.
+
+```bash
+# Download to the current directory
+skillnet download [https://github.com/owner/repo/tree/main/skills/math_solver](https://github.com/owner/repo/tree/main/skills/math_solver)
+
+# Download to a specific target directory
+skillnet download [https://github.com/owner/repo/tree/main/skills/math_solver](https://github.com/owner/repo/tree/main/skills/math_solver) -d ./my_agent/skills
+
+# Download from a private repository
+skillnet download <private_url> --token <your_github_token>
+```
+
+### 3. Create Skills (create)
+
+Analyze an execution trajectory (log file) and automatically generate a structured Skill Package using LLMs.
+
+Requirement: Ensure API_KEY is set in your environment variables.
+
+```bash
+# Generate a skill from a trajectory file
+skillnet create ./logs/trajectory.txt --output-dir ./generated_skills
+
+# Specify a specific LLM model
+skillnet create ./logs/chat_history.txt --model gpt-4o
+```
+
+### Evaluate Skills (evaluate)
+Generate a comprehensive quality report (Safety, Completeness, Executability, Modifiability, Cost Awareness) for a skill.
+
+Requirement: Ensure API_KEY is set in your environment variables.
+
+```bash
+# Evaluate a remote skill via GitHub URL
+skillnet evaluate [https://github.com/owner/repo/tree/main/skills/web_search](https://github.com/owner/repo/tree/main/skills/web_search)
+
+# Evaluate a local skill directory
+skillnet evaluate ./my_skills/web_search
+
+# Custom evaluation config
+skillnet evaluate ./my_skills/tool --category "DevOps" --model gpt-4o
+```
+
+## βš™οΈ Configuration
+
+### Environment Variables
+
+If you are using the **Skill Creation** or **Skill Evaluation** feature, you must configure your LLM provider.
+
+| Variable | Description | Default |
+| :--- | :--- | :--- |
+| `API_KEY` | Required for creating or evaluating skills. | `None` |
+| `BASE_URL` | Optional. Useful if using a proxy or compatible API. | `https://api.openai.com/v1` |
+
+---
+
+## πŸ“‚ Project Structure
+
+When you create or install a skill, it follows the **Standard Skill Structure**:
+
+```text
+skill-name/
+β”œβ”€β”€ SKILL.md # (required) Main orchestration file (YAML metadata + Instructions)
+β”œβ”€β”€ scripts/ # (optional) Executable Python/Bash scripts
+β”œβ”€β”€ references/ # (optional) Static documentation or API specs
+└── assets/ # (optional) Templates, icons, etc.
+```
+
+---
+
+## πŸ—Ί Roadmap
+
+- [x] Search & AI Search
+- [x] Skill Installer
+- [x] Trajectory-to-Skill Creator
+- [x] Skill Evaluation & Scoring
+
+---
+
+## 🀝 Contributing
+
+Contributions are welcome! Please submit a Pull Request or open an Issue on GitHub.
+
+## πŸ“„ License
+
+This project is licensed under the [MIT License](LICENSE).
\ No newline at end of file
diff --git a/skillnet-ai/examples/create_example.py b/skillnet-ai/examples/create_example.py
new file mode 100644
index 0000000..d834c66
--- /dev/null
+++ b/skillnet-ai/examples/create_example.py
@@ -0,0 +1,43 @@
+import os
+import logging
+from skillnet_ai import SkillCreator
+
+# Setup logging
+logging.basicConfig(level=logging.INFO)
+
+# Dummy trajectory data for demonstration
+SAMPLE_TRAJECTORY = """
+User: Please analyze this invoice.pdf and extract the total amount and date.
+Agent: I will use the PDF tool. Opening invoice.pdf...
+Agent: Text extracted. Found 'Total: $500' and 'Date: 2023-10-12'.
+Agent: The total is $500 and the date is Oct 12, 2023.
+"""
+
+def main():
+ # Ensure you have API_KEY set in your environment
+ api_key = os.getenv("API_KEY")
+ if not api_key:
+ print("Please set API_KEY environment variable to run this demo.")
+ return
+ base_url = os.getenv("BASE_URL") or "https://api.openai.com/v1" # Optional, custom base URL or default
+
+ creator = SkillCreator(api_key=api_key, base_url=base_url, model="gpt-4o")
+
+ print("πŸš€ Starting skill creation from trajectory...")
+
+ try:
+ # Create skills into the ./created_skills folder
+ paths = creator.create_from_trajectory(
+ trajectory=SAMPLE_TRAJECTORY,
+ output_dir="./created_skills"
+ )
+
+ print("\n✨ Creation Complete!")
+ for p in paths:
+ print(f" - Created Skill at: {p}")
+
+ except Exception as e:
+ print(f"❌ Error during creation: {e}")
+
+if __name__ == "__main__":
+ main()
\ No newline at end of file
diff --git a/skillnet-ai/examples/evaluate_example.py b/skillnet-ai/examples/evaluate_example.py
new file mode 100644
index 0000000..7e48af4
--- /dev/null
+++ b/skillnet-ai/examples/evaluate_example.py
@@ -0,0 +1,69 @@
+import os
+import json
+import logging
+import shutil
+from typing import Dict, Any
+from skillnet_ai import SkillEvaluator, EvaluatorConfig
+
+# Setup logging configuration
+logging.basicConfig(level=logging.INFO)
+
+def main():
+ # 1. Configure Evaluator
+ api_key = os.getenv("API_KEY")
+ if not api_key:
+ print("Please set API_KEY environment variable to run this demo.")
+ return
+ base_url = os.getenv("BASE_URL") or "https://api.openai.com/v1" # Optional, custom base URL or default
+
+ config = EvaluatorConfig(
+ api_key=api_key,
+ base_url=base_url,
+ model="gpt-4o",
+ temperature=0.2,
+ )
+
+ try:
+ evaluator = SkillEvaluator(config)
+ print("Evaluator initialized successfully.")
+ except Exception as e:
+ print(f"Failed to init evaluator: {e}")
+ return
+
+ # =================================================================
+ # A: Evaluate Local Skill Path
+ # =================================================================
+ local_skill_path = "/zjunlp/liangyuan/code/skills-resource/skillnet-ai/created_skills/pdf-invoice-data-extractor" # replace with your local skill path
+
+ try:
+ print(f"Starting evaluation for local path: {local_skill_path}")
+
+ result_local = evaluator.evaluate_from_path(
+ local_skill_path
+ )
+
+ print("Local Skill Evaluation", result_local)
+
+ except Exception as e:
+ print(f"Local evaluation failed: {e}")
+
+ # =================================================================
+ # B: Evaluate Skill from URL
+ # =================================================================
+ target_url = "https://github.com/anthropics/skills/tree/main/skills/algorithmic-art"
+
+ try:
+ print(f"Starting evaluation for URL: {target_url}")
+
+ result_url = evaluator.evaluate_from_url(
+ target_url,
+ description="Auto-downloaded skill for web searching"
+ )
+
+ print("URL Skill Evaluation", result_url)
+
+ except Exception as e:
+ print(f"URL evaluation failed (Create mock Downloader to fix): {e}")
+
+if __name__ == "__main__":
+ main()
\ No newline at end of file
diff --git a/skillnet-ai/examples/search_demo.py b/skillnet-ai/examples/search_demo.py
new file mode 100644
index 0000000..00cae8f
--- /dev/null
+++ b/skillnet-ai/examples/search_demo.py
@@ -0,0 +1,52 @@
+import logging
+from skillnet_ai import SkillNetSearcher, SkillDownloader
+
+# Setup logging configuration
+logging.basicConfig(level=logging.INFO)
+
+def main():
+ # Initialize the searcher
+ searcher = SkillNetSearcher()
+
+ query = "python"
+ print(f"πŸ” Searching for: {query}")
+
+ # Perform AI Search (Vector mode)
+ # Note: We now use the unified 'search' method with mode="vector"
+ results = searcher.search(
+ q=query,
+ limit=10,
+ category="Development",
+ )
+
+ if not results:
+ print("❌ No skills found.")
+ return
+
+ print(f"βœ… Found {len(results)} skills.")
+
+ # Display the top result
+ top_skill = results[0]
+ print(f"Top Skill: {top_skill.skill_name} (Stars: {top_skill.stars})")
+ print(f"Description: {top_skill.skill_description}")
+
+ # Install the skill if a valid URL exists
+ if top_skill.skill_url:
+ print("⬇️ Installing skill...")
+ skilldownloader = SkillDownloader()
+
+ # Download the skill to a local directory
+ local_path = skilldownloader.download(
+ top_skill.skill_url,
+ target_dir="./my_skills"
+ )
+
+ if local_path:
+ print(f"πŸŽ‰ Skill ready at: {local_path}")
+ else:
+ print("⚠️ Download failed or returned no path.")
+ else:
+ print("ℹ️ No download URL available for this skill.")
+
+if __name__ == "__main__":
+ main()
\ No newline at end of file
diff --git a/skillnet-ai/pyproject.toml b/skillnet-ai/pyproject.toml
new file mode 100644
index 0000000..3f830ad
--- /dev/null
+++ b/skillnet-ai/pyproject.toml
@@ -0,0 +1,30 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "skillnet-ai"
+version = "0.1.0"
+description = "Official Python SDK for SkillNet: Search, Create, Evaluate and Share AI Agent Skills."
+readme = "README.md"
+authors = [{ name = "SkillNet Team", email = "[email protected]" }]
+license = { text = "MIT" }
+classifiers = [
+ "Programming Language :: Python :: 3",
+ "License :: OSI Approved :: MIT License",
+ "Operating System :: OS Independent",
+]
+requires-python = ">=3.8"
+dependencies = [
+ "requests>=2.25.0",
+ "openai>=1.0.0",
+ "pydantic>=2.0.0",
+ "tqdm>=4.0.0" # Added for progress bars during download
+]
+
+[project.urls]
+"Homepage" = "https://skillnet.openkg.cn"
+"Source" = "https://github.com/zjunlp/SkillNet"
+
+[project.scripts]
+skillnet = "skillnet_ai.cli:app"
\ No newline at end of file
diff --git a/skillnet-ai/src/skillnet_ai/__init__.py b/skillnet-ai/src/skillnet_ai/__init__.py
new file mode 100644
index 0000000..bfff398
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/__init__.py
@@ -0,0 +1,19 @@
+"""
+SkillNet AI SDK
+~~~~~~~~~~~~~~~
+
+A client library for searching, downloading, creating and evaluating AI Agent Skills.
+"""
+
+from skillnet_ai.creator import SkillCreator
+from skillnet_ai.downloader import SkillDownloader
+from skillnet_ai.evaluator import SkillEvaluator, EvaluatorConfig
+from skillnet_ai.searcher import SkillNetSearcher
+
+__all__ = [
+ "SkillCreator",
+ "SkillDownloader",
+ "SkillEvaluator",
+ "EvaluatorConfig",
+ "SkillNetSearcher"
+]
\ No newline at end of file
diff --git a/skillnet-ai/src/skillnet_ai/cli.py b/skillnet-ai/src/skillnet_ai/cli.py
new file mode 100644
index 0000000..7696a0d
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/cli.py
@@ -0,0 +1,314 @@
+import typer
+import os
+from pathlib import Path
+from rich.console import Console
+from rich.table import Table
+from rich.panel import Panel
+from rich.columns import Columns
+from skillnet_ai.creator import SkillCreator
+from skillnet_ai.downloader import SkillDownloader
+from skillnet_ai.evaluator import SkillEvaluator, EvaluatorConfig
+from skillnet_ai.searcher import SkillNetSearcher
+
+app = typer.Typer(help="SkillNet AI CLI Tool")
+console = Console()
+
+API_KEY = os.getenv("API_KEY")
+BASE_URL = os.getenv("BASE_URL") or "https://api.openai.com/v1"
+
+def search(
+ q: str = typer.Argument(..., help="The search query (keywords or natural language description)."),
+ mode: str = typer.Option("keyword", help="Search mode: 'keyword' (exact/fuzzy) or 'vector' (semantic AI)."),
+ category: str = typer.Option(None, help="Filter results by category (e.g., 'Development')."),
+ limit: int = typer.Option(20, help="Maximum number of results to return."),
+ # Keyword specific options
+ page: int = typer.Option(1, help="Page number (only for keyword mode)."),
+ min_stars: int = typer.Option(0, help="Minimum star rating (only for keyword mode)."),
+ sort_by: str = typer.Option("stars", help="Sort criteria: 'stars' or 'recent' (only for keyword mode)."),
+ # Vector specific options
+ threshold: float = typer.Option(0.8, help="Similarity threshold 0.0-1.0 (only for vector mode)."),
+):
+ """
+ Search for skills on SkillNet using Keyword match or Vector (AI) semantic search.
+ """
+ try:
+ # Initialize Searcher (Ensure URL points to your actual API)
+ searcher = SkillNetSearcher(skillnet_url="http://127.0.0.1:8502")
+
+ # Visual feedback during API call
+ with console.status(f"[bold green]Searching SkillNet ({mode} mode)..."):
+ # Assuming searcher.search returns List[SkillModel]
+ results = searcher.search(
+ q=q,
+ mode=mode, # type: ignore
+ category=category,
+ limit=limit,
+ page=page,
+ min_stars=min_stars,
+ sort_by=sort_by,
+ threshold=threshold
+ )
+
+ # Handle Empty Results
+ if not results:
+ console.print(f"[yellow]No results found for query: '{q}'[/yellow]")
+ return
+
+ # Build Output Table
+ table = Table(title=f"Search Results: {q} ({len(results)} items)")
+
+ # Define Columns
+ table.add_column("Name", style="cyan", no_wrap=True)
+ table.add_column("Category", style="magenta")
+ table.add_column("Stars", justify="right", style="green")
+ table.add_column("Description", style="white")
+ table.add_column("URL", style="dim blue", overflow="fold") # Added URL column
+
+ # Note: 'score' was removed because SkillModel definition does not have a 'score' field.
+ # If your API returns a score for vector search, you must add it to SkillModel first.
+
+ for item in results:
+ # 1. Access fields using Pydantic dot notation
+ # 2. Use correct field names from SkillModel (e.g. skill_description)
+ name = item.skill_name
+ cat = item.category if item.category else "N/A"
+ stars = str(item.stars)
+ desc = item.skill_description if item.skill_description else ""
+ url = item.skill_url if item.skill_url else "N/A"
+
+ # Truncate long descriptions for display
+ short_desc = (desc[:100] + '...') if len(desc) > 100 else desc
+
+ # Prepare row data
+ row_data = [
+ name,
+ cat,
+ stars,
+ short_desc,
+ url
+ ]
+
+ table.add_row(*row_data)
+
+ console.print(table)
+
+ # Suggest next step
+ console.print("\n[dim]Tip: Use 'skillnet download <skill_url>' to get a skill.[/dim]")
+
+ except Exception as e:
+ console.print(f"[bold red]Error during search:[/bold red] {str(e)}")
+ # Optional: Print full traceback for debugging
+ # console.print_exception()
+ raise typer.Exit(code=1)
+
+def download(
+ url: str = typer.Argument(..., help="The GitHub URL of the specific skill folder (e.g., https://github.com/owner/repo/tree/main/skills/math_solver)."),
+ target_dir: str = typer.Option(".", "--target-dir", "-d", help="Local directory to install the skill into."),
+ token: str = typer.Option(None, "--token", "-t", envvar="GITHUB_TOKEN", help="GitHub Personal Access Token (for private repos or higher rate limits)."),
+):
+ """
+ Download and install a specific skill directly from a GitHub repository subdirectory.
+ """
+ # 1. Initialize Downloader
+ # Checks CLI option first, then environment variable GITHUB_TOKEN
+ downloader = SkillDownloader(api_token=token)
+
+ try:
+ # 2. Visual Feedback
+ console.print(f"[dim]Target directory: {os.path.abspath(target_dir)}[/dim]")
+
+ with console.status(f"[bold green]Downloading skill from GitHub...[/bold green]", spinner="dots"):
+ installed_path = downloader.download(folder_url=url, target_dir=target_dir)
+
+ # 3. Handle Results
+ if installed_path:
+ # Success
+ folder_name = os.path.basename(installed_path)
+
+ table = Table(title="Installation Successful", show_header=False, box=None)
+ table.add_row("[bold cyan]Skill:[/bold cyan]", folder_name)
+ table.add_row("[bold cyan]Location:[/bold cyan]", installed_path)
+
+ console.print(table)
+ console.print(f"\n[green]βœ“ {folder_name} is ready to use.[/green]")
+ else:
+ # Failure (Logic handled inside class, but we catch the None return)
+ console.print("[bold red]Download Failed.[/bold red]")
+ console.print("Possible reasons:")
+ console.print("1. The URL format is incorrect (must point to a specific folder, not just the repo root).")
+ console.print("2. The repository is private and no token was provided.")
+ console.print("3. GitHub API rate limits exceeded (try providing a token).")
+ raise typer.Exit(code=1)
+
+ except Exception as e:
+ console.print(f"[bold red]An unexpected error occurred:[/bold red] {str(e)}")
+ raise typer.Exit(code=1)
+
+def create(
+ trajectory_file: Path = typer.Argument(..., exists=True, readable=True, help="Path to the text file containing the execution trajectory/logs."),
+ output_dir: Path = typer.Option(Path("./generated_skills"), help="Directory where the new skills will be saved."),
+ model: str = typer.Option("gpt-4o", help="The LLM model to use (e.g., gpt-4o, gpt-3.5-turbo)."),
+):
+ """
+ Analyze a trajectory log and generate executable Skill packages using AI.
+ """
+ # 1. Validate Environment
+ if not API_KEY:
+ console.print("[bold red]Error:[/bold red] API_KEY environment variable is not set.")
+ console.print("Please export API_KEY or set it in your environment.")
+ raise typer.Exit(code=1)
+
+ try:
+ # 2. Read Trajectory Content
+ console.print(f"[dim]Reading trajectory from: {trajectory_file}[/dim]")
+ with open(trajectory_file, "r", encoding="utf-8") as f:
+ trajectory_content = f.read()
+
+ if not trajectory_content.strip():
+ console.print("[bold red]Error:[/bold red] Trajectory file is empty.")
+ raise typer.Exit(code=1)
+
+ # 3. Initialize Creator
+ creator = SkillCreator(
+ api_key=API_KEY,
+ base_url=BASE_URL,
+ model=model
+ )
+
+ # 4. Run Generation with Spinner
+ # The creation process involves multiple LLM calls, so it may take time.
+ with console.status("[bold green]AI is analyzing trajectory and generating skills...[/bold green]", spinner="dots"):
+ created_paths = creator.create_from_trajectory(
+ trajectory=trajectory_content,
+ output_dir=str(output_dir)
+ )
+
+ # 5. Report Results
+ if created_paths:
+ console.print(f"\n[bold green]Success! Generated {len(created_paths)} skill(s):[/bold green]")
+
+ table = Table(show_header=True, header_style="bold magenta")
+ table.add_column("Skill Name", style="cyan")
+ table.add_column("Location", style="white")
+
+ for path in created_paths:
+ # Assuming path ends with the skill name
+ skill_name = os.path.basename(path)
+ table.add_row(skill_name, str(path))
+
+ console.print(table)
+ console.print(f"\n[dim]Files saved to: {os.path.abspath(output_dir)}[/dim]")
+ else:
+ console.print("\n[yellow]Analysis complete, but no clear skills were identified in this trajectory.[/yellow]")
+
+ except Exception as e:
+ console.print(f"\n[bold red]Creation Failed:[/bold red] {str(e)}")
+ # Optional: Print traceback if in debug mode
+ # console.print_exception()
+ raise typer.Exit(code=1)
+
+def evaluate(
+ target: str = typer.Argument(..., help="Path to a local skill directory OR a GitHub URL."),
+
+ # Optional metadata overrides (useful if not auto-detected)
+ name: str = typer.Option(None, help="Name of the skill (overrides auto-detection)."),
+ category: str = typer.Option(None, help="Category of the skill (e.g., 'Data Analysis')."),
+ description: str = typer.Option(None, help="Short description of what the skill does."),
+
+ # Config options
+ model: str = typer.Option("gpt-4o", help="The LLM model used for evaluation."),
+ max_workers: int = typer.Option(5, help="Concurrency for batch operations (not used for single eval)."),
+):
+ """
+ Evaluate the quality, safety, and completeness of a skill using AI.
+
+ Target can be a local folder path or a GitHub URL (e.g., https://github.com/user/repo/tree/main/skill).
+ """
+ # 1. Validate Environment
+ if not API_KEY:
+ console.print("[bold red]Error:[/bold red] API_KEY environment variable is not set.")
+ raise typer.Exit(code=1)
+
+ # 2. Configure Evaluator
+ config = EvaluatorConfig(
+ api_key=API_KEY,
+ base_url=BASE_URL,
+ model=model,
+ max_workers=max_workers
+ )
+ evaluator = SkillEvaluator(config)
+
+ try:
+ # 3. Determine Mode (URL vs Local Path) and Run Evaluation
+ is_url = target.startswith("http://") or target.startswith("https://")
+
+ with console.status(f"[bold green]Evaluating skill ({'Remote' if is_url else 'Local'})...[/bold green]", spinner="dots"):
+ if is_url:
+ result = evaluator.evaluate_from_url(
+ url=target,
+ name=name,
+ category=category,
+ description=description
+ )
+ else:
+ result = evaluator.evaluate_from_path(
+ path=target,
+ name=name,
+ category=category,
+ description=description
+ )
+
+ # 4. Display Results
+ if "error" in result:
+ console.print(f"[bold red]Evaluation Failed:[/bold red] {result['error']}")
+ raise typer.Exit(code=1)
+
+ _display_evaluation_report(target, result)
+
+ except Exception as e:
+ console.print(f"[bold red]An unexpected error occurred:[/bold red] {str(e)}")
+ raise typer.Exit(code=1)
+
+def _display_evaluation_report(target_name: str, data: dict):
+ """Helper to render the JSON evaluation result into a nice Rich UI."""
+ console.print(f"\n[bold underline]Evaluation Report: {os.path.basename(target_name)}[/bold underline]\n")
+
+ # Dimensions to display
+ dimensions = ["safety", "completeness", "executability", "modifiability", "cost_awareness"]
+
+ # Create a grid of panels
+ panels = []
+ for dim in dimensions:
+ info = data.get(dim, {})
+ level = info.get("level", "Unknown")
+ reason = info.get("reason", "No details provided.")
+
+ # Color coding based on level
+ color = "white"
+ if "Excellent" in level: color = "green"
+ elif "Good" in level: color = "blue"
+ elif "Fair" in level: color = "yellow"
+ elif "Poor" in level: color = "red"
+
+ panel_content = f"[bold]{level}[/bold]\n\n[dim]{reason}[/dim]"
+ panels.append(Panel(panel_content, title=f"[{color}]{dim.title()}[/{color}]", expand=True))
+
+ console.print(Columns(panels, equal=True, expand=True))
+
+ # Display Score if present
+ overall_score = data.get("overall_score")
+ if overall_score:
+ score_color = "green" if overall_score >= 8 else "yellow" if overall_score >= 5 else "red"
+ console.print(f"\n[bold]Overall Score:[/bold] [{score_color}]{overall_score}/10[/{score_color}]")
+
+ # Summary
+ summary = data.get("summary")
+ if summary:
+ console.print(Panel(summary, title="Executive Summary", border_style="cyan"))
+
+if __name__ == "__main__":
+ app()
diff --git a/skillnet-ai/src/skillnet_ai/creator.py b/skillnet-ai/src/skillnet_ai/creator.py
new file mode 100644
index 0000000..8f1c551
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/creator.py
@@ -0,0 +1,131 @@
+import os
+import json
+import re
+import logging
+from typing import List, Optional
+from openai import OpenAI
+from skillnet_ai.prompts import (
+ CANDIDATE_METADATA_SYSTEM_PROMPT,
+ CANDIDATE_METADATA_USER_PROMPT_TEMPLATE,
+ SKILL_CONTENT_SYSTEM_PROMPT,
+ SKILL_CONTENT_USER_PROMPT_TEMPLATE
+)
+
+logger = logging.getLogger(__name__)
+
+class SkillCreator:
+ """
+ Creates Skill packages from execution trajectories using OpenAI-compatible LLMs.
+ """
+
+ def __init__(self, api_key: Optional[str] = None, base_url: Optional[str] = None, model: str = "gpt-4o"):
+ self.api_key = api_key or os.getenv("API_KEY")
+ self.base_url = base_url or os.getenv("BASE_URL") or "https://api.openai.com/v1"
+ self.model = model
+
+ if not self.api_key:
+ raise ValueError("API Key is missing. Please provide it in init or set API_KEY environment variable.")
+
+ self.client = OpenAI(api_key=self.api_key, base_url=self.base_url)
+
+ def _get_llm_response(self, messages: List[dict]) -> str:
+ """Helper to call LLM and get string content."""
+ try:
+ response = self.client.chat.completions.create(
+ model=self.model,
+ messages=messages
+ )
+ return response.choices[0].message.content
+ except Exception as e:
+ logger.error(f"LLM Call Failed: {e}")
+ raise
+
+ def create_from_trajectory(self, trajectory: str, output_dir: str = ".") -> List[str]:
+ """
+ Main entry point: Analyze trajectory and create skill files.
+
+ Args:
+ trajectory: The string content of the user's action log/trajectory.
+ output_dir: The directory where skills should be saved.
+
+ Returns:
+ List of paths to the created skill directories.
+ """
+ logger.info("Step 1: Analyzing trajectory to identify skills...")
+
+ # 1. Create Metadata
+ meta_messages = [
+ {"role": "system", "content": CANDIDATE_METADATA_SYSTEM_PROMPT},
+ {"role": "user", "content": CANDIDATE_METADATA_USER_PROMPT_TEMPLATE.format(trajectory=trajectory)}
+ ]
+
+ raw_meta_response = self._get_llm_response(meta_messages)
+ candidates = self._parse_candidate_metadata(raw_meta_response)
+
+ if not candidates:
+ logger.warning("No skills identified in the trajectory.")
+ return []
+
+ created_paths = []
+
+ # 2. Create Content for each candidate
+ for cand in candidates:
+ name = cand.get("name")
+ description = cand.get("description")
+ logger.info(f"Creating content for skill: {name}...")
+
+ content_messages = [
+ {"role": "system", "content": SKILL_CONTENT_SYSTEM_PROMPT},
+ {"role": "user", "content": SKILL_CONTENT_USER_PROMPT_TEMPLATE.format(
+ trajectory=trajectory, name=name, description=description
+ )}
+ ]
+
+ raw_content_response = self._get_llm_response(content_messages)
+
+ # 3. Parse and Save Files
+ self._save_skill_files(raw_content_response, output_dir)
+ created_paths.append(os.path.join(output_dir, name))
+
+ return created_paths
+
+ def _parse_candidate_metadata(self, llm_output: str) -> List[dict]:
+ """Extract JSON from the LLM output tags."""
+ try:
+ # Look for content between <Skill_Candidate_Metadata> tags
+ if "<Skill_Candidate_Metadata>" in llm_output:
+ json_str = llm_output.split("<Skill_Candidate_Metadata>")[1].split("</Skill_Candidate_Metadata>")[0]
+ else:
+ # Fallback: try to find the first JSON list block
+ json_str = llm_output
+
+ # clean markdown code blocks if present
+ json_str = json_str.replace("```json", "").replace("```", "").strip()
+ return json.loads(json_str)
+ except Exception as e:
+ logger.error(f"Failed to parse metadata JSON: {e}")
+ return []
+
+ def _save_skill_files(self, llm_output: str, output_base_dir: str):
+ """Parse the FILE blocks and write them to disk."""
+ # Regex to find: ## FILE: path \n ```lang \n content \n ```
+ pattern = re.compile(r'##\s*FILE:\s*(.+?)\s*\n```(?:\w*)\n(.*?)```', re.DOTALL)
+ matches = pattern.findall(llm_output)
+
+ if not matches:
+ logger.warning("No file blocks found in LLM output.")
+ return
+
+ for file_path, content in matches:
+ file_path = file_path.strip()
+ full_path = os.path.join(output_base_dir, file_path)
+
+ # Create directory if missing
+ os.makedirs(os.path.dirname(full_path), exist_ok=True)
+
+ try:
+ with open(full_path, 'w', encoding='utf-8') as f:
+ f.write(content)
+ logger.info(f"Saved: {full_path}")
+ except IOError as e:
+ logger.error(f"Failed to write {full_path}: {e}")
\ No newline at end of file
diff --git a/skillnet-ai/src/skillnet_ai/downloader.py b/skillnet-ai/src/skillnet_ai/downloader.py
new file mode 100644
index 0000000..14232bf
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/downloader.py
@@ -0,0 +1,145 @@
+import requests
+import os
+import logging
+from typing import Optional, List, Dict
+
+logger = logging.getLogger(__name__)
+
+class SkillDownloader:
+ """
+ A class to handle downloading specific subdirectories from GitHub repositories
+ and installing them as local skills.
+ """
+
+ def __init__(self, api_token: Optional[str] = None):
+ """
+ Initialize the downloader.
+
+ Args:
+ api_token: Optional GitHub Personal Access Token to avoid rate limits
+ and access private repositories.
+ """
+ self.api_token = api_token
+ self.session = requests.Session()
+ self.session.headers.update({
+ "Accept": "application/vnd.github.v3+json"
+ })
+ if self.api_token:
+ self.session.headers.update({"Authorization": f"token {self.api_token}"})
+
+ def download(self, folder_url: str, target_dir: str = ".") -> Optional[str]:
+ """
+ Download a specific GitHub subdirectory to the target directory.
+
+ Args:
+ folder_url: The full URL to the GitHub folder.
+ target_dir: The local directory to install the skill into.
+
+ Returns:
+ The absolute path to the installed skill directory, or None if failed.
+ """
+ try:
+ # 1. Parse the URL
+ parsed_info = self._parse_github_url(folder_url)
+ if not parsed_info:
+ return None
+
+ owner, repo, ref, dir_path, folder_name = parsed_info
+ logger.info(f"Parsing: {owner}/{repo} @ {ref} -> {dir_path}")
+
+ # 2. Get file tree
+ files_to_download = self._get_file_tree(owner, repo, ref, dir_path)
+ if not files_to_download:
+ logger.warning("No matching files found or API error.")
+ return None
+
+ logger.info(f"Found {len(files_to_download)} files, starting download...")
+
+ # 3. Download files
+ success_count = 0
+ for file_info in files_to_download:
+ if self._download_single_file(owner, repo, ref, dir_path, file_info, folder_name, target_dir):
+ success_count += 1
+
+ if success_count == 0:
+ logger.error("Failed to download any files.")
+ return None
+
+ final_path = os.path.abspath(os.path.join(target_dir, folder_name))
+ logger.info(f"βœ… Skill installed successfully at: {final_path}")
+ return final_path
+
+ except Exception as e:
+ logger.error(f"Critical error during installation: {e}")
+ return None
+
+ def _parse_github_url(self, url: str) -> Optional[tuple]:
+ """
+ Parses the GitHub URL into components.
+ Expected format: https://github.com/owner/repo/tree/ref/path/to/dir
+ """
+ parts = url.rstrip('/').split('/')
+ if len(parts) < 7:
+ logger.error(f"Invalid GitHub URL format: {url}")
+ return None
+
+ # parts[2] is usually 'github.com'
+ owner, repo = parts[3], parts[4]
+ ref = parts[6] # branch or commit hash
+ dir_path = "/".join(parts[7:])
+ folder_name = parts[-1]
+
+ return owner, repo, ref, dir_path, folder_name
+
+ def _get_file_tree(self, owner: str, repo: str, ref: str, dir_path: str) -> List[Dict]:
+ """
+ Fetches the recursive file tree from GitHub API and filters for the target directory.
+ """
+ api_url = f"https://api.github.com/repos/{owner}/{repo}/git/trees/{ref}?recursive=1"
+
+ response = self.session.get(api_url)
+ if response.status_code != 200:
+ logger.error(f"GitHub API error: {response.status_code} - {response.text}")
+ return []
+
+ tree = response.json().get('tree', [])
+
+ # Filter files that are inside the target dir_path and are blobs (files, not folders)
+ return [
+ item for item in tree
+ if item['path'].startswith(dir_path) and item['type'] == 'blob'
+ ]
+
+ def _download_single_file(self, owner: str, repo: str, ref: str, dir_path: str, file_info: Dict, folder_name: str, target_dir: str) -> bool:
+ """
+ Downloads a single file from raw.githubusercontent.com.
+ """
+ # Construct raw URL
+ # Note: Handling private repos might require using the API content endpoint instead of raw.githubusercontent
+ raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{ref}/{file_info['path']}"
+
+ # Calculate local path
+ relative_path = file_info['path'].replace(f"{dir_path}/", "")
+ # Remove leading slash if present to avoid path join issues
+ relative_path = relative_path.lstrip('/')
+
+ local_file_path = os.path.join(target_dir, folder_name, relative_path)
+
+ # Create parent directories
+ os.makedirs(os.path.dirname(local_file_path), exist_ok=True)
+
+ try:
+ # If it's a private repo, raw.githubusercontent might need token in header
+ # The session already has the token if provided.
+ file_resp = self.session.get(raw_url)
+
+ if file_resp.status_code == 200:
+ with open(local_file_path, "wb") as f:
+ f.write(file_resp.content)
+ return True
+ else:
+ logger.warning(f"Failed to download {raw_url}: {file_resp.status_code}")
+ return False
+ except Exception as e:
+ logger.error(f"Exception downloading {raw_url}: {e}")
+ return False
\ No newline at end of file
diff --git a/skillnet-ai/src/skillnet_ai/evaluator.py b/skillnet-ai/src/skillnet_ai/evaluator.py
new file mode 100644
index 0000000..36c6447
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/evaluator.py
@@ -0,0 +1,329 @@
+import json
+import logging
+import os
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from dataclasses import dataclass
+from typing import Dict, Any, List, Optional
+
+from openai import OpenAI
+from tqdm import tqdm
+
+from skillnet_ai.downloader import SkillDownloader
+from skillnet_ai.prompts import SKILL_EVALUATION_PROMPT
+
+logger = logging.getLogger(__name__)
+
+
+# ==========================================================================
+# Configuration and data models
+# ==========================================================================
+
+@dataclass
+class EvaluatorConfig:
+ """Configuration for the skill evaluator."""
+ api_key: str
+ base_url: str = "https://api.openai.com/v1"
+ model: str = "gpt-4o"
+ cache_dir: str = ".skill_cache"
+ max_workers: int = 5
+ temperature: float = 0.3
+
+
+@dataclass
+class Skill:
+ """Unified representation of a skill."""
+ path: str # Local path to the skill root directory
+ name: str
+ description: Optional[str] = None
+ category: Optional[str] = None
+ url: Optional[str] = None # Original URL (when created from URL)
+
+ @classmethod
+ def from_url(cls, url: str, downloader: 'SkillDownloader',
+ cache_dir: str, **kwargs) -> 'Skill':
+ """Create a Skill from a GitHub URL."""
+ normalized_url = cls._normalize_url(url)
+ if not normalized_url:
+ raise ValueError(f"Invalid GitHub URL: {url}")
+ # Download to local cache
+ local_path = downloader.download(normalized_url, target_dir=cache_dir)
+ if not local_path:
+ raise RuntimeError(f"Failed to download: {url}")
+ # Derive skill name from URL if not provided
+ name = kwargs.get('name') or normalized_url.rstrip('/').split('/')[-1]
+
+ return cls(
+ path=local_path,
+ name=name,
+ url=url,
+ description=kwargs.get('description'),
+ category=kwargs.get('category')
+ )
+
+ @classmethod
+ def from_path(cls, path: str, **kwargs) -> 'Skill':
+ """Create a Skill from a local directory path."""
+ abs_path = os.path.abspath(path)
+ if not os.path.isdir(abs_path):
+ raise ValueError(f"Invalid skill path: {path}")
+
+ name = kwargs.get('name') or os.path.basename(abs_path)
+
+ return cls(
+ path=abs_path,
+ name=name,
+ description=kwargs.get('description'),
+ category=kwargs.get('category')
+ )
+
+ @staticmethod
+ def _normalize_url(url: str) -> Optional[str]:
+ """Normalize GitHub URL to /tree/ format."""
+ if not url:
+ return None
+ if "/blob/" in url:
+ return url.replace("/blob/", "/tree/")
+ if "/tree/" in url:
+ return url
+ return None
+
+
+# ==========================================================================
+# Skill content loader
+# ==========================================================================
+
+class SkillLoader:
+ """Load SKILL.md and script files for a skill."""
+
+ @staticmethod
+ def load_skill_md(skill_dir: str, max_chars: int = 12000) -> Optional[str]:
+ """Load SKILL.md content with optional truncation."""
+ path = SkillLoader._find_file(skill_dir, "skill.md")
+ if not path:
+ logger.warning(f"SKILL.md not found in {skill_dir}")
+ return None
+
+ with open(path, 'r', encoding='utf-8', errors='ignore') as f:
+ content = f.read()
+
+ if len(content) > max_chars:
+ content = content[:max_chars] + "\n\n...[truncated]..."
+
+ return content
+
+ @staticmethod
+ def load_scripts(skill_dir: str, max_files: int = 5,
+ max_chars: int = 1200) -> List[Dict[str, str]]:
+ """Load a sample of files under the scripts directory."""
+ scripts = []
+
+ for root, _, files in os.walk(skill_dir):
+ if "scripts" not in root.split(os.sep):
+ continue
+
+ for filename in files:
+ if len(scripts) >= max_files:
+ return scripts
+
+ filepath = os.path.join(root, filename)
+ rel_path = os.path.relpath(filepath, skill_dir)
+
+ try:
+ with open(filepath, 'r', encoding='utf-8', errors='ignore') as f:
+ content = f.read(max_chars)
+ scripts.append({"path": rel_path, "content": content})
+ except Exception as e:
+ logger.warning(f"Skip {filepath}: {e}")
+
+ return scripts
+
+ @staticmethod
+ def _find_file(directory: str, filename: str) -> Optional[str]:
+ """Recursively find a file in directory (case-insensitive)."""
+ for root, _, files in os.walk(directory):
+ for f in files:
+ if f.lower() == filename.lower():
+ return os.path.join(root, f)
+ return None
+
+
+# ==========================================================================
+# Prompt builder
+# ==========================================================================
+
+class PromptBuilder:
+ """Build prompts for skill evaluation."""
+
+ @staticmethod
+ def build(skill: Skill, skill_md: Optional[str],
+ scripts: List[Dict[str, str]]) -> str:
+ """Build the evaluation prompt for a given skill."""
+ skill_md_block = skill_md or "[SKILL.md not found]"
+
+ if scripts:
+ scripts_block = "\n".join([
+ f"# {s['path']}\n{s['content']}\n" for s in scripts
+ ])
+ else:
+ scripts_block = "[No scripts found]"
+
+ return SKILL_EVALUATION_PROMPT.format(
+ skill_name=skill.name,
+ skill_description=skill.description or "N/A",
+ category=skill.category or "N/A",
+ repo_name="N/A",
+ author="N/A",
+ skill_md_block=skill_md_block,
+ scripts_block=scripts_block
+ )
+
+
+# ==========================================================================
+# LLM client
+# ==========================================================================
+
+class LLMClient:
+ """Thin wrapper around the OpenAI client for evaluation calls."""
+
+ def __init__(self, config: EvaluatorConfig):
+ self.client = OpenAI(api_key=config.api_key, base_url=config.base_url)
+ self.model = config.model
+ self.temperature = config.temperature
+
+ def evaluate(self, prompt: str) -> Dict[str, Any]:
+ """Call the LLM with the given prompt and parse JSON response."""
+ messages = [
+ {
+ "role": "system",
+ "content": (
+ "You are an expert evaluator of AI Agent Skills. "
+ "Follow the JSON schema and constraints exactly. "
+ "Use ONLY the provided metadata, SKILL.md, and scripts snippets."
+ )
+ },
+ {"role": "user", "content": prompt}
+ ]
+
+ try:
+ response = self.client.chat.completions.create(
+ model=self.model,
+ messages=messages,
+ response_format={"type": "json_object"},
+ temperature=self.temperature
+ )
+ raw_response = response.choices[0].message.content
+ return json.loads(raw_response)
+ except Exception as e:
+ logger.error(f"LLM call failed: {e}")
+ raise
+
+
+# ==========================================================================
+# Core evaluator
+# ==========================================================================
+
+class SkillEvaluator:
+ """
+ Main entry point for evaluating AI skills.
+
+ Typical usage:
+ config = EvaluatorConfig(api_key="your-key")
+ evaluator = SkillEvaluator(config)
+
+ # Single skill from URL
+ skill = Skill.from_url("https://github.com/.../skill", evaluator.downloader, config.cache_dir)
+ result = evaluator.evaluate(skill)
+
+ # Single skill from local path
+ skill = Skill.from_path("/path/to/skill")
+ result = evaluator.evaluate(skill)
+
+ # Batch evaluation
+ skills = [skill1, skill2, skill3]
+ results = evaluator.evaluate_batch(skills)
+ """
+
+ def __init__(self, config: EvaluatorConfig):
+ """Initialize the evaluator with configuration."""
+ if not config.api_key:
+ raise ValueError("API key is required")
+
+ self.config = config
+ self.downloader = SkillDownloader()
+ self.loader = SkillLoader()
+ self.prompt_builder = PromptBuilder()
+ self.llm_client = LLMClient(config)
+
+ def evaluate(self, skill: Skill) -> Dict[str, Any]:
+ """
+ Evaluate a single skill.
+
+ Args:
+ skill: A Skill instance to evaluate.
+
+ Returns:
+ A dict containing the evaluation result.
+ """
+ try:
+ # Load content
+ skill_md = self.loader.load_skill_md(skill.path)
+ scripts = self.loader.load_scripts(skill.path)
+
+ # Build prompt
+ prompt = self.prompt_builder.build(skill, skill_md, scripts)
+
+ # Call LLM
+ return self.llm_client.evaluate(prompt)
+
+ except Exception as e:
+ logger.error(f"Evaluation failed for {skill.name}: {e}")
+ return self._create_error_result(str(e))
+
+ def evaluate_batch(self, skills: List[Skill]) -> List[Dict[str, Any]]:
+ """
+ Evaluate multiple skills in parallel.
+
+ Args:
+ skills: List of Skill objects.
+
+ Returns:
+ List of evaluation results in the same order as input.
+ """
+ results = [None] * len(skills)
+
+ with ThreadPoolExecutor(max_workers=self.config.max_workers) as executor:
+ future_to_idx = {
+ executor.submit(self.evaluate, skill): idx
+ for idx, skill in enumerate(skills)
+ }
+
+ with tqdm(total=len(skills), desc="Evaluating skills") as pbar:
+ for future in as_completed(future_to_idx):
+ idx = future_to_idx[future]
+ results[idx] = future.result()
+ pbar.update(1)
+
+ return results
+
+ def evaluate_from_url(self, url: str, **kwargs) -> Dict[str, Any]:
+ """Convenience helper: create and evaluate a skill from a URL."""
+ skill = Skill.from_url(url, self.downloader, self.config.cache_dir, **kwargs)
+ return self.evaluate(skill)
+
+ def evaluate_from_path(self, path: str, **kwargs) -> Dict[str, Any]:
+ """Convenience helper: create and evaluate a skill from a local path."""
+ skill = Skill.from_path(path, **kwargs)
+ return self.evaluate(skill)
+
+ @staticmethod
+ def _create_error_result(error_msg: str) -> Dict[str, Any]:
+ """Create a default error-shaped evaluation result."""
+ error_item = {"level": "Poor", "reason": f"Evaluation failed: {error_msg}"}
+ return {
+ "error": error_msg,
+ "safety": error_item,
+ "completeness": error_item,
+ "executability": error_item,
+ "modifiability": error_item,
+ "cost_awareness": error_item
+ }
\ No newline at end of file
diff --git a/skillnet-ai/src/skillnet_ai/models.py b/skillnet-ai/src/skillnet_ai/models.py
new file mode 100644
index 0000000..827d71f
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/models.py
@@ -0,0 +1,41 @@
+from typing import Optional, List, Literal
+from pydantic import BaseModel, Field
+
+class SkillModel(BaseModel):
+ """Represents a Skill object returned from the search API."""
+ skill_name: str
+ skill_description: Optional[str] = None
+ author: Optional[str] = None
+ stars: int = 0
+ skill_url: Optional[str] = None
+ category: Optional[str] = None
+
+class MetaModel(BaseModel):
+ """
+ Pagination and query metadata.
+ Contains fields for both Keyword and Vector search modes.
+ """
+ # Common fields
+ query: Optional[str] = None
+ search_mode: str = "keyword"
+ category: Optional[str] = None
+ limit: int = 20
+ total: int = 0
+
+ # Keyword mode specific fields (may be None in Vector mode)
+ page: Optional[int] = None
+ min_stars: Optional[int] = None
+ sort_by: Optional[str] = None
+ sort_order: Optional[str] = None
+
+ # Vector mode specific fields (may be None in Keyword mode)
+ threshold: Optional[float] = None
+
+ class Config:
+ extra = "ignore"
+
+class SearchResponse(BaseModel):
+ """Wrapper for the search API response."""
+ data: List[SkillModel]
+ meta: MetaModel
+ success: bool
\ No newline at end of file
diff --git a/skillnet-ai/src/skillnet_ai/prompts.py b/skillnet-ai/src/skillnet_ai/prompts.py
new file mode 100644
index 0000000..da33842
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/prompts.py
@@ -0,0 +1,236 @@
+"""
+This module contains the prompt templates used for generating skills from trajectories.
+"""
+
+# Prompt to extract metadata (candidates) from a raw trajectory
+CANDIDATE_METADATA_SYSTEM_PROMPT = "You are a helpful assistant."
+
+CANDIDATE_METADATA_USER_PROMPT_TEMPLATE = """
+Your goal is to analyze an interaction trajectory and extract **reusable Skills**.
+
+A "Skill" is a modular, self-contained package that extends the agent's capabilities (e.g., "PDF Processor", "Market Analyzer", "Code Reviewer").
+
+# Core Objective
+1. Analyze the trajectory to identify distinct **capabilities** or **workflows**.
+2. For EACH distinct capability, extract exactly ONE corresponding **Skill Metadata** entry.
+
+*Note: Avoid over-fragmentation. If the trajectory is a coherent workflow (e.g., "analyze PDF and summarize"), create ONE skill for the whole process rather than splitting it into tiny steps, unless the steps are distinct independent domains.*
+
+# Input Data
+**Execution Trajectory:**
+{trajectory}
+
+# Step 1: Skill Identification
+Identify skills that are:
+- **Reusable**: Useful for future, similar requests.
+- **Modular**: Self-contained with clear inputs and outputs.
+- **Domain Specific**: Provides specialized knowledge or workflow logic.
+
+# Step 2: Metadata Extraction Rules
+For EACH identified skill, generate metadata with:
+
+### `name` requirements:
+- **kebab-case** (e.g., `financial-report-generator`, `code-refactor-tool`).
+- Concise but descriptive.
+
+### `description` requirements (CRITICAL):
+This description acts as the **Trigger** for the AI to know WHEN to use this skill.
+It must be a **When-To-Use** statement containing:
+1. **Context**: The specific situation or user intent (e.g., "When the user asks to analyze a PDF...").
+2. **Capabilities**: What the skill provides (e.g., "...extracts text and summarizes financial metrics").
+3. **Triggers**: Specific keywords or file types associated with this skill.
+
+# Output Format:
+[
+ {{
+ "name": "example-skill-name",
+ "description": "Comprehensive trigger description explaining precisely WHEN and WHY to load this skill."
+ }},
+ ...
+]
+
+Keep your output in the format below:
+<Skill_Candidate_Metadata>
+your generated candidate metadata list in JSON format here
+</Skill_Candidate_Metadata>
+"""
+
+# Prompt to generate actual file content for a specific skill
+SKILL_CONTENT_SYSTEM_PROMPT = "You are an expert Technical Writer specializing in creating SKILL for AI agents."
+
+SKILL_CONTENT_USER_PROMPT_TEMPLATE = """
+Your task is to generate a **skill package** based on the provided execution trajectory, skill name, and skill description.
+This includes the main `SKILL.md` orchestration file and any necessary bundled resources (scripts, references, assets).
+
+# Input Data
+1. **Trajectory:** {trajectory}
+2. **Skill Name:** {name}
+3. **Skill Description:** {description}
+
+# Skill Structure Standard
+You must output the skill using the following directory structure:
+
+```text
+skill-name/
+β”œβ”€β”€ SKILL.md (required)
+β”‚ β”œβ”€β”€ YAML frontmatter metadata (required)
+β”‚ β”‚ β”œβ”€β”€ name: (required)
+β”‚ β”‚ └── description: (required)
+β”‚ └── Markdown instructions (required)
+└── Bundled Resources (optional)
+ β”œβ”€β”€ scripts/ - Executable code (Python/Bash/etc.)
+ β”œβ”€β”€ references/ - Documentation intended to be loaded into context as needed
+ └── assets/ - Files used in output (templates, icons, fonts, etc.)
+```
+
+# Core Design Principles
+1. Context is a Public Good: Be concise. Only add context in SKILL.md that is essential.
+2. Progressive Disclosure:
+- Keep SKILL.md lean.
+- Offload heavy documentation/schemas to references/.
+- Offload repeatable, deterministic logic to scripts/.
+3. Degrees of Freedom:
+- Use scripts (Low Freedom) for fragile, error-prone, or strict sequence tasks found in the trajectory.
+- Use text instructions (High Freedom) for creative decisions.
+
+# Output Format (STRICT)
+You must output the files using the following strict format so that a script can parse and save them.
+For every file (including SKILL.md, scripts, references, etc.), use this exact pattern:
+
+## FILE: <directory_name>/<path_to_file>
+```<language_tag_if_applicable>
+<file_content_here>
+```
+
+**Example Output:**
+
+## FILE: pdf-processor/SKILL.md
+```yaml
+---
+name: pdf-processor
+description: Extracts text from PDFs and summarizes them.
+---
+# Instructions
+1. Run the extraction script.
+2. Summarize the output.
+```
+
+## FILE: pdf-processor/scripts/extract.py
+```python
+import pdfplumber
+# ... code ...
+```
+
+## FILE: pdf-processor/references/api_docs.md
+```markdown
+# API Documentation
+...
+```
+
+Now, generate the complete skill package based on the provided trajectory, name, and description.
+"""
+
+
+SKILL_EVALUATION_PROMPT = """You will evaluate an AI Agent Skill using the metadata and (if present) the SKILL.md content and scripts snippets below.
+- Judge clarity, safety, and practical usability based on what is explicitly stated.
+- If information is missing, reflect that in the rating and explain the uncertainty in the reason.
+
+=== METADATA ===
+- Name: {skill_name}
+- Description: {skill_description}
+- Category: {category}
+
+=== SKILL.md ===
+{skill_md_block}
+
+=== SCRIPTS/ ===
+{scripts_block}
+
+Rating scale (3 levels only):
+- Good: Clear, safe-by-default, and practically usable as described; low ambiguity; low risk.
+- Average: Some ambiguity, missing caveats, or limited detail; likely usable but with notable gaps/risks.
+- Poor: Vague, risky, misleading, or missing critical information; likely unsafe or impractical as described.
+
+Evaluation dimensions and how to judge them:
+
+1) Safety
+ Assess potential harm and misuse given what is described.
+ Signals for Good:
+ - Avoids destructive actions by default; includes safety checks/confirmations; mentions scope limits.
+ - Does not encourage credential exfiltration, system damage, bypassing security, or unsafe automation.
+ Signals for Average:
+ - Benign domain, but no mention of safeguards for operations that could be risky (e.g., deployments, deletions).
+ Signals for Poor:
+ - Mentions or implies dangerous actions without safeguards (e.g., "delete/reset/disable security/steal/exploit").
+ - Encourages unsafe tool usage or bypassing best practices.
+
+2) Completeness
+ Assess whether the description covers the essential steps/constraints to achieve its stated goal.
+ Signals for Good:
+ - Clear goal + clear steps/inputs/outputs; mentions prerequisites (env, packages, permissions) when relevant.
+ - Mentions failure modes or edge cases at least briefly when they matter.
+ Signals for Average:
+ - Goal is clear, but steps/prereqs/outputs are underspecified; assumes context the user may not have.
+ Signals for Poor:
+ - Too vague to act on; missing core steps; unclear what "done" looks like.
+
+3) Executability
+ Assess whether an agent could realistically execute the described workflow with typical tools.
+ Signals for Good:
+ - Concrete actions and artifacts (commands, files, parameters); minimal ambiguity.
+ - Avoids "hand-wavy" steps like "just configure X" without specifying how/where.
+ Signals for Average:
+ - Generally executable, but contains ambiguous steps or missing tool/environment assumptions.
+ Signals for Poor:
+ - Non-actionable ("optimize it", "make it work") with no operational detail; depends on unspecified systems.
+
+4) Modifiability
+ Assess how easy it would be to adjust/reuse/compose this Skill as described.
+ Signals for Good:
+ - Narrow, modular scope; clearly defined inputs/outputs; low coupling; safe to combine with other Skills.
+ - Mentions configuration points or parameters rather than hard-coding assumptions.
+ Signals for Average:
+ - Some reusable parts, but unclear boundaries or assumptions; moderate coupling to a specific repo/tooling.
+ Signals for Poor:
+ - Overly broad or tightly coupled; unclear how to adapt; likely to conflict with other workflows.
+
+5) Cost-awareness
+ Assess whether the described approach is mindful of time/compute/money and operational overhead, given its domain.
+ For clearly lightweight domains (e.g., documentation, brainstorming, simple text-only workflows) with no heavy data/infra hints:
+ - Good: The task is inherently low-cost and the description does not suggest heavy loops, huge datasets, or expensive external services.
+ For potentially heavy domains (e.g., data processing, infra, large-scale agents, external APIs, long-running jobs):
+ - Good: Explicitly mentions batching/limits/caching/scope control or otherwise shows cost awareness.
+ - Average: No explicit cost control is mentioned, but nothing suggests obviously wasteful behavior.
+ - Poor: Encourages wasteful or unrealistic workflows without acknowledging cost/limits (e.g., "run huge jobs repeatedly", "scan all repos constantly").
+
+Output requirements:
+- Return STRICT JSON only (no prose, no markdown).
+- Use exactly these keys: safety, completeness, executability, modifiability, cost_awareness.
+- Each key must contain: level (Good/Average/Poor) and reason (1-2 sentences).
+- The reason must cite concrete evidence from the provided content (metadata/SKILL.md/scripts), not imagined details.
+
+Return the evaluation results in JSON format exactly like this:
+{{
+ "safety": {{
+ "level": "Good/Average/Poor",
+ "reason": "Reason for the rating (1-2 sentences)"
+ }},
+ "completeness": {{
+ "level": "Good/Average/Poor",
+ "reason": "Reason for the rating (1-2 sentences)"
+ }},
+ "executability": {{
+ "level": "Good/Average/Poor",
+ "reason": "Reason for the rating (1-2 sentences)"
+ }},
+ "modifiability": {{
+ "level": "Good/Average/Poor",
+ "reason": "Reason for the rating (1-2 sentences)"
+ }},
+ "cost_awareness": {{
+ "level": "Good/Average/Poor",
+ "reason": "Reason for the rating (1-2 sentences)"
+ }}
+}}
+
+Remember: STRICT JSON only."""
\ No newline at end of file
diff --git a/skillnet-ai/src/skillnet_ai/searcher.py b/skillnet-ai/src/skillnet_ai/searcher.py
new file mode 100644
index 0000000..dfd0013
--- /dev/null
+++ b/skillnet-ai/src/skillnet_ai/searcher.py
@@ -0,0 +1,100 @@
+import requests
+import logging
+from typing import Optional, List, Dict, Any, Literal
+from skillnet_ai.models import SearchResponse
+
+# Configure logger
+logger = logging.getLogger(__name__)
+
+class SkillNetSearcher:
+ """
+ Skiil Searcher for interacting with the SkillNet Search API.
+ """
+
+ def __init__(self, skillnet_url: str = "https://skillnet.openkg.cn"):
+ self.skillnet_url = skillnet_url.rstrip("/")
+ self.session = requests.Session()
+ self.session.headers.update({
+ "User-Agent": "SkillNet-Python-SDK/0.1.0"
+ })
+
+ def search(
+ self,
+ q: str,
+ mode: Literal["keyword", "vector"] = "keyword",
+ category: Optional[str] = None,
+ limit: int = 20,
+ # --- Keyword mode ---
+ page: int = 1,
+ min_stars: int = 0,
+ sort_by: str = "stars",
+ # --- Vector mode ---
+ threshold: float = 0.8
+ ) -> List[Dict[str, Any]]:
+ """
+ Unified search for skills (supports both Keyword and AI Vector search).
+
+ Args:
+ q: Search query (Keywords or Natural Language).
+ mode: 'keyword' for exact/fuzzy match, 'vector' for semantic AI search.
+ category: Filter by category (e.g., 'Development').
+ limit: Items per page (keyword) or Match count (vector).
+
+ # Keyword specific:
+ page: Page number.
+ min_stars: Minimum stars filter.
+ sort_by: Sort field ('stars' or 'recent').
+
+ # Vector specific:
+ threshold: Similarity threshold (0.0 to 1.0).
+
+ Returns:
+ A list of skill dictionaries.
+ """
+ endpoint = f"{self.skillnet_url}/search"
+
+ # 1. Construct base parameters
+ params = {
+ "q": q,
+ "mode": mode,
+ "category": category,
+ "limit": limit
+ }
+
+ # 2. Add mode-specific parameters
+ if mode == "keyword":
+ params.update({
+ "page": page,
+ "min_stars": min_stars,
+ "sort_by": sort_by,
+ })
+ elif mode == "vector":
+ params.update({
+ "threshold": threshold
+ })
+
+ # 3. Clean None values
+ params = {k: v for k, v in params.items() if v is not None}
+
+ try:
+ logger.debug(f"Searching skills ({mode} mode) with params: {params}")
+
+ # Make the request
+ response = self.session.get(endpoint, params=params, timeout=15)
+ response.raise_for_status()
+
+ # Parse response
+ search_res = SearchResponse(**response.json())
+
+ if not search_res.success:
+ logger.warning(f"Search API returned success=False")
+ return []
+
+ return search_res.data
+
+ except requests.exceptions.RequestException as e:
+ logger.error(f"Search request failed: {e}")
+ raise
+ except Exception as e:
+ logger.error(f"Data parsing failed: {e}")
+ raise
\ No newline at end of file