Skip to content

Create, Evaluate, and Connect AI Skills

14.6 KB
Raw Blame
{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "958ee3e3",
   "metadata": {},
   "source": [
    "# SkillNet AI SDK Usage Tutorial\n",
    "\n",
    "This notebook demonstrates the process of using the `skillnet-ai` Python SDK in a general scenario.\n",
    "Key features include:\n",
    "1. **Search**: Find skills using keyword or semantic search\n",
    "2. **Download**: Install skills from GitHub\n",
    "3. **Create**: Create new skills from conversation logs, documents, or prompts\n",
    "4. **Evaluate**: Evaluate skill quality across multiple dimensions\n",
    "5. **Analyze**: Analyze skill relationships in the local skill library\n",
    "6. **Route**: Select task-relevant skills from an analyzed library\n",
    "\n",
    "## 0. Installation and Environment Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "51a0220c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install skillnet-ai\n",
    "# Run this notebook from the checkout's examples/ directory.\n",
    "%pip install -e \"../skillnet-ai[graph]\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "748acf46",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SkillNetClient initialized successfully.\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from skillnet_ai import SkillNetClient\n",
    "\n",
    "# Use environment variables or settings saved by skillnet configure.\n",
    "# Create, evaluate and analyze use API_KEY, BASE_URL and SKILLNET_MODEL.\n",
    "# Analyze also requires the EMBEDDING_* settings described in section 5.\n",
    "client = SkillNetClient()\n",
    "\n",
    "print(\"SkillNetClient initialized successfully.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f5c5567",
   "metadata": {},
   "source": [
    "## 1. Search Skills (Search)\n",
    "\n",
    "SkillNet supports two search modes:\n",
    "*   **Keyword**: Traditional fuzzy keyword matching\n",
    "*   **Vector**: Semantic vector retrieval"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "cb8079fd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- Keyword Search: 'pdf' ---\n",
      "Name: biorxiv-database (Stars: 17616)\n",
      "Name: treatment-plans (Stars: 16346)\n",
      "Name: clinical-decision-support (Stars: 16091)\n",
      "Name: scientific-visualization (Stars: 16082)\n",
      "Name: literature-review (Stars: 16059)\n",
      "Name: pdf (Stars: 15845)\n",
      "Name: plotly (Stars: 15805)\n",
      "Name: pdf-processing-pro (Stars: 15803)\n",
      "Name: pdf-processing (Stars: 15439)\n",
      "Name: pdf (Stars: 13720)\n",
      "Name: pdf-processing (Stars: 9170)\n",
      "Name: pdf-extractor (Stars: 8003)\n",
      "Name: biorxiv-database (Stars: 6637)\n",
      "Name: treatment-plans (Stars: 6170)\n",
      "Name: clinical-decision-support (Stars: 6069)\n",
      "Name: literature-review (Stars: 6041)\n",
      "Name: etetoolkit (Stars: 5879)\n",
      "Name: paper-2-web (Stars: 5867)\n",
      "Name: pptx-posters (Stars: 5810)\n",
      "Name: markitdown (Stars: 5761)\n",
      "\n",
      "--- Vector Search: 'Help me analyze financial PDF reports' ---\n",
      "Found: financial-document-parser (Stars: 95)\n",
      "URL: https://github.com/Microck/ordinary-claude-skills/blob/76c86cb21a36c0cc10c213d991aa1c9498b363c8/skills_categorized/payment/financial-document-parser\n"
     ]
    }
   ],
   "source": [
    "# 1. Standard Keywords Match\n",
    "print(\"--- Keyword Search: 'pdf' ---\")\n",
    "results = client.search(q=\"pdf\")\n",
    "\n",
    "if results:\n",
    "    for skill in results:\n",
    "        print(f\"Name: {skill.skill_name} (Stars: {skill.stars})\")\n",
    "\n",
    "# 2. Semantic Search\n",
    "print(\"\\n--- Vector Search: 'Help me analyze financial PDF reports' ---\")\n",
    "results = client.search(q=\"Help me analyze financial PDF reports\", mode=\"vector\")\n",
    "\n",
    "if results:\n",
    "    top_skill = results[0]\n",
    "    print(f\"Found: {top_skill.skill_name} (Stars: {top_skill.stars})\")\n",
    "    print(f\"URL: {top_skill.skill_url}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5724546",
   "metadata": {},
   "source": [
    "## 2. Download Skills (Download)\n",
    "\n",
    "You can download skills directly from a GitHub URL to a local directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "c31f3c93",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading skill from https://github.com/anthropics/skills/tree/main/skills/skill-creator...\n",
      "✅ Skill successfully installed at: e:\\科研\\SkillNet相关\\SkillNet\\skillnet-ai\\examples\\my_downloaded_skills\\skill-creator\n",
      "Files in directory: ['LICENSE.txt', 'references', 'scripts', 'SKILL.md']\n"
     ]
    }
   ],
   "source": [
    "# Define download target directory\n",
    "download_dir = \"./my_downloaded_skills\"\n",
    "\n",
    "# Example: Download anthropics' skill-creator skill\n",
    "# Note: This requires network access to GitHub\n",
    "skill_url = \"https://github.com/anthropics/skills/tree/main/skills/skill-creator\"\n",
    "\n",
    "try:\n",
    "    print(f\"Downloading skill from {skill_url}...\")\n",
    "    local_path = client.download(url=skill_url, target_dir=download_dir)\n",
    "    print(f\"✅ Skill successfully installed at: {local_path}\")\n",
    "    \n",
    "    # View downloaded files\n",
    "    if os.path.exists(local_path):\n",
    "        print(\"Files in directory:\", os.listdir(local_path))\n",
    "except Exception as e:\n",
    "    print(f\"❌ Download failed: {e}\")\n",
    "    # Note: If network is unreachable, this step may fail, but won't affect later demonstrations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bb6bd05",
   "metadata": {},
   "source": [
    "## 3. Create Skills (Create)\n",
    "\n",
    "Automatically generate standardized skill packages using LLMs. Supports various sources:\n",
    "*   Conversation Trajectory/Logs\n",
    "*   GitHub Repository\n",
    "*   Office Documents (PDF/Word/PPT)\n",
    "*   Natural Language Prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "e8e86da0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--- Creating skill from trajectory ---\n",
      "Created 1 skills:\n",
      "- ./my_created_skills\\file-format-renamer\n",
      "\n",
      "--- Creating skill from prompt ---\n",
      "- ./my_created_skills\\celsius-to-fahrenheit-converter\n"
     ]
    }
   ],
   "source": [
    "# Set output directory for generated skills\n",
    "created_skills_dir = \"./my_created_skills\"\n",
    "\n",
    "# Example 1: Create from Conversation Log (Trajectory)\n",
    "trajectory_log = \"\"\"\n",
    "User: I need to rename all .jpg files in this folder to .png.\n",
    "Agent: I will write a python script to iterate through the folder...\n",
    "Agent: Script executed. Renamed 5 files.\n",
    "\"\"\"\n",
    "\n",
    "print(\"--- Creating skill from trajectory ---\")\n",
    "try:\n",
    "    # This step requires a valid API_KEY\n",
    "    created_paths = client.create(\n",
    "        trajectory_content=trajectory_log,\n",
    "        output_dir=created_skills_dir\n",
    "    )\n",
    "    print(f\"Created {len(created_paths)} skills:\")\n",
    "    for path in created_paths:\n",
    "        print(f\"- {path}\")\n",
    "except Exception as e:\n",
    "    print(f\"Creation failed (check API Key): {e}\")\n",
    "\n",
    "# Example 2: Create directly from Prompt\n",
    "print(\"\\n--- Creating skill from prompt ---\")\n",
    "try:\n",
    "    created_paths_prompt = client.create(\n",
    "        prompt=\"Create a tool that converts degrees Celsius to Fahrenheit\",\n",
    "        output_dir=created_skills_dir\n",
    "    )\n",
    "    for path in created_paths_prompt:\n",
    "        print(f\"- {path}\")\n",
    "except Exception as e:\n",
    "    print(f\"Creation failed: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a79d93b",
   "metadata": {},
   "source": [
    "## 4. Evaluate Skills (Evaluate)\n",
    "\n",
    "Evaluate skill quality across multiple dimensions (Safety, Completeness, Executability, etc.)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e676694c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Evaluate the skill just downloaded or created\n",
    "# Here we try to evaluate a skill in the download directory (if it exists)\n",
    "import glob\n",
    "\n",
    "# Find an existing skill path to evaluate\n",
    "target_skill = None\n",
    "if os.path.exists(download_dir) and os.listdir(download_dir):\n",
    "    # Get the first subdirectory\n",
    "    subdirs = [f.path for f in os.scandir(download_dir) if f.is_dir()]\n",
    "    if subdirs:\n",
    "        target_skill = subdirs[0]\n",
    "\n",
    "if target_skill:\n",
    "    print(f\"--- Evaluating skill at: {target_skill} ---\")\n",
    "    try:\n",
    "        # This step requires a valid API_KEY\n",
    "        eval_result = client.evaluate(target=target_skill)\n",
    "        print(\"Evaluation Result:\")\n",
    "        print(eval_result)\n",
    "    except Exception as e:\n",
    "        print(f\"Evaluation failed (check API Key): {e}\")\n",
    "else:\n",
    "    print(\"No local skill found to evaluate. Please download or create one first.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe2e1e53",
   "metadata": {},
   "source": [
    "## 5. Skill Relationship Analysis (Analyze)\n",
    "\n",
    "Extract scenarios and capabilities, then build an index containing the relationship\n",
    "graph and Wiki. `compose_with` is directed from predecessor to successor;\n",
    "`similar_to` describes comparable capabilities under their respective constraints.\n",
    "\n",
    "Use the `graph` extra installed above. Configure the analysis model with `API_KEY`,\n",
    "`BASE_URL` and `SKILLNET_MODEL`, and embeddings with `EMBEDDING_API_KEY`,\n",
    "`EMBEDDING_BASE_URL` and `EMBEDDING_MODEL`, or use saved settings.\n",
    "See [endpoint configuration](../skillnet-ai/README.md#analysis-and-routing-endpoints).\n",
    "\n",
    "The independent skills created in section 3 may have no relationships; their index\n",
    "is still usable for routing. See [analyze_example.py](analyze_example.py) for a\n",
    "small library demonstrating scenario-specific handoffs and similar capabilities.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f1513bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Analyze the directory of created skills\n",
    "analyze_target_dir = created_skills_dir\n",
    "analysis = None\n",
    "\n",
    "if os.path.exists(analyze_target_dir) and len(os.listdir(analyze_target_dir)) > 0:\n",
    "    print(f\"--- Analyzing relationships in: {analyze_target_dir} ---\")\n",
    "    try:\n",
    "        analysis = client.analyze(skills_dir=analyze_target_dir)\n",
    "        print(f\"Index: {analysis.index_dir}\")\n",
    "        print(f\"Skills analyzed: {analysis.skill_count}\")\n",
    "        for relation_type, count in analysis.relation_counts.items():\n",
    "            print(f\"{relation_type}: {count}\")\n",
    "        print(f\"Cache hits: {analysis.cache_hits}\")\n",
    "        snapshot_name = (analysis.index_dir / \"CURRENT\").read_text(encoding=\"utf-8\").strip()\n",
    "        print(f\"Full graph: {analysis.index_dir / snapshot_name / 'graph.json'}\")\n",
    "\n",
    "    except Exception as e:\n",
    "        print(f\"Analysis failed: {e}\")\n",
    "else:\n",
    "    print(\"No skills directory found to analyze.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "routing-guide",
   "metadata": {},
   "source": [
    "## 6. Select Skills for a Task (Route)\n",
    "\n",
    "Reuse the index from section 5 to select up to `k` skills. This example makes one\n",
    "route call for the temperature-conversion skill created earlier. Selection order\n",
    "is not execution order; the router may select fewer skills or none.\n",
    "\n",
    "Install **one** Explorer backend in this notebook's Python environment:\n",
    "\n",
    "- Claude: `%pip install -e \"../skillnet-ai[graph,claude]\"`\n",
    "- Codex: `%pip install -e \"../skillnet-ai[graph,codex]\"`\n",
    "\n",
    "Set `SKILLNET_EXPLORER_BACKEND` (`claude` or `codex`, default `claude`),\n",
    "`SKILLNET_EXPLORER_API_KEY`, `SKILLNET_EXPLORER_BASE_URL` and\n",
    "`SKILLNET_EXPLORER_MODEL`, or use saved settings. The endpoint must support the\n",
    "chosen SDK's protocol. Keep the embedding settings from section 5.\n",
    "See [endpoint configuration](../skillnet-ai/README.md#analysis-and-routing-endpoints).\n",
    "\n",
    "Routing calls the embedding and Explorer APIs. It reuses the index without\n",
    "rebuilding it or executing skills; most waiting comes from the SDK's model calls.\n",
    "The cell below displays elapsed time for your configuration. For a multi-step\n",
    "task and an example with existing intermediate inputs, see\n",
    "[route_example.py](route_example.py).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "routing-example",
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import perf_counter\n",
    "\n",
    "if analysis is None:\n",
    "    raise RuntimeError(\"Complete section 5 successfully before routing.\")\n",
    "\n",
    "# Reload settings if you configured the Explorer after initializing the client.\n",
    "client = SkillNetClient()\n",
    "started = perf_counter()\n",
    "result = client.route(\n",
    "    query=\"Convert temperatures from degrees Celsius to Fahrenheit.\",\n",
    "    index_dir=analysis.index_dir,\n",
    "    k=3,\n",
    ")\n",
    "print(f\"Selected {len(result.skills)} skill(s) in {perf_counter() - started:.1f}s\")\n",
    "if not result.skills:\n",
    "    print(\"No suitable skills selected.\")\n",
    "for skill in result.skills:\n",
    "    print(f\"\\n{skill.name} ({skill.skill_id})\")\n",
    "    print(f\"Path: {skill.path}\")\n",
    "    print(f\"Reason: {skill.reason}\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "skillnet",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}