Skip to content

Create, Evaluate, and Connect AI Skills

Update prompts.py

Xu Haoming4b5a01d4298b
diff --git a/skillnet-ai/src/skillnet_ai/prompts.py b/skillnet-ai/src/skillnet_ai/prompts.py
index 88be845..92ea868 100644
--- a/skillnet-ai/src/skillnet_ai/prompts.py
+++ b/skillnet-ai/src/skillnet_ai/prompts.py
@@ -203,6 +203,7 @@ Evaluation dimensions and how to judge them (apply these rules even if the overa
Additional guidance for Executability:
- **Do NOT rate Poor solely because "No runnable python scripts found"**. Many skills (security guidelines, ideation, policies, design workflows) are instruction-only: the agent reads SKILL.md and follows the guidance with typical tools. For such skills, if the instructions are clear and actionable, executability should be Good.
- If script execution fails due to an obvious documentation placeholder in an example command (e.g., tokens like "[options]", "<file>", "<pattern>", "{{path}}") or an argument parsing error caused by such placeholders, do NOT automatically set executability to Poor. Prefer Average and explain that the script likely needs real inputs or a concrete runnable example; only use Poor if there is additional evidence the workflow is not realistically executable.
+ - If script execution fails due to missing Python dependencies (e.g., errors like "ModuleNotFoundError" or "ImportError" such as "No module named 'X'"), treat this primarily as an unmet prerequisite in the evaluator/runtime environment (which may not support auto-installing dependencies) rather than a flaw in the Skill package itself. Prefer executability = Average and explicitly cite the missing dependency and the need to pre-install it. Do NOT speculate that the Skill's installation instructions are "incorrect" or contain "typos" based on guesswork; only call them incorrect if the provided content explicitly demonstrates a contradiction. Otherwise describe the gap as an environment/setup prerequisite.
- If you detect any CRITICAL CORRECTNESS ERROR in a core formula, algorithm, or code snippet (e.g., Python using "^" for exponentiation or other language-level mistakes that would produce wrong results or runtime failures), executability MUST be "Poor".
- If allowed_tools grants broader permissions than what the Skill clearly needs (e.g., allows "bash" or other powerful tools but the described workflow and examples do not require them), reduce executability by at least one level due to environment/permission ambiguity.
- When reading formulas and code snippets, audit them line-by-line in the context of their target language and typical runtime environment; if you find subtle traps or inconsistencies that would mislead an implementer or cause incorrect behavior, choose a lower (more conservative) executability rating.