Fully autonomous AI Agents system capable of performing complex penetration testing tasks
Go ai-agentsai-security-toolanthropicautonomous-agentsgolanggptgraphqlmulti-agent-systemoffensive-securityopen-sourceopenaipenetration-testingpenetration-testing-toolsreactsecurity-automationsecurity-testingsecurity-toolsself-hosted
feat(provider): add OpenCode provider configuration and testing report
Dmitry Ng454569a8eb9f
diff --git a/.vscode/launch.json b/.vscode/launch.jsonindex dfa9ebed..becb5cd7 100644--- a/.vscode/launch.json+++ b/.vscode/launch.json@@ -79,6 +79,7 @@// "-config", "${workspaceFolder}/examples/configs/vllm-mixed.provider.yml",// "-config", "${workspaceFolder}/examples/configs/hcnsec.provider.yml",// "-config", "${workspaceFolder}/examples/configs/custom-openai.provider.yml",+ // "-config", "${workspaceFolder}/examples/configs/opencode.provider.yml",// "-config", "${workspaceFolder}/examples/configs/openrouter.provider.yml",// "-config", "${workspaceFolder}/examples/configs/deepinfra.provider.yml",// "-config", "${workspaceFolder}/examples/configs/novita.provider.yml",@@ -104,6 +105,7 @@// "-report", "${workspaceFolder}/examples/tests/minimax-report.md",// "-report", "${workspaceFolder}/examples/tests/hcnsec-report.md",// "-report", "${workspaceFolder}/examples/tests/custom-openai-report.md",+ // "-report", "${workspaceFolder}/examples/tests/opencode-report.md",// "-report", "${workspaceFolder}/examples/tests/openrouter-report.md",// "-report", "${workspaceFolder}/examples/tests/deepinfra-report.md",// "-report", "${workspaceFolder}/examples/tests/novita-report.md",diff --git a/Dockerfile b/Dockerfileindex 8a1894f7..04ae03c2 100644--- a/Dockerfile+++ b/Dockerfile@@ -174,6 +174,7 @@ COPY examples/configs/ollama-llama318b-instruct.provider.yml /opt/pentagi/conf/COPY examples/configs/ollama-llama318b.provider.yml /opt/pentagi/conf/COPY examples/configs/ollama-qwen332b-fp16-tc.provider.yml /opt/pentagi/conf/COPY examples/configs/ollama-qwq32b-fp16-tc.provider.yml /opt/pentagi/conf/+COPY examples/configs/opencode.provider.yml /opt/pentagi/conf/COPY examples/configs/openrouter.provider.yml /opt/pentagi/conf/COPY examples/configs/orcarouter.provider.yml /opt/pentagi/conf/COPY examples/configs/vllm-mixed.provider.yml /opt/pentagi/conf/diff --git a/README.md b/README.mdindex 83775430..74ccf132 100644--- a/README.md+++ b/README.md@@ -75,7 +75,7 @@ You can watch the video **PentAGI overview**:- Persistent Storage. All commands and outputs are stored in PostgreSQL with [pgvector](https://hub.docker.com/r/vxcontrol/pgvector) extension.- Scalable Architecture. Microservices-based design supporting horizontal scaling.- Self-Hosted Solution. Complete control over your deployment and data.-- Flexible Authentication. Support for 10+ LLM providers ([OpenAI](https://platform.openai.com/), [Anthropic](https://www.anthropic.com/), [Google AI/Gemini](https://ai.google.dev/), [AWS Bedrock](https://aws.amazon.com/bedrock/), [Ollama](https://ollama.com/), [DeepSeek](https://www.deepseek.com/en/), [GLM](https://z.ai/), [Kimi](https://platform.moonshot.ai/), [Qwen](https://www.alibabacloud.com/en/), [MiniMax](https://www.minimax.io/), Custom) plus aggregators ([OpenRouter](https://openrouter.ai/), [DeepInfra](https://deepinfra.com/), [Atlas Cloud](https://www.atlascloud.ai/)). For production local deployments, see our [vLLM + Qwen3.5-27B-FP8 guide](examples/guides/vllm-qwen35-27b-fp8.md).+- Flexible Authentication. Support for 10+ LLM providers ([OpenAI](https://platform.openai.com/), [Anthropic](https://www.anthropic.com/), [Google AI/Gemini](https://ai.google.dev/), [AWS Bedrock](https://aws.amazon.com/bedrock/), [Ollama](https://ollama.com/), [DeepSeek](https://www.deepseek.com/en/), [GLM](https://z.ai/), [Kimi](https://platform.moonshot.ai/), [Qwen](https://www.alibabacloud.com/en/), [MiniMax](https://www.minimax.io/), Custom) plus aggregators ([OpenRouter](https://openrouter.ai/), [DeepInfra](https://deepinfra.com/), [Atlas Cloud](https://www.atlascloud.ai/), [OpenCode Go plan](https://opencode.ai/en/go)). For production local deployments, see our [vLLM + Qwen3.5-27B-FP8 guide](examples/guides/vllm-qwen35-27b-fp8.md).- API Token Authentication. Secure Bearer token system for programmatic access to REST and GraphQL APIs.- Quick Deployment. Easy setup through [Docker Compose](https://docs.docker.com/compose/) with comprehensive environment configuration.@@ -3236,14 +3236,14 @@ docker run --rm \#### Using Pre-configured Providers-The Docker image comes with built-in support for major providers (OpenAI, Anthropic, Gemini, Ollama) and pre-configured provider files for additional services (OpenRouter, OrcaRouter, DeepInfra, DeepSeek, Moonshot, Novita, xAI):+The Docker image comes with built-in support for major providers (OpenAI, Anthropic, Gemini, Ollama) and pre-configured provider files for additional services (OpenRouter, OpenCode, Atlas, OrcaRouter, DeepInfra, DeepSeek, Moonshot, Novita, xAI):```bash# Test with OpenRouter configurationdocker exec -it pentagi /opt/pentagi/bin/ctester -config /opt/pentagi/conf/openrouter.provider.yml-# Test with OrcaRouter configuration-docker exec -it pentagi /opt/pentagi/bin/ctester -config /opt/pentagi/conf/orcarouter.provider.yml+# Test with OpenCode Go plan configuration+docker exec -it pentagi /opt/pentagi/bin/ctester -config /opt/pentagi/conf/opencode.provider.yml# Test with DeepInfra configurationdocker exec -it pentagi /opt/pentagi/bin/ctester -config /opt/pentagi/conf/deepinfra.provider.yml@@ -3300,11 +3300,11 @@ docker exec -it pentagi /opt/pentagi/bin/ctester -config /opt/pentagi/conf/ollamTo use these configurations, your `.env` file only needs to contain:```-LLM_SERVER_URL=https://openrouter.ai/api/v1 # or https://api.deepinfra.com/v1/openai or https://api.openai.com/v1 or https://api.novita.ai/openai or https://api.atlascloud.ai/v1 or https://api.orcarouter.ai/v1 or https://api.x.ai/v1+LLM_SERVER_URL=https://openrouter.ai/api/v1 # or https://api.deepinfra.com/v1/openai or https://api.openai.com/v1 or https://opencode.ai/zen/go/v1 or https://api.novita.ai/openai or https://api.atlascloud.ai/v1 or https://api.orcarouter.ai/v1 or https://api.x.ai/v1LLM_SERVER_KEY=your_api_keyLLM_SERVER_MODEL= # Leave empty, as models are specified in the config-LLM_SERVER_CONFIG_PATH=/opt/pentagi/conf/openrouter.provider.yml # or deepinfra.provider.ymll or custom-openai.provider.yml or novita.provider.yml or atlas.provider.yml or orcarouter.provider.yml or xai.provider.yml-LLM_SERVER_PROVIDER= # Provider name for LiteLLM proxy (e.g., openrouter, deepseek, moonshot, novita, orcarouter, xai)+LLM_SERVER_CONFIG_PATH=/opt/pentagi/conf/openrouter.provider.yml # or deepinfra.provider.ymll or opencode.provider.ymll or custom-openai.provider.yml or novita.provider.yml or atlas.provider.yml or orcarouter.provider.yml or xai.provider.yml+LLM_SERVER_PROVIDER= # Provider name for LiteLLM proxy (e.g., openrouter, deepseek, moonshot, novita, opencode, orcarouter, xai)LLM_SERVER_LEGACY_REASONING=false # Controls reasoning format, for OpenAI must be true (default: false)LLM_SERVER_PRESERVE_REASONING=false # Preserve reasoning content in multi-turn conversations (required by Moonshot, default: false)@@ -3423,6 +3423,7 @@ When using LiteLLM proxy, set the corresponding `*_PROVIDER` variable to enable- `moonshot` - for Kimi models (`KIMI_PROVIDER=moonshot` → `moonshot/kimi-k2.5`)- `dashscope` - for Qwen models (`QWEN_PROVIDER=dashscope` → `dashscope/qwen-plus`)- `openai`, `anthropic`, `gemini` - for major cloud providers+- `opencode` - for OpenCode Go plan- `openrouter` - for OpenRouter aggregator- `orcarouter` - for OrcaRouter aggregator- `deepinfra` - for DeepInfra hostingdiff --git a/examples/configs/opencode.provider.yml b/examples/configs/opencode.provider.ymlnew file mode 100644index 00000000..10463849--- /dev/null+++ b/examples/configs/opencode.provider.yml@@ -0,0 +1,130 @@+simple:+ model: mimo-v2.5+ temperature: 0.3+ top_p: 0.9+ n: 1+ max_tokens: 8192+ extra_body:+ thinking:+ type: disabled++simple_json:+ model: mimo-v2.5+ temperature: 0.3+ top_p: 0.9+ n: 1+ max_tokens: 4096+ json: true+ extra_body:+ thinking:+ type: disabled++primary_agent:+ model: deepseek-v4-pro+ n: 1+ max_tokens: 16384+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enabled++assistant:+ model: deepseek-v4-pro+ n: 1+ max_tokens: 16384+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enabled++generator:+ model: deepseek-v4-pro+ n: 1+ max_tokens: 32768+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enabled++refiner:+ model: deepseek-v4-pro+ n: 1+ max_tokens: 32768+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enabled++adviser:+ model: deepseek-v4-pro+ n: 1+ max_tokens: 16384+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enabled++reflector:+ model: deepseek-v4-flash+ temperature: 0.5+ top_p: 0.9+ n: 1+ max_tokens: 8192+ extra_body:+ thinking:+ type: disabled++searcher:+ model: deepseek-v4-flash+ temperature: 0.5+ top_p: 0.9+ n: 1+ max_tokens: 4096+ extra_body:+ thinking:+ type: disabled++enricher:+ model: deepseek-v4-flash+ temperature: 0.5+ top_p: 0.9+ n: 1+ max_tokens: 4096+ extra_body:+ thinking:+ type: disabled++coder:+ model: deepseek-v4-pro+ n: 1+ max_tokens: 20480+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enabled++installer:+ model: deepseek-v4-flash+ n: 1+ max_tokens: 12288+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enabled++pentester:+ model: deepseek-v4-pro+ n: 1+ max_tokens: 16384+ reasoning:+ effort: high+ extra_body:+ thinking:+ type: enableddiff --git a/examples/tests/opencode-report.md b/examples/tests/opencode-report.mdnew file mode 100644index 00000000..a061fb1e--- /dev/null+++ b/examples/tests/opencode-report.md@@ -0,0 +1,556 @@+# LLM Agent Testing Report++Generated: Tue, 04 Aug 2026 17:52:20 UTC++## Overall Results++| Agent | Model | Reasoning | Success Rate | Average Latency |+|-------|-------|-----------|--------------|-----------------|+| simple | mimo-v2.5 | true | 24/24 (100.00%) | 3.686s |+| simple_json | mimo-v2.5 | false | 7/7 (100.00%) | 6.841s |+| primary_agent | deepseek-v4-pro | true | 24/24 (100.00%) | 2.749s |+| assistant | deepseek-v4-pro | true | 24/24 (100.00%) | 2.568s |+| generator | deepseek-v4-pro | true | 24/24 (100.00%) | 2.708s |+| refiner | deepseek-v4-pro | true | 24/24 (100.00%) | 2.575s |+| adviser | deepseek-v4-pro | true | 24/24 (100.00%) | 2.367s |+| reflector | deepseek-v4-flash | true | 24/24 (100.00%) | 1.641s |+| searcher | deepseek-v4-flash | true | 24/24 (100.00%) | 1.649s |+| enricher | deepseek-v4-flash | true | 24/24 (100.00%) | 1.620s |+| coder | deepseek-v4-pro | true | 24/24 (100.00%) | 2.590s |+| installer | deepseek-v4-flash | true | 24/24 (100.00%) | 1.822s |+| pentester | deepseek-v4-pro | true | 24/24 (100.00%) | 2.605s |++**Total**: 295/295 (100.00%) successful tests+**Overall average latency**: 2.487s++## Detailed Results++### simple (mimo-v2.5)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 5.883s | |+| Text Transform Uppercase | ✅ Pass | 4.343s | |+| Count from 1 to 5 | ✅ Pass | 4.122s | |+| Math Calculation | ✅ Pass | 3.864s | |+| Basic Echo Function | ✅ Pass | 3.019s | |+| Streaming Simple Math Streaming | ✅ Pass | 2.553s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 3.197s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.549s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.365s | |+| Search Query Function | ✅ Pass | 2.015s | |+| Ask Advice Function | ✅ Pass | 2.025s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.728s | |+| Basic Context Memory Test | ✅ Pass | 2.968s | |+| Function Argument Memory Test | ✅ Pass | 1.561s | |+| Function Response Memory Test | ✅ Pass | 1.278s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.662s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 2.393s | |+| Read a file, then edit it via unified diff | ✅ Pass | 4.179s | |+| Penetration Testing Methodology | ✅ Pass | 5.625s | |+| Vulnerability Assessment Tools | ✅ Pass | 9.226s | |+| SQL Injection Attack Type | ✅ Pass | 7.439s | |+| Penetration Testing Framework | ✅ Pass | 4.882s | |+| Web Application Security Scanner | ✅ Pass | 3.887s | |+| Penetration Testing Tool Selection | ✅ Pass | 4.694s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 3.686s++---++### simple_json (mimo-v2.5)++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Vulnerability Report Memory Test | ✅ Pass | 5.392s | |+| Person Information JSON | ✅ Pass | 1.719s | |+| Project Information JSON | ✅ Pass | 1.303s | |+| User Profile JSON | ✅ Pass | 2.431s | |+| Streaming Person Information JSON Streaming | ✅ Pass | 1.374s | |+| JSON Array Response Without Schema | ✅ Pass | 32.140s | |++#### Capability Tests++| Test | Capability | Result | Latency | Note |+|------|------------|--------|---------|------|+| Structured Output With JSON Schema | structured_output | ✅ Pass | 3.525s | |++**Summary**: 7/7 (100.00%) successful tests++**Average latency**: 6.841s++---++### primary_agent (deepseek-v4-pro)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 1.795s | |+| Text Transform Uppercase | ✅ Pass | 1.751s | |+| Count from 1 to 5 | ✅ Pass | 1.684s | |+| Math Calculation | ✅ Pass | 1.966s | |+| Basic Echo Function | ✅ Pass | 1.984s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.624s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.726s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.163s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.286s | |+| Search Query Function | ✅ Pass | 2.207s | |+| Ask Advice Function | ✅ Pass | 2.326s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.885s | |+| Basic Context Memory Test | ✅ Pass | 2.461s | |+| Function Argument Memory Test | ✅ Pass | 1.767s | |+| Function Response Memory Test | ✅ Pass | 2.339s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 3.399s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 2.215s | |+| Read a file, then edit it via unified diff | ✅ Pass | 5.852s | |+| Penetration Testing Methodology | ✅ Pass | 3.285s | |+| Vulnerability Assessment Tools | ✅ Pass | 9.214s | |+| SQL Injection Attack Type | ✅ Pass | 3.741s | |+| Penetration Testing Framework | ✅ Pass | 3.013s | |+| Web Application Security Scanner | ✅ Pass | 2.833s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.446s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 2.749s++---++### assistant (deepseek-v4-pro)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 2.218s | |+| Text Transform Uppercase | ✅ Pass | 1.915s | |+| Count from 1 to 5 | ✅ Pass | 1.788s | |+| Math Calculation | ✅ Pass | 1.848s | |+| Basic Echo Function | ✅ Pass | 1.976s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.601s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.786s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.336s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.348s | |+| Search Query Function | ✅ Pass | 2.075s | |+| Ask Advice Function | ✅ Pass | 2.073s | |+| Streaming Search Query Function Streaming | ✅ Pass | 2.031s | |+| Basic Context Memory Test | ✅ Pass | 2.063s | |+| Function Argument Memory Test | ✅ Pass | 2.068s | |+| Function Response Memory Test | ✅ Pass | 1.961s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.920s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 2.182s | |+| Read a file, then edit it via unified diff | ✅ Pass | 5.868s | |+| Penetration Testing Methodology | ✅ Pass | 3.612s | |+| Vulnerability Assessment Tools | ✅ Pass | 6.697s | |+| SQL Injection Attack Type | ✅ Pass | 2.484s | |+| Penetration Testing Framework | ✅ Pass | 3.598s | |+| Web Application Security Scanner | ✅ Pass | 1.871s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.303s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 2.568s++---++### generator (deepseek-v4-pro)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 2.051s | |+| Text Transform Uppercase | ✅ Pass | 2.001s | |+| Count from 1 to 5 | ✅ Pass | 1.791s | |+| Math Calculation | ✅ Pass | 1.544s | |+| Basic Echo Function | ✅ Pass | 1.966s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.640s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.907s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.238s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.337s | |+| Search Query Function | ✅ Pass | 2.173s | |+| Ask Advice Function | ✅ Pass | 2.341s | |+| Streaming Search Query Function Streaming | ✅ Pass | 2.026s | |+| Basic Context Memory Test | ✅ Pass | 2.881s | |+| Function Argument Memory Test | ✅ Pass | 2.004s | |+| Function Response Memory Test | ✅ Pass | 1.953s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 3.369s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.852s | |+| Read a file, then edit it via unified diff | ✅ Pass | 6.872s | |+| Penetration Testing Methodology | ✅ Pass | 3.517s | |+| Vulnerability Assessment Tools | ✅ Pass | 7.270s | |+| SQL Injection Attack Type | ✅ Pass | 2.615s | |+| Penetration Testing Framework | ✅ Pass | 3.764s | |+| Web Application Security Scanner | ✅ Pass | 2.607s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.266s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 2.708s++---++### refiner (deepseek-v4-pro)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 2.014s | |+| Text Transform Uppercase | ✅ Pass | 1.849s | |+| Count from 1 to 5 | ✅ Pass | 1.946s | |+| Math Calculation | ✅ Pass | 1.772s | |+| Basic Echo Function | ✅ Pass | 1.842s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.569s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.883s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.039s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.522s | |+| Search Query Function | ✅ Pass | 2.253s | |+| Ask Advice Function | ✅ Pass | 2.164s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.933s | |+| Basic Context Memory Test | ✅ Pass | 2.625s | |+| Function Argument Memory Test | ✅ Pass | 2.138s | |+| Function Response Memory Test | ✅ Pass | 2.165s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 3.266s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 2.034s | |+| Read a file, then edit it via unified diff | ✅ Pass | 5.432s | |+| Penetration Testing Methodology | ✅ Pass | 3.023s | |+| Vulnerability Assessment Tools | ✅ Pass | 7.027s | |+| SQL Injection Attack Type | ✅ Pass | 1.861s | |+| Penetration Testing Framework | ✅ Pass | 3.503s | |+| Web Application Security Scanner | ✅ Pass | 2.864s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.055s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 2.575s++---++### adviser (deepseek-v4-pro)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 2.105s | |+| Text Transform Uppercase | ✅ Pass | 1.905s | |+| Count from 1 to 5 | ✅ Pass | 1.790s | |+| Math Calculation | ✅ Pass | 1.711s | |+| Basic Echo Function | ✅ Pass | 1.962s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.581s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.914s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.500s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.423s | |+| Search Query Function | ✅ Pass | 1.798s | |+| Ask Advice Function | ✅ Pass | 2.050s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.925s | |+| Basic Context Memory Test | ✅ Pass | 2.906s | |+| Function Argument Memory Test | ✅ Pass | 1.827s | |+| Function Response Memory Test | ✅ Pass | 2.000s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.765s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.994s | |+| Read a file, then edit it via unified diff | ✅ Pass | 5.798s | |+| Penetration Testing Methodology | ✅ Pass | 3.265s | |+| Vulnerability Assessment Tools | ✅ Pass | 2.125s | |+| SQL Injection Attack Type | ✅ Pass | 1.931s | |+| Penetration Testing Framework | ✅ Pass | 3.723s | |+| Web Application Security Scanner | ✅ Pass | 2.619s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.168s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 2.367s++---++### reflector (deepseek-v4-flash)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 1.504s | |+| Text Transform Uppercase | ✅ Pass | 1.061s | |+| Count from 1 to 5 | ✅ Pass | 1.305s | |+| Math Calculation | ✅ Pass | 1.039s | |+| Basic Echo Function | ✅ Pass | 1.373s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.329s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.145s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.904s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 1.892s | |+| Search Query Function | ✅ Pass | 1.410s | |+| Ask Advice Function | ✅ Pass | 1.482s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.395s | |+| Basic Context Memory Test | ✅ Pass | 2.046s | |+| Function Argument Memory Test | ✅ Pass | 1.326s | |+| Function Response Memory Test | ✅ Pass | 1.188s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.115s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.240s | |+| Read a file, then edit it via unified diff | ✅ Pass | 3.529s | |+| Penetration Testing Methodology | ✅ Pass | 2.082s | |+| Vulnerability Assessment Tools | ✅ Pass | 3.821s | |+| SQL Injection Attack Type | ✅ Pass | 1.225s | |+| Penetration Testing Framework | ✅ Pass | 1.196s | |+| Web Application Security Scanner | ✅ Pass | 1.281s | |+| Penetration Testing Tool Selection | ✅ Pass | 1.485s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 1.641s++---++### searcher (deepseek-v4-flash)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 1.730s | |+| Text Transform Uppercase | ✅ Pass | 1.125s | |+| Count from 1 to 5 | ✅ Pass | 1.067s | |+| Math Calculation | ✅ Pass | 1.210s | |+| Basic Echo Function | ✅ Pass | 1.649s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.177s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.379s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.614s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 1.704s | |+| Search Query Function | ✅ Pass | 1.836s | |+| Ask Advice Function | ✅ Pass | 1.652s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.415s | |+| Basic Context Memory Test | ✅ Pass | 1.638s | |+| Function Argument Memory Test | ✅ Pass | 1.380s | |+| Function Response Memory Test | ✅ Pass | 1.164s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 1.771s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.303s | |+| Read a file, then edit it via unified diff | ✅ Pass | 4.305s | |+| Penetration Testing Methodology | ✅ Pass | 1.201s | |+| Vulnerability Assessment Tools | ✅ Pass | 3.574s | |+| SQL Injection Attack Type | ✅ Pass | 1.253s | |+| Penetration Testing Framework | ✅ Pass | 1.467s | |+| Web Application Security Scanner | ✅ Pass | 1.408s | |+| Penetration Testing Tool Selection | ✅ Pass | 1.542s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 1.649s++---++### enricher (deepseek-v4-flash)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 1.181s | |+| Text Transform Uppercase | ✅ Pass | 1.918s | |+| Count from 1 to 5 | ✅ Pass | 1.154s | |+| Math Calculation | ✅ Pass | 1.134s | |+| Basic Echo Function | ✅ Pass | 1.340s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.069s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.501s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.747s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 1.846s | |+| Search Query Function | ✅ Pass | 1.649s | |+| Ask Advice Function | ✅ Pass | 1.378s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.847s | |+| Basic Context Memory Test | ✅ Pass | 1.439s | |+| Function Argument Memory Test | ✅ Pass | 1.299s | |+| Function Response Memory Test | ✅ Pass | 1.073s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.009s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.379s | |+| Read a file, then edit it via unified diff | ✅ Pass | 3.807s | |+| Penetration Testing Methodology | ✅ Pass | 1.163s | |+| Vulnerability Assessment Tools | ✅ Pass | 4.037s | |+| SQL Injection Attack Type | ✅ Pass | 1.165s | |+| Penetration Testing Framework | ✅ Pass | 1.171s | |+| Web Application Security Scanner | ✅ Pass | 1.101s | |+| Penetration Testing Tool Selection | ✅ Pass | 1.451s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 1.620s++---++### coder (deepseek-v4-pro)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 1.750s | |+| Text Transform Uppercase | ✅ Pass | 1.944s | |+| Count from 1 to 5 | ✅ Pass | 1.841s | |+| Math Calculation | ✅ Pass | 1.683s | |+| Basic Echo Function | ✅ Pass | 2.016s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.628s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 2.298s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.437s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.144s | |+| Search Query Function | ✅ Pass | 2.338s | |+| Ask Advice Function | ✅ Pass | 2.100s | |+| Streaming Search Query Function Streaming | ✅ Pass | 2.352s | |+| Basic Context Memory Test | ✅ Pass | 2.277s | |+| Function Argument Memory Test | ✅ Pass | 2.404s | |+| Function Response Memory Test | ✅ Pass | 2.149s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 3.142s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 2.159s | |+| Read a file, then edit it via unified diff | ✅ Pass | 5.470s | |+| Penetration Testing Methodology | ✅ Pass | 3.792s | |+| Vulnerability Assessment Tools | ✅ Pass | 6.231s | |+| SQL Injection Attack Type | ✅ Pass | 1.975s | |+| Penetration Testing Framework | ✅ Pass | 3.089s | |+| Web Application Security Scanner | ✅ Pass | 2.527s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.395s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 2.590s++---++### installer (deepseek-v4-flash)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 1.266s | |+| Text Transform Uppercase | ✅ Pass | 1.353s | |+| Count from 1 to 5 | ✅ Pass | 1.448s | |+| Math Calculation | ✅ Pass | 1.556s | |+| Basic Echo Function | ✅ Pass | 1.578s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.299s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.994s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.900s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 1.531s | |+| Search Query Function | ✅ Pass | 1.864s | |+| Ask Advice Function | ✅ Pass | 1.934s | |+| Streaming Search Query Function Streaming | ✅ Pass | 1.659s | |+| Basic Context Memory Test | ✅ Pass | 2.764s | |+| Function Argument Memory Test | ✅ Pass | 1.408s | |+| Function Response Memory Test | ✅ Pass | 1.847s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.287s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.778s | |+| Read a file, then edit it via unified diff | ✅ Pass | 3.633s | |+| Penetration Testing Methodology | ✅ Pass | 1.422s | |+| Vulnerability Assessment Tools | ✅ Pass | 2.364s | |+| SQL Injection Attack Type | ✅ Pass | 1.802s | |+| Penetration Testing Framework | ✅ Pass | 1.336s | |+| Web Application Security Scanner | ✅ Pass | 1.525s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.178s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 1.822s++---++### pentester (deepseek-v4-pro)++#### Basic Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| Simple Math | ✅ Pass | 2.151s | |+| Text Transform Uppercase | ✅ Pass | 1.506s | |+| Count from 1 to 5 | ✅ Pass | 1.940s | |+| Math Calculation | ✅ Pass | 1.607s | |+| Basic Echo Function | ✅ Pass | 2.126s | |+| Streaming Simple Math Streaming | ✅ Pass | 1.636s | |+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 3.010s | |+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.324s | |++#### Advanced Tests++| Test | Result | Latency | Error |+|------|--------|---------|-------|+| JSON Response Function | ✅ Pass | 2.441s | |+| Search Query Function | ✅ Pass | 2.244s | |+| Ask Advice Function | ✅ Pass | 2.284s | |+| Streaming Search Query Function Streaming | ✅ Pass | 2.421s | |+| Basic Context Memory Test | ✅ Pass | 1.935s | |+| Function Argument Memory Test | ✅ Pass | 2.131s | |+| Function Response Memory Test | ✅ Pass | 2.362s | |+| Penetration Testing Memory with Tool Call | ✅ Pass | 3.331s | |+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.897s | |+| Read a file, then edit it via unified diff | ✅ Pass | 5.897s | |+| Penetration Testing Methodology | ✅ Pass | 4.382s | |+| Vulnerability Assessment Tools | ✅ Pass | 3.823s | |+| SQL Injection Attack Type | ✅ Pass | 2.507s | |+| Penetration Testing Framework | ✅ Pass | 4.464s | |+| Web Application Security Scanner | ✅ Pass | 1.676s | |+| Penetration Testing Tool Selection | ✅ Pass | 2.410s | |++**Summary**: 24/24 (100.00%) successful tests++**Average latency**: 2.605s++---+