Skip to content

Fully autonomous AI Agents system capable of performing complex penetration testing tasks

feat(provider): add qwen3.8-27b fp8 provider configuration and testing report

Dmitry Ng8b2deb6a53f8
diff --git a/Dockerfile b/Dockerfile
index 5fdb657a..1762fbe2 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -182,6 +182,8 @@ COPY examples/configs/vllm-qwen3.5-27b-fp8-no-think.provider.yml /opt/pentagi/co
COPY examples/configs/vllm-qwen3.5-27b-fp8.provider.yml /opt/pentagi/conf/
COPY examples/configs/vllm-qwen3.6-27b-fp8-no-think.provider.yml /opt/pentagi/conf/
COPY examples/configs/vllm-qwen3.6-27b-fp8.provider.yml /opt/pentagi/conf/
+COPY examples/configs/vllm-qwen3.8-27b-fp8-no-think.provider.yml /opt/pentagi/conf/
+COPY examples/configs/vllm-qwen3.8-27b-fp8.provider.yml /opt/pentagi/conf/
COPY examples/configs/vllm-qwen3.6-35b-a3b-fp8-no-think.provider.yml /opt/pentagi/conf/
COPY examples/configs/vllm-qwen3.6-35b-a3b-fp8.provider.yml /opt/pentagi/conf/
COPY examples/configs/vllm-qwen332b-fp16.provider.yml /opt/pentagi/conf/
diff --git a/examples/configs/vllm-qwen3.8-27b-fp8-no-think.provider.yml b/examples/configs/vllm-qwen3.8-27b-fp8-no-think.provider.yml
new file mode 100644
index 00000000..169e71c9
--- /dev/null
+++ b/examples/configs/vllm-qwen3.8-27b-fp8-no-think.provider.yml
@@ -0,0 +1,193 @@
+# Qwen3.8-27B FP8 Provider Configuration - NON-THINKING MODE
+# Based on official Qwen recommendations for vLLM inference
+# Architecture: Hybrid 75% DeltaNet + 25% Full Attention (48+16 layers)
+# Context: 262K native, expandable to 1M with YaRN
+# Vision: VLM with Vision Encoder (uses VRAM even for text-only tasks)
+#
+# Non-thinking mode is disabled via extra_body parameter
+# Recommended sampling parameters:
+# - General tasks: temp=0.7, top_p=0.8, top_k=20, min_p=0.0, pp=1.5, rp=1.0
+# - Reasoning tasks: temp=1.0, top_p=0.95, top_k=20, min_p=0.0, pp=1.5, rp=1.0
+
+simple:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+simple_json:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ json: true
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+primary_agent:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+assistant:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+generator:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+refiner:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+adviser:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+reflector:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+searcher:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+enricher:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+coder:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+installer:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+pentester:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
diff --git a/examples/configs/vllm-qwen3.8-27b-fp8.provider.yml b/examples/configs/vllm-qwen3.8-27b-fp8.provider.yml
new file mode 100644
index 00000000..46f974a5
--- /dev/null
+++ b/examples/configs/vllm-qwen3.8-27b-fp8.provider.yml
@@ -0,0 +1,174 @@
+# Qwen3.8-27B FP8 Provider Configuration - THINKING MODE (default)
+# Based on official Qwen recommendations for vLLM inference
+# Architecture: Hybrid 75% DeltaNet + 25% Full Attention (48+16 layers)
+# Context: 262K native, expandable to 1M with YaRN
+# Vision: VLM with Vision Encoder (uses VRAM even for text-only tasks)
+#
+# Thinking mode is enabled by default (no extra_body needed)
+# Recommended sampling parameters:
+# - General tasks: temp=1.0, top_p=0.95, top_k=20, min_p=0.0, pp=1.5, rp=1.0
+# - Precise coding: temp=0.6, top_p=0.95, top_k=20, min_p=0.0, pp=0.0, rp=1.0
+#
+# Non-thinking mode is disabled via extra_body parameter
+# Recommended sampling parameters:
+# - General tasks: temp=0.7, top_p=0.8, top_k=20, min_p=0.0, pp=1.5, rp=1.0
+# - Reasoning tasks: temp=1.0, top_p=0.95, top_k=20, min_p=0.0, pp=1.5, rp=1.0
+
+simple:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+simple_json:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ json: true
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+primary_agent:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+
+assistant:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+
+generator:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+
+refiner:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+
+adviser:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+
+reflector:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 1.0
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+searcher:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+enricher:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.7
+ top_k: 20
+ top_p: 0.8
+ min_p: 0.0
+ presence_penalty: 1.5
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+ extra_body:
+ chat_template_kwargs:
+ enable_thinking: false
+
+coder:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.6
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 0.0
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+
+installer:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.6
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 0.0
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
+
+pentester:
+ model: "Qwen/Qwen3.8-27B-FP8"
+ temperature: 0.6
+ top_k: 20
+ top_p: 0.95
+ min_p: 0.0
+ presence_penalty: 0.0
+ repetition_penalty: 1.0
+ n: 1
+ max_tokens: 32768
diff --git a/examples/tests/vllm-qwen3.8-27b-fp8.report.md b/examples/tests/vllm-qwen3.8-27b-fp8.report.md
new file mode 100644
index 00000000..efccbe0f
--- /dev/null
+++ b/examples/tests/vllm-qwen3.8-27b-fp8.report.md
@@ -0,0 +1,556 @@
+# LLM Agent Testing Report
+
+Generated: Tue, 01 Sep 2026 23:37:44 UTC
+
+## Overall Results
+
+| Agent | Model | Reasoning | Success Rate | Average Latency |
+|-------|-------|-----------|--------------|-----------------|
+| simple | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 1.785s |
+| simple_json | Qwen/Qwen3.8-27B-FP8 | false | 7/7 (100.00%) | 1.118s |
+| primary_agent | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 2.237s |
+| assistant | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 2.523s |
+| generator | Qwen/Qwen3.8-27B-FP8 | true | 23/24 (95.83%) | 2.683s |
+| refiner | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 2.410s |
+| adviser | Qwen/Qwen3.8-27B-FP8 | true | 23/24 (95.83%) | 2.376s |
+| reflector | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 1.898s |
+| searcher | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 1.802s |
+| enricher | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 1.897s |
+| coder | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 2.011s |
+| installer | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 2.063s |
+| pentester | Qwen/Qwen3.8-27B-FP8 | true | 24/24 (100.00%) | 2.354s |
+
+**Total**: 293/295 (99.32%) successful tests
+**Overall average latency**: 2.145s
+
+## Detailed Results
+
+### simple (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Math Calculation | ✅ Pass | 0.675s | |
+| Simple Math | ✅ Pass | 0.693s | |
+| Count from 1 to 5 | ✅ Pass | 0.781s | |
+| Streaming Simple Math Streaming | ✅ Pass | 0.634s | |
+| Basic Echo Function | ✅ Pass | 0.913s | |
+| Text Transform Uppercase | ✅ Pass | 2.412s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.155s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 3.483s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 0.943s | |
+| Search Query Function | ✅ Pass | 0.978s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.034s | |
+| Ask Advice Function | ✅ Pass | 1.539s | |
+| Function Response Memory Test | ✅ Pass | 0.548s | |
+| Basic Context Memory Test | ✅ Pass | 2.058s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 0.826s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.961s | |
+| Function Argument Memory Test | ✅ Pass | 3.931s | |
+| SQL Injection Attack Type | ✅ Pass | 0.437s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 4.282s | |
+| Penetration Testing Methodology | ✅ Pass | 2.616s | |
+| Vulnerability Assessment Tools | ✅ Pass | 2.596s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.094s | |
+| Web Application Security Scanner | ✅ Pass | 1.955s | |
+| Penetration Testing Framework | ✅ Pass | 4.273s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 1.785s
+
+---
+
+### simple_json (Qwen/Qwen3.8-27B-FP8)
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Vulnerability Report Memory Test | ✅ Pass | 1.692s | |
+| Person Information JSON | ✅ Pass | 0.966s | |
+| User Profile JSON | ✅ Pass | 0.909s | |
+| Streaming Person Information JSON Streaming | ✅ Pass | 0.870s | |
+| Project Information JSON | ✅ Pass | 1.149s | |
+| JSON Array Response Without Schema | ✅ Pass | 1.343s | |
+
+#### Capability Tests
+
+| Test | Capability | Result | Latency | Note |
+|------|------------|--------|---------|------|
+| Structured Output With JSON Schema | structured_output | ✅ Pass | 0.892s | |
+
+**Summary**: 7/7 (100.00%) successful tests
+
+**Average latency**: 1.118s
+
+---
+
+### primary_agent (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Count from 1 to 5 | ✅ Pass | 1.328s | |
+| Text Transform Uppercase | ✅ Pass | 1.328s | |
+| Simple Math | ✅ Pass | 1.349s | |
+| Math Calculation | ✅ Pass | 1.415s | |
+| Streaming Simple Math Streaming | ✅ Pass | 1.090s | |
+| Basic Echo Function | ✅ Pass | 1.235s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.622s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 3.944s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Search Query Function | ✅ Pass | 1.443s | |
+| Ask Advice Function | ✅ Pass | 1.799s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.535s | |
+| JSON Response Function | ✅ Pass | 3.751s | |
+| Function Argument Memory Test | ✅ Pass | 1.457s | |
+| Basic Context Memory Test | ✅ Pass | 2.154s | |
+| Function Response Memory Test | ✅ Pass | 1.319s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 2.397s | |
+| Penetration Testing Methodology | ✅ Pass | 0.933s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 5.765s | |
+| Vulnerability Assessment Tools | ✅ Pass | 2.567s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 5.546s | |
+| SQL Injection Attack Type | ✅ Pass | 2.476s | |
+| Web Application Security Scanner | ✅ Pass | 1.527s | |
+| Penetration Testing Framework | ✅ Pass | 3.435s | |
+| Penetration Testing Tool Selection | ✅ Pass | 2.271s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 2.237s
+
+---
+
+### assistant (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Simple Math | ✅ Pass | 1.332s | |
+| Math Calculation | ✅ Pass | 1.334s | |
+| Basic Echo Function | ✅ Pass | 1.202s | |
+| Streaming Simple Math Streaming | ✅ Pass | 1.156s | |
+| Text Transform Uppercase | ✅ Pass | 2.722s | |
+| Count from 1 to 5 | ✅ Pass | 3.416s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.448s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 3.530s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Search Query Function | ✅ Pass | 1.581s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.297s | |
+| JSON Response Function | ✅ Pass | 3.869s | |
+| Ask Advice Function | ✅ Pass | 2.206s | |
+| Basic Context Memory Test | ✅ Pass | 2.121s | |
+| Function Response Memory Test | ✅ Pass | 1.431s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.907s | |
+| Function Argument Memory Test | ✅ Pass | 4.516s | |
+| Penetration Testing Methodology | ✅ Pass | 1.172s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 4.219s | |
+| SQL Injection Attack Type | ✅ Pass | 1.288s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 6.439s | |
+| Penetration Testing Framework | ✅ Pass | 2.035s | |
+| Web Application Security Scanner | ✅ Pass | 1.611s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.740s | |
+| Vulnerability Assessment Tools | ✅ Pass | 5.979s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 2.523s
+
+---
+
+### generator (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Text Transform Uppercase | ✅ Pass | 1.330s | |
+| Math Calculation | ✅ Pass | 1.334s | |
+| Count from 1 to 5 | ✅ Pass | 1.364s | |
+| Streaming Simple Math Streaming | ✅ Pass | 1.084s | |
+| Basic Echo Function | ✅ Pass | 1.225s | |
+| Simple Math | ✅ Pass | 2.721s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.528s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.464s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Search Query Function | ✅ Pass | 1.433s | |
+| JSON Response Function | ✅ Pass | 3.961s | |
+| Function Argument Memory Test | ✅ Pass | 1.427s | |
+| Ask Advice Function | ✅ Pass | 3.155s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.880s | |
+| Function Response Memory Test | ✅ Pass | 3.027s | |
+| Basic Context Memory Test | ✅ Pass | 4.700s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 5.436s | |
+| Penetration Testing Methodology | ✅ Pass | 1.045s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 4.671s | |
+| Penetration Testing Memory with Tool Call | ❌ Fail | 6.220s | expected function 'generate\_report' not found in tool calls: expected function generate\_report not found in tool calls |
+| SQL Injection Attack Type | ✅ Pass | 2.412s | |
+| Penetration Testing Framework | ✅ Pass | 1.787s | |
+| Web Application Security Scanner | ✅ Pass | 1.883s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.640s | |
+| Vulnerability Assessment Tools | ✅ Pass | 7.663s | |
+
+**Summary**: 23/24 (95.83%) successful tests
+
+**Average latency**: 2.683s
+
+---
+
+### refiner (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Text Transform Uppercase | ✅ Pass | 1.397s | |
+| Count from 1 to 5 | ✅ Pass | 1.534s | |
+| Simple Math | ✅ Pass | 2.358s | |
+| Basic Echo Function | ✅ Pass | 1.260s | |
+| Streaming Simple Math Streaming | ✅ Pass | 1.721s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.234s | |
+| Math Calculation | ✅ Pass | 4.525s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 3.785s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 1.654s | |
+| Search Query Function | ✅ Pass | 1.606s | |
+| Ask Advice Function | ✅ Pass | 2.278s | |
+| Function Argument Memory Test | ✅ Pass | 1.310s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 2.535s | |
+| Basic Context Memory Test | ✅ Pass | 2.234s | |
+| Function Response Memory Test | ✅ Pass | 1.151s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.745s | |
+| Penetration Testing Methodology | ✅ Pass | 1.788s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 4.988s | |
+| SQL Injection Attack Type | ✅ Pass | 1.054s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 5.005s | |
+| Penetration Testing Framework | ✅ Pass | 1.813s | |
+| Web Application Security Scanner | ✅ Pass | 1.433s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.758s | |
+| Vulnerability Assessment Tools | ✅ Pass | 7.658s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 2.410s
+
+---
+
+### adviser (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Text Transform Uppercase | ✅ Pass | 1.088s | |
+| Simple Math | ✅ Pass | 1.466s | |
+| Streaming Simple Math Streaming | ✅ Pass | 0.894s | |
+| Count from 1 to 5 | ✅ Pass | 2.582s | |
+| Basic Echo Function | ✅ Pass | 1.406s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.589s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 2.498s | |
+| Math Calculation | ✅ Pass | 4.471s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Search Query Function | ✅ Pass | 1.170s | |
+| JSON Response Function | ✅ Pass | 2.452s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.248s | |
+| Ask Advice Function | ✅ Pass | 1.952s | |
+| Basic Context Memory Test | ✅ Pass | 1.899s | |
+| Function Response Memory Test | ✅ Pass | 1.051s | |
+| Function Argument Memory Test | ✅ Pass | 3.146s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.829s | |
+| Penetration Testing Memory with Tool Call | ❌ Fail | 4.554s | expected function 'generate\_report' not found in tool calls: expected function generate\_report not found in tool calls |
+| Read a file, then edit it via unified diff | ✅ Pass | 4.938s | |
+| Vulnerability Assessment Tools | ✅ Pass | 3.113s | |
+| SQL Injection Attack Type | ✅ Pass | 2.245s | |
+| Web Application Security Scanner | ✅ Pass | 1.385s | |
+| Penetration Testing Framework | ✅ Pass | 2.082s | |
+| Penetration Testing Methodology | ✅ Pass | 6.063s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.891s | |
+
+**Summary**: 23/24 (95.83%) successful tests
+
+**Average latency**: 2.376s
+
+---
+
+### reflector (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Text Transform Uppercase | ✅ Pass | 0.619s | |
+| Simple Math | ✅ Pass | 1.328s | |
+| Basic Echo Function | ✅ Pass | 0.920s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 0.597s | |
+| Count from 1 to 5 | ✅ Pass | 3.171s | |
+| Streaming Simple Math Streaming | ✅ Pass | 1.870s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.120s | |
+| Math Calculation | ✅ Pass | 4.451s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 0.961s | |
+| Search Query Function | ✅ Pass | 1.089s | |
+| Basic Context Memory Test | ✅ Pass | 0.588s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.033s | |
+| Ask Advice Function | ✅ Pass | 1.458s | |
+| Function Argument Memory Test | ✅ Pass | 0.564s | |
+| Function Response Memory Test | ✅ Pass | 0.548s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 0.729s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.398s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 3.892s | |
+| SQL Injection Attack Type | ✅ Pass | 0.883s | |
+| Web Application Security Scanner | ✅ Pass | 0.453s | |
+| Penetration Testing Methodology | ✅ Pass | 4.558s | |
+| Penetration Testing Framework | ✅ Pass | 2.611s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.081s | |
+| Vulnerability Assessment Tools | ✅ Pass | 7.624s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 1.898s
+
+---
+
+### searcher (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Count from 1 to 5 | ✅ Pass | 1.328s | |
+| Text Transform Uppercase | ✅ Pass | 1.329s | |
+| Streaming Simple Math Streaming | ✅ Pass | 0.669s | |
+| Basic Echo Function | ✅ Pass | 1.010s | |
+| Simple Math | ✅ Pass | 3.172s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 0.781s | |
+| Math Calculation | ✅ Pass | 4.365s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 3.054s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 1.179s | |
+| Search Query Function | ✅ Pass | 0.745s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 0.943s | |
+| Basic Context Memory Test | ✅ Pass | 0.563s | |
+| Ask Advice Function | ✅ Pass | 1.527s | |
+| Function Argument Memory Test | ✅ Pass | 0.629s | |
+| Function Response Memory Test | ✅ Pass | 0.546s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 0.816s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.085s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 4.082s | |
+| SQL Injection Attack Type | ✅ Pass | 0.832s | |
+| Vulnerability Assessment Tools | ✅ Pass | 3.904s | |
+| Penetration Testing Methodology | ✅ Pass | 4.750s | |
+| Web Application Security Scanner | ✅ Pass | 1.616s | |
+| Penetration Testing Framework | ✅ Pass | 2.233s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.082s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 1.802s
+
+---
+
+### enricher (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Count from 1 to 5 | ✅ Pass | 1.065s | |
+| Text Transform Uppercase | ✅ Pass | 1.329s | |
+| Simple Math | ✅ Pass | 1.330s | |
+| Basic Echo Function | ✅ Pass | 0.991s | |
+| Streaming Simple Math Streaming | ✅ Pass | 0.425s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 0.542s | |
+| Math Calculation | ✅ Pass | 4.082s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.998s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 1.129s | |
+| Search Query Function | ✅ Pass | 1.139s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 0.890s | |
+| Basic Context Memory Test | ✅ Pass | 0.600s | |
+| Function Argument Memory Test | ✅ Pass | 0.544s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 0.560s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 2.222s | |
+| Function Response Memory Test | ✅ Pass | 2.842s | |
+| Ask Advice Function | ✅ Pass | 5.690s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 3.884s | |
+| SQL Injection Attack Type | ✅ Pass | 0.659s | |
+| Vulnerability Assessment Tools | ✅ Pass | 3.113s | |
+| Penetration Testing Methodology | ✅ Pass | 4.511s | |
+| Penetration Testing Framework | ✅ Pass | 2.157s | |
+| Web Application Security Scanner | ✅ Pass | 1.599s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.214s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 1.897s
+
+---
+
+### coder (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Simple Math | ✅ Pass | 1.404s | |
+| Count from 1 to 5 | ✅ Pass | 1.449s | |
+| Text Transform Uppercase | ✅ Pass | 2.357s | |
+| Basic Echo Function | ✅ Pass | 1.171s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.529s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.452s | |
+| Streaming Simple Math Streaming | ✅ Pass | 2.454s | |
+| Math Calculation | ✅ Pass | 4.059s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 1.662s | |
+| Search Query Function | ✅ Pass | 1.734s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.461s | |
+| Ask Advice Function | ✅ Pass | 1.834s | |
+| Basic Context Memory Test | ✅ Pass | 1.192s | |
+| Function Argument Memory Test | ✅ Pass | 1.309s | |
+| Function Response Memory Test | ✅ Pass | 1.084s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 2.879s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 4.429s | |
+| Penetration Testing Methodology | ✅ Pass | 1.896s | |
+| Vulnerability Assessment Tools | ✅ Pass | 2.157s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 4.604s | |
+| SQL Injection Attack Type | ✅ Pass | 1.394s | |
+| Penetration Testing Framework | ✅ Pass | 1.899s | |
+| Web Application Security Scanner | ✅ Pass | 1.226s | |
+| Penetration Testing Tool Selection | ✅ Pass | 1.624s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 2.011s
+
+---
+
+### installer (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Simple Math | ✅ Pass | 1.393s | |
+| Count from 1 to 5 | ✅ Pass | 1.496s | |
+| Text Transform Uppercase | ✅ Pass | 2.357s | |
+| Basic Echo Function | ✅ Pass | 1.316s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 1.327s | |
+| Streaming Simple Math Streaming | ✅ Pass | 2.171s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 2.592s | |
+| Math Calculation | ✅ Pass | 4.120s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 1.514s | |
+| Search Query Function | ✅ Pass | 1.360s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.297s | |
+| Ask Advice Function | ✅ Pass | 1.785s | |
+| Function Response Memory Test | ✅ Pass | 0.841s | |
+| Basic Context Memory Test | ✅ Pass | 1.683s | |
+| Function Argument Memory Test | ✅ Pass | 1.590s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.889s | |
+| Penetration Testing Methodology | ✅ Pass | 1.180s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 4.879s | |
+| Vulnerability Assessment Tools | ✅ Pass | 2.090s | |
+| SQL Injection Attack Type | ✅ Pass | 1.628s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 4.476s | |
+| Penetration Testing Framework | ✅ Pass | 1.645s | |
+| Web Application Security Scanner | ✅ Pass | 1.548s | |
+| Penetration Testing Tool Selection | ✅ Pass | 3.318s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 2.063s
+
+---
+
+### pentester (Qwen/Qwen3.8-27B-FP8)
+
+#### Basic Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| Simple Math | ✅ Pass | 1.328s | |
+| Count from 1 to 5 | ✅ Pass | 2.495s | |
+| Basic Echo Function | ✅ Pass | 1.184s | |
+| Streaming Basic Echo Function Streaming | ✅ Pass | 1.231s | |
+| Streaming Simple Math Streaming | ✅ Pass | 2.407s | |
+| Streaming Count from 1 to 3 Streaming | ✅ Pass | 2.194s | |
+| Text Transform Uppercase | ✅ Pass | 5.145s | |
+| Math Calculation | ✅ Pass | 4.120s | |
+
+#### Advanced Tests
+
+| Test | Result | Latency | Error |
+|------|--------|---------|-------|
+| JSON Response Function | ✅ Pass | 1.439s | |
+| Search Query Function | ✅ Pass | 1.372s | |
+| Streaming Search Query Function Streaming | ✅ Pass | 1.404s | |
+| Function Argument Memory Test | ✅ Pass | 1.428s | |
+| Ask Advice Function | ✅ Pass | 3.082s | |
+| Function Response Memory Test | ✅ Pass | 1.289s | |
+| Cybersecurity Workflow Memory Test | ✅ Pass | 1.727s | |
+| Basic Context Memory Test | ✅ Pass | 4.196s | |
+| Penetration Testing Memory with Tool Call | ✅ Pass | 3.987s | |
+| Penetration Testing Methodology | ✅ Pass | 1.242s | |
+| Vulnerability Assessment Tools | ✅ Pass | 2.143s | |
+| SQL Injection Attack Type | ✅ Pass | 1.953s | |
+| Read a file, then edit it via unified diff | ✅ Pass | 5.018s | |
+| Penetration Testing Framework | ✅ Pass | 1.447s | |
+| Web Application Security Scanner | ✅ Pass | 1.384s | |
+| Penetration Testing Tool Selection | ✅ Pass | 3.277s | |
+
+**Summary**: 24/24 (100.00%) successful tests
+
+**Average latency**: 2.354s
+
+---
+