Skip to content

Fully autonomous AI Agents system capable of performing complex penetration testing tasks

20.9 KB
Raw Blame

LLM Agent Testing Report

Generated: Thu, 24 Sep 2026 08:38:11 UTC

Overall Results

Agent Model Reasoning Success Rate Average Latency
simple kimi-k2.6 false 25/25 (100.00%) 3.748s
simple_json kimi-k2.6 false 8/8 (100.00%) 1.320s
primary_agent kimi-k2.7-code-highspeed true 25/25 (100.00%) 1.745s
assistant kimi-k2.7-code-highspeed true 25/25 (100.00%) 1.026s
generator kimi-k2.7-code true 25/25 (100.00%) 3.723s
refiner kimi-k2.7-code true 25/25 (100.00%) 2.745s
adviser kimi-k3 true 16/16 (100.00%) 1.055s
reflector kimi-k2.6 false 16/16 (100.00%) 4.864s
searcher kimi-k2.6 false 25/25 (100.00%) 3.741s
enricher kimi-k2.6 false 25/25 (100.00%) 2.631s
coder kimi-k2.7-code true 25/25 (100.00%) 3.966s
installer kimi-k2.6 true 25/25 (100.00%) 5.654s
pentester kimi-k2.7-code true 25/25 (100.00%) 2.514s

Total: 290/290 (100.00%) successful tests Overall average latency: 3.078s

Detailed Results

simple (kimi-k2.6)

Basic Tests

Test Result Latency Error
Text Transform Uppercase ✅ Pass 0.979s
Streaming Simple Math Streaming ✅ Pass 0.983s
Math Calculation ✅ Pass 1.012s
Count from 1 to 5 ✅ Pass 1.132s
Simple Math ✅ Pass 1.226s
Streaming Count from 1 to 3 Streaming ✅ Pass 1.275s
Basic Echo Function ✅ Pass 1.451s
Streaming Basic Echo Function Streaming ✅ Pass 1.383s
Answer Stops At The Output Limit ✅ Pass 46.631s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.241s
Search Query Function ✅ Pass 0.223s
Ask Advice Function ✅ Pass 0.799s
Streaming Search Query Function Streaming ✅ Pass 0.645s
Basic Context Memory Test ✅ Pass 0.219s
Function Argument Memory Test ✅ Pass 0.947s
Function Response Memory Test ✅ Pass 1.026s
Penetration Testing Memory with Tool Call ✅ Pass 2.428s
Cybersecurity Workflow Memory Test ✅ Pass 7.045s
Read a file, then edit it via unified diff ✅ Pass 0.443s
SQL Injection Attack Type ✅ Pass 1.271s
Penetration Testing Tool Selection ✅ Pass 1.725s
Vulnerability Assessment Tools ✅ Pass 4.364s
Web Application Security Scanner ✅ Pass 4.994s
Penetration Testing Methodology ✅ Pass 6.220s
Penetration Testing Framework ✅ Pass 5.023s

Summary: 25/25 (100.00%) successful tests

Average latency: 3.748s


simple_json (kimi-k2.6)

Advanced Tests

Test Result Latency Error
Vulnerability Report Memory Test ✅ Pass 2.182s
Person Information JSON ✅ Pass 1.328s
Project Information JSON ✅ Pass 1.220s
User Profile JSON ✅ Pass 1.572s
JSON Array Response Without Schema ✅ Pass 1.742s
Streaming Person Information JSON Streaming ✅ Pass 1.244s

Capability Tests

Test Capability Result Latency Note
Structured Output Refuses A Non-Object Schema structured_output ✅ Pass 0.000s
Structured Output With JSON Schema structured_output ✅ Pass 1.270s

Summary: 8/8 (100.00%) successful tests

Average latency: 1.320s


primary_agent (kimi-k2.7-code-highspeed)

Basic Tests

Test Result Latency Error
Text Transform Uppercase ✅ Pass 0.980s
Simple Math ✅ Pass 1.148s
Count from 1 to 5 ✅ Pass 1.219s
Basic Echo Function ✅ Pass 1.034s
Math Calculation ✅ Pass 1.517s
Streaming Simple Math Streaming ✅ Pass 1.243s
Streaming Count from 1 to 3 Streaming ✅ Pass 1.497s
Streaming Basic Echo Function Streaming ✅ Pass 1.385s
Answer Stops At The Output Limit ✅ Pass 9.233s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 1.098s
Search Query Function ✅ Pass 1.091s
Ask Advice Function ✅ Pass 1.398s
Streaming Search Query Function Streaming ✅ Pass 1.133s
Function Argument Memory Test ✅ Pass 1.090s
Basic Context Memory Test ✅ Pass 1.142s
Cybersecurity Workflow Memory Test ✅ Pass 1.032s
Function Response Memory Test ✅ Pass 1.492s
Penetration Testing Memory with Tool Call ✅ Pass 1.527s
Read a file, then edit it via unified diff ✅ Pass 3.486s
Penetration Testing Methodology ✅ Pass 1.276s
SQL Injection Attack Type ✅ Pass 1.133s
Penetration Testing Framework ✅ Pass 1.321s
Vulnerability Assessment Tools ✅ Pass 2.204s
Web Application Security Scanner ✅ Pass 2.090s
Penetration Testing Tool Selection ✅ Pass 1.852s

Summary: 25/25 (100.00%) successful tests

Average latency: 1.745s


assistant (kimi-k2.7-code-highspeed)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.221s
Text Transform Uppercase ✅ Pass 0.219s
Count from 1 to 5 ✅ Pass 0.220s
Math Calculation ✅ Pass 0.221s
Basic Echo Function ✅ Pass 0.219s
Streaming Simple Math Streaming ✅ Pass 0.228s
Streaming Count from 1 to 3 Streaming ✅ Pass 1.116s
Streaming Basic Echo Function Streaming ✅ Pass 1.039s
Answer Stops At The Output Limit ✅ Pass 9.241s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.229s
Search Query Function ✅ Pass 0.224s
Ask Advice Function ✅ Pass 0.229s
Streaming Search Query Function Streaming ✅ Pass 0.225s
Basic Context Memory Test ✅ Pass 0.229s
Function Argument Memory Test ✅ Pass 1.059s
Function Response Memory Test ✅ Pass 1.045s
Penetration Testing Memory with Tool Call ✅ Pass 1.178s
Cybersecurity Workflow Memory Test ✅ Pass 1.197s
Read a file, then edit it via unified diff ✅ Pass 2.846s
Penetration Testing Methodology ✅ Pass 0.219s
Vulnerability Assessment Tools ✅ Pass 0.210s
SQL Injection Attack Type ✅ Pass 0.219s
Penetration Testing Framework ✅ Pass 0.216s
Web Application Security Scanner ✅ Pass 1.355s
Penetration Testing Tool Selection ✅ Pass 2.243s

Summary: 25/25 (100.00%) successful tests

Average latency: 1.026s


generator (kimi-k2.7-code)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 1.258s
Text Transform Uppercase ✅ Pass 1.743s
Math Calculation ✅ Pass 1.931s
Count from 1 to 5 ✅ Pass 2.488s
Basic Echo Function ✅ Pass 2.435s
Streaming Simple Math Streaming ✅ Pass 2.017s
Streaming Basic Echo Function Streaming ✅ Pass 1.569s
Streaming Count from 1 to 3 Streaming ✅ Pass 2.236s
Answer Stops At The Output Limit ✅ Pass 42.701s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.218s
Search Query Function ✅ Pass 0.222s
Ask Advice Function ✅ Pass 0.216s
Streaming Search Query Function Streaming ✅ Pass 0.224s
Basic Context Memory Test ✅ Pass 0.223s
Penetration Testing Memory with Tool Call ✅ Pass 1.920s
Function Response Memory Test ✅ Pass 2.039s
Cybersecurity Workflow Memory Test ✅ Pass 2.525s
Function Argument Memory Test ✅ Pass 2.823s
Read a file, then edit it via unified diff ✅ Pass 0.437s
Web Application Security Scanner ✅ Pass 1.866s
SQL Injection Attack Type ✅ Pass 2.280s
Penetration Testing Tool Selection ✅ Pass 2.775s
Penetration Testing Framework ✅ Pass 4.448s
Vulnerability Assessment Tools ✅ Pass 5.202s
Penetration Testing Methodology ✅ Pass 7.272s

Summary: 25/25 (100.00%) successful tests

Average latency: 3.723s


refiner (kimi-k2.7-code)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.222s
Text Transform Uppercase ✅ Pass 0.221s
Count from 1 to 5 ✅ Pass 0.229s
Math Calculation ✅ Pass 0.229s
Streaming Simple Math Streaming ✅ Pass 0.219s
Basic Echo Function ✅ Pass 0.238s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.222s
Streaming Basic Echo Function Streaming ✅ Pass 0.238s
Answer Stops At The Output Limit ✅ Pass 42.333s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.222s
Search Query Function ✅ Pass 0.226s
Ask Advice Function ✅ Pass 0.216s
Streaming Search Query Function Streaming ✅ Pass 0.223s
Basic Context Memory Test ✅ Pass 0.222s
Function Argument Memory Test ✅ Pass 1.266s
Function Response Memory Test ✅ Pass 2.271s
Penetration Testing Memory with Tool Call ✅ Pass 2.306s
Cybersecurity Workflow Memory Test ✅ Pass 2.466s
Read a file, then edit it via unified diff ✅ Pass 0.457s
Penetration Testing Methodology ✅ Pass 2.892s
Web Application Security Scanner ✅ Pass 0.226s
Penetration Testing Tool Selection ✅ Pass 0.216s
SQL Injection Attack Type ✅ Pass 2.330s
Penetration Testing Framework ✅ Pass 3.547s
Vulnerability Assessment Tools ✅ Pass 5.369s

Summary: 25/25 (100.00%) successful tests

Average latency: 2.745s


adviser (kimi-k3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.232s
Simple Math ✅ Pass 0.234s
Text Transform Uppercase ✅ Pass 0.220s
Count from 1 to 5 ✅ Pass 0.221s
Math Calculation ✅ Pass 0.225s
Streaming Simple Math Streaming ✅ Pass 0.218s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.225s

Advanced Tests

Test Result Latency Error
Basic Context Memory Test ✅ Pass 0.225s
Cybersecurity Workflow Memory Test ✅ Pass 3.851s
Function Response Memory Test ✅ Pass 4.230s
Function Argument Memory Test ✅ Pass 5.906s
Penetration Testing Methodology ✅ Pass 0.222s
Vulnerability Assessment Tools ✅ Pass 0.215s
SQL Injection Attack Type ✅ Pass 0.211s
Penetration Testing Framework ✅ Pass 0.212s
Web Application Security Scanner ✅ Pass 0.218s

Summary: 16/16 (100.00%) successful tests

Average latency: 1.055s


reflector (kimi-k2.6)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.994s
Text Transform Uppercase ✅ Pass 1.164s
Math Calculation ✅ Pass 0.903s
Count from 1 to 5 ✅ Pass 1.150s
Streaming Simple Math Streaming ✅ Pass 0.944s
Streaming Count from 1 to 3 Streaming ✅ Pass 1.114s
Answer Stops At The Output Limit ✅ Pass 47.627s

Advanced Tests

Test Result Latency Error
Basic Context Memory Test ✅ Pass 1.300s
Function Argument Memory Test ✅ Pass 1.129s
Function Response Memory Test ✅ Pass 1.006s
Cybersecurity Workflow Memory Test ✅ Pass 0.879s
SQL Injection Attack Type ✅ Pass 1.169s
Penetration Testing Framework ✅ Pass 3.339s
Penetration Testing Methodology ✅ Pass 5.206s
Vulnerability Assessment Tools ✅ Pass 5.164s
Web Application Security Scanner ✅ Pass 4.723s

Summary: 16/16 (100.00%) successful tests

Average latency: 4.864s


searcher (kimi-k2.6)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.216s
Text Transform Uppercase ✅ Pass 0.220s
Count from 1 to 5 ✅ Pass 0.216s
Math Calculation ✅ Pass 0.221s
Streaming Simple Math Streaming ✅ Pass 0.215s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.233s
Basic Echo Function ✅ Pass 1.291s
Streaming Basic Echo Function Streaming ✅ Pass 1.207s
Answer Stops At The Output Limit ✅ Pass 47.414s

Advanced Tests

Test Result Latency Error
Basic Context Memory Test ✅ Pass 0.217s
Search Query Function ✅ Pass 1.293s
JSON Response Function ✅ Pass 1.552s
Function Argument Memory Test ✅ Pass 0.982s
Streaming Search Query Function Streaming ✅ Pass 1.285s
Ask Advice Function ✅ Pass 1.633s
Function Response Memory Test ✅ Pass 1.199s
Cybersecurity Workflow Memory Test ✅ Pass 0.892s
Penetration Testing Memory with Tool Call ✅ Pass 2.075s
Read a file, then edit it via unified diff ✅ Pass 4.938s
SQL Injection Attack Type ✅ Pass 0.221s
Penetration Testing Tool Selection ✅ Pass 1.575s
Web Application Security Scanner ✅ Pass 4.075s
Penetration Testing Framework ✅ Pass 4.747s
Penetration Testing Methodology ✅ Pass 7.495s
Vulnerability Assessment Tools ✅ Pass 8.107s

Summary: 25/25 (100.00%) successful tests

Average latency: 3.741s


enricher (kimi-k2.6)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.226s
Text Transform Uppercase ✅ Pass 0.232s
Count from 1 to 5 ✅ Pass 0.223s
Math Calculation ✅ Pass 0.226s
Basic Echo Function ✅ Pass 0.213s
Streaming Simple Math Streaming ✅ Pass 0.214s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.222s
Streaming Basic Echo Function Streaming ✅ Pass 0.221s
Answer Stops At The Output Limit ✅ Pass 47.444s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.221s
Search Query Function ✅ Pass 0.227s
Ask Advice Function ✅ Pass 0.216s
Streaming Search Query Function Streaming ✅ Pass 0.218s
Basic Context Memory Test ✅ Pass 0.222s
Function Response Memory Test ✅ Pass 0.977s
Function Argument Memory Test ✅ Pass 1.129s
Cybersecurity Workflow Memory Test ✅ Pass 1.162s
Penetration Testing Memory with Tool Call ✅ Pass 2.308s
Read a file, then edit it via unified diff ✅ Pass 4.422s
Penetration Testing Methodology ✅ Pass 0.220s
Vulnerability Assessment Tools ✅ Pass 0.220s
SQL Injection Attack Type ✅ Pass 0.224s
Penetration Testing Framework ✅ Pass 0.219s
Penetration Testing Tool Selection ✅ Pass 0.225s
Web Application Security Scanner ✅ Pass 4.339s

Summary: 25/25 (100.00%) successful tests

Average latency: 2.631s


coder (kimi-k2.7-code)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 1.382s
Text Transform Uppercase ✅ Pass 1.671s
Count from 1 to 5 ✅ Pass 2.281s
Math Calculation ✅ Pass 1.987s
Basic Echo Function ✅ Pass 2.582s
Streaming Simple Math Streaming ✅ Pass 2.048s
Streaming Count from 1 to 3 Streaming ✅ Pass 2.248s
Streaming Basic Echo Function Streaming ✅ Pass 2.117s
Answer Stops At The Output Limit ✅ Pass 38.323s

Advanced Tests

Test Result Latency Error
Search Query Function ✅ Pass 1.805s
JSON Response Function ✅ Pass 2.613s
Basic Context Memory Test ✅ Pass 1.737s
Ask Advice Function ✅ Pass 1.869s
Streaming Search Query Function Streaming ✅ Pass 1.999s
Function Response Memory Test ✅ Pass 1.778s
Function Argument Memory Test ✅ Pass 2.466s
Penetration Testing Memory with Tool Call ✅ Pass 2.243s
Cybersecurity Workflow Memory Test ✅ Pass 2.460s
Read a file, then edit it via unified diff ✅ Pass 5.130s
SQL Injection Attack Type ✅ Pass 1.613s
Penetration Testing Methodology ✅ Pass 3.551s
Penetration Testing Tool Selection ✅ Pass 2.042s
Penetration Testing Framework ✅ Pass 3.659s
Web Application Security Scanner ✅ Pass 3.631s
Vulnerability Assessment Tools ✅ Pass 5.915s

Summary: 25/25 (100.00%) successful tests

Average latency: 3.966s


installer (kimi-k2.6)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 2.216s
Text Transform Uppercase ✅ Pass 3.286s
Count from 1 to 5 ✅ Pass 2.514s
Math Calculation ✅ Pass 1.891s
Basic Echo Function ✅ Pass 2.169s
Streaming Simple Math Streaming ✅ Pass 2.190s
Streaming Basic Echo Function Streaming ✅ Pass 1.847s
Streaming Count from 1 to 3 Streaming ✅ Pass 4.440s
Answer Stops At The Output Limit ✅ Pass 37.054s

Advanced Tests

Test Result Latency Error
Search Query Function ✅ Pass 2.351s
Streaming Search Query Function Streaming ✅ Pass 2.115s
JSON Response Function ✅ Pass 2.404s
Ask Advice Function ✅ Pass 2.770s
Basic Context Memory Test ✅ Pass 2.770s
Function Argument Memory Test ✅ Pass 2.897s
Function Response Memory Test ✅ Pass 2.644s
Cybersecurity Workflow Memory Test ✅ Pass 3.504s
Penetration Testing Memory with Tool Call ✅ Pass 4.919s
Read a file, then edit it via unified diff ✅ Pass 7.809s
Penetration Testing Tool Selection ✅ Pass 2.645s
SQL Injection Attack Type ✅ Pass 5.982s
Vulnerability Assessment Tools ✅ Pass 9.477s
Web Application Security Scanner ✅ Pass 8.751s
Penetration Testing Methodology ✅ Pass 9.875s
Penetration Testing Framework ✅ Pass 12.822s

Summary: 25/25 (100.00%) successful tests

Average latency: 5.654s


pentester (kimi-k2.7-code)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.217s
Simple Math ✅ Pass 1.423s
Text Transform Uppercase ✅ Pass 1.637s
Math Calculation ✅ Pass 2.115s
Count from 1 to 5 ✅ Pass 2.401s
Basic Echo Function ✅ Pass 2.543s
Streaming Simple Math Streaming ✅ Pass 1.913s
Streaming Basic Echo Function Streaming ✅ Pass 1.873s
Streaming Count from 1 to 3 Streaming ✅ Pass 2.047s

Advanced Tests

Test Result Latency Error
Search Query Function ✅ Pass 2.048s
Ask Advice Function ✅ Pass 2.023s
Streaming Search Query Function Streaming ✅ Pass 1.847s
JSON Response Function ✅ Pass 2.665s
Basic Context Memory Test ✅ Pass 2.079s
Function Response Memory Test ✅ Pass 2.141s
Function Argument Memory Test ✅ Pass 2.540s
Cybersecurity Workflow Memory Test ✅ Pass 2.107s
Penetration Testing Memory with Tool Call ✅ Pass 2.180s
Read a file, then edit it via unified diff ✅ Pass 4.619s
SQL Injection Attack Type ✅ Pass 1.991s
Vulnerability Assessment Tools ✅ Pass 4.093s
Penetration Testing Methodology ✅ Pass 5.939s
Penetration Testing Tool Selection ✅ Pass 2.770s
Penetration Testing Framework ✅ Pass 3.857s
Web Application Security Scanner ✅ Pass 3.782s

Summary: 25/25 (100.00%) successful tests

Average latency: 2.514s