Skip to content

Fully autonomous AI Agents system capable of performing complex penetration testing tasks

20.9 KB
Raw Blame

LLM Agent Testing Report

Generated: Thu, 24 Sep 2026 22:33:42 UTC

Overall Results

Agent Model Reasoning Success Rate Average Latency
simple MiniMax-M2.7 true 25/25 (100.00%) 0.657s
simple_json MiniMax-M2.7 true 8/8 (100.00%) 0.827s
primary_agent MiniMax-M3 true 25/25 (100.00%) 0.674s
assistant MiniMax-M3 true 25/25 (100.00%) 0.711s
generator MiniMax-M2.7 true 25/25 (100.00%) 0.625s
refiner MiniMax-M2.7 true 25/25 (100.00%) 0.603s
adviser MiniMax-M2.7 true 16/16 (100.00%) 0.583s
reflector MiniMax-M3 true 16/16 (100.00%) 0.747s
searcher MiniMax-M3 true 25/25 (100.00%) 0.576s
enricher MiniMax-M2.7 true 25/25 (100.00%) 0.552s
coder MiniMax-M2.7 true 25/25 (100.00%) 0.645s
installer MiniMax-M3 true 24/25 (96.00%) 0.506s
pentester MiniMax-M2.7 true 25/25 (100.00%) 0.596s

Total: 289/290 (99.66%) successful tests Overall average latency: 0.626s

Detailed Results

simple (MiniMax-M2.7)

Basic Tests

Test Result Latency Error
Streaming Simple Math Streaming ✅ Pass 0.333s
Count from 1 to 5 ✅ Pass 0.345s
Text Transform Uppercase ✅ Pass 0.388s
Math Calculation ✅ Pass 0.398s
Simple Math ✅ Pass 0.400s
Basic Echo Function ✅ Pass 0.403s
Answer Stops At The Output Limit ✅ Pass 0.416s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.528s
Streaming Basic Echo Function Streaming ✅ Pass 0.268s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.223s
Ask Advice Function ✅ Pass 0.226s
Search Query Function ✅ Pass 0.229s
Basic Context Memory Test ✅ Pass 0.210s
Streaming Search Query Function Streaming ✅ Pass 0.271s
Function Argument Memory Test ✅ Pass 1.565s
Function Response Memory Test ✅ Pass 1.607s
Cybersecurity Workflow Memory Test ✅ Pass 3.164s
Penetration Testing Memory with Tool Call ✅ Pass 3.597s
Read a file, then edit it via unified diff ✅ Pass 0.469s
Penetration Testing Methodology ✅ Pass 0.222s
Vulnerability Assessment Tools ✅ Pass 0.216s
SQL Injection Attack Type ✅ Pass 0.242s
Penetration Testing Framework ✅ Pass 0.235s
Web Application Security Scanner ✅ Pass 0.232s
Penetration Testing Tool Selection ✅ Pass 0.228s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.657s


simple_json (MiniMax-M2.7)

Advanced Tests

Test Result Latency Error
Vulnerability Report Memory Test ✅ Pass 5.240s
Person Information JSON ✅ Pass 0.220s
Project Information JSON ✅ Pass 0.211s
User Profile JSON ✅ Pass 0.216s
JSON Array Response Without Schema ✅ Pass 0.221s
Streaming Person Information JSON Streaming ✅ Pass 0.284s

Capability Tests

Test Capability Result Latency Note
Structured Output Refuses A Non-Object Schema structured_output ✅ Pass 0.000s
Structured Output With JSON Schema structured_output ✅ Pass 0.222s

Summary: 8/8 (100.00%) successful tests

Average latency: 0.827s


primary_agent (MiniMax-M3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.220s
Simple Math ✅ Pass 0.225s
Count from 1 to 5 ✅ Pass 0.223s
Text Transform Uppercase ✅ Pass 0.255s
Math Calculation ✅ Pass 0.251s
Basic Echo Function ✅ Pass 0.238s
Streaming Simple Math Streaming ✅ Pass 0.222s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.260s
Streaming Basic Echo Function Streaming ✅ Pass 0.263s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.216s
Search Query Function ✅ Pass 0.212s
Ask Advice Function ✅ Pass 0.210s
Streaming Search Query Function Streaming ✅ Pass 0.267s
Basic Context Memory Test ✅ Pass 0.221s
Function Argument Memory Test ✅ Pass 1.328s
Penetration Testing Memory with Tool Call ✅ Pass 1.871s
Function Response Memory Test ✅ Pass 2.213s
Cybersecurity Workflow Memory Test ✅ Pass 6.263s
Read a file, then edit it via unified diff ✅ Pass 0.469s
Penetration Testing Methodology ✅ Pass 0.229s
Vulnerability Assessment Tools ✅ Pass 0.216s
SQL Injection Attack Type ✅ Pass 0.235s
Penetration Testing Framework ✅ Pass 0.250s
Web Application Security Scanner ✅ Pass 0.242s
Penetration Testing Tool Selection ✅ Pass 0.244s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.674s


assistant (MiniMax-M3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.222s
Simple Math ✅ Pass 0.219s
Math Calculation ✅ Pass 0.220s
Count from 1 to 5 ✅ Pass 0.232s
Text Transform Uppercase ✅ Pass 0.232s
Basic Echo Function ✅ Pass 0.213s
Streaming Simple Math Streaming ✅ Pass 0.227s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.310s
Streaming Basic Echo Function Streaming ✅ Pass 0.316s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.839s
Search Query Function ✅ Pass 0.781s
Ask Advice Function ✅ Pass 0.215s
Basic Context Memory Test ✅ Pass 0.228s
Streaming Search Query Function Streaming ✅ Pass 0.281s
Penetration Testing Memory with Tool Call ✅ Pass 1.980s
Function Argument Memory Test ✅ Pass 2.364s
Function Response Memory Test ✅ Pass 2.407s
Cybersecurity Workflow Memory Test ✅ Pass 4.557s
Read a file, then edit it via unified diff ✅ Pass 0.435s
SQL Injection Attack Type ✅ Pass 0.224s
Penetration Testing Methodology ✅ Pass 0.276s
Vulnerability Assessment Tools ✅ Pass 0.279s
Penetration Testing Framework ✅ Pass 0.223s
Penetration Testing Tool Selection ✅ Pass 0.243s
Web Application Security Scanner ✅ Pass 0.245s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.711s


generator (MiniMax-M2.7)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.234s
Simple Math ✅ Pass 0.239s
Count from 1 to 5 ✅ Pass 0.240s
Text Transform Uppercase ✅ Pass 0.246s
Math Calculation ✅ Pass 0.213s
Basic Echo Function ✅ Pass 0.213s
Streaming Simple Math Streaming ✅ Pass 0.234s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.311s
Streaming Basic Echo Function Streaming ✅ Pass 0.301s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.284s
Search Query Function ✅ Pass 0.235s
Ask Advice Function ✅ Pass 0.232s
Streaming Search Query Function Streaming ✅ Pass 0.284s
Basic Context Memory Test ✅ Pass 0.225s
Function Response Memory Test ✅ Pass 2.030s
Function Argument Memory Test ✅ Pass 2.228s
Penetration Testing Memory with Tool Call ✅ Pass 2.690s
Cybersecurity Workflow Memory Test ✅ Pass 3.078s
Read a file, then edit it via unified diff ✅ Pass 0.432s
Penetration Testing Methodology ✅ Pass 0.216s
Vulnerability Assessment Tools ✅ Pass 0.310s
Penetration Testing Framework ✅ Pass 0.299s
SQL Injection Attack Type ✅ Pass 0.302s
Web Application Security Scanner ✅ Pass 0.279s
Penetration Testing Tool Selection ✅ Pass 0.262s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.625s


refiner (MiniMax-M2.7)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.226s
Simple Math ✅ Pass 0.224s
Text Transform Uppercase ✅ Pass 0.226s
Count from 1 to 5 ✅ Pass 0.225s
Math Calculation ✅ Pass 0.208s
Basic Echo Function ✅ Pass 0.215s
Streaming Simple Math Streaming ✅ Pass 0.210s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.280s
Streaming Basic Echo Function Streaming ✅ Pass 0.265s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.217s
Search Query Function ✅ Pass 0.219s
Ask Advice Function ✅ Pass 0.219s
Basic Context Memory Test ✅ Pass 0.212s
Streaming Search Query Function Streaming ✅ Pass 0.269s
Function Response Memory Test ✅ Pass 1.879s
Function Argument Memory Test ✅ Pass 2.700s
Cybersecurity Workflow Memory Test ✅ Pass 2.243s
Penetration Testing Memory with Tool Call ✅ Pass 3.116s
Read a file, then edit it via unified diff ✅ Pass 0.427s
Penetration Testing Methodology ✅ Pass 0.258s
Vulnerability Assessment Tools ✅ Pass 0.213s
Penetration Testing Framework ✅ Pass 0.226s
Web Application Security Scanner ✅ Pass 0.260s
SQL Injection Attack Type ✅ Pass 0.260s
Penetration Testing Tool Selection ✅ Pass 0.272s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.603s


adviser (MiniMax-M2.7)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.226s
Simple Math ✅ Pass 0.235s
Text Transform Uppercase ✅ Pass 0.239s
Count from 1 to 5 ✅ Pass 0.210s
Math Calculation ✅ Pass 0.215s
Streaming Simple Math Streaming ✅ Pass 0.217s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.330s

Advanced Tests

Test Result Latency Error
Basic Context Memory Test ✅ Pass 0.222s
Function Argument Memory Test ✅ Pass 1.883s
Function Response Memory Test ✅ Pass 2.011s
Cybersecurity Workflow Memory Test ✅ Pass 2.066s
Penetration Testing Methodology ✅ Pass 0.265s
Vulnerability Assessment Tools ✅ Pass 0.282s
SQL Injection Attack Type ✅ Pass 0.291s
Penetration Testing Framework ✅ Pass 0.301s
Web Application Security Scanner ✅ Pass 0.334s

Summary: 16/16 (100.00%) successful tests

Average latency: 0.583s


reflector (MiniMax-M3)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.225s
Text Transform Uppercase ✅ Pass 0.242s
Answer Stops At The Output Limit ✅ Pass 0.251s
Count from 1 to 5 ✅ Pass 0.252s
Math Calculation ✅ Pass 0.239s
Streaming Simple Math Streaming ✅ Pass 0.218s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.264s

Advanced Tests

Test Result Latency Error
Basic Context Memory Test ✅ Pass 0.226s
Cybersecurity Workflow Memory Test ✅ Pass 0.960s
Function Argument Memory Test ✅ Pass 1.765s
Function Response Memory Test ✅ Pass 5.757s
Vulnerability Assessment Tools ✅ Pass 0.315s
Penetration Testing Methodology ✅ Pass 0.324s
SQL Injection Attack Type ✅ Pass 0.305s
Penetration Testing Framework ✅ Pass 0.301s
Web Application Security Scanner ✅ Pass 0.298s

Summary: 16/16 (100.00%) successful tests

Average latency: 0.747s


searcher (MiniMax-M3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.219s
Simple Math ✅ Pass 0.221s
Text Transform Uppercase ✅ Pass 0.224s
Count from 1 to 5 ✅ Pass 0.222s
Math Calculation ✅ Pass 0.219s
Basic Echo Function ✅ Pass 0.212s
Streaming Simple Math Streaming ✅ Pass 0.220s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.278s
Streaming Basic Echo Function Streaming ✅ Pass 0.263s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.212s
Search Query Function ✅ Pass 0.284s
Ask Advice Function ✅ Pass 0.288s
Basic Context Memory Test ✅ Pass 0.214s
Streaming Search Query Function Streaming ✅ Pass 0.263s
Penetration Testing Memory with Tool Call ✅ Pass 1.261s
Function Argument Memory Test ✅ Pass 1.715s
Cybersecurity Workflow Memory Test ✅ Pass 1.688s
Function Response Memory Test ✅ Pass 4.528s
Read a file, then edit it via unified diff ✅ Pass 0.446s
Penetration Testing Methodology ✅ Pass 0.281s
Vulnerability Assessment Tools ✅ Pass 0.223s
SQL Injection Attack Type ✅ Pass 0.219s
Penetration Testing Framework ✅ Pass 0.221s
Web Application Security Scanner ✅ Pass 0.226s
Penetration Testing Tool Selection ✅ Pass 0.231s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.576s


enricher (MiniMax-M2.7)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.250s
Simple Math ✅ Pass 0.252s
Text Transform Uppercase ✅ Pass 0.242s
Count from 1 to 5 ✅ Pass 0.249s
Math Calculation ✅ Pass 0.235s
Basic Echo Function ✅ Pass 0.217s
Streaming Simple Math Streaming ✅ Pass 0.213s
Streaming Basic Echo Function Streaming ✅ Pass 0.302s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.320s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.227s
Search Query Function ✅ Pass 0.215s
Ask Advice Function ✅ Pass 0.221s
Streaming Search Query Function Streaming ✅ Pass 0.266s
Basic Context Memory Test ✅ Pass 0.216s
Function Response Memory Test ✅ Pass 1.497s
Function Argument Memory Test ✅ Pass 1.913s
Penetration Testing Memory with Tool Call ✅ Pass 2.809s
Cybersecurity Workflow Memory Test ✅ Pass 2.369s
Read a file, then edit it via unified diff ✅ Pass 0.435s
Penetration Testing Methodology ✅ Pass 0.234s
Vulnerability Assessment Tools ✅ Pass 0.228s
SQL Injection Attack Type ✅ Pass 0.226s
Penetration Testing Framework ✅ Pass 0.225s
Web Application Security Scanner ✅ Pass 0.219s
Penetration Testing Tool Selection ✅ Pass 0.216s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.552s


coder (MiniMax-M2.7)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.225s
Answer Stops At The Output Limit ✅ Pass 0.230s
Text Transform Uppercase ✅ Pass 0.244s
Count from 1 to 5 ✅ Pass 0.254s
Math Calculation ✅ Pass 0.242s
Basic Echo Function ✅ Pass 0.221s
Streaming Simple Math Streaming ✅ Pass 0.225s
Streaming Basic Echo Function Streaming ✅ Pass 0.293s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.337s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.214s
Search Query Function ✅ Pass 0.219s
Ask Advice Function ✅ Pass 0.212s
Basic Context Memory Test ✅ Pass 0.213s
Streaming Search Query Function Streaming ✅ Pass 0.267s
Function Argument Memory Test ✅ Pass 2.096s
Function Response Memory Test ✅ Pass 2.836s
Cybersecurity Workflow Memory Test ✅ Pass 2.742s
Penetration Testing Memory with Tool Call ✅ Pass 3.164s
Read a file, then edit it via unified diff ✅ Pass 0.518s
Penetration Testing Methodology ✅ Pass 0.218s
Vulnerability Assessment Tools ✅ Pass 0.219s
SQL Injection Attack Type ✅ Pass 0.230s
Penetration Testing Framework ✅ Pass 0.228s
Web Application Security Scanner ✅ Pass 0.232s
Penetration Testing Tool Selection ✅ Pass 0.231s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.645s


installer (MiniMax-M3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.223s
Simple Math ✅ Pass 0.239s
Text Transform Uppercase ✅ Pass 0.238s
Count from 1 to 5 ✅ Pass 0.238s
Math Calculation ✅ Pass 0.218s
Basic Echo Function ✅ Pass 0.219s
Streaming Simple Math Streaming ✅ Pass 0.217s
Streaming Basic Echo Function Streaming ✅ Pass 0.266s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.299s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.222s
Search Query Function ✅ Pass 0.233s
Ask Advice Function ✅ Pass 0.230s
Streaming Search Query Function Streaming ✅ Pass 0.273s
Basic Context Memory Test ✅ Pass 0.214s
Penetration Testing Memory with Tool Call ✅ Pass 1.436s
Function Argument Memory Test ✅ Pass 1.545s
Cybersecurity Workflow Memory Test ❌ Fail 2.015s expected text 'example.com' not found; stop reason: stop
Function Response Memory Test ✅ Pass 2.573s
Read a file, then edit it via unified diff ✅ Pass 0.438s
Penetration Testing Methodology ✅ Pass 0.210s
Vulnerability Assessment Tools ✅ Pass 0.214s
SQL Injection Attack Type ✅ Pass 0.216s
Penetration Testing Framework ✅ Pass 0.214s
Web Application Security Scanner ✅ Pass 0.224s
Penetration Testing Tool Selection ✅ Pass 0.224s

Summary: 24/25 (96.00%) successful tests

Average latency: 0.506s


pentester (MiniMax-M2.7)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.232s
Simple Math ✅ Pass 0.227s
Text Transform Uppercase ✅ Pass 0.229s
Count from 1 to 5 ✅ Pass 0.221s
Math Calculation ✅ Pass 0.216s
Basic Echo Function ✅ Pass 0.217s
Streaming Simple Math Streaming ✅ Pass 0.211s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.284s
Streaming Basic Echo Function Streaming ✅ Pass 0.292s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.250s
Search Query Function ✅ Pass 0.218s
Ask Advice Function ✅ Pass 0.221s
Streaming Search Query Function Streaming ✅ Pass 0.274s
Basic Context Memory Test ✅ Pass 0.245s
Function Argument Memory Test ✅ Pass 1.904s
Read a file, then edit it via unified diff ✅ Pass 0.439s
Function Response Memory Test ✅ Pass 2.182s
Cybersecurity Workflow Memory Test ✅ Pass 2.453s
Penetration Testing Memory with Tool Call ✅ Pass 3.180s
Penetration Testing Methodology ✅ Pass 0.233s
Vulnerability Assessment Tools ✅ Pass 0.229s
SQL Injection Attack Type ✅ Pass 0.228s
Penetration Testing Framework ✅ Pass 0.224s
Web Application Security Scanner ✅ Pass 0.239s
Penetration Testing Tool Selection ✅ Pass 0.237s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.596s