Skip to content

Fully autonomous AI Agents system capable of performing complex penetration testing tasks

21.1 KB
Raw Blame

LLM Agent Testing Report

Generated: Thu, 24 Sep 2026 21:38:14 UTC

Overall Results

Agent Model Reasoning Success Rate Average Latency
simple glm-5.3-flash true 25/25 (100.00%) 0.621s
simple_json glm-5.3-flash true 8/8 (100.00%) 0.397s
primary_agent glm-5.3 true 25/25 (100.00%) 2.774s
assistant glm-5.3-flash true 25/25 (100.00%) 0.556s
generator glm-5.3 true 25/25 (100.00%) 3.530s
refiner glm-5.3 true 25/25 (100.00%) 2.588s
adviser glm-5.3 true 16/16 (100.00%) 1.895s
reflector glm-5.3-flash true 16/16 (100.00%) 0.378s
searcher glm-5.3-flash true 24/25 (96.00%) 0.545s
enricher glm-5.3-flash true 24/25 (96.00%) 0.518s
coder glm-5.3 true 25/25 (100.00%) 0.650s
installer glm-5.3-flash true 25/25 (100.00%) 0.554s
pentester glm-5.3 true 25/25 (100.00%) 0.671s

Total: 288/290 (99.31%) successful tests Overall average latency: 1.258s

Detailed Results

simple (glm-5.3-flash)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.250s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.250s
Basic Echo Function ✅ Pass 0.250s
Answer Stops At The Output Limit ✅ Pass 0.263s
Text Transform Uppercase ✅ Pass 0.278s
Count from 1 to 5 ✅ Pass 0.278s
Streaming Simple Math Streaming ✅ Pass 0.290s
Math Calculation ✅ Pass 0.292s
Streaming Basic Echo Function Streaming ✅ Pass 0.240s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.188s
Search Query Function ✅ Pass 0.189s
Ask Advice Function ✅ Pass 0.188s
Streaming Search Query Function Streaming ✅ Pass 0.173s
Basic Context Memory Test ✅ Pass 0.172s
Function Argument Memory Test ✅ Pass 1.158s
Cybersecurity Workflow Memory Test ✅ Pass 1.209s
Function Response Memory Test ✅ Pass 1.344s
Penetration Testing Memory with Tool Call ✅ Pass 6.988s
Read a file, then edit it via unified diff ✅ Pass 0.358s
Penetration Testing Methodology ✅ Pass 0.214s
SQL Injection Attack Type ✅ Pass 0.206s
Vulnerability Assessment Tools ✅ Pass 0.219s
Penetration Testing Framework ✅ Pass 0.175s
Web Application Security Scanner ✅ Pass 0.175s
Penetration Testing Tool Selection ✅ Pass 0.173s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.621s


simple_json (glm-5.3-flash)

Advanced Tests

Test Result Latency Error
Vulnerability Report Memory Test ✅ Pass 2.090s
Person Information JSON ✅ Pass 0.182s
Project Information JSON ✅ Pass 0.178s
User Profile JSON ✅ Pass 0.182s
JSON Array Response Without Schema ✅ Pass 0.176s
Streaming Person Information JSON Streaming ✅ Pass 0.186s

Capability Tests

Test Capability Result Latency Note
Structured Output Refuses A Non-Object Schema structured_output ✅ Pass 0.000s
Structured Output With JSON Schema structured_output ✅ Pass 0.177s

Summary: 8/8 (100.00%) successful tests

Average latency: 0.397s


primary_agent (glm-5.3)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.199s
Answer Stops At The Output Limit ✅ Pass 0.206s
Text Transform Uppercase ✅ Pass 0.224s
Math Calculation ✅ Pass 0.221s
Count from 1 to 5 ✅ Pass 0.235s
Basic Echo Function ✅ Pass 0.229s
Streaming Simple Math Streaming ✅ Pass 0.233s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.202s
Streaming Basic Echo Function Streaming ✅ Pass 0.197s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.185s
Search Query Function ✅ Pass 0.170s
Ask Advice Function ✅ Pass 0.173s
Streaming Search Query Function Streaming ✅ Pass 0.174s
Basic Context Memory Test ✅ Pass 0.171s
Function Response Memory Test ✅ Pass 16.103s
Cybersecurity Workflow Memory Test ✅ Pass 15.361s
Function Argument Memory Test ✅ Pass 16.206s
Penetration Testing Memory with Tool Call ✅ Pass 17.403s
Read a file, then edit it via unified diff ✅ Pass 0.359s
Vulnerability Assessment Tools ✅ Pass 0.186s
Penetration Testing Methodology ✅ Pass 0.188s
SQL Injection Attack Type ✅ Pass 0.179s
Penetration Testing Framework ✅ Pass 0.178s
Web Application Security Scanner ✅ Pass 0.184s
Penetration Testing Tool Selection ✅ Pass 0.181s

Summary: 25/25 (100.00%) successful tests

Average latency: 2.774s


assistant (glm-5.3-flash)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.178s
Math Calculation ✅ Pass 0.186s
Basic Echo Function ✅ Pass 0.213s
Text Transform Uppercase ✅ Pass 0.240s
Count from 1 to 5 ✅ Pass 0.225s
Answer Stops At The Output Limit ✅ Pass 0.252s
Streaming Simple Math Streaming ✅ Pass 0.235s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.280s
Streaming Basic Echo Function Streaming ✅ Pass 0.316s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.323s
Search Query Function ✅ Pass 0.224s
Ask Advice Function ✅ Pass 0.184s
Streaming Search Query Function Streaming ✅ Pass 0.224s
Basic Context Memory Test ✅ Pass 0.198s
Function Argument Memory Test ✅ Pass 1.109s
Function Response Memory Test ✅ Pass 1.482s
Cybersecurity Workflow Memory Test ✅ Pass 1.300s
Penetration Testing Memory with Tool Call ✅ Pass 5.293s
Read a file, then edit it via unified diff ✅ Pass 0.362s
Penetration Testing Methodology ✅ Pass 0.185s
Vulnerability Assessment Tools ✅ Pass 0.171s
SQL Injection Attack Type ✅ Pass 0.191s
Penetration Testing Framework ✅ Pass 0.171s
Web Application Security Scanner ✅ Pass 0.174s
Penetration Testing Tool Selection ✅ Pass 0.177s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.556s


generator (glm-5.3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.272s
Math Calculation ✅ Pass 0.266s
Simple Math ✅ Pass 0.266s
Count from 1 to 5 ✅ Pass 0.266s
Text Transform Uppercase ✅ Pass 0.266s
Basic Echo Function ✅ Pass 0.255s
Streaming Simple Math Streaming ✅ Pass 0.291s
Streaming Basic Echo Function Streaming ✅ Pass 0.348s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.355s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.172s
Search Query Function ✅ Pass 0.176s
Ask Advice Function ✅ Pass 0.173s
Streaming Search Query Function Streaming ✅ Pass 0.172s
Basic Context Memory Test ✅ Pass 0.173s
Function Argument Memory Test ✅ Pass 13.103s
Function Response Memory Test ✅ Pass 22.170s
Cybersecurity Workflow Memory Test ✅ Pass 22.870s
Penetration Testing Memory with Tool Call ✅ Pass 25.186s
Read a file, then edit it via unified diff ✅ Pass 0.359s
Penetration Testing Methodology ✅ Pass 0.177s
Vulnerability Assessment Tools ✅ Pass 0.173s
SQL Injection Attack Type ✅ Pass 0.173s
Penetration Testing Framework ✅ Pass 0.199s
Web Application Security Scanner ✅ Pass 0.196s
Penetration Testing Tool Selection ✅ Pass 0.191s

Summary: 25/25 (100.00%) successful tests

Average latency: 3.530s


refiner (glm-5.3)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.303s
Count from 1 to 5 ✅ Pass 0.303s
Text Transform Uppercase ✅ Pass 0.303s
Answer Stops At The Output Limit ✅ Pass 0.303s
Math Calculation ✅ Pass 0.209s
Basic Echo Function ✅ Pass 0.174s
Streaming Simple Math Streaming ✅ Pass 0.200s
Streaming Basic Echo Function Streaming ✅ Pass 0.205s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.214s

Advanced Tests

Test Result Latency Error
Search Query Function ✅ Pass 0.204s
JSON Response Function ✅ Pass 0.204s
Ask Advice Function ✅ Pass 0.198s
Streaming Search Query Function Streaming ✅ Pass 0.194s
Basic Context Memory Test ✅ Pass 0.197s
Function Response Memory Test ✅ Pass 13.149s
Function Argument Memory Test ✅ Pass 14.333s
Penetration Testing Memory with Tool Call ✅ Pass 15.139s
Cybersecurity Workflow Memory Test ✅ Pass 17.307s
Read a file, then edit it via unified diff ✅ Pass 0.354s
Penetration Testing Methodology ✅ Pass 0.177s
Vulnerability Assessment Tools ✅ Pass 0.192s
SQL Injection Attack Type ✅ Pass 0.206s
Penetration Testing Framework ✅ Pass 0.206s
Web Application Security Scanner ✅ Pass 0.213s
Penetration Testing Tool Selection ✅ Pass 0.192s

Summary: 25/25 (100.00%) successful tests

Average latency: 2.588s


adviser (glm-5.3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.192s
Text Transform Uppercase ✅ Pass 0.205s
Simple Math ✅ Pass 0.205s
Count from 1 to 5 ✅ Pass 0.189s
Math Calculation ✅ Pass 0.167s
Streaming Simple Math Streaming ✅ Pass 0.175s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.189s

Advanced Tests

Test Result Latency Error
Basic Context Memory Test ✅ Pass 0.186s
Cybersecurity Workflow Memory Test ✅ Pass 2.271s
Function Argument Memory Test ✅ Pass 16.510s
Function Response Memory Test ✅ Pass 9.049s
Penetration Testing Methodology ✅ Pass 0.192s
Vulnerability Assessment Tools ✅ Pass 0.186s
SQL Injection Attack Type ✅ Pass 0.203s
Web Application Security Scanner ✅ Pass 0.194s
Penetration Testing Framework ✅ Pass 0.202s

Summary: 16/16 (100.00%) successful tests

Average latency: 1.895s


reflector (glm-5.3-flash)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.180s
Answer Stops At The Output Limit ✅ Pass 0.189s
Math Calculation ✅ Pass 0.205s
Count from 1 to 5 ✅ Pass 0.209s
Text Transform Uppercase ✅ Pass 0.209s
Streaming Simple Math Streaming ✅ Pass 0.187s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.175s

Advanced Tests

Test Result Latency Error
Basic Context Memory Test ✅ Pass 0.178s
Function Argument Memory Test ✅ Pass 0.959s
Function Response Memory Test ✅ Pass 1.216s
Cybersecurity Workflow Memory Test ✅ Pass 1.340s
Penetration Testing Methodology ✅ Pass 0.190s
Vulnerability Assessment Tools ✅ Pass 0.184s
SQL Injection Attack Type ✅ Pass 0.188s
Penetration Testing Framework ✅ Pass 0.207s
Web Application Security Scanner ✅ Pass 0.225s

Summary: 16/16 (100.00%) successful tests

Average latency: 0.378s


searcher (glm-5.3-flash)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.186s
Text Transform Uppercase ✅ Pass 0.193s
Simple Math ✅ Pass 0.193s
Count from 1 to 5 ✅ Pass 0.188s
Streaming Simple Math Streaming ✅ Pass 0.194s
Basic Echo Function ✅ Pass 0.201s
Math Calculation ✅ Pass 0.201s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.168s
Streaming Basic Echo Function Streaming ✅ Pass 0.207s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.235s
Search Query Function ❌ Fail 0.183s expected function 'search' not found in tool calls: function search was called with arguments that do not match: invalid argument 'query': expected...
Ask Advice Function ✅ Pass 0.219s
Basic Context Memory Test ✅ Pass 0.267s
Streaming Search Query Function Streaming ✅ Pass 0.340s
Function Response Memory Test ✅ Pass 1.018s
Function Argument Memory Test ✅ Pass 1.294s
Cybersecurity Workflow Memory Test ✅ Pass 1.221s
Penetration Testing Memory with Tool Call ✅ Pass 5.577s
Read a file, then edit it via unified diff ✅ Pass 0.373s
Penetration Testing Methodology ✅ Pass 0.225s
Vulnerability Assessment Tools ✅ Pass 0.213s
SQL Injection Attack Type ✅ Pass 0.175s
Penetration Testing Framework ✅ Pass 0.171s
Web Application Security Scanner ✅ Pass 0.171s
Penetration Testing Tool Selection ✅ Pass 0.203s

Summary: 24/25 (96.00%) successful tests

Average latency: 0.545s


enricher (glm-5.3-flash)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.190s
Answer Stops At The Output Limit ✅ Pass 0.191s
Text Transform Uppercase ✅ Pass 0.172s
Basic Echo Function ✅ Pass 0.179s
Math Calculation ✅ Pass 0.188s
Count from 1 to 5 ✅ Pass 0.188s
Streaming Simple Math Streaming ✅ Pass 0.173s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.202s
Streaming Basic Echo Function Streaming ✅ Pass 0.215s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.294s
Search Query Function ❌ Fail 0.176s expected function 'search' not found in tool calls: function search was called with arguments that do not match: invalid argument 'query': expected...
Ask Advice Function ✅ Pass 0.171s
Streaming Search Query Function Streaming ✅ Pass 0.180s
Basic Context Memory Test ✅ Pass 0.191s
Function Argument Memory Test ✅ Pass 1.162s
Function Response Memory Test ✅ Pass 1.208s
Cybersecurity Workflow Memory Test ✅ Pass 1.218s
Penetration Testing Memory with Tool Call ✅ Pass 5.049s
Read a file, then edit it via unified diff ✅ Pass 0.372s
Penetration Testing Methodology ✅ Pass 0.202s
Vulnerability Assessment Tools ✅ Pass 0.235s
SQL Injection Attack Type ✅ Pass 0.236s
Penetration Testing Framework ✅ Pass 0.190s
Web Application Security Scanner ✅ Pass 0.173s
Penetration Testing Tool Selection ✅ Pass 0.183s

Summary: 24/25 (96.00%) successful tests

Average latency: 0.518s


coder (glm-5.3)

Basic Tests

Test Result Latency Error
Simple Math ✅ Pass 0.183s
Answer Stops At The Output Limit ✅ Pass 0.202s
Text Transform Uppercase ✅ Pass 0.190s
Math Calculation ✅ Pass 0.203s
Count from 1 to 5 ✅ Pass 0.203s
Basic Echo Function ✅ Pass 0.180s
Streaming Simple Math Streaming ✅ Pass 0.201s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.213s
Streaming Basic Echo Function Streaming ✅ Pass 0.220s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.175s
Search Query Function ✅ Pass 0.174s
Ask Advice Function ✅ Pass 0.175s
Streaming Search Query Function Streaming ✅ Pass 0.173s
Basic Context Memory Test ✅ Pass 0.177s
Function Response Memory Test ✅ Pass 1.680s
Function Argument Memory Test ✅ Pass 1.927s
Cybersecurity Workflow Memory Test ✅ Pass 2.139s
Read a file, then edit it via unified diff ✅ Pass 0.356s
Penetration Testing Memory with Tool Call ✅ Pass 6.389s
Penetration Testing Methodology ✅ Pass 0.183s
Vulnerability Assessment Tools ✅ Pass 0.171s
SQL Injection Attack Type ✅ Pass 0.172s
Penetration Testing Framework ✅ Pass 0.176s
Web Application Security Scanner ✅ Pass 0.186s
Penetration Testing Tool Selection ✅ Pass 0.181s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.650s


installer (glm-5.3-flash)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.204s
Simple Math ✅ Pass 0.208s
Text Transform Uppercase ✅ Pass 0.216s
Count from 1 to 5 ✅ Pass 0.218s
Math Calculation ✅ Pass 0.193s
Basic Echo Function ✅ Pass 0.176s
Streaming Simple Math Streaming ✅ Pass 0.189s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.202s
Streaming Basic Echo Function Streaming ✅ Pass 0.208s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.174s
Search Query Function ✅ Pass 0.181s
Ask Advice Function ✅ Pass 0.182s
Streaming Search Query Function Streaming ✅ Pass 0.186s
Basic Context Memory Test ✅ Pass 0.185s
Function Argument Memory Test ✅ Pass 1.125s
Function Response Memory Test ✅ Pass 1.326s
Cybersecurity Workflow Memory Test ✅ Pass 1.841s
Read a file, then edit it via unified diff ✅ Pass 0.380s
Penetration Testing Memory with Tool Call ✅ Pass 5.387s
Penetration Testing Methodology ✅ Pass 0.181s
Vulnerability Assessment Tools ✅ Pass 0.182s
SQL Injection Attack Type ✅ Pass 0.174s
Penetration Testing Framework ✅ Pass 0.174s
Web Application Security Scanner ✅ Pass 0.167s
Penetration Testing Tool Selection ✅ Pass 0.168s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.554s


pentester (glm-5.3)

Basic Tests

Test Result Latency Error
Answer Stops At The Output Limit ✅ Pass 0.188s
Simple Math ✅ Pass 0.190s
Text Transform Uppercase ✅ Pass 0.191s
Count from 1 to 5 ✅ Pass 0.190s
Math Calculation ✅ Pass 0.172s
Basic Echo Function ✅ Pass 0.175s
Streaming Simple Math Streaming ✅ Pass 0.179s
Streaming Count from 1 to 3 Streaming ✅ Pass 0.188s
Streaming Basic Echo Function Streaming ✅ Pass 0.188s

Advanced Tests

Test Result Latency Error
JSON Response Function ✅ Pass 0.218s
Search Query Function ✅ Pass 0.174s
Ask Advice Function ✅ Pass 0.179s
Streaming Search Query Function Streaming ✅ Pass 0.183s
Basic Context Memory Test ✅ Pass 0.189s
Function Response Memory Test ✅ Pass 1.873s
Function Argument Memory Test ✅ Pass 2.092s
Cybersecurity Workflow Memory Test ✅ Pass 1.847s
Read a file, then edit it via unified diff ✅ Pass 0.365s
Penetration Testing Methodology ✅ Pass 0.172s
Vulnerability Assessment Tools ✅ Pass 0.180s
SQL Injection Attack Type ✅ Pass 0.193s
Penetration Testing Framework ✅ Pass 0.180s
Web Application Security Scanner ✅ Pass 0.185s
Penetration Testing Tool Selection ✅ Pass 0.179s
Penetration Testing Memory with Tool Call ✅ Pass 6.898s

Summary: 25/25 (100.00%) successful tests

Average latency: 0.671s