LLM Agent Testing Report
Generated: Fri, 25 Sep 2026 00:15:09 UTC
Overall Results
| Agent |
Model |
Reasoning |
Success Rate |
Average Latency |
| simple |
claude-haiku-4-5 |
false |
25/25 (100.00%) |
1.981s |
| simple_json |
claude-haiku-4-5 |
false |
8/8 (100.00%) |
1.036s |
| primary_agent |
claude-sonnet-5 |
true |
26/26 (100.00%) |
2.799s |
| assistant |
claude-sonnet-5 |
true |
26/26 (100.00%) |
2.936s |
| generator |
claude-opus-4-8 |
true |
26/26 (100.00%) |
4.156s |
| refiner |
claude-opus-4-8 |
true |
26/26 (100.00%) |
3.751s |
| adviser |
claude-sonnet-5 |
true |
17/17 (100.00%) |
3.577s |
| reflector |
claude-haiku-4-5 |
true |
16/16 (100.00%) |
3.148s |
| searcher |
claude-haiku-4-5 |
true |
25/25 (100.00%) |
2.677s |
| enricher |
claude-haiku-4-5 |
false |
25/25 (100.00%) |
2.044s |
| coder |
claude-sonnet-5 |
true |
26/26 (100.00%) |
2.848s |
| installer |
claude-sonnet-5 |
true |
26/26 (100.00%) |
2.841s |
| pentester |
claude-sonnet-5 |
true |
26/26 (100.00%) |
3.007s |
Total: 298/298 (100.00%) successful tests
Overall average latency: 2.912s
Detailed Results
simple (claude-haiku-4-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Math Calculation |
✅ Pass |
0.874s |
|
| Simple Math |
✅ Pass |
0.888s |
|
| Streaming Simple Math Streaming |
✅ Pass |
0.890s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
0.891s |
|
| Text Transform Uppercase |
✅ Pass |
0.909s |
|
| Count from 1 to 5 |
✅ Pass |
1.241s |
|
| Basic Echo Function |
✅ Pass |
1.271s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.166s |
|
| Answer Stops At The Output Limit |
✅ Pass |
15.809s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| JSON Response Function |
✅ Pass |
1.270s |
|
| Search Query Function |
✅ Pass |
0.850s |
|
| Basic Context Memory Test |
✅ Pass |
0.808s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
0.913s |
|
| Function Argument Memory Test |
✅ Pass |
0.661s |
|
| Function Response Memory Test |
✅ Pass |
0.625s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
0.718s |
|
| Ask Advice Function |
✅ Pass |
1.991s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
1.547s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
3.774s |
|
| SQL Injection Attack Type |
✅ Pass |
0.796s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.178s |
|
| Penetration Testing Methodology |
✅ Pass |
2.452s |
|
| Web Application Security Scanner |
✅ Pass |
2.119s |
|
| Vulnerability Assessment Tools |
✅ Pass |
3.221s |
|
| Penetration Testing Framework |
✅ Pass |
2.641s |
|
Summary: 25/25 (100.00%) successful tests
Average latency: 1.981s
simple_json (claude-haiku-4-5)
Advanced Tests
| Test |
Result |
Latency |
Error |
| Vulnerability Report Memory Test |
✅ Pass |
0.952s |
|
| Person Information JSON |
✅ Pass |
0.824s |
|
| Project Information JSON |
✅ Pass |
0.964s |
|
| User Profile JSON |
✅ Pass |
1.240s |
|
| JSON Array Response Without Schema |
✅ Pass |
0.906s |
|
| Streaming Person Information JSON Streaming |
✅ Pass |
0.997s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Structured Output Refuses A Non-Object Schema |
structured_output |
✅ Pass |
0.000s |
|
| Structured Output With JSON Schema |
structured_output |
✅ Pass |
2.397s |
|
Summary: 8/8 (100.00%) successful tests
Average latency: 1.036s
primary_agent (claude-sonnet-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Text Transform Uppercase |
✅ Pass |
1.263s |
|
| Math Calculation |
✅ Pass |
1.000s |
|
| Simple Math |
✅ Pass |
1.492s |
|
| Basic Echo Function |
✅ Pass |
1.726s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.184s |
|
| Count from 1 to 5 |
✅ Pass |
2.454s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.510s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.779s |
|
| Answer Stops At The Output Limit |
✅ Pass |
18.677s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| JSON Response Function |
✅ Pass |
1.508s |
|
| Search Query Function |
✅ Pass |
1.754s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
1.458s |
|
| Ask Advice Function |
✅ Pass |
1.833s |
|
| Basic Context Memory Test |
✅ Pass |
1.390s |
|
| Function Argument Memory Test |
✅ Pass |
1.116s |
|
| Function Response Memory Test |
✅ Pass |
0.964s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
0.804s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
2.535s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
4.239s |
|
| SQL Injection Attack Type |
✅ Pass |
1.923s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.583s |
|
| Penetration Testing Methodology |
✅ Pass |
3.646s |
|
| Web Application Security Scanner |
✅ Pass |
3.270s |
|
| Penetration Testing Framework |
✅ Pass |
4.360s |
|
| Vulnerability Assessment Tools |
✅ Pass |
6.727s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
2.562s |
|
Summary: 26/26 (100.00%) successful tests
Average latency: 2.799s
assistant (claude-sonnet-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Text Transform Uppercase |
✅ Pass |
1.132s |
|
| Simple Math |
✅ Pass |
1.370s |
|
| Math Calculation |
✅ Pass |
0.917s |
|
| Count from 1 to 5 |
✅ Pass |
2.172s |
|
| Basic Echo Function |
✅ Pass |
1.757s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.449s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.480s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.993s |
|
| Answer Stops At The Output Limit |
✅ Pass |
18.052s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| JSON Response Function |
✅ Pass |
1.527s |
|
| Search Query Function |
✅ Pass |
1.589s |
|
| Ask Advice Function |
✅ Pass |
1.690s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
1.568s |
|
| Function Response Memory Test |
✅ Pass |
1.024s |
|
| Function Argument Memory Test |
✅ Pass |
1.072s |
|
| Basic Context Memory Test |
✅ Pass |
1.572s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.147s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
2.984s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
4.620s |
|
| SQL Injection Attack Type |
✅ Pass |
2.002s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.855s |
|
| Vulnerability Assessment Tools |
✅ Pass |
5.407s |
|
| Penetration Testing Methodology |
✅ Pass |
5.796s |
|
| Penetration Testing Framework |
✅ Pass |
4.730s |
|
| Web Application Security Scanner |
✅ Pass |
4.508s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
2.895s |
|
Summary: 26/26 (100.00%) successful tests
Average latency: 2.936s
generator (claude-opus-4-8)
Basic Tests
| Test |
Result |
Latency |
Error |
| Count from 1 to 5 |
✅ Pass |
1.350s |
|
| Text Transform Uppercase |
✅ Pass |
1.769s |
|
| Simple Math |
✅ Pass |
1.836s |
|
| Math Calculation |
✅ Pass |
1.167s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.171s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.174s |
|
| Basic Echo Function |
✅ Pass |
2.307s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.868s |
|
| Answer Stops At The Output Limit |
✅ Pass |
30.394s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| JSON Response Function |
✅ Pass |
1.745s |
|
| Search Query Function |
✅ Pass |
1.789s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
1.644s |
|
| Ask Advice Function |
✅ Pass |
2.367s |
|
| Function Argument Memory Test |
✅ Pass |
1.388s |
|
| Function Response Memory Test |
✅ Pass |
1.073s |
|
| Basic Context Memory Test |
✅ Pass |
1.567s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.302s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
3.787s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
6.241s |
|
| Penetration Testing Methodology |
✅ Pass |
5.411s |
|
| Vulnerability Assessment Tools |
✅ Pass |
7.381s |
|
| Penetration Testing Tool Selection |
✅ Pass |
5.637s |
|
| SQL Injection Attack Type |
✅ Pass |
6.556s |
|
| Web Application Security Scanner |
✅ Pass |
6.281s |
|
| Penetration Testing Framework |
✅ Pass |
6.726s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
4.122s |
|
Summary: 26/26 (100.00%) successful tests
Average latency: 4.156s
refiner (claude-opus-4-8)
Basic Tests
| Test |
Result |
Latency |
Error |
| Simple Math |
✅ Pass |
1.328s |
|
| Count from 1 to 5 |
✅ Pass |
1.165s |
|
| Text Transform Uppercase |
✅ Pass |
1.331s |
|
| Math Calculation |
✅ Pass |
1.117s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.116s |
|
| Basic Echo Function |
✅ Pass |
1.967s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.919s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.480s |
|
| Answer Stops At The Output Limit |
✅ Pass |
30.239s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| JSON Response Function |
✅ Pass |
2.356s |
|
| Search Query Function |
✅ Pass |
2.388s |
|
| Ask Advice Function |
✅ Pass |
2.508s |
|
| Function Argument Memory Test |
✅ Pass |
1.091s |
|
| Basic Context Memory Test |
✅ Pass |
1.218s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
3.793s |
|
| Function Response Memory Test |
✅ Pass |
1.794s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.549s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
4.052s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
4.725s |
|
| SQL Injection Attack Type |
✅ Pass |
3.770s |
|
| Penetration Testing Methodology |
✅ Pass |
6.680s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.856s |
|
| Web Application Security Scanner |
✅ Pass |
2.248s |
|
| Vulnerability Assessment Tools |
✅ Pass |
7.482s |
|
| Penetration Testing Framework |
✅ Pass |
4.295s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
4.056s |
|
Summary: 26/26 (100.00%) successful tests
Average latency: 3.751s
adviser (claude-sonnet-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Simple Math |
✅ Pass |
1.265s |
|
| Text Transform Uppercase |
✅ Pass |
1.288s |
|
| Math Calculation |
✅ Pass |
1.340s |
|
| Count from 1 to 5 |
✅ Pass |
2.118s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.464s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.340s |
|
| Answer Stops At The Output Limit |
✅ Pass |
19.117s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| Function Argument Memory Test |
✅ Pass |
1.522s |
|
| Function Response Memory Test |
✅ Pass |
0.948s |
|
| Basic Context Memory Test |
✅ Pass |
2.910s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
2.748s |
|
| SQL Injection Attack Type |
✅ Pass |
1.909s |
|
| Web Application Security Scanner |
✅ Pass |
3.889s |
|
| Penetration Testing Methodology |
✅ Pass |
5.324s |
|
| Penetration Testing Framework |
✅ Pass |
4.514s |
|
| Vulnerability Assessment Tools |
✅ Pass |
5.654s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
3.446s |
|
Summary: 17/17 (100.00%) successful tests
Average latency: 3.577s
reflector (claude-haiku-4-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Simple Math |
✅ Pass |
0.924s |
|
| Text Transform Uppercase |
✅ Pass |
0.816s |
|
| Count from 1 to 5 |
✅ Pass |
1.037s |
|
| Math Calculation |
✅ Pass |
1.155s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.025s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.125s |
|
| Answer Stops At The Output Limit |
✅ Pass |
18.402s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| Basic Context Memory Test |
✅ Pass |
1.152s |
|
| Function Argument Memory Test |
✅ Pass |
1.203s |
|
| Function Response Memory Test |
✅ Pass |
1.470s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.313s |
|
| SQL Injection Attack Type |
✅ Pass |
2.528s |
|
| Vulnerability Assessment Tools |
✅ Pass |
5.107s |
|
| Penetration Testing Methodology |
✅ Pass |
5.361s |
|
| Penetration Testing Framework |
✅ Pass |
4.245s |
|
| Web Application Security Scanner |
✅ Pass |
3.492s |
|
Summary: 16/16 (100.00%) successful tests
Average latency: 3.148s
searcher (claude-haiku-4-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Simple Math |
✅ Pass |
1.048s |
|
| Text Transform Uppercase |
✅ Pass |
1.072s |
|
| Math Calculation |
✅ Pass |
0.877s |
|
| Count from 1 to 5 |
✅ Pass |
2.131s |
|
| Basic Echo Function |
✅ Pass |
1.452s |
|
| Streaming Simple Math Streaming |
✅ Pass |
0.919s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.236s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.561s |
|
| Answer Stops At The Output Limit |
✅ Pass |
17.417s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| Search Query Function |
✅ Pass |
1.128s |
|
| JSON Response Function |
✅ Pass |
1.634s |
|
| Ask Advice Function |
✅ Pass |
1.243s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
1.479s |
|
| Basic Context Memory Test |
✅ Pass |
1.282s |
|
| Function Argument Memory Test |
✅ Pass |
1.207s |
|
| Function Response Memory Test |
✅ Pass |
1.456s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.312s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
1.934s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
4.047s |
|
| SQL Injection Attack Type |
✅ Pass |
2.336s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.881s |
|
| Penetration Testing Framework |
✅ Pass |
3.955s |
|
| Penetration Testing Methodology |
✅ Pass |
4.970s |
|
| Vulnerability Assessment Tools |
✅ Pass |
5.260s |
|
| Web Application Security Scanner |
✅ Pass |
4.075s |
|
Summary: 25/25 (100.00%) successful tests
Average latency: 2.677s
enricher (claude-haiku-4-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Simple Math |
✅ Pass |
0.677s |
|
| Text Transform Uppercase |
✅ Pass |
0.751s |
|
| Count from 1 to 5 |
✅ Pass |
1.019s |
|
| Math Calculation |
✅ Pass |
0.746s |
|
| Basic Echo Function |
✅ Pass |
1.230s |
|
| Streaming Simple Math Streaming |
✅ Pass |
0.773s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
0.796s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
0.917s |
|
| Answer Stops At The Output Limit |
✅ Pass |
17.005s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| Search Query Function |
✅ Pass |
0.956s |
|
| JSON Response Function |
✅ Pass |
1.233s |
|
| Ask Advice Function |
✅ Pass |
0.881s |
|
| Basic Context Memory Test |
✅ Pass |
0.817s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
0.970s |
|
| Function Response Memory Test |
✅ Pass |
0.763s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
0.783s |
|
| Function Argument Memory Test |
✅ Pass |
1.784s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
1.700s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
4.046s |
|
| SQL Injection Attack Type |
✅ Pass |
0.813s |
|
| Vulnerability Assessment Tools |
✅ Pass |
2.650s |
|
| Web Application Security Scanner |
✅ Pass |
2.158s |
|
| Penetration Testing Methodology |
✅ Pass |
3.528s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.460s |
|
| Penetration Testing Framework |
✅ Pass |
2.630s |
|
Summary: 25/25 (100.00%) successful tests
Average latency: 2.044s
coder (claude-sonnet-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Simple Math |
✅ Pass |
1.298s |
|
| Text Transform Uppercase |
✅ Pass |
1.407s |
|
| Count from 1 to 5 |
✅ Pass |
1.131s |
|
| Math Calculation |
✅ Pass |
1.020s |
|
| Basic Echo Function |
✅ Pass |
1.673s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.335s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.451s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.662s |
|
| Answer Stops At The Output Limit |
✅ Pass |
19.048s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| Streaming Search Query Function Streaming |
✅ Pass |
1.422s |
|
| Basic Context Memory Test |
✅ Pass |
1.311s |
|
| JSON Response Function |
✅ Pass |
1.702s |
|
| Function Argument Memory Test |
✅ Pass |
1.102s |
|
| Search Query Function |
✅ Pass |
1.626s |
|
| Ask Advice Function |
✅ Pass |
1.748s |
|
| Function Response Memory Test |
✅ Pass |
1.056s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.173s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
2.974s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
4.579s |
|
| SQL Injection Attack Type |
✅ Pass |
2.046s |
|
| Penetration Testing Tool Selection |
✅ Pass |
2.171s |
|
| Penetration Testing Methodology |
✅ Pass |
4.087s |
|
| Web Application Security Scanner |
✅ Pass |
3.726s |
|
| Vulnerability Assessment Tools |
✅ Pass |
5.611s |
|
| Penetration Testing Framework |
✅ Pass |
5.141s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
2.532s |
|
Summary: 26/26 (100.00%) successful tests
Average latency: 2.848s
installer (claude-sonnet-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Simple Math |
✅ Pass |
1.268s |
|
| Text Transform Uppercase |
✅ Pass |
0.963s |
|
| Count from 1 to 5 |
✅ Pass |
0.999s |
|
| Math Calculation |
✅ Pass |
1.077s |
|
| Basic Echo Function |
✅ Pass |
1.651s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.397s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.450s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.630s |
|
| Answer Stops At The Output Limit |
✅ Pass |
19.376s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| Search Query Function |
✅ Pass |
1.595s |
|
| JSON Response Function |
✅ Pass |
1.612s |
|
| Ask Advice Function |
✅ Pass |
1.742s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
1.674s |
|
| Function Argument Memory Test |
✅ Pass |
0.898s |
|
| Basic Context Memory Test |
✅ Pass |
1.485s |
|
| Function Response Memory Test |
✅ Pass |
1.092s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.249s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
2.692s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
5.661s |
|
| SQL Injection Attack Type |
✅ Pass |
2.723s |
|
| Penetration Testing Methodology |
✅ Pass |
5.018s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.630s |
|
| Web Application Security Scanner |
✅ Pass |
2.487s |
|
| Penetration Testing Framework |
✅ Pass |
3.700s |
|
| Vulnerability Assessment Tools |
✅ Pass |
6.283s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
2.490s |
|
Summary: 26/26 (100.00%) successful tests
Average latency: 2.841s
pentester (claude-sonnet-5)
Basic Tests
| Test |
Result |
Latency |
Error |
| Text Transform Uppercase |
✅ Pass |
1.372s |
|
| Count from 1 to 5 |
✅ Pass |
0.934s |
|
| Simple Math |
✅ Pass |
1.536s |
|
| Math Calculation |
✅ Pass |
1.060s |
|
| Streaming Simple Math Streaming |
✅ Pass |
1.354s |
|
| Streaming Count from 1 to 3 Streaming |
✅ Pass |
1.360s |
|
| Basic Echo Function |
✅ Pass |
1.548s |
|
| Streaming Basic Echo Function Streaming |
✅ Pass |
1.608s |
|
| Answer Stops At The Output Limit |
✅ Pass |
19.345s |
|
Advanced Tests
| Test |
Result |
Latency |
Error |
| JSON Response Function |
✅ Pass |
1.562s |
|
| Search Query Function |
✅ Pass |
1.580s |
|
| Function Argument Memory Test |
✅ Pass |
0.896s |
|
| Basic Context Memory Test |
✅ Pass |
1.441s |
|
| Streaming Search Query Function Streaming |
✅ Pass |
1.662s |
|
| Ask Advice Function |
✅ Pass |
1.878s |
|
| Function Response Memory Test |
✅ Pass |
0.990s |
|
| Cybersecurity Workflow Memory Test |
✅ Pass |
1.093s |
|
| Penetration Testing Memory with Tool Call |
✅ Pass |
2.511s |
|
| Read a file, then edit it via unified diff |
✅ Pass |
5.174s |
|
| Penetration Testing Tool Selection |
✅ Pass |
1.559s |
|
| SQL Injection Attack Type |
✅ Pass |
2.172s |
|
| Web Application Security Scanner |
✅ Pass |
4.206s |
|
| Penetration Testing Framework |
✅ Pass |
5.018s |
|
| Penetration Testing Methodology |
✅ Pass |
6.336s |
|
| Vulnerability Assessment Tools |
✅ Pass |
7.407s |
|
Capability Tests
| Test |
Capability |
Result |
Latency |
Note |
| Adaptive Thinking Produces Reasoning |
adaptive_thinking |
✅ Pass |
2.559s |
|
Summary: 26/26 (100.00%) successful tests
Average latency: 3.007s