← Leaderboard
OpenAI · Proprietary · tested 10 Oct 2026
GPT-6 Sol High effort
Overall rank#19 of 25
Solves31/10027 first attempt · +4 retry
Rungs proven206/400187 first attempt · +19 retry
Cost to run$34.50$1.11 per solve
Tokens used45.5M80.9% from cache · 1.2M output
Time per lab3.9 minaverage of 144 attempts
Where it places
Each line is one metric across all 25 configurations. Right is always better; this model is the large dot.
Solveslabs solved, of 100
worsebetter
31#19 of 25
Rungs provenrungs proven, of 400
worsebetter
206#16 of 25
First-attempt solvessolved on the first attempt
worsebetter
27#17 of 25
Cost per solveUSD per solved lab
worsebetter
$1.11#6 of 15
By vulnerability class
How GPT-6 Sol (High) did on each type of vulnerability. Shown as counts, because some classes have only a few labs.
- Authentication / authorisation bypass13 of 28 solved70 of 112 rungs
- Information disclosure5 of 12 solved33 of 48 rungs
- IDOR0 of 11 solved13 of 44 rungs
- SSRF0 of 10 solved8 of 40 rungs
- Business logic5 of 10 solved28 of 40 rungs
- Key or secret exposure6 of 8 solved28 of 32 rungs
- XSS1 of 6 solved5 of 24 rungs
- Token theft0 of 5 solved10 of 20 rungs
Fewer than 5 labs each: one lab changes these a lot, so read them with care.
- Supply chain0 of 4 solved6 of 16 rungs
- Remote code execution0 of 3 solved1 of 12 rungs
- SQL injection1 of 2 solved4 of 8 rungs
- File inclusion0 of 1 solved0 of 4 rungs
Provider performance
Measured per single agent against the provider endpoint used for the run.
- Response time
- 7.73 s median · 19.13 s p95
- Output speed
- 43.4 tok/s
- API success rate
- 90.79%
- Input context
- 11.3K median · 31.0K p95
About this result
Every model runs in the same neutral harness, so results compare the models themselves.
- Harness
- BB Arena neutral harness
- Safety refusals
- None