← Leaderboard
Alibaba · Proprietary · tested 17 Nov 2026
Qwen4 Max Max effort
Overall rank#8 of 19
Solves9/1004 first attempt · +5 retry
Rungs proven132/40095 first attempt · +37 retry
Cost to run$169$18.78 per solve
Tokens used718.8M93.0% from cache · 14.5M output
Time per lab24.6 minaverage of 196 attempts
Where it places
Each line is one metric across all 19 ranked configurations. Right is always better; this model is the large dot.
Solveslabs solved, of 100
worsebetter
9#8 of 19
Rungs provenrungs proven, of 400
worsebetter
132#5 of 19
First-attempt solvessolved on the first attempt
worsebetter
4#12 of 19
Cost per solveUSD per solved lab
worsebetter
$18.78#11 of 18
By vulnerability class
How Qwen4 Max (Max) did on each type of vulnerability. Shown as counts: each class has 10 labs, so one lab is 10% of a class.
- Access control3 of 10 solved24 of 40 rungs
- Authentication & sessions0 of 10 solved14 of 40 rungs
- Injection: queries & data1 of 10 solved14 of 40 rungs
- Code execution0 of 10 solved9 of 40 rungs
- Server-side access0 of 10 solved6 of 40 rungs
- Browser-side1 of 10 solved12 of 40 rungs
- Infrastructure & edge1 of 10 solved11 of 40 rungs
- Business logic & races2 of 10 solved17 of 40 rungs
- Secrets, crypto & supply chain1 of 10 solved17 of 40 rungs
- AI agents0 of 10 solved8 of 40 rungs
Provider performance
Measured per single agent against the provider endpoint used for the run.
- Response time
- 13.89 s median · 43.17 s p95
- Output speed
- 55.8 tok/s
- API success rate
- 98.91%
- Input context
- 43.9K median · 104.8K p95
About this result
Every model runs in the same neutral harness, so results compare the models themselves.
- Harness
- BB Arena neutral harness
- Safety refusals
- None