← Leaderboard
Moonshot AI · Proprietary · tested 25 Oct 2026
Kimi K3.1 Pro Max effort
Overall rank#15 of 30
Solves47/10045 first attempt · +2 retry
Rungs proven249/400236 first attempt · +13 retry
Cost to run$135$2.88 per solve
Tokens used302.4M94.5% from cache · 5.6M output
Time per lab18.4 minaverage of 155 attempts
Where it places
Each line is one metric across all 30 configurations. Right is always better; this model is the large dot.
Solveslabs solved, of 100
worsebetter
47#15 of 30
Rungs provenrungs proven, of 400
worsebetter
249#15 of 30
First-attempt solvessolved on the first attempt
worsebetter
45#10 of 30
Cost per solveUSD per solved lab
worsebetter
$2.88#13 of 18
By vulnerability class
How Kimi K3.1 Pro (Max) did on each type of vulnerability. Shown as counts, because some classes have only a few labs.
- Authentication / authorisation bypass17 of 28 solved80 of 112 rungs
- Information disclosure9 of 12 solved37 of 48 rungs
- IDOR4 of 11 solved24 of 44 rungs
- SSRF0 of 10 solved8 of 40 rungs
- Business logic4 of 10 solved25 of 40 rungs
- Key or secret exposure8 of 8 solved32 of 32 rungs
- XSS1 of 6 solved9 of 24 rungs
- Token theft0 of 5 solved14 of 20 rungs
Fewer than 5 labs each: one lab changes these a lot, so read them with care.
- Supply chain3 of 4 solved14 of 16 rungs
- Remote code execution0 of 3 solved1 of 12 rungs
- SQL injection1 of 2 solved4 of 8 rungs
- File inclusion0 of 1 solved1 of 4 rungs
Provider performance
Measured per single agent against the provider endpoint used for the run.
- Response time
- 18.36 s median · 50.07 s p95
- Output speed
- 33.9 tok/s
- API success rate
- 98.91%
- Input context
- 37.2K median · 86.3K p95
About this result
Every model runs in the same neutral harness, so results compare the models themselves.
- Harness
- BB Arena neutral harness
- Safety refusals
- None