← Leaderboard
Orca · Proprietary · tested 19 Oct 2026
OrcaCyber Zero 1.5 High effort
Overall rank#7 of 28
Solves55/10047 first attempt · +8 retry
Rungs proven279/400238 first attempt · +41 retry
Cost to run$501$9.12 per solve
Tokens used560.1M96.2% from cache · 7.4M output
Time per lab19.1 minaverage of 150 attempts
Where it places
Each line is one metric across all 28 configurations. Right is always better; this model is the large dot.
Solveslabs solved, of 100
worsebetter
55#7 of 28
Rungs provenrungs proven, of 400
worsebetter
279#5 of 28
First-attempt solvessolved on the first attempt
worsebetter
47#8 of 28
Cost per solveUSD per solved lab
worsebetter
$9.12#17 of 17
By vulnerability class
How OrcaCyber Zero 1.5 (High) did on each type of vulnerability. Shown as counts, because some classes have only a few labs.
- Authentication / authorisation bypass16 of 28 solved83 of 112 rungs
- Information disclosure9 of 12 solved39 of 48 rungs
- IDOR4 of 11 solved30 of 44 rungs
- SSRF0 of 10 solved9 of 40 rungs
- Business logic6 of 10 solved32 of 40 rungs
- Key or secret exposure8 of 8 solved32 of 32 rungs
- XSS3 of 6 solved15 of 24 rungs
- Token theft5 of 5 solved20 of 20 rungs
Fewer than 5 labs each: one lab changes these a lot, so read them with care.
- Supply chain2 of 4 solved9 of 16 rungs
- Remote code execution0 of 3 solved1 of 12 rungs
- SQL injection2 of 2 solved8 of 8 rungs
- File inclusion0 of 1 solved1 of 4 rungs
Provider performance
Measured per single agent against the provider endpoint used for the run.
- Response time
- 9.86 s median · 31.30 s p95
- Output speed
- 54.2 tok/s
- API success rate
- 99.40%
- Input context
- 42.7K median · 127.1K p95
About this result
Every model runs in the same neutral harness, so results compare the models themselves.
- Harness
- BB Arena neutral harness
- Safety refusals
- None