Provider comparisons · DeepSeek V4.1 Flash
DeepSeek V4.1 Flash API providers compared on agentic security tasks
Many providers serve DeepSeek V4.1 Flash and say it's the same model. We ran each of them on the same BB Arena security labs, in the same harness, to see what you actually get: solve rate, speed, reliability and cost.
- Labs
- 10
- BB Arena security labs
- Attempts per provider
- 40
- 4 per lab, each from scratch
- Time limit
- 25 min
- per attempt
- Reasoning
- Max
- same harness for every provider
Providers compared
DeepSeek's own API is pinned on top as the reference; the rest are sorted by solve rate. Click a column to sort.
| 2DeepSeekReference: DeepSeek's own API | Native | Original | 45.0%18 of 40 | 2.98 | 50.0%20 of 40 | 144 | 621 | 4.4 | 99.5% | 1.2B | $8.32 | $0.46 | 0 |
| 1ParasailVia OpenRouter, pinned | FP8 | Original | 45.0%18 of 40 | 3.00 | 50.0%20 of 40 | 162 | 837 | 3.6 | 99.5% | 983.2M | $14.02 | $0.78 | 0 |
| 2DeepSeek via OpenRouterVia OpenRouter, pinned | Native | Original | 45.0%18 of 40 | 2.98 | 42.5%17 of 40 | 143 | 750 | 4.4 | 99.5% | 946.4M | $6.75 | $0.37 | 0 |
| 4WaferVia OpenRouter, pinned | MXFP4 | Original | 42.5%17 of 40 | 2.90 | 30.0%12 of 40 | 174 | 854 | 3.2 | 98.7% | 1.1B | $11.09 | $0.65 | 7 |
| 5PerchReseller, own API | Native | Original | 40.0%16 of 40 | 2.93 | 50.0%20 of 40 | 123 | 865 | 4.8 | 99.5% | 882.1M | $12.76 | $0.80 | 0 |
| 6MistralVia OpenRouter, pinned | FP8 | Original | 40.0%16 of 40 | 2.88 | 45.0%18 of 40 | 194 | 918 | 3.3 | 99.6% | 1.3B | $14.16 | $0.88 | 0 |
| 7BoundlessReseller, own API | Native | Original | 40.0%16 of 40 | 2.83 | 55.0%22 of 40 | 107 | 716 | 5.9 | 99.4% | 656.3M | $8.27 | $0.52 | 1 |
| 8OrcaCyber Flash 1.0Reseller, own API | Undeclared | Abliterated | 37.5%15 of 40 | 2.83 | 50.0%20 of 40 | 185 | 1036 | 3.5 | 89.6% | 1.3B | $282.50 | $18.83 | 0 |
| 9OrcaRouter latestReseller, own API | Native | Original | 37.5%15 of 40 | 2.75 | 60.0%24 of 40 | 92 | 654 | 6.9 | 94.7% | 582.3M | $8.69 | $0.58 | 0 |
| 10DeepInfraVia OpenRouter, pinned | FP8 | Original | 32.5%13 of 40 | 2.53 | 65.0%26 of 40 | 121 | 761 | 5.0 | 99.5% | 805.4M | $4.33 | $0.33 | 0 |
| 11NovitaVia OpenRouter, pinned | FP8 | Original | 32.5%13 of 40 | 2.43 | 62.5%25 of 40 | 117 | 682 | 5.3 | 98.9% | 882.3M | $7.98 | $0.61 | 0 |
| 12CommandCodeReseller, own API | Undeclared | Original | 30.0%12 of 40 | 2.33 | 70.0%28 of 40 | 83 | 511 | 6.7 | 98.4% | 615.0M | $11.73 | $0.98 | 0 |
| 12Lyber AbliteratedReseller, own API | FP8 | Abliterated | 30.0%12 of 40 | 2.33 | 67.5%27 of 40 | 232 | 866 | 1.8 | 99.6% | 1.7B | $54.76 | $4.56 | 0 |
| 14Boundless AbliteratedReseller, own API | Native | Abliterated | 27.5%11 of 40 | 2.10 | 75.0%30 of 40 | 96 | 595 | 4.4 | 99.5% | 820.8M | $14.35 | $1.30 | 0 |
| 15OrcaRouter 1017Reseller, own API | Native | Original | 25.0%10 of 40 | 2.45 | 77.5%31 of 40 | 89 | 687 | 7.2 | 95.1% | 603.1M | $8.68 | $0.87 | 0 |
| 16abliteration.ai AbliteratedReseller, own API | FP8 | Abliterated | 25.0%10 of 40 | 2.33 | 75.0%30 of 40 | 97 | 697 | 5.7 | 98.7% | 698.3M | $86.35 | $8.63 | 0 |
| 17OpenCodeReseller, own API | Undeclared | Original | 20.0%8 of 40 | 2.18 | 77.5%31 of 40 | 72 | 440 | 8.1 | 92.9% | 440.7M | $16.24 | $2.03 | 0 |
| 18MorphVia OpenRouter, pinned | FP8 | Original | 17.5%7 of 40 | 1.80 | 17.5%7 of 40 | 198 | 1049 | 3.3 | 98.0% | 401.0M | $6.44 | $0.92 | 25 |
| 19SiliconFlowVia OpenRouter, pinned | FP8 | Original | 15.0%6 of 40 | 1.93 | 85.0%34 of 40 | 77 | 441 | 6.8 | 98.6% | 623.6M | $8.61 | $1.44 | 0 |
| 20Ollama CloudReseller, own API | Undeclared | Original | 12.5%5 of 40 | 2.03 | 90.0%36 of 40 | 59 | 313 | 11.2 | 99.0% | 371.5M | $3.52 | $0.70 | 0 |
| 21Silk Compute AbliteratedReseller, own API | FP8 | Abliterated | 10.0%4 of 40 | 1.93 | 90.0%36 of 40 | 48 | 402 | 8.0 | 98.4% | 314.4M | $15.06 | $3.77 | 0 |
| –OpenInference Could not sustain the loadVia OpenRouter, pinned | FP4 | Original | 0.0%0 of 40 | 0.08 | n/a | 122 | 118 | 2.9 | 89.9% | 6.9M | $0.33 | n/a | 40 |
| –Baidu Could not sustain the loadVia OpenRouter, pinned | FP8 | Original | 0.0%0 of 40 | 0.03 | n/a | 60 | 106 | 3.6 | 64.6% | 1.7M | $0.25 | n/a | 40 |
Every provider is scored out of 40 attempts (10 labs, 4 each), so a gap of a few points is noise. Cost is each provider's own API price for the tokens we used, off-peak discounts included where the provider has them (prices as listed 16 Nov 2026).
By vulnerability class
Labs solved per type of vulnerability, to compare providers where it matters to you. Each cell is the attempts fully solved out of the attempts on that class's labs (4 per lab).
| Provider | Access control2 labs | Auth & sessions2 labs | Server-side2 labs | Secrets & crypto2 labs | Logic & races1 lab | AI agents1 lab |
|---|---|---|---|---|---|---|
| 2DeepSeek | 1/8 | 2/8 | 4/8 | 7/8 | 3/4 | 1/4 |
| 1Parasail | 1/8 | 4/8 | 3/8 | 7/8 | 3/4 | 0/4 |
| 2DeepSeek via OpenRouter | 0/8 | 3/8 | 4/8 | 8/8 | 3/4 | 0/4 |
| 4Wafer | 2/8 | 3/8 | 5/8 | 5/8 | 2/4 | 0/4 |
| 5Perch | 3/8 | 4/8 | 2/8 | 5/8 | 2/4 | 0/4 |
| 6Mistral | 2/8 | 2/8 | 4/8 | 5/8 | 3/4 | 0/4 |
| 7Boundless | 2/8 | 1/8 | 4/8 | 5/8 | 4/4 | 0/4 |
| 8OrcaCyber Flash 1.0Abliterated | 0/8 | 2/8 | 4/8 | 6/8 | 3/4 | 0/4 |
| 9OrcaRouter latest | 1/8 | 2/8 | 5/8 | 4/8 | 3/4 | 0/4 |
| 10DeepInfra | 0/8 | 3/8 | 3/8 | 6/8 | 1/4 | 0/4 |
| 11Novita | 2/8 | 1/8 | 4/8 | 4/8 | 2/4 | 0/4 |
| 12CommandCode | 0/8 | 1/8 | 5/8 | 3/8 | 3/4 | 0/4 |
| 12Lyber AbliteratedAbliterated | 0/8 | 1/8 | 5/8 | 5/8 | 1/4 | 0/4 |
| 14Boundless AbliteratedAbliterated | 0/8 | 2/8 | 3/8 | 6/8 | 0/4 | 0/4 |
| 15OrcaRouter 1017 | 1/8 | 1/8 | 2/8 | 6/8 | 0/4 | 0/4 |
| 16abliteration.ai AbliteratedAbliterated | 1/8 | 2/8 | 2/8 | 3/8 | 2/4 | 0/4 |
| 17OpenCode | 0/8 | 0/8 | 3/8 | 4/8 | 1/4 | 0/4 |
| 18Morph | 0/8 | 0/8 | 3/8 | 4/8 | 0/4 | 0/4 |
| 19SiliconFlow | 0/8 | 2/8 | 4/8 | 0/8 | 0/4 | 0/4 |
| 20Ollama Cloud | 0/8 | 2/8 | 2/8 | 1/8 | 0/4 | 0/4 |
| 21Silk Compute AbliteratedAbliterated | 0/8 | 1/8 | 0/8 | 2/8 | 1/4 | 0/4 |
| –OpenInference | 0/8 | 0/8 | 0/8 | 0/8 | 0/4 | 0/4 |
| –Baidu | 0/8 | 0/8 | 0/8 | 0/8 | 0/4 | 0/4 |
Notes, method and limitsWhat went wrong per provider, how we tested, and what the numbers can and cannot tell you
What went wrong
- Wafer. 7 cut-off replies.
- Boundless. 1 cut-off reply.
- OrcaCyber Flash 1.0. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights.
- OrcaRouter latest. Slow output (92 tok/s vs 144 on DeepSeek's API).
- CommandCode. Slow output (83 tok/s vs 144 on DeepSeek's API).
- Lyber Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights.
- Boundless Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights; slow output (96 tok/s vs 144 on DeepSeek's API).
- OrcaRouter 1017. Slow output (89 tok/s vs 144 on DeepSeek's API).
- abliteration.ai Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights; slow output (97 tok/s vs 144 on DeepSeek's API).
- OpenCode. Slow output (72 tok/s vs 144 on DeepSeek's API).
- Morph. 25 cut-off replies.
- SiliconFlow. Slow output (77 tok/s vs 144 on DeepSeek's API).
- Ollama Cloud. 56% of requests rate limited; slow output (59 tok/s vs 144 on DeepSeek's API).
- Silk Compute Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights; slow output (48 tok/s vs 144 on DeepSeek's API).
- OpenInference. Rate limited (90% of requests); stopped, all 40 attempts count as failures.
- Baidu. Rate limited (81% of requests); stopped, all 40 attempts count as failures.
How we tested
- Tested 15-22 Nov 2026: 23 providers.
- DeepSeek V4.1 Flash at maximum reasoning, the same dated release everywhere, in our leaderboard's harness: a target, a shell, no hints, 25 minutes per attempt.
- 10 BB Arena security labs, 4 attempts each per provider, 10 attempts at once. OpenRouter providers were pinned with fallbacks off.
- A provider's own failures count against it; they are never retried.
- Out of time
- attempts the 25-minute limit ended while the model was still working.
- Speed
- output tokens per second per busy worker.
- Throughput
- output tokens per second across 10 attempts at once.
- Latency
- median time of one model request.
- Cache
- share of input served from the provider's prompt cache.
- Failures
- attempts the provider ended: a rate limit, cut-off reply or provider error.
Limits
- 40 attempts per provider only show large differences.
- Providers ran at different times; load and rate limits change during the day.
- Precision is as the provider declares it.