Provider comparisons · DeepSeek V4.1 Flash

DeepSeek V4.1 Flash API providers compared on agentic security tasks

Many providers serve DeepSeek V4.1 Flash and say it's the same model. We ran each of them on the same BB Arena security labs, in the same harness, to see what you actually get: solve rate, speed, reliability and cost.

Labs
10
BB Arena security labs
Attempts per provider
40
4 per lab, each from scratch
Time limit
25 min
per attempt
Reasoning
Max
same harness for every provider

Providers compared

DeepSeek's own API is pinned on top as the reference; the rest are sorted by solve rate. Click a column to sort.

2DeepSeekReference: DeepSeek's own APINativeOriginal45.0%18 of 402.9850.0%20 of 401446214.499.5%1.2B$8.32$0.460
1ParasailVia OpenRouter, pinnedFP8Original45.0%18 of 403.0050.0%20 of 401628373.699.5%983.2M$14.02$0.780
2DeepSeek via OpenRouterVia OpenRouter, pinnedNativeOriginal45.0%18 of 402.9842.5%17 of 401437504.499.5%946.4M$6.75$0.370
4WaferVia OpenRouter, pinnedMXFP4Original42.5%17 of 402.9030.0%12 of 401748543.298.7%1.1B$11.09$0.657
5PerchReseller, own APINativeOriginal40.0%16 of 402.9350.0%20 of 401238654.899.5%882.1M$12.76$0.800
6MistralVia OpenRouter, pinnedFP8Original40.0%16 of 402.8845.0%18 of 401949183.399.6%1.3B$14.16$0.880
7BoundlessReseller, own APINativeOriginal40.0%16 of 402.8355.0%22 of 401077165.999.4%656.3M$8.27$0.521
8OrcaCyber Flash 1.0Reseller, own APIUndeclaredAbliterated37.5%15 of 402.8350.0%20 of 4018510363.589.6%1.3B$282.50$18.830
9OrcaRouter latestReseller, own APINativeOriginal37.5%15 of 402.7560.0%24 of 40926546.994.7%582.3M$8.69$0.580
10DeepInfraVia OpenRouter, pinnedFP8Original32.5%13 of 402.5365.0%26 of 401217615.099.5%805.4M$4.33$0.330
11NovitaVia OpenRouter, pinnedFP8Original32.5%13 of 402.4362.5%25 of 401176825.398.9%882.3M$7.98$0.610
12CommandCodeReseller, own APIUndeclaredOriginal30.0%12 of 402.3370.0%28 of 40835116.798.4%615.0M$11.73$0.980
12Lyber AbliteratedReseller, own APIFP8Abliterated30.0%12 of 402.3367.5%27 of 402328661.899.6%1.7B$54.76$4.560
14Boundless AbliteratedReseller, own APINativeAbliterated27.5%11 of 402.1075.0%30 of 40965954.499.5%820.8M$14.35$1.300
15OrcaRouter 1017Reseller, own APINativeOriginal25.0%10 of 402.4577.5%31 of 40896877.295.1%603.1M$8.68$0.870
16abliteration.ai AbliteratedReseller, own APIFP8Abliterated25.0%10 of 402.3375.0%30 of 40976975.798.7%698.3M$86.35$8.630
17OpenCodeReseller, own APIUndeclaredOriginal20.0%8 of 402.1877.5%31 of 40724408.192.9%440.7M$16.24$2.030
18MorphVia OpenRouter, pinnedFP8Original17.5%7 of 401.8017.5%7 of 4019810493.398.0%401.0M$6.44$0.9225
19SiliconFlowVia OpenRouter, pinnedFP8Original15.0%6 of 401.9385.0%34 of 40774416.898.6%623.6M$8.61$1.440
20Ollama CloudReseller, own APIUndeclaredOriginal12.5%5 of 402.0390.0%36 of 405931311.299.0%371.5M$3.52$0.700
21Silk Compute AbliteratedReseller, own APIFP8Abliterated10.0%4 of 401.9390.0%36 of 40484028.098.4%314.4M$15.06$3.770

Every provider is scored out of 40 attempts (10 labs, 4 each), so a gap of a few points is noise. Cost is each provider's own API price for the tokens we used, off-peak discounts included where the provider has them (prices as listed 16 Nov 2026).

By vulnerability class

Labs solved per type of vulnerability, to compare providers where it matters to you. Each cell is the attempts fully solved out of the attempts on that class's labs (4 per lab).

ProviderAccess control2 labsAuth & sessions2 labsServer-side2 labsSecrets & crypto2 labsLogic & races1 labAI agents1 lab
2DeepSeek1/82/84/87/83/41/4
1Parasail1/84/83/87/83/40/4
2DeepSeek via OpenRouter0/83/84/88/83/40/4
4Wafer2/83/85/85/82/40/4
5Perch3/84/82/85/82/40/4
6Mistral2/82/84/85/83/40/4
7Boundless2/81/84/85/84/40/4
8OrcaCyber Flash 1.0Abliterated0/82/84/86/83/40/4
9OrcaRouter latest1/82/85/84/83/40/4
10DeepInfra0/83/83/86/81/40/4
11Novita2/81/84/84/82/40/4
12CommandCode0/81/85/83/83/40/4
12Lyber AbliteratedAbliterated0/81/85/85/81/40/4
14Boundless AbliteratedAbliterated0/82/83/86/80/40/4
15OrcaRouter 10171/81/82/86/80/40/4
16abliteration.ai AbliteratedAbliterated1/82/82/83/82/40/4
17OpenCode0/80/83/84/81/40/4
18Morph0/80/83/84/80/40/4
19SiliconFlow0/82/84/80/80/40/4
20Ollama Cloud0/82/82/81/80/40/4
21Silk Compute AbliteratedAbliterated0/81/80/82/81/40/4
0% solved100%Best in classWith 4 to 8 attempts a cell, one or two solves either way is noise.
Notes, method and limitsWhat went wrong per provider, how we tested, and what the numbers can and cannot tell you

What went wrong

  • Wafer. 7 cut-off replies.
  • Boundless. 1 cut-off reply.
  • OrcaCyber Flash 1.0. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights.
  • OrcaRouter latest. Slow output (92 tok/s vs 144 on DeepSeek's API).
  • CommandCode. Slow output (83 tok/s vs 144 on DeepSeek's API).
  • Lyber Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights.
  • Boundless Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights; slow output (96 tok/s vs 144 on DeepSeek's API).
  • OrcaRouter 1017. Slow output (89 tok/s vs 144 on DeepSeek's API).
  • abliteration.ai Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights; slow output (97 tok/s vs 144 on DeepSeek's API).
  • OpenCode. Slow output (72 tok/s vs 144 on DeepSeek's API).
  • Morph. 25 cut-off replies.
  • SiliconFlow. Slow output (77 tok/s vs 144 on DeepSeek's API).
  • Ollama Cloud. 56% of requests rate limited; slow output (59 tok/s vs 144 on DeepSeek's API).
  • Silk Compute Abliterated. A modified model: DeepSeek V4.1 Flash with its refusal behaviour removed (abliterated), not the original weights; slow output (48 tok/s vs 144 on DeepSeek's API).
  • OpenInference. Rate limited (90% of requests); stopped, all 40 attempts count as failures.
  • Baidu. Rate limited (81% of requests); stopped, all 40 attempts count as failures.

How we tested

  • Tested 15-22 Nov 2026: 23 providers.
  • DeepSeek V4.1 Flash at maximum reasoning, the same dated release everywhere, in our leaderboard's harness: a target, a shell, no hints, 25 minutes per attempt.
  • 10 BB Arena security labs, 4 attempts each per provider, 10 attempts at once. OpenRouter providers were pinned with fallbacks off.
  • A provider's own failures count against it; they are never retried.
Out of time
attempts the 25-minute limit ended while the model was still working.
Speed
output tokens per second per busy worker.
Throughput
output tokens per second across 10 attempts at once.
Latency
median time of one model request.
Cache
share of input served from the provider's prompt cache.
Failures
attempts the provider ended: a rate limit, cut-off reply or provider error.

Limits

  • 40 attempts per provider only show large differences.
  • Providers ran at different times; load and rate limits change during the day.
  • Precision is as the provider declares it.