← DeepSeek V4 Pro · API · Max effort
Evidenceacceptedthird_party_leaderboard
SWE-bench Verified
SWE-bench+ / openlm.ai — DeepSeek-V4
ev_2026_07_dsv4pro_swe_openlm
Result
76.2%
Confidence B
- Tested
- 2026-04-23
- Runs
- 1
Configuration
How it was measured
- Exact model
- deepseek-v4-pro
- Reasoning
- max
- Agent harness
- mini-swe-agent / board default
- Tool access
- agent harness (board standard)
- Prompting
- board default agent protocol
SECONDARY STR — openlm.ai SWE-bench+ mirror. Prefer Vals mini-swe primary (ev_2026_07_dsv4pro_swe_vals 77.4%) for peer comparisons.
Caveats
Known limitations
- Secondary to Vals mini-swe primary (same harness family, slightly different board).
- Do not equate with vendor SWE Verified 80.6% (different harness/config).
Weighting
Evidence weight factors
The engine weights each evidence row by these factors (0–1) when fusing scores — higher is more trusted.
relevance0.90
quality0.75
recency0.90
health0.80
comparability0.75
reliability0.75
Benchmark
SWE-bench Verified
Human-filtered 500-instance subset of SWE-bench evaluating whether models can resolve real GitHub issues in existing repositories under a controlled agent harness.
Benchmark source →