← Gemini 3.1 Pro · API · High thinking
Evidenceacceptedvendor_reported
SWE-bench Verified
Google DeepMind — Gemini 3.1 Pro · SWE Verified
ev_2026_07_gem31pro_swe_vendor
Result
80.6%
Confidence B
- Tested
- 2026-02-19
- Runs
- 1
Configuration
How it was measured
- Exact model
- gemini-3.1-pro-preview
- Reasoning
- high
- Agent harness
- vendor_gemini_swe
- Tool access
- agentic coding tools
- Prompting
- vendor published agent eval
Secondary STR — vendor-reported SWE-bench Verified for Gemini 3.1 Pro.
Caveats
Known limitations
- Vendor-reported; cannot receive confidence S.
- Harness ≠ openlm / Vals mini-swe rows.
Weighting
Evidence weight factors
The engine weights each evidence row by these factors (0–1) when fusing scores — higher is more trusted.
relevance0.90
quality0.55
recency0.85
health0.80
comparability0.65
reliability0.70
Benchmark
SWE-bench Verified
Human-filtered 500-instance subset of SWE-bench evaluating whether models can resolve real GitHub issues in existing repositories under a controlled agent harness.
Benchmark source →