← GPT-5.6 Sol · API · Max effort
Evidenceacceptedvendor_reported
SWE-bench Pro
OpenAI GPT-5.6 launch table — SWE-Bench Pro
ev_2026_07_gpt56sol_swe_pro_vendor
Result
64.6%
Confidence B
- Tested
- 2026-07-09
- Runs
- 1
Configuration
How it was measured
- Exact model
- gpt-5.6-sol
- Reasoning
- max
- Agent harness
- vendor_openai_swe_pro
- Tool access
- agentic coding tools (vendor harness)
- Prompting
- OpenAI internal / published eval harness (not mini-swe-agent)
Secondary STR (vendor) — lower weight vs independent Vals Verified.
Caveats
Known limitations
- Vendor-reported; cannot receive confidence S.
- Harness not mini-swe-agent — do not average with Vals Verified as same board.
Weighting
Evidence weight factors
The engine weights each evidence row by these factors (0–1) when fusing scores — higher is more trusted.
relevance0.85
quality0.55
recency1.00
health0.80
comparability0.65
reliability0.70
Benchmark
SWE-bench Pro
Harder SWE-bench variant with multi-file diffs from actively maintained repositories; used by vendors and labs as an agentic coding stress test.
Benchmark source →