← Gemini 3.1 Pro · API · High thinking
Evidenceacceptedvendor_reported
MMMU-Pro
Google DeepMind model card — Gemini 3.1 Pro · MMMU-Pro (no tools)
ev_2026_07_gem31pro_mmmu_vendor
Result
80.5%
Confidence B
- Tested
- 2026-02-19
- Runs
- 1
Configuration
How it was measured
- Exact model
- gemini-3.1-pro-preview
- Reasoning
- high
- Agent harness
- none
- Tool access
- none (no tools)
- Prompting
- vendor MMMU-Pro
Secondary multimodal — vendor MMMU-Pro (no tools). Prefer AA primary.
Caveats
Known limitations
- Vendor-reported; cannot receive confidence S.
- Prefer independent AA MMMU-Pro (ev_2026_07_gem31pro_aa_mmmu) as primary.
Weighting
Evidence weight factors
The engine weights each evidence row by these factors (0–1) when fusing scores — higher is more trusted.
relevance0.90
quality0.55
recency0.85
health0.85
comparability0.80
reliability0.70
Benchmark
MMMU-Pro
Multimodal understanding and reasoning: college-level questions that require interpreting images (diagrams, charts, photos, figures) alongside text. MMMU-Pro is a harder, cleaner follow-on to MMMU that reduces shortcut strategies. Primary ModelQuest signal for visual / UI-relevant understanding when formal design rubrics are unavailable.
Benchmark source →