How ModelQuest scores models

• Builds, not nicknames — scores attach to exact API / product / effort configurations.
• Evidence before opinion — every claim is traceable to evidence records.
• No false precision — ranges, confidence grades, and insufficient evidence are first-class.
• Honest flaws — profiles include weaknesses and workarounds.

Each attribute has an absolute raw aggregate (evidence fusion) and a peer public score used on sheets, Duels, and Quest Fit.

// Peer min-max (meth_v0_1_1) — multi-Build peer set
public = 40 + ((raw − min_peer) / (max_peer − min_peer)) × 55

// Band ≈ 40–95 within the *current* scored peer set
// Single peer: absolute provisional (or fixed 70 if forced)

• Peer public ≈ 40 means lowest in the scored peer set on that axis — not “failed the benchmark.”
• Peer public ≈ 95 means highest among current peers — ranks reshuffle when Builds are added.
• Always re-run npm run score:all after new evidence so peer maps stay consistent.

Contribution =
  NormalisedResult
  × Relevance × Quality × Recency
  × Health × Comparability × Reliability
  × mapping_weight

Vendor-reported results cannot receive confidence S. Agent scores must name the harness (e.g. mini-swe-agent).

Reasoning uses hierarchical fusion so hard closed-book HLE does not naively crush GPQA:

• Primary (GPQA Diamond): 55%
• Secondary (HLE / LiveBench): 35%
• Bleed (coding-suite leakage): ≤10%

HLE uses frontier-relative normalisation (closed-book anchor ~55% → 100 on the HLE-local scale).

Fit is a weighted blend of this Quest’s attribute/skill weights (plus optional cost/tempo soft prefs). Inputs for attributes are peer-public scores. Missing weights are redistributed; low coverage lowers Fit confidence. A ≥4-point lead marks a decisive winner.

Party suggestions stack a primary Fit pick with a ceiling Build and a value/volume Build — multi-model by design, not a single #1 forever.

• Peer min-max runs across currently scored launch Builds (Codex sheets), not the full 12-Build slate until each has evidence.
• Prefer insufficient evidence over inventing scores.
• Full narrative in the repo: methodology/ModelQuest_Methodology_v0.1.md