Denominators and what they mean
Every resolve rate on this site is published twice, with a 95% interval on each:
- per attempted — resolved ÷ every instance the run launched. This is the leaderboard convention: an infrastructure failure counts against the run.
- per gradeable — resolved ÷ (verdicts + empty patches). Attempts the framework failed (harness crash, invalid grade) are removed from the denominator and reported; an empty patch stays, because "the agent produced nothing" is a real outcome.
When the two differ, the gap is the framework's own failure rate, and it is on the page rather than absorbed into "unresolved". When they coincide, as they do for the five MiniMax runs, no attempt was excluded.
| harness | attempted | gradeable | resolved | rate / attempted | rate / gradeable | cost | $ / resolved | tokens in / out |
|---|---|---|---|---|---|---|---|---|
| OpenCode 0c9303b3 | 500 | 500 | 385 | 77.0% [73.1, 80.5] | 77.0% [73.1, 80.5] | $26.29 | $0.068 | 534.1M / 4.4M |
| custom minimal agent 05bd05b8 | 500 | 500 | 384 | 76.8% [72.9, 80.3] | 76.8% [72.9, 80.3] | $33.98 | $0.088 | 683.1M / 5.8M |
| Claude Code 1f722383 | 500 | 500 | 384 | 76.8% [72.9, 80.3] | 76.8% [72.9, 80.3] | $39.50 | $0.103 | 36.7M / 6.0M |
| Codex CLI c10df654 | 500 | 500 | 383 | 76.6% [72.7, 80.1] | 76.6% [72.7, 80.1] | $30.63 | $0.080 | 635.6M / 4.7M |
| mini-swe-agent b4338f67 | 500 | 500 | 379 | 75.8% [71.9, 79.3] | 75.8% [71.9, 79.3] | $32.72 | $0.086 | 616.9M / 5.8M |
Intervals
The interval is the Wilson score interval at 95%. At n = 500 a rate near 77% carries about ±3.7 percentage points. Two harnesses whose intervals overlap are not shown as ranked; the charts draw the intervals so the reader can see that.
What is never done
- No attempt is dropped from both denominators silently.
- No rate is published without its n.
- No null is coerced to zero: a run with no cost instrumentation renders as not measured, never as free.