Head-to-head
gemma4:26b vs mistral-small:24b
Both models answered the same one-shot tasks under identical rules — no retries, answers published unedited. Two separate axes, never blended: does the code actually run, and how do blind-scored text answers hold up against a written gold answer.
| Model | Runnable sims | Blind text avg | Failure modes |
|---|---|---|---|
| gemma4:26b | 0% (0/1) | 9.61 (31 scored) | BLANK 1 |
| mistral-small:24b | 33% (28/85) | 8.65 (31 scored) | ERR 26 · STAT 11 · BLANK 9 · HANG 3 · DNF 8 |
Biggest contrasts
Rounds where the two models diverged most — open them and read both answers side by side.
| Round | gemma4:26b | mistral-small:24b |
|---|---|---|
| R36 · Continue the sequence (doubling gaps) | ★10 | ★2 |
| R134 · Smallest number from three remainders | ★10 | ★3 |
| R160 · Critique a generic slogan in 2 sentences + give 2 better alternatives | ★10 | ★4 |
| R148 · Table QA: sum, filter, max over order data | ★10 | ★4 |
| R133 · Three friends, three pets | ★10 | ★6 |
→ Interactive round-by-round comparison · → Full leaderboard
Generated 2026-07-31 from the published rounds. Harness failures are excluded from every base — a broken cluster node is our fault, not the model's.