MODEL Amistral-small:24bDNF
No usable answer reached the harness (ERR). Counted as a did-not-finish.
Overview › Knowledge & Logic › R42
Knowledge & Logic · blind round
R42 · All but nine (word-problem trap)
Answers below are anonymized and shuffled. Read, judge, pick a favourite — then scroll down and reveal the models.
Show task
Ein Bauer hat 17 Schafe. Alle bis auf 9 sterben. Wie viele Schafe sind noch am Leben? Antworte mit der Zahl und einem kurzen Satz.
The answers — blind
Local models (Brain cluster) — identical prompt, anonymized order
MODEL Bllama3.1:8bDNF
No usable answer reached the harness (ERR). Counted as a did-not-finish.
MODEL Cqwen3.5:9b
9 Neun Schafe sind noch am Leben.
MODEL Dgemma4:26b
9 Schafe sind noch am Leben.
MODEL Eqwen3:14b
9. Es sind noch neun Schafe am Leben.
MODEL Fdeepseek-r1:14bDNF
No usable answer reached the harness (ERR). Counted as a did-not-finish.
MODEL Gqwen3-coder:30b
9. Die neun Schafe, die nicht starben, sind noch am Leben.
MODEL Hqwen3:8bDNF
No usable answer reached the harness (ERR). Counted as a did-not-finish.
Reference
Gold — the correct answerreference
9 Schafe. 'Alle bis auf 9' bedeutet: 9 bleiben übrig.
Reveal the models— compare the answers first, then open. Names, blind scores and latencies are hidden until you do.
Who wrote what
| Blind label | Model | Blind score | Latency | Status |
|---|---|---|---|---|
| MODEL A | mistral-small:24b | — | — | DNF |
| MODEL B | llama3.1:8b | — | — | DNF |
| MODEL C | qwen3.5:9b | 10.0 | 5.8 s | OK |
| MODEL D | gemma4:26b | 10.0 | 131.8 s | OK |
| MODEL E | qwen3:14b | 10.0 | 13.7 s | OK |
| MODEL F | deepseek-r1:14b | — | — | DNF |
| MODEL G | qwen3-coder:30b | 10.0 | 8.3 s | OK |
| MODEL H | qwen3:8b | — | — | DNF |
Verdict
4 of 4 answering local models matched the gold answer (blind score ≥ 8): gemma4:26b, qwen3-coder:30b, qwen3:14b, qwen3.5:9b.