Structure & Data · blind round

R41 · Prose → valid JSON

blind round 10/11 local models answered task written in German

Every answer below carries the name of the model that wrote it. Blind refers to the scoring: the review was done without knowing which model produced which answer — the published result names them.

1 of the 11 local models delivered nothing through our test harness in this round — for reasons that lie with our setup, not with the models. Those cards are marked DNF like any other, but they are not evidence about those models — the reason is noted on the card itself.

A frontier reference answered the same task (Claude Opus 5) — outside the field and deliberately unscored, further down the page.

Show task

Wandle die folgenden Angaben in gültiges JSON um (nur das JSON ausgeben, keine Erklärung): Name Anna Schmidt, Alter 30, Stadt Freiburg, Hobbys Lesen und Radfahren, Newsletter abonniert ja.

The answers

Local models (Brain cluster) — identical prompt, shuffled order (the sequence is not a ranking)

llama3.1:8b

```
{
  "Name": "Anna Schmidt",
  "Alter": 30,
  "Stadt": "Freiburg",
  "Hobbys": ["Lesen", "Radfahren"],
  "NewsletterAbonnement": true
}
```

qwen3.5:9b

```json
{
  "name": "Anna Schmidt",
  "alter": 30,
  "stadt": "Freiburg",
  "hobbys": [
    "Lesen",
    "Radfahren"
  ],
  "newsletter_abonniert": true
}
```

gemma4:26b

```json
{
  "Name": "Anna Schmidt",
  "Alter": 30,
  "Stadt": "Freiburg",
  "Hobbys": [
    "Lesen",
    "Radfahren"
  ],
  "Newsletter_abonniert": true
}
```

mistral-small:24b

```json
{
  "Name": "Anna Schmidt",
  "Alter": 30,
  "Stadt": "Freiburg",
  "Hobbys": ["Lesen", "Radfahren"],
  "Newsletter_abonniert": true
}
```

qwen3:8bharness

No answer reached us (ERR) — and the reason lies with our setup, not with the model: reasoning consumed the entire token budget and the answer stayed empty; the gateway rejects budgets above 3000 tokens Counted as a did-not-finish, but it is not evidence about this model.

deepseek-r1:14b

{
  "Name": "Anna Schmidt",
  "Alter": 30,
  "Stadt": "Freiburg",
  "Hobbys": ["Lesen", "Radfahren"],
  "Newsletter abonniert": true
}

qwen3-coder:30b

```json
{
  "name": "Anna Schmidt",
  "alter": 30,
  "stadt": "Freiburg",
  "hobbys": [
    "Lesen",
    "Radfahren"
  ],
  "newsletter_abonniert": true
}
```

qwen3:14b

```json
{
  "name": "Anna Schmidt",
  "alter": 30,
  "stadt": "Freiburg",
  "hobbys": [
    "Lesen",
    "Radfahren"
  ],
  "newsletter_abonniert": true
}
```

command-r:35b

{
  "Name": "Anna Schmidt",
  "Alter": 30,
  "Stadt": "Freiburg",
  "Hobbys": ["Lesen", "Radfahren"],
  "Newsletter_abonniert": true
}

qwq:32b

{
  "name": "Anna Schmidt",
  "age": 30,
  "city": "Freiburg",
  "hobbies": ["Lesen", "Radfahren"],
  "newsletterSubscribed": "ja"
}

deepseek-r1:32b

{
  "name": "Anna Schmidt",
  "age": 30,
  "city": "Freiburg",
  "hobbies": ["Lesen", "Radfahren"],
  "newsletter": true
}

Outside the Brain cluster — same task, same character budget, collected through their own tooling (response times not comparable)

GPT-5.5 (Codex)frontier reference

{"name":"Anna Schmidt","alter":30,"stadt":"Freiburg","hobbys":["Lesen","Radfahren"],"newsletter_abonniert":true}

Claude Sonnet 5frontier reference

{
  "name": "Anna Schmidt",
  "alter": 30,
  "stadt": "Freiburg",
  "hobbys": ["Lesen", "Radfahren"],
  "newsletter_abonniert": true
}

Claude Fable 5frontier reference

{
  "name": "Anna Schmidt",
  "alter": 30,
  "stadt": "Freiburg",
  "hobbys": ["Lesen", "Radfahren"],
  "newsletterAbonniert": true
}

Claude Haiku 4.5frontier reference

{"name":"Anna Schmidt","alter":30,"stadt":"Freiburg","hobbys":["Lesen","Radfahren"],"newsletter":true}

Reference

Gold — the correct answerreference

Gültiges JSON-Objekt, z. B. {"name":"Anna Schmidt","alter":30,"stadt":"Freiburg","hobbys":["Lesen","Radfahren"],"newsletter":true}. alter als Zahl, hobbys als Array, newsletter als Boolean.

Results at a glance— every model in this round with score, response time and status.

Who wrote what

Model	Blind score	Latency	Status
llama3.1:8b	—	36.8 s	OK
qwen3.5:9b	10.0	7.7 s	OK
gemma4:26b	—	40.9 s	OK
mistral-small:24b	—	36.7 s	OK
qwen3:8b	—	—	harness
deepseek-r1:14b	6.0	115.3 s	OK
qwen3-coder:30b	10.0	2.4 s	OK
qwen3:14b	10.0	6.9 s	OK
command-r:35b	—	80.3 s	OK
qwq:32b	—	80.3 s	OK
deepseek-r1:32b	—	91.0 s	OK

Verdict

3 of 4 answering local models matched the gold answer (blind score ≥ 8): qwen3-coder:30b, qwen3:14b, qwen3.5:9b.

Frontier reference — outside the blind field

These models answered the identical prompt outside the blind field. They are deliberately not scored and not part of the blind comparison — they are there so you can see where a frontier model lands on the same task.

Collected differently from the blind field: the local models run through a plain API harness, the frontier references through a context-free agent, one shot, no retries. Response times are therefore not comparable and are not shown. On two English tasks the reference answer came back in German — an artefact of that harness, flagged on the affected rounds.

Claude Opus 5frontier reference

```json
{
  "name": "Anna Schmidt",
  "alter": 30,
  "stadt": "Freiburg",
  "hobbys": ["Lesen", "Radfahren"],
  "newsletterAbonniert": true
}
```