| |

Qwen 3.6 35B-A3B earned a place, with a large supervision asterisk

Engine/model: oMLX, Qwen3.6-35B-A3B-4bit

Qwen 3.6 35B-A3B finished the ten-call qualification in 93.9 seconds. That number includes four repository packets containing almost 90,000 prompt tokens. The six shorter workflow cases had a median latency of 0.73 seconds, while the repository reviews landed at a 21.9-second median.

That speed is useful. The quality result is more complicated.

Methodology correction: This article originally reported 4/6 under scorer v1. Cloud controls exposed two benchmark defects: a hidden literal-keyword requirement and case-sensitive action enums. Under versioned scorer v2, this model scores 6/6. This supersedes the earlier workflow total; original receipts remain preserved, and the code-review adjudication and verdict are unchanged.

Corrected two-panel chart for Qwen3.6-35B-A3B-4bit: workflow score 6 of 6 under scorer v2, with independently adjudicated code-review counts.
Scorer v2 changes this workflow result from 4/6 to 6/6 by removing a hidden literal-keyword penalty and normalizing action capitalization. Code-review adjudication is unchanged.

Better evidence handling, familiar schema mistakes

The model passed four of six domain cases. It cleared the literal-evidence hotel test, which is where several smaller models lost exact quotations or mixed subjects. It also handled two of three structured portfolio actions and one of two casual community replies.

Both misses had appeared elsewhere in the campaign. One community reply failed to name the required team or preserve the requested stance. The portfolio case used uppercase HOLD instead of the schema’s lowercase enum. Neither failure required deep reasoning to avoid, which makes them more irritating, not less. Production automation needs exact contracts.

The review validator rejected all four runs

The model produced 25 repository recommendations. None of the four complete review runs passed the deterministic grounding validator. Source adjudication found four useful recommendations, ten that needed qualification, and eleven that were false or materially wrong.

The useful set was practical: an unused generated style file, repeated per-record duplicate queries, and missing test coverage in two small projects. The bad set included a supposed hardcoded secret that was actually a redaction marker in the review packet. Another finding claimed an image validator forgot to check the JPEG start marker even though the cited function does exactly that. Other recommendations ignored nearby input validation, configuration loading, or comments explaining why some filtering must happen in memory.

This is the sort of reviewer that can save time if a human treats every result as a lead. It is not a reviewer I would let file issues on its own.

Nearly 100,000 tokens in under two minutes

The ten calls processed 98,460 tokens: 90,810 prompt tokens and 7,650 completion tokens. Most of that was source context, not generated prose. The model was concise and exceptionally fast on the oMLX engine. That efficiency gives it a real advantage over slower dense models when the task is bounded and verification is cheap.

Provisional verdict: likely keep

The current label is likely_keep. It passed the evidence-heavy extraction case, handled most structured decisions, and offered enough useful review leads to justify a supervised role. Its review precision is nowhere near good enough for autonomous maintenance, security triage, or migration advice.

The right job for this model is rapid first-pass work behind hard validators: extract, classify, propose, then check. Used that way, the speed is an asset. Remove the checks and the confident mistakes arrive just as quickly.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *