Qwen 3.5 4B was quick, but its code review got stuck in a loop
Experiment note: This is a qualification report for model-generated work. Every repository recommendation was checked against frozen source before publication. The findings describe the model, not changes anyone should apply without review.
Small models are tempting agent workers. They load quickly, take little disk space, and can make routine jobs feel almost free. The catch is simple: a fast answer is only useful if the answer survives contact with the source.
This round tested omlx / Qwen3.5-4B-MLX-4bit on an M5 Max. Six short cases covered evidence extraction, structured portfolio decisions, and casual sports replies. Four larger cases asked it to review unfamiliar repositories. The split was sharp. It could handle several narrow instructions. Once the context widened, it started repeating itself and mistaking whatever was visible in the packet for the whole project.
Methodology correction: This article originally reported 4/6 under scorer v1. Cloud controls exposed two benchmark defects: a hidden literal-keyword requirement and case-sensitive action enums. Under versioned scorer v2, this model scores 5/6. This supersedes the earlier workflow total; original receipts remain preserved, and the code-review adjudication and verdict are unchanged.
Four domain passes out of six
The model passed 4/6 domain cases, or 66.7 percent.
Both casual fan replies passed. Two of the three structured portfolio cases passed as well. The miss there was tiny but decisive: the required action enum was lowercase, and the model returned uppercase HOLD. A person understands the answer. A controller rejects it.
The evidence-extraction case also failed. The response contained several good-looking claims, but one supposedly exact quotation was not literal, leaving too few valid claims. This is the kind of failure that makes a fluent evidence packet dangerous. The prose looks finished while the provenance contract has already broken.
The six calls used 1,858 tokens and took 50.06 seconds altogether. The median call finished in 1.78 seconds; the long evidence case accounted for most of the wait.
Twenty-nine recommendations, very little signal
The repository reviews returned 29 recommendations. None of the four review runs passed the deterministic validator.
Independent source review found two useful recommendations and three that had a real concern buried under an exaggerated claim. The remaining 24 were false or materially wrong.
The repetition was the most obvious problem. In one packet, the model produced the same confused recommendation six times about generated Python cache files and package structure. In another, it repeated a missing-test claim five times even though the exact test file appeared in the frozen manifest.
Several errors were more confident. It called environment-backed configuration an exposed secret. It treated normal Python package names as broken dependencies. It recommended enabling an image feature that is deliberately disabled for static export. It also blamed duplicate framework configuration for build trouble, but the pinned application compiled and exported successfully.
That leaves a false-or-materially-wrong rate of 82.8 percent. Quote matching would not have rescued it. Some recommendations pointed at real lines and still misunderstood what those lines meant.
The four review calls consumed 97,352 tokens and took 125.79 seconds. Across the full run, the model handled 99,210 tokens in 175.85 seconds, just under three minutes. That speed is appealing. The cleanup bill is not.
Provisional verdict: deletion candidate
omlx / Qwen3.5-4B-MLX-4bit is now a deletion candidate, not an approved deletion.
I can imagine a narrow role for it where the output is tiny, the schema is strict, and deterministic validation catches every mistake. The campaign already has other small-model lanes competing for that job, though. On broad review, this one generated far too much confident noise to justify trust or a protected slot.
No model will be removed from this result alone. Exact deletion still requires separate authorization, and a changed-condition retest could move the verdict. For now, the 4B model has shown speed without enough judgment.
