Qwen 3.5 9B was faster than expected and less reliable than the 4B
Experiment note: This is a qualification report for model-generated work. Every repository recommendation was checked against frozen source before publication. The findings describe the model, not changes anyone should apply without review.
Model size is a lousy substitute for qualification. The 9B candidate in this campaign had more room to reason than its 4B sibling and finished the workload faster. It also passed fewer of the small, tightly specified jobs.
The candidate was omlx / Qwen3.5-9B-MLX-4bit, running on an M5 Max. I gave it the same six domain cases and four repository-review packets used elsewhere in the campaign. The result was not subtle enough to blame on one awkward prompt: it missed literal evidence, broke a simple enum contract, failed both casual-reply checks, and treated packet boundaries as source-code defects.
Methodology correction: This article originally reported 2/6 under scorer v1. Cloud controls exposed two benchmark defects: a hidden literal-keyword requirement and case-sensitive action enums. Under versioned scorer v2, this model scores 4/6. This supersedes the earlier workflow total; original receipts remain preserved, and the code-review adjudication and verdict are unchanged.
Two domain passes out of six
The overall domain score was 2/6, or 33.3 percent.
The two passes came from structured portfolio decisions. It respected the supplied values for a capped purchase and a partial sale. The third portfolio case asked for a lowercase hold action; the model returned uppercase HOLD, which made the response invalid for the controller.
The evidence case failed because one quotation was not literal and too few claims remained valid. Both casual sports replies also failed their required stance or subject checks. One of them sounded energetic enough, but sounding like a fan was not the whole assignment. The contract still mattered.
Those six calls used 1,932 tokens and took 21.96 seconds. The median was 1.04 seconds. This was a genuinely quick model on short work, but the pass rate makes that number hard to celebrate.
A few good leads inside a noisy review
The four repository reviews produced 32 recommendations. One review passed the deterministic validator.
After checking every recommendation against the pinned source, three were plainly useful. Ten more raised a real concern but exaggerated severity, assumed a product requirement that had never been stated, or proposed more machinery than the code needed. Nineteen were false or materially wrong.
The useful leads were ordinary and welcome: a static deployment workflow skipped the build, one project lacked tests, and a container lacked a readiness check.
The bad calls were much louder. The model mistook excerpts in the review packet for files that ended mid-function, even though the pinned files were complete. It said a container entry file and migration runner were missing, overlooking framework output and the migration code that ran during database creation. It warned that matching React 19 type packages were invalid. It predicted a build failure from duplicate framework configuration, then the pinned application built and exported without that failure.
Security language made the same problem worse. One recommendation jumped from an ordinary SQLite configuration to database corruption. Another invoked SQL injection even though the query layer used bound parameters and request constraints. A dramatic label did not make either claim true.
The review calls used 97,031 tokens and took 118.87 seconds. The full qualification consumed 98,963 tokens in 140.83 seconds, about two minutes and twenty-one seconds. Most of that work bought recommendations that still needed careful source review.
Provisional verdict: deletion candidate
omlx / Qwen3.5-9B-MLX-4bit is a deletion candidate, not an approved deletion.
Its speed is real, and three useful review leads prove it was not merely producing random text. But it lost too many bounded contracts and misunderstood too much surrounding code. The 9B size did not translate into a safer worker here; on the six domain cases, it was worse than the 4B candidate tested alongside it.
A changed prompt, a different quantization, or another engine may tell a different story later in the campaign. This exact model and engine combination has not earned a protected fleet role yet. Nothing will be removed without separate authorization.
