Qwen 3.5 27B Q8 passed the workflows, then made me check everything
ollama / qwen3.5:27b-q8_0 did the disciplined part well. It extracted literal evidence, made three valid structured portfolio decisions, and wrote two constrained fan replies without wandering outside the prompt. Under scorer v2, that is six passes from six cases.
Its code review was messier. Five recommendations were useful. Ten were false or materially wrong. The remaining nine needed qualification before I would hand them to a developer.
That split makes this model useful, but expensive in the least fun way: it saves discovery time and then charges some of it back as verification.
Six workflow passes, with the old score preserved
The raw receipts carry a v1 score of 4/6. Scorer v2 corrects the total to 6/6.
One paper-trading answer used uppercase HOLD, with empty trade fields exactly as the case required. The old scorer rejected the capitalization. V2 strips whitespace and casefolds the action before checking the enum. The other correction removes an unstated keyword test from the casual Bears-fan prompt. The answer clearly took the requested side and even named the team, but v1 still failed it because a second hidden literal was absent.
These are benchmark corrections, not retroactive changes to the model output. Both totals remain visible: 4/6 under v1 and 6/6 under v2.
The timing was respectable for the six bounded calls. They took 62.08 seconds in total, with a 4.49-second median. Most of the wait came from the long evidence packet; the short structured calls were much quicker.
The source check changed the story
The model returned 24 findings from four repository snapshots. Two review documents passed the quote-and-schema validator. I then inspected every recommendation in its surrounding source.
Five survived as useful. The model correctly called out a scraper failure that could stop a whole batch, a missing backend test suite, swallowed exception details in an ingestion loop, distance sorting that materializes every matching row before pagination, and a session-secret fallback that invalidates sessions after restarts or across workers.
Nine findings contained a smaller, defensible suggestion but oversold it. Dynamic SQL identifiers would benefit from an allowlist, for example, but the model labeled them a critical injection path without showing that an attacker controlled those keys. A migration shortcut could hide a partially applied manual schema change, though the actual migration runs inside a transaction. A split dependency layout deserved cleanup, but it did not prevent the pinned application from building.
Ten findings failed the source check. The model repeatedly interpreted packet truncation as broken source, even though the full files were present. It claimed static export could not support client-side interactions; the application compiled, generated its pages, and exported successfully. It treated a redaction marker in the review packet as proof of a committed secret, while the pinned code reads the value from an environment variable. It also proposed replacing same-origin mutation checks with a token without demonstrating any bypass of the existing role and origin gates.
This is why I do not use quote validity as a proxy for code-review accuracy. The model usually pointed at a real line. It often told the wrong story about it.
Provisional verdict: likely keep
The broad reviews consumed 96,450 tokens and 545.41 seconds, with a median of 143.36 seconds per repository. The full ten-call run used 98,394 tokens and took 607.49 seconds, a little over ten minutes.
That is slower than the 35B Ollama candidate tested alongside it, and the semantic error rate is still too high for unsupervised review. The 8-bit model nevertheless earned a provisional likely keep because it went 6/6 on the bounded workflow suite and found five concrete issues that survived source inspection. Direct comparison with the matching family on the other engine still matters, especially given this model’s 27.91 GiB footprint.
I would give it evidence extraction, structured controller work, and narrow review questions with an exact acceptance gate. I would not give it a whole private repository and treat the resulting severity labels as decisions.
No deletion is authorized by this test.
