GPT-OSS 20B scores five of six once it has room to finish
Engine/model: Ollama, gpt-oss:20b
Corrected result. GPT-OSS 20B passes five of six bounded workflows. The 0/6 originally reported here was a measurement artifact, not a capability finding: the harness allowed only 320 output tokens for five of the six cases, and this model spends tokens reasoning before it writes. It was cut off mid-answer and then scored as a failure. Given 4,000 tokens it completes cleanly and passes every case except one.
Its repository-review lane does genuinely fail, for a reason worth separating out: it consumed the entire 8,000-token allowance on all four repositories while emitting zero visible characters. Nothing to adjudicate — but not silence from inability; silence from reasoning that never reaches the output.
Budget correction — formally confirms this article’s operational finding. The audit now records this tag as budget-limited: all six workflow calls and all four review calls hit the configured completion ceiling. The stored output shows correct, schema-valid JSON with verbatim quotes being produced and then cut off mid-object, so the 0/6 total measures the serving configuration rather than the model’s ability. The verdict remains retest required, which is precisely what a raised-budget re-run now tests. No deletion is authorized or implied.
Ten completed calls, no usable artifact
The failure pattern was consistent. The hotel case ended as invalid JSON. Both short community replies exposed empty text. All three portfolio cases were invalid JSON. The four repository reviews each recorded an empty visible content field after reporting the full 3,200 completion tokens.
The six domain calls took 96.2 seconds, with a 15.8-second median. The review calls took another 182.8 seconds. Across the run, the endpoint reported 93,722 tokens in 279.0 seconds, but those token counts did not buy a deliverable.
This looks less like ten independent reasoning failures and more like one bad serving contract repeated ten times. A reasoning-capable model can spend the response budget internally and leave the visible channel empty. The campaign records only the observable result, so it cannot recover or grade thoughts that never became output.
No code-review score is better than a fake one
The review chart shows zero useful, zero qualified, and zero wrong findings. That is not a perfect precision score. It means there were no recommendations at all. Treating silence as clean review would reward the exact behavior this test is supposed to catch.
The useful result here is operational: this Ollama model tag is not qualified under the current completion ceiling and reasoning behavior. Retrying the same request unchanged would only produce a more expensive copy of the same evidence.
Verdict: retest required, and the retest has now happened
The label stays retest_required, because the review lane is still unusable and the workflow lane has a single genuine miss (a portfolio action outside the permitted enum, at 342 tokens, finishing normally). But the earlier claim that this 12.85 GiB tag has no production role is withdrawn. On workflow tasks with an adequate allowance it is one of the stronger configurations measured — ahead of several much larger models.
The remaining fix is reasoning suppression on the review task, or a substantially larger allowance with separate visible-output checking. This is not a family-wide verdict and it is not deletion authorization.
This is not a family-wide verdict, and it is not deletion authorization. It is a precise statement about one engine, one model tag, and one failed serving configuration.
