Hotel Step 3 Automation: Local Qwen vs Budget Cloud

I wanted Step 3 of the hotel-review pipeline to become a reusable unattended subsystem rather than a pile of cohort-specific scripts. I also wanted a concrete answer to a practical question: how much time does local inference save in API spend, and how little money does a budget cloud model need to save a meaningful amount of wall-clock time?

The result is a manifest-driven controller that can run the same hash-frozen Step 3 contract through local Ollama or OpenAI-compatible cloud providers. It dispatches one native Hermes Kanban card at a time, validates every input and output hash, keeps credentials outside contracts, and records tokens, latency, retries, failures, and modeled API cost.

The controlled comparison

All three routes scored the same frozen Hampton Inn Tomah packet. Each received the identical rubric, Step 2 evidence ledger, three accepted price observations, mobility enrichment, 2,500-token output ceiling, and disclosure batches of three. The runs were sequential, so local GPU contention and overlapping cloud calls did not distort the timing comparison.

Route	Model time	Calls	Input tokens	Output tokens	Modeled API cost	Speed	Scored weight
Local Qwen 3.8	340.45s	7	56,334	6,492	$0	1.00×	95%
Alibaba Qwen 3.8 Flash	70.70s	7	60,155	5,459	$0.011589	4.82×	85%
DeepSeek V4 Pro	73.70s	7	63,100	5,561	$0.052657	4.62×	74%

Alibaba saved about four minutes and thirty seconds per hotel for roughly 1.2 cents. DeepSeek was similarly fast but cost about 4.5 times as much for this particular structured task.

Local inference has no API charge, but “free” should not be confused with zero economic cost. The table does not price electricity, hardware amortization, queue delay, or the opportunity cost of occupying the local model host for nearly six minutes.

What the projections look like

• 10 hotels: local 56.74 minutes; Alibaba 11.78 minutes and $0.1159; DeepSeek 12.28 minutes and $0.5266.
• 100 hotels: local 9.46 hours; Alibaba 1.96 hours and $1.1589; DeepSeek 2.05 hours and $5.2657.

Those are linear projections from one controlled property, not a throughput guarantee. Different property classes and evidence volumes may change both latency and token use.

The outputs did not agree completely

All three routes agreed exactly on value, price, location, guest reputation, rooms, service, amenities, and leaving character/design unsupported. Their price result was the same: subscore 4 in the $$ band.

The main difference was evidence sufficiency. Local Qwen scored 95% of rubric weight and left only character/design unknown. Alibaba scored 85%, leaving car-free mobility, character/design, and operational friction unknown. DeepSeek scored 74%, additionally leaving walkability and on-site dining unknown.

That does not automatically make the local result better. In a fail-closed evidence pipeline, a more conservative unknown can be preferable to a confident inference. Alibaba and DeepSeek agreed on 10 of 12 criterion status/subscore pairs; local and Alibaba agreed on 9 of 12.

What became reusable

The new run-step3-matrix command accepts a frozen JSON manifest containing subjects, routes, file hashes, cost rates, output limits, and execution policy. It supports two modes:

• compare: run every route and produce a side-by-side receipt.
• fallback: try ordered routes and stop after the first valid terminal artifact set.

Every route receives a fresh immutable workspace and a sealed native Kanban contract. Contracts contain Vaultwarden item aliases but no credential values. The stage controller independently verifies the rubric, ledger, enrichment artifacts, scorecard, assessment, gap manifest, receipts, and telemetry.

The implementation is commit bde594b on the hotel-pipeline cleanup branch. The final repository gate passed 274 tests plus Ruff and whitespace checks.

Failures the first real run uncovered

The first cloud run failed usefully. Alibaba and DeepSeek both exhausted the visible output budget because the reusable client had not carried forward their provider-specific controls for disabling hidden reasoning. Alibaba requires enable_thinking: false; DeepSeek requires thinking: {"type":"disabled"} for this bounded JSON route.

The next attempt completed model inference but hit a second legacy assumption: both the Step 3 controller and the Kanban stage executor still required cloud calls to equal zero. That rule made sense when Step 3 was deliberately local-only, but it became wrong once an explicitly approved cloud route existed.

The fixed invariant is route-aware: local routes must report zero cloud calls, while a cloud route must name its provider and report positive cloud_calls == calls. The zero-cloud guarantee was narrowed rather than silently deleted.

All failed lineages remain preserved. Their token-based costs are labeled unavailable because the old failure receipt discarded usage; they are excluded from the successful steady-state comparison rather than guessed.

Routing decision

Local Qwen 3.8 remains the default when queue depth and latency are acceptable. Alibaba Qwen 3.8 Flash becomes the first budget-cloud fallback for a bounded local failure, a backlogged queue, or a deadline-sensitive batch. DeepSeek V4 Pro does not justify its higher price for routine Step 3 scoring, but it remains interesting for editorial prose, where an earlier blind comparison favored its writing.

The next honest confidence gate is a different property class using the frozen controller without further prompt or validator tuning. One hotel is enough to measure a real trade-off; it is not enough to declare universal routing.

Pricing sources and artifact identity

• Alibaba Cloud Model Studio pricing
• DeepSeek API pricing

Successful-route costs use provider-reported token counts and the listed rates; they are modeled list-price values rather than account-balance deltas.

Comparison report SHA-256: fbee16242ef74782a6d2b8b1d8fd31d9cc7164b879497ecbcbea47e3ed5aea3c