Which Local Model Can Turn a Hotel into a Source List?

.ob{border-collapse:collapse;width:100%;font-size:14px}.ob th,.ob td{border:1px solid #d8dee4;padding:7px;vertical-align:top}.ob th{background:#f6f8fa;text-align:left}.ok{background:#e9f8ee}.warn{background:#fff8dc}.bad{background:#fdecec}.note{border-left:4px solid #4b74c6;padding:10px 14px;background:#f5f8ff}.metric{display:inline-block;padding:7px 10px;margin:3px;background:#f2f4f7;border-radius:5px}.mono,pre{font-family:ui-monospace,SFMono-Regular,Menlo,monospace}pre{white-space:pre-wrap;overflow-wrap:anywhere;font-size:12px}details{margin:10px 0}.small{font-size:13px;color:#4b5563}

Question tested: Starting with only a hotel name and full street address, which installed local text model can formulate searches and produce a validated list of websites for later extraction?

This benchmark covers source discovery and selection—not factual extraction. The model wrote the search plan; a deterministic controller executed its queries through one search backend, returned only that model’s results, rendered selected URLs, and scored the resulting corpus.

Result grid

Model	Machine	Queries	Selection	Score	Valid guest	Rendered guest	Model seconds	Outcome
gemma4:31b	remote-10.7.6.39	accepted	accepted	93.00	5	4	78.72	Usable minimum met: 5 valid / 4 rendered guest sources
qwen3.5:9b	remote-10.7.6.39	accepted	accepted	92.00	4	3	18.85	Usable minimum met: 4 valid / 3 rendered guest sources
qwen3.5:9b	local-mac-mini	accepted	accepted	92.00	4	3	43.74	Usable minimum met: 4 valid / 3 rendered guest sources
qwen3.5:9b-64k	local-mac-mini	accepted	accepted	92.00	4	3	45.60	Usable minimum met: 4 valid / 3 rendered guest sources
mistral-small:24b	local-mac-mini	accepted	accepted	88.00	3	3	113.96	Usable minimum met: 3 valid / 3 rendered guest sources
gemma4:e4b	remote-10.7.6.39	accepted	accepted	87.00	4	3	23.27	Usable minimum met: 4 valid / 3 rendered guest sources
qwen3:8b	local-mac-mini	accepted	accepted	87.00	4	3	37.22	Usable minimum met: 4 valid / 3 rendered guest sources
qwen3.8:latest	remote-10.7.6.39	accepted	accepted	83.00	3	3	44.26	Usable minimum met: 3 valid / 3 rendered guest sources
qwen3.6:27b	local-mac-mini	accepted	accepted	72.00	2	2	147.22	Insufficient corpus: 2 valid / 2 rendered guest sources
qwen3.6:latest	local-mac-mini	accepted	accepted	70.00	2	1	142.27	Insufficient corpus: 2 valid / 1 rendered guest sources
qwen3.5:35b	remote-10.7.6.39	accepted	accepted	67.00	2	2	50.67	Insufficient corpus: 2 valid / 2 rendered guest sources
edtorre/muse-glimmer-agent:latest	remote-10.7.6.39	accepted	accepted	65.00	2	1	42.46	Insufficient corpus: 2 valid / 1 rendered guest sources
qwen3.6:latest	remote-10.7.6.39	accepted	accepted	65.00	2	1	48.67	Insufficient corpus: 2 valid / 1 rendered guest sources
nemotron-3.5-lightning:30b	remote-10.7.6.39	accepted	accepted	61.00	1	1	64.42	Insufficient corpus: 1 valid / 1 rendered guest sources
gemma3:12b	local-mac-mini	accepted	accepted	61.00	1	1	66.94	Insufficient corpus: 1 valid / 1 rendered guest sources
muse-glimmer:30b	remote-10.7.6.39	accepted	accepted	59.00	1	0	42.43	Insufficient corpus: 1 valid / 0 rendered guest sources
nemotron-3.5-lightning:30b-mlx	local-mac-mini	request_error	not_run	0.00	—	—	0.04	Query-plan contract failed
muse-glimmer:30b-mlx	local-mac-mini	request_error	not_run	0.00	—	—	0.06	Query-plan contract failed
gemma4:e2b-mlx	local-mac-mini	invalid_output	not_run	0.00	—	—	3.98	Query-plan contract failed
gemma4:e4b-mlx	local-mac-mini	invalid_output	not_run	0.00	—	—	5.47	Query-plan contract failed
gemma4:26b-16k	local-mac-mini	invalid_output	not_run	0.00	—	—	11.85	Query-plan contract failed
gemma4:26b-mlx	local-mac-mini	invalid_output	not_run	0.00	—	—	12.17	Query-plan contract failed
gpt-oss:20b	local-mac-mini	invalid_output	not_run	0.00	—	—	15.57	Query-plan contract failed
gemma4:12b-mlx	local-mac-mini	invalid_output	not_run	0.00	—	—	15.96	Query-plan contract failed
gemma4:12b-mlx-chat	local-mac-mini	invalid_output	not_run	0.00	—	—	18.55	Query-plan contract failed
mo-shakib/clearwriter:latest	local-mac-mini	accepted	invalid_output	0.00	—	—	24.11	Source-selection contract failed
qwen3.8:27b-mlx	local-mac-mini	invalid_output	not_run	0.00	—	—	32.53	Query-plan contract failed
qwen3:14b	local-mac-mini	accepted	invalid_output	0.00	—	—	61.21	Source-selection contract failed

Open the full report with every query plan, selected URL, raw model response, scoring receipt, and retrieval result →

Provisional winner

Remote gemma4:31b scored 93.00/100. It found the correct Hilton property page, five valid property-level guest sources, and four that successfully rendered. It lost points because one of its three supplemental official choices was the events page rather than a preferred rooms, dining, or amenities source.

What worked

• Eight model routes met the actual minimum: correct canonical page and at least three usable independent or guest sources.
• The existing qwen3.5:9b route was strong: 92/100 locally and remotely. It found the correct official package and four valid guest corpora, with three rendering successfully.
• Local mistral-small:24b selected only three guest sources, but all three were valid and usable.
• Every schema-valid selection found the correct canonical Hilton property page. The main separator was guest-source diversity and retrieval quality.

What failed

• Ten routes failed before searching because their query-plan output was malformed or the Ollama request failed.
• Two more produced valid query plans but malformed final selections.
• Several models counted multiple Tripadvisor photo, feature, or individual-review URLs as separate guest corpora. Those are property-specific but are not independent property-level source sets.
• No model earned 100. Even the winner had one weaker supplemental official selection and one selected guest page that did not render.

Same-model inference time

These totals cover the two model calls—query planning plus source selection—and exclude the shared search and rendering controller.

Exact model	Local seconds	Remote seconds	Remote speedup
qwen3.5:9b	43.74	18.85	2.32×
qwen3.6:latest	142.27	48.67	2.92×

Conclusion

Yes: a local model can perform Step 1 when the controller owns search execution and retrieval. The best first candidate is remote gemma4:31b; the practical Mac-mini candidate is the existing qwen3.5:9b or qwen3.5:9b-64k. This is one hotel and one run per route, so it qualifies candidates rather than proving general reliability. The next reliability test should use the same frozen controller contract on different hotel types.

Read every model’s findings →