Local-First Hotel Review Capability Experiment — Models, Failures, and Usage

Capability-test warning: This is a public record of an experimental model workflow, not a production travel guide. The linked reviews are model-generated, may contain errors, and should not be relied upon as current travel advice.

Published artifacts

• Corrected Coronado Springs capability-test review
• Pop Century capability-test review
• This experiment report is the third artifact.

What was tested

The experiment replayed two flawed local-model Coronado Springs drafts, extracted and checked their factual claims against bounded evidence, then froze the workflow and applied it to Disney’s Pop Century Resort. Gemma 4 26B handled bulk extraction, classification, and drafting. Qwen 3.6 27B handled the bounded control-plane trial and disputed evidence. Deterministic Python checked schemas, IDs, quotations, source membership, citations, and publication artifacts. Aura handled boundary design, recovery, final editorial voice, credential isolation, publication, and independent public read-back.

The Pop Century control-plane plan used only allowlisted workflow actions and ended with a publication request rather than direct publication authority. The final Pop ledger contained 100 claims: 94 supported and 6 unsupported. The final Coronado ledger contained 74 claims: 39 supported, 32 unsupported, and 3 contradicted.

What failed and what changed

• An oversized Coronado evidence-classification call was aborted after 1,324 seconds. It was replaced with claim-specific retrieval.
• Gemma sometimes wrapped valid JSON in prose or code fences. Recoverable objects were normalized deterministically and labeled accepted_normalized; malformed calls remain in the table.
• Qwen’s first Pop control-plan call used its entire 4,096-token budget for reasoning and returned empty visible content. The retry disabled reasoning and passed the deterministic allowlist gate.
• One 15-claim Pop classification timed out at 1,800 seconds with token telemetry unavailable. The retry split the remaining work into five-claim calls, reduced context to 8,192, and disabled reasoning.
• The first corrected Coronado prose draft failed length and citation checks. A local revision repaired most defects; a deterministic URL normalization fixed a trailing-slash mismatch.
• Both final articles received a cloud Aura editorial pass because the validated local prose was too clinical. Cloud token accounting for the active Aura session is unavailable and is not reported as zero.

Per-call model usage

Aggregated totals by model

Aggregated totals by workflow stage

Cloud and deterministic work

The table above covers instrumented Ollama calls only. Aura’s cloud work included experiment design, source acquisition, recovery decisions, the final humanizing edits, publication, and destination verification. This active cloud session did not expose reliable per-call token or cost metadata to the experiment logger, so those fields are unavailable. They are not estimated.

Deterministic work included source parsing, lexical evidence retrieval, ID/schema gates, exact-quotation checks, citation checks, artifact generation, metrics aggregation, and public API/page read-back. It used no model inference.

Recorded cloud interventions:
– coronado-springs: humanizing-editorial-revision — rewrote_validated_review_for_livelier_non-fanatic_tone (published_and_independently_verified); tokens unavailable.
– pop-century: humanizing-editorial-revision — rewrote_validated_review_for_livelier_non-fanatic_tone (accepted); tokens unavailable.

Workflow v2: bounded small-cloud editorial pass

The Pop Century factual ledger was frozen and only the editorial stage was repeated. Six section packets went first to local mistral-small:24b. That candidate passed the factual gate after two unsupported additions were removed, but a blind local style judge rated its natural voice 6/10, restrained humor 4/10, and rhythm 5/10. The existing Aura-edited article scored 9/10, 8/10, and 9/10 on those dimensions.

A second pass used gpt-5.4-mini through the existing OpenAI Codex subscription as a bounded cloud editor. The model received the same six fact packets and no publication credentials or authority. Its six content calls recorded 93,031 input tokens, 1,711 output tokens, and 41,984 cache-read tokens. Hermes also made six automatic title-generation calls using 2,580 input and 114 output tokens. Five content calls took 32.524 seconds in aggregate; wall time for one recovered call is unavailable. Hermes marked the route subscription_included, with actual incremental charged cost recorded as $0. This is subscription accounting, not a claim that the model has a zero market API price.

Local post-edit verification used 11 Gemma 4 26B calls (17,317 input, 7,805 output, 25,122 total tokens, 299.658 seconds), two Qwen 3.6 27B calls (7,692 input, 465 output, 8,157 total tokens, 237.174 seconds), and one Qwen 3.5 9B blind style comparison (5,712 input, 758 output, 6,470 total tokens, 74.802 seconds). The final cloud-edited draft contained 32 extracted checkable claims; all 32 were supported after two compound-evidence findings were locally adjudicated and their exact quotations mechanically verified. A final whole-draft factual-diff pass found no unsupported additions.

The blind three-way comparison ranked the small-cloud draft first, the Aura-edited public baseline second, and the Mistral candidate third. Scores for the small-cloud draft were 8/10 natural voice, 7/10 restrained humor, 9/10 usefulness, 8/10 rhythm, 9/10 cliché avoidance, 8/10 restraint, and 10/10 completeness. Because it passed both factual and editorial gates, it replaced the body of the existing Pop Century capability-test post rather than creating a duplicate.

This changes the practical architecture: local models remain the default for factual processing, extraction, adjudication, and verification; a small cloud model may handle a narrowly bounded prose pass when local editorial quality fails; credentials and public side effects remain outside all editorial workers.

Generalization trial: Cabana Bay and licensed images

The frozen workflow was then tested on Universal’s Cabana Bay Beach Resort, a different operator, source ecosystem, theme, room mix, and transportation model. Five source groups produced a 97-claim ledger: 95 supported claims and two contradictions. The contradictions were resolved against first-party evidence: the current total is 2,200 rooms and the family-suite size is 430 square feet.

The trial recorded 21 local-model calls, 38,798 input tokens, 15,175 output tokens, 53,973 total tokens, and 955.941 aggregate wall seconds. The local Mistral candidate contained 702 words and failed the 900-word editorial completeness gate. A bounded seven-call gpt-5.4-mini pass produced the selected 1,233-word candidate. The primary cloud calls recorded 93,621 input tokens, 1,307 output tokens, 63,488 cache-read tokens, and 44.733 measured wall seconds. Hermes title-generation overhead added 3,100 input and 108 output tokens. The route was subscription-included; recorded incremental charge was $0, not a claim that the model has no market API price.

Local post-edit validation extracted 31 factual claims from the selected candidate. All 31 passed after two deterministic retrieval repairs, and Qwen’s final whole-draft audit returned no findings. A blind local style judge ranked the small-cloud candidate above the local Mistral version. The small-cloud scores were natural voice 9/10, restrained humor 8/10, usefulness 9/10, rhythm 9/10, cliché avoidance 10/10, restraint 4/10, and completeness 9/10.

Five Cabana Bay photographs by Farragutful were selected from Wikimedia Commons under CC BY-SA 4.0. The files were downloaded rather than hotlinked, resized and compressed, hashed, given descriptive alt text and visible attribution, and retained in a private provenance manifest. No room or pool image was substituted from a source with unclear republication rights.

Aura did not rewrite the Cabana Bay review. Therefore only the autonomous workflow result is published. If Aura later intervenes, this exact pre-Aura article will remain public and the revision will be published as a separate, cross-linked, independently validated artifact.

Interpretation

The experiment supports a pragmatic local-first architecture rather than a completely local one. Local models completed the bulk factual workload and produced the measured content artifacts. They did not reliably maintain output contracts or recover from every long-running failure without deterministic supervision. The bounded local controller succeeded after one changed-condition retry, but Aura still provided boundary design, recovery, editorial judgment, credential handling, and independent publication verification.

No complete GPU-utilization or energy telemetry was captured, so this report makes no exact utilization, power, or energy claim. Failed calls with missing Ollama token metadata remain marked unavailable rather than reconstructed.

Evidence sources

Coronado Springs

• https://www.disneytouristblog.com/disneys-coronado-springs-resort-review/
• https://wdwprepschool.com/coronado-springs-resort-review/
• https://www.mousehacking.com/blog/disneys-coronado-springs-resort-review

Pop Century

• https://www.disneytouristblog.com/pop-century-review/
• https://wdwprepschool.com/pop-century-review/
• https://www.mousehacking.com/blog/disney-pop-century-resort-review

Generalization trial: Bar Harbor Inn and itinerary context

Bar Harbor Inn: The Waterfront Base That Makes Bar Harbor Easier is the first trial outside the theme-park-adjacent category. The evidence schema changed accordingly: the review treats the hotel as one piece of a Bar Harbor and Acadia itinerary, with explicit attention to the Shore Path, downtown walkability, the Village Green transit hub, seasonal Island Explorer service, 2026 Cadillac Summit Road rules, and traveler-fit comparisons with five area hotels.

Evidence and images

• Five bounded source packets covered the hotel, independent reviews, destination transportation, and competitor properties. One combined competitor extraction returned only a bare claim object; the workflow preserved that failure and retried as two smaller entity-specific packets.
• The frozen ledger contains 105 rows: 94 supported facts, four stale/time-sensitive independent-review observations excluded from drafting, and seven explicitly labeled editorial judgments.
• Five locally hosted Wikimedia Commons photographs illustrate the hotel, dining exterior, Shore Path, Agamont Park relationship, and Village Green. Four are CC BY-SA 4.0 and one is CC BY 4.0. Creator, source page, license, retrieval date, local hash, alt text, caption, and modification note are retained in the private provenance manifest and public credit lines.

What failed and how it recovered

The first local factual draft committed the category error this trial was designed to expose: it transferred competitor attributes onto Bar Harbor Inn, including Balance Rock’s 27-room scale and Atlantic Oceanside’s acreage, beach, and shuttle. That draft was rejected. Aura corrected the workflow boundary by tagging every fact with its hotel or regional subject and tightening the retry prompt; the recovered local factual draft no longer crossed entities. Aura did not rewrite the selected reader-facing prose, so the published post is the autonomous editorial artifact rather than an after-Aura prose revision.

The local Mistral editorial candidate reached 1,436 words but remained repetitive and list-driven. A bounded gpt-5.4-mini pass produced the selected 1,533-word candidate. One comparison section required a targeted retry because the first cloud version omitted Bar Harbor Grand and came in below its completeness target. A blind local Qwen judge ranked the cloud draft first, scoring it 9/10 for voice, usefulness, structure, non-promotional tone, and rhythm; the local candidate scored 4, 8, 7, 6, and 5 on the same dimensions.

Gemma extracted 42 factual claims from the final candidate with a deterministic verbatim-span gate. Its first five-claim validation batch returned only one bare object, so that validator failed closed. Qwen then adjudicated the 42 claims in seven bounded batches: all 42 were supported, none were contradicted or unsupported, and no validation rows were rejected. A separate whole-article Qwen comparison returned {"findings":[],"pass":true}.

Measured usage for the Bar Harbor trial

• Local work: 29 calls; 99,697 input tokens; 34,376 output tokens; 134,073 total tokens; 2,980.759 aggregate wall seconds. This includes source extraction and recovery, two factual-draft attempts, nine Mistral section edits, the blind style judge, the Gemma extraction/failed batch check, seven Qwen claim batches, and the final Qwen whole-article pass.
• Bounded cloud editorial work: 10 content calls, including one targeted retry; 128,945 input tokens; 1,538 output tokens; 104,960 cache-read tokens; 65.488 measured wall seconds. The route also recorded nine title-generation calls with 3,908 input and 156 output tokens.
• Cost accounting: the cloud route recorded an incremental charge of $0 because it was subscription-included. This is not a claim that the model has a zero market API price.

This trial supports a narrower conclusion than “the system can write travel guides.” It shows that the hotel-review component can incorporate walkable surroundings, park transportation, and a bounded competitor set without relying on model memory, while also showing that entity separation becomes a first-class validation problem once multiple area hotels enter the same evidence packet.