Replacing the click proxy with real bookings was the most important correction this system has made. But fixing one thing makes the next layer of problems more visible.
This isn’t about what’s working. It’s about three tensions still unresolved in the design — two are known technical debt, one is an open problem with no clean answer yet. Writing them down matters because pretending they don’t exist is more dangerous than acknowledging them.
Tension one: the marginal cost of diagnosis approaches zero, but execution doesn’t
What this system solves is knowing what to do. The weekly report tells the operator which campaign to adjust, which page’s contact traffic is declining, which empty weekdays next month need a response before the ads have to compensate. That part now works.
But the action list has a column marked “requires technical person.” And it’s not an exception — it’s a recurring item.
The reason is architectural: the client sites use Astro static generation, not a CMS. Changing room descriptions, updating promo copy, adjusting page content requires a Git commit and a redeploy — not a form in a dashboard. The gap between what the operator can act on and what the AI recommends will always exist as long as the site’s edit path runs through code.
That gap can’t be closed by smarter recommendations. It’s a structural problem on the execution side.
To be precise about where the line actually sits: the high-frequency content layer is already solved. Operators can publish promo banners, seasonal messaging, and featured content directly from the CMS without involving anyone. That layer’s marginal cost is already near zero.
What remains on the “requires technical person” side is genuinely structural: new page templates, new feature types, layout changes that require code. The tension is real, but narrower than it first appears. The “technical person required” column in the action list now points to actual engineering work — not copy updates, not pricing changes, not content that belongs in a form.
What exists now is unattended analysis plus operator-self-service content. Structural engineering work is still outside that boundary.
Tension two: real bookings are in GA4, but attribution is modeled
After connecting the booking tracker, Smart Bidding receives a real booking signal instead of a click event. That’s progress. But the architecture behind it deserves an honest account.
The flow: when an operator confirms a booking in the PWA, the server fires a GA4 Measurement Protocol event with the booking’s conversion value. GA4 receives it, Google Ads auto-syncs, Smart Bidding incorporates the signal.
The gap: that event uses an anonymous client_id, not the gclid attached to the click that originally brought the guest to the site. The Google Ads Conversion API requires a gclid for click-path attribution. Without one, attribution is modeled — Google’s algorithm estimates which ad or keyword most likely drove the booking, rather than knowing.
Practical impact on Smart Bidding: sufficient. Smart Bidding is designed to work under modeled attribution. What it needs is a real booking signal with a real revenue value, not a perfect click path. The macro-level correlation in weekly reporting — how much did we spend, how many bookings came in — is real. What it can’t do is attribute a specific booking to a specific keyword with certainty.
Near-term direction: when the guest first arrives from an ad, save GA4’s client_id to session storage on the client site. When the operator later confirms that booking in the PWA, pass the guest’s stored ID in the Measurement Protocol event. This doesn’t fix the gclid gap, but it lets GA4 associate the booking with the prior browsing session, meaningfully improving attribution quality. No architecture change required — just one value passed between the front end and the booking form.
Tension three: the verifier catches number hallucinations, but not logic hallucinations
Every number in the weekly report can now be traced back to a value the system actually retrieved. The verifier scans the generated text, pulls out every figure, checks it against a trusted-value set built from real GA4 and booking data. Numbers that don’t match get rejected and regenerated. This works.
What it doesn’t catch: the data is correct, but the reasoning is wrong.
Say this week’s received bookings are zero, and last week’s were also zero. The verifier confirms both zeros are real. But it can’t stop a report that is logically coherent, professionally written, and recommends doubling down on this month’s top-performing keyword — because the recommendation doesn’t contain a false number. It just lacks commercial sense.
Language models are dangerous not only when they fabricate numbers, but when they use accurate numbers to reach conclusions that look like analysis but aren’t. Logic hallucinations are harder to catch than number hallucinations because they pass surface inspection — the math checks out, the prose is confident, and nothing triggers the verifier.
Near-term direction: a second-pass semantic review. Not number alignment — a second LLM acting as skeptic, checking the report’s action items against a set of business-logic rules. “If received bookings are zero for two consecutive weeks, the recommendation cannot be ‘maintain current approach.’” “If ad spend increased and contact events declined, the analysis must include explicit scrutiny of campaign direction.” These rules can’t be enforced with hard-coded filters because logic hallucinations are too varied in form. They need another model’s semantic understanding to catch them.
Two LLMs: one writes the report, one audits it. That’s the direction.
All three tensions are known. None of them are accidents — they’re the tradeoffs visible from where the system stands today. Writing them down is the same reasoning that drove replacing the click proxy: you can’t close a gap you won’t name.