Every number in an automated B&B report has to trace back to a real value the system actually fetched. That rule exists because a language model writing prose about analytics will, given the chance, invent a clean-looking statistic. The verifier in this pipeline walks the rendered report, extracts every number, and checks each one against a set of trusted values pulled from GA4 and the booking data. If a number has no source, the report is rejected and re-rendered.
That machinery is the subject of an earlier piece — A Digital Operations Stack for Hengchun B&Bs — and I won’t re-explain it here. This is about what broke when I added a new mandatory section to the report, and the two decisions that came out of fixing it: how to make the verifier trust a percentage it never directly saw, and what should happen when a report genuinely can’t be verified.
The section that didn’t exist before
The weekly report already answered “who came to the site” and “did the ads bring them.” What it didn’t do was connect those into a single diagnostic the owner could read top to bottom: of the people who arrived, how many engaged, and of those, how many took a contact action. I added a fixed four-layer funnel health check — traffic structure (new vs returning users), engagement depth (engaged sessions over total sessions), per-event conversion, and the contact-intent rate — and made it a section the report must contain. Skip it, and the report fails.
The contact-intent rate needed its own definition. The raw conversion event set includes things that aren’t really contact: a booking_click is interest, not a message sent; a facebook_click is a weaker signal; a Google-reviews tap is research. So I carved out a configurable subset — phone, LINE, WeChat taps — as the numerator for “this visitor actually tried to reach the owner,” and let each property override it in its config. That distinction matters because the whole point of the funnel is to separate people who looked from people who acted, and lumping a review-page tap in with a phone call quietly inflates the action rate.
The verifier rejected the percentages it asked for
Here’s where it got interesting. The funnel ratios are stored as three-decimal floats — 0.847, 0.632 — computed in an earlier stage so the model never has to do arithmetic. The report cites them as percentages: “engagement rate 63.2%.” Reasonable. Except the verifier’s trusted-values list contained 0.632, not 63.2. From its point of view, 63.2% was a number with no source — an orphan — and it flagged the report.
The obvious fix would have been to force the report to show its work: write every percentage as engagement 142 ÷ 225 = 63.2%, so the verifier could match the operands. I had that rule in place, and it was wrong. It pushed the owner-facing report toward arithmetic clutter — parentheses full of raw session counts in a document meant for someone who keeps a paper notebook. The whole design goal of these reports is that the person reading them shouldn’t need to do or check math.
So instead I taught the verifier the relationship. When it walks the data and finds a float between roughly 0.01 and 1, it now also pushes the ×100 rounded form into the trusted set. 0.632 enters the trusted list, and so does 63.2. The percentage is no longer an orphan because the verifier understands that a ratio and its percentage are the same fact in two costumes. That’s a small change — three lines in the walk function — but it moved the burden to the right place. The report stays clean; the verifier got smarter about what “the same number” means.
The bounds matter. Below 0.01 and the ×100 form collides with legitimate small integers; at exactly 1.0 it’s a degenerate 100% that doesn’t need the trick. Keeping it strictly inside the open interval avoids polluting the trusted set with values that would let a genuinely fabricated number slip through.
What happens when verification can’t pass
The second decision was harder. Until now, when the verifier exhausted its retries — couldn’t reconcile the report after the maximum render attempts — the pipeline aborted with a non-zero exit. The report for that property simply didn’t exist that week. For a system that runs unattended on a weekly cron across multiple properties, that’s a bad failure mode: a single stubborn orphan number could blank out a property’s report entirely, and the first I’d know is the absence.
I changed it to fail open. When verification is exhausted, the run no longer dies. It marks the report flagged, reads the leftover orphan and jargon details from the last failure snapshot, and prepends a banner to the top of the report naming exactly which numbers couldn’t be sourced. The report still gets produced.
But a flagged report must never reach the owner. An unverified report with potentially fabricated numbers is worse than no report — it erodes the one thing this whole system runs on, which is that the owner can trust what’s in front of them without checking it. So the routing changed too. The email step now reads the run summary, detects the flagged status, and restricts delivery to maintainers only. The owner sees nothing; I see a report with a warning banner telling me precisely what to investigate.
That split — produce the artifact, quarantine it, route it to the person who can fix it — is the difference between a pipeline that’s robust to its own imperfection and one that’s brittle. A crash gives me a stack trace and a missing report. A flagged report gives me the actual content, the specific unsourceable numbers, and a guarantee the client never saw it.
Why this exists at all
None of this is a feature anyone asked for. The owner asked for one thing: tell me whether the money I’m spending on ads is working, in language I understand. The funnel check, the verifier, the trust-extension, the fail-open routing — that’s the scaffolding required to make a language model’s analytics prose trustworthy enough to act on, on a weekly schedule, across a portfolio of small businesses, with no analyst in the loop.
For a single B&B, paying an engineer plus an analyst plus someone to package it into plain language every week was never on the table — the cost was an order of magnitude beyond what the business could justify. The reason this work happens at all is that the marginal cost of adding a property to the pipeline is now close to zero, and the verifier is what makes that scale safe. Without it, automated analytics prose is a liability that compounds with every property you add.
The day I let the verifier trust a percentage it had only ever seen as a decimal, I also let it stop arguing with reports that were correct.