Why Most Agency Reports Lie (And What To Show Instead)

DTC founders commonly bring us 20-slide monthly agency reporting decks and ask whether the numbers are "real." The pattern is consistent across brands at $1M-$10M ARR: the deck opens with millions of impressions, a high-looking blended ROAS, a YoY lift in "engagement," and a screenshot of Klaviyo email revenue claimed as "owned channel attribution." Reconcile against Shopify net revenue and the platforms typically sum to more than the brand actually earned. The ROAS number usually uses an attribution window that double-counts Klaviyo flows already firing on the same orders. The impression count is real and useless. This is the default shape of agency reporting in 2026, and it is why founders stop trusting agencies right before they fire them. AgencyAnalytics' own benchmark survey of reporting practices documents what agencies themselves report — transparency is the single most-cited factor in client retention, and the gap between what agencies report and what clients trust is where most renewals quietly fail (via AgencyAnalytics).

The five lies (in increasing order of cynicism)

Across the agency monthly decks operators bring us — clients we end up taking on, brands we decline, founders who pay for a scoped read of their previous agency's work — the same patterns recur. The shortest answer to why agency reports lie is selection bias: every metric that survives editing into a client deck is the metric that flattered the agency, and every uncomfortable number quietly moved to an appendix nobody opens. Naming the patterns is the first defense.

Lie #1: Blended ROAS without the methodology footnote

"4.7x blended ROAS" is the most common opening number in a paid-media deck and the most meaningless one. Blended against what? Total Shopify revenue divided by total ad spend? Then it includes organic, direct, returning customers, email flows, and word of mouth — none of which the agency drove. Platform-reported ROAS aggregated across channels? Then it triple-counts the customers who clicked a Meta ad, opened a Klaviyo flow email, and converted through a Google branded-search click.

There is no honest blended ROAS without a methodology statement attached. We require ours to specify: which window (1-day click, 7-day click, 28-day click + 1-day view), which channels are pooled, and whether organic/direct/returning are excluded. If a previous agency's report doesn't say, assume the favorable interpretation was chosen.

Lie #2: Impression and engagement vanity

Impressions cost the agency nothing to inflate. Reach, frequency, "video views to 50%," engagement rate — all of these grow when you add a $5/day reach campaign to a paused ad set. None of them correlate with marginal revenue at most DTC scales. A $4M ARR brand does not need 14.2M monthly impressions; it needs roughly 60-200 marginal customers depending on AOV.

Engagement metrics are slightly worse because they sound like business outcomes ("a 38% lift in engagement") when they are platform-internal scores designed to encourage spend. We see at least one engagement-as-headline-KPI deck per month. We have never seen one that survived a founder's first question of "did Shopify revenue grow?"

Lie #3: Last-click double counting between Klaviyo and paid

This one is structurally embedded. Meta's CAPI sends a Purchase event with a click ID from a Meta ad. Klaviyo's tracking attributes the same Purchase to a flow email opened 20 minutes before checkout. Both platforms report the revenue to themselves. If the agency sums "Meta-reported revenue + Klaviyo-reported revenue + Google-reported revenue" without de-duplicating, the total can easily exceed actual Shopify revenue by a wide margin. The reconciliation gap is consistent across DTC brands of any meaningful scale.

The honest reconciliation method: pick one source of truth (we use Shopify net revenue or the merchant-processor settlement) and treat platform numbers as estimates of relative channel contribution, not absolute revenue. The math gets uncomfortable. That discomfort is the work.

Lie #4: Year-over-year framing through seasonality

Clean beauty seasonality is brutal. Q4 outperforms Q2 by 2-3x. Mother's Day weekend can be 6% of annual revenue. If an agency presents "YoY same-month" without flagging that the prior year ran a 25%-off sale and this year didn't, you'll read it as a performance change when it's a calendar artifact.

The defensible framing: present rolling 13-week trend lines, mark promotional events on the chart, and reconcile against the same-week-prior-year only with a methodology note about what was different (price, discount, product mix, channel mix). It takes longer to produce. It cannot be faked.

Lie #5: Incrementality claims without a holdout

The most cynical pattern. The agency runs Performance Max and Demand Gen on top of an existing Google Ads structure, sees a 20% total revenue lift, and claims the lift is "incremental to PMax." Maybe it is. Maybe the PMax campaign cannibalized branded search, demand-genned the email subscribers, or ran during a launch that would have lifted revenue anyway. Without a deliberate holdout (geo split, audience split, or time-period control), "incremental" is just "additional."

We will not claim incrementality without a holdout. We tell clients this explicitly during onboarding, and when they ask "but the previous agency said X was incremental," we explain why the previous claim was unfalsifiable. Some clients find this annoying. Those clients usually leave for a competitor that will tell them what they want to hear, and we are okay with that.

The diagnostic: how to read an agency client reporting deck in 8 minutes

Founders ask us how to evaluate a report without becoming an analyst. The shortcut on what to look for in agency reports — and the five agency reporting red flags worth flagging out loud the next time a deck lands in your inbox — look at the same five things in this order.

Eight minutes. Five questions. The deck either passes or it doesn't.

Marketing dashboard software vendors (and what they hide)

A specific aside on the tool category that produces most of these reports: marketing dashboard software — the agency reporting tools most boutiques run on — is sold by vendors whose product is "make the agency look organized to the client," which is structurally different from "tell the client what's actually happening." Whatagraph, AgencyAnalytics, DashThis, Looker Studio, and the rest of the SMB-focused marketing dashboard software stack all face the same incentive — agencies are the buyers, the products optimize for what makes agency-output-to-clients look good.

Three specific things most marketing dashboard software hides by default:

• The Shopify reconciliation gap. Platform-summed revenue minus actual Shopify revenue. A 25% gap is normal. Most tools don't surface this as a panel; you have to build it yourself with calculated fields.
• Marginal CAC. Blended CAC is the default. New-customer CAC requires extra setup most agencies skip because the resulting number is less flattering.
• Connector failure history. When a Klaviyo or Meta connector silently drops data, the dashboard shows zeros without flagging "data may be missing." The chart looks like a performance drop; the reality is a tracking gap.

The point isn't that marketing dashboard software is malicious. It's that the tools are built for agencies-as-buyers, the agencies are built for clients-who-want-good-news, and the resulting reports omit the uncomfortable numbers by design rather than by accident. Reading any agency report requires asking what the dashboard isn't showing you, not just what it is.

What to show instead

Honest reporting is shorter than dishonest reporting. What honest marketing agency reports look like in practice: six core panels and three diagnostic panels, period. No 22-slide deck, no executive-summary slide that contradicts the data behind it. The cores:

Panel 1: Net revenue reconciliation

Three numbers side by side: Shopify net revenue (period), platform-reported revenue summed (raw), platform-reported revenue with platform double-counts removed. The gap between the raw sum and the reconciled sum is the platform double-count. It should not be hidden. (Today this work generally lives in a commerce source-of-truth alongside paid-channel reporting; the deduplication math itself is the panel, regardless of which tool produces it.)

Panel 2: Marginal CAC, 13-week trend

New-customer spend over new customers acquired, weekly. Trend line. Marked with any promotional event. No "blended CAC" alternative offered.

Panel 3: Contribution margin per channel

Revenue (de-duplicated) − COGS − channel spend − fulfillment. Per channel. Negative numbers shown in red. This is the panel founders ask for and most agencies do not produce.

Panel 4: Channel mix and saturation curves

How much spend, how much marginal revenue per dollar at the current spend level. If Meta is at saturation (next dollar returns 30 cents), founder should see it before the agency proposes increasing the budget.

Panel 5: Cohort revenue (90/365)

New-customer cohorts plotted against their 90-day and 365-day revenue. This is where retention quality shows up. Our LTV by cohort note covers the math.

Panel 6: Tracking health

EMQ scores, GA4-Shopify reconciliation percentage, CAPI event volume vs expected. Reporting honesty starts with the pipes. If tracking is degraded, everything above is suspect. These signals typically come from the underlying tracking layer (Meta Events Manager, GA4 DebugView, server-side gateway logs) rather than the reporting dashboard itself.

The three diagnostic panels (audience saturation, creative fatigue indicators, promotional-period flags) appear when relevant and stay hidden when not. The point is the absence of theater.

What we refuse to do in agency client reporting

We do not produce client decks designed to be screen-shared in a Tuesday meeting and never opened again. A few specific refusals:

No "executive summary" page that contradicts the data behind it. We have seen decks where slide 1 says "ROAS up 22%" and slide 14 shows the methodology that, applied honestly, produces a 4% lift. We will not write the slide 1 version. If a number doesn't survive its own footnote, it doesn't go in the deck.

No screenshot-only reports. Pasting Meta Ads Manager screenshots into a PDF is what an agency does when they don't have a data layer. It also lets them crop. We send live dashboards (Level or Looker Studio) so the client can drill down and we cannot crop.

No vanity benchmarks against "industry average." "Your CTR is 1.4x the industry average" is meaningless without sourcing the benchmark and matching the methodology. Half the "industry average" numbers floating around are aggregated by ad-tech vendors selling the average back to you.

No "we'll explain the methodology if you ask" approach. Methodology lives on page 2, in writing. Founders should not have to ask for transparency.

No padding the report with our own performance. A monthly deck is the client's data, not our trophy case. We do not insert "Marketing Bar accomplishments" slides into operational reports.

What good looks like 90 days in

The shape of a reporting fix that's working: the monthly report shrinks rather than grows, the contribution margin per channel panel reveals at least one channel running unprofitably that the previous deck reported as positive, the founder reallocates spend out of the unprofitable channel into one that's actually compounding, and net new customers per month rises at the same total spend. The deck gets shorter and the bank account gets bigger. That's the trade.

A second pattern, from a different angle: once the dashboard stops reporting platform-blended ROAS and starts showing contribution-margin-positive new-customer revenue per channel, the operator team gains the confidence to scale spend on the channels that are actually working — because the underlying reporting honesty surfaces which audience and creative combinations carry the marginal dollar. The reporting doesn't scale spend. The reporting lets the founder trust the numbers enough to scale spend.

Five questions to ask your current agency before renewal

If you're a founder evaluating whether to renew, walk through these in writing. The answers — or the lack of them — tell you everything.

1. Show me the methodology page from last month's deck. If it doesn't exist, you have your answer. Methodology is not a verbal explanation in a follow-up call; it's a written page that's part of the report.
2. Which number on the report reconciles to Shopify net revenue? There should be exactly one number that matches, and it should be the headline. If the answer is "well, you have to subtract X and add Y," the report isn't anchored to a commerce source of truth.
3. What was the marginal CAC last month, and how does it compare to the same period 90 days ago? If only blended CAC is on the deck, ask why. The honest answer is that marginal CAC requires the agency to separate new-customer spend from total spend — which most agencies don't track that cleanly.
4. When was the last holdout test, and what did it find? If "never" or "we did one a year ago," there's no real incrementality data. The agency is reporting correlation as causation.
5. Show me a channel running unprofitably on contribution margin. Every paid program has at least one. An agency that claims every channel is profitable either doesn't have the COGS data or is hiding the unprofitable one because the conversation would be uncomfortable.

The agency that can answer all five in under five minutes is a partner. The agency that gets defensive on any of them is selling the report, not the work.

What separates a working agency report from a broken one

Pattern recognition, vendor-agnostic, drawn from the structural differences not the cosmetic ones:

Working reports open with reconciliation. Broken reports open with a vanity number (impressions, blended ROAS, engagement lift). The first slide of a working deck is the gap between Shopify and platform-reported revenue. The first slide of a broken deck is "Q3 highlights."

Working reports get shorter over time. Broken reports get longer because each new metric the client questions adds a slide rather than replacing one. A two-year-old monthly deck at the same agency has grown from 12 slides to 31. The growth is selection bias compounded.

Working reports include negative numbers. Broken reports never do. If every panel shows green, the agency is filtering. Real DTC operations include unprofitable channels, declining cohorts, and broken weeks. The report should show them.

Working reports name the next decision. Each report ends with "based on this, we recommend X" — a specific budget shift, creative-test prioritization, or pause decision. Broken reports end with "looking forward to next month's plan." If the report doesn't drive a decision this Monday, it's theater.

Working reports survive a CFO walking in mid-meeting. If the founder's CFO can ask any question in any room and the answer is documented in the deck or in the methodology page, the report is working. If the CFO's question requires "let me get back to you," the report wasn't built for interrogation.

The unspoken cost of dishonest reporting

The operational cost of bad reporting compounds in ways founders rarely calculate. Three specific costs worth pricing in when evaluating whether to fix the reporting layer:

Cost 1: Bad budget decisions. A channel reported as profitable that's actually unprofitable doesn't just waste the spend on that channel — it diverts spend away from the channels that could absorb it. Six months of misallocated $20K/month is $120K of opportunity cost on top of the wasted spend. The reporting cost is the budget cost squared.

Cost 2: Slower scale. Founders scale spend when they trust the numbers. Founders who don't trust the numbers cap spend at the conservative limit because the downside of scaling on bad data is worse than the upside of scaling correctly. Operators on honest reporting consistently scale earlier than operators on suspect reporting, and the compounding difference over 12-18 months is large.

Cost 3: Agency churn. Founders who can't reconcile agency numbers don't usually voice the concern — they quietly start interviewing replacements. By the time the agency hears feedback, the founder has already decided. The agency lost the account three months before the conversation. The cost is the LTV of the account, which on a $5M ARR client at a boutique-tier monthly retainer is meaningful.

Where to next

If you want the operational side of how to actually build the dashboard, see our cross-channel marketing dashboard comparison and the LTV by cohort dashboard reality check. The service page for the reporting work itself lives at custom reporting.

If you want to see the honest version of a cross-channel marketing report — paid-channel focused, no theater — that is what Level is built for today. The current scope is Meta, Google Ads, and TikTok in a single live view; Klaviyo and Shopify dedup is on the roadmap.