Performance Spotlight: How Agencies Are Testing for Incrementality

Summary

The article argues that marketing measurement must separate true incremental impact from outcomes that would have occurred anyway. While channels and tools evolve, the core advertising principle remains: assess results against a counterfactual. Incrementality testing—or lift testing—provides that counterfactual. The mechanics are relatively simple: randomly hold ads back from a test group, let the rest of the audience experience normal campaigns, then compare behavior between groups. The difference is attributed to advertising causality, rather than seasonality, promotions, or existing demand.

Agencies are increasingly operationalizing this. C-4 Analytics, Tinuiti, WPP Media, and AdYogi are cited as examples of agencies running lift studies as standard practice across different markets and methods. The common thread is not a single test design but a commitment to learning and acting on causal evidence.

The Tinuiti/DSW example illustrates the operational value. DSW sells online and through more than 500 stores, but its Meta campaigns only counted online sales. When ads drove in-store purchases, Meta’s system did not receive those signals and therefore could not optimize to find similar in-store buyers. Tinuiti rebuilt the account into a single campaign and fed DSW’s in-store sales data into Meta so the system could optimize for both online and in-store purchases.

Because this was a major account change, Tinuiti validated it with an incrementality test against the prior setup. The new structure delivered 23% more total purchases, 45% more in-store purchases, and 15% better return on ad spend. The article notes that retail sales fluctuate for many reasons, so a strong month alone would not prove causation. The lift test provided a defensible comparison.

Actionable takeaways for ad ops leaders: build holdout-based lift testing into major campaign or account changes; ensure offline and online conversion signals are fed back to ad platforms when they matter; and judge performance by causal lift, not just platform-reported ROAS or month-over-month gains. This makes optimization decisions more defensible and scalable.

Analyst Note

The Tinuiti/DSW case is a useful signal that incrementality testing is moving from a periodic audit to an operational input for campaign structure decisions. What's notable here isn't the 23% lift figure itself but that the agency used a controlled holdout to validate a major account restructuring—combining online and in-store sales data in a single Meta campaign—rather than relying on post-hoc attribution. This matters now because signal loss from privacy changes has made platform-reported conversions less reliable, and advertisers are under pressure to justify spend with causal evidence.

The underlying principle—separating baseline demand from ad-driven lift—isn't new, but the operationalization is. The competitive angle: agencies that can productize lift studies are differentiating on measurement rigor, not just media buying. The key implication for UA and monetization teams is that offline conversion data is no longer a nice-to-have for retail accounts; it's a prerequisite for training ad systems on the full purchase journey.

Worth watching is whether lift studies become a standard line item in agency retainers, and how platforms respond when advertisers can independently verify incremental impact.