Retargeting Incrementality: Did Those People Need the Ad?

A.The selection effect, stated plainly

Every retargeting audience is defined by an action on your site: visited, viewed a product, abandoned a cart. Those actions are the strongest predictors of purchase you have.

So the audience converts at several times the rate of cold traffic, and it would do so with no advertising at all. When the ad platform then reports conversions from that audience, it is reporting the base rate of an already-committed group and attributing it to the impression.

The narrower and more intent-heavy the audience, the better the reported return and the worse the incrementality. Cart abandoners are the extreme case: the highest reported ROAS in the account, and the group with the highest chance of returning unprompted.

B.View-through, which compounds it

Retargeting campaigns lean heavily on view-through conversions, because display inventory is cheap and click rates are low. Under a view-through window, someone who never clicked but scrolled past an ad within the window is counted.

Consider what that means for a cart abandoner. They were coming back. In the intervening days they browse the web, an ad renders in a feed they do not look at, they return directly and purchase. The platform records a view-through conversion. Nothing about that chain is false, and the causal claim inside it is unsupported.

The same mechanism drives the broader gap covered in why paid social reports more conversions than you had.

C.Why last-click makes analytics agree with the platform

You might expect your own analytics to act as a check. Usually it does the opposite.

A returning visitor who clicks a retargeting ad arrives with that campaign as their most recent source. Under last non-direct click, the retargeting ad takes full credit for a customer that content, search or word of mouth originally found. Both systems now agree, and both are crediting the last thing to happen before an outcome that was already underway. See first-touch versus last-touch attribution.

A.How to run a holdout

Take the audience that would normally be retargeted. Randomly assign a share of it, somewhere between 10% and 20% depending on volume, to see no retargeting ads at all. Run for long enough to cover your typical purchase cycle, which usually means at least four weeks and often longer.

Compare conversion rate and revenue per person between the two groups. The difference is incrementality. Not modelled, not attributed: measured.

Randomisation is the entire validity of the test. A holdout by geography, device or time period is not a holdout, because those groups differ in ways that affect conversion independently.

B.Reading the result without fooling yourself

Three things go wrong when people read holdouts.

Ending it early. The treated group converts faster because ads accelerate people who were coming anyway. Stop at two weeks and you measure acceleration and call it lift. Run past the full purchase cycle and much of the apparent gap closes.

Underpowering it. A 10% holdout of a small audience produces a confidence interval wide enough to contain both zero and a heroic result. Compute the sample size first, the same way you would for any test. Our sample size guide applies unchanged.

Measuring the wrong outcome. Lift in purchases is the beginning. Lift in revenue matters more, and lift in customers still paying six months later matters most, because a channel that pulls forward low-value purchases can show positive lift on count and negative lift on value.

A.Exclude the people who were coming back

The cheapest improvement available. If someone has returned to your site twice unprompted in the last week, they do not need an ad and any conversion they produce will be miscounted as caused.

Excluding high-intent returners lowers your reported ROAS and raises your actual contribution, which is a trade most teams find politically difficult and is nearly always correct.

B.Target the unresolved objection, not the visit

A visit is a weak targeting signal because it says nothing about why they left. The behaviour before the exit says a great deal, and it maps onto genuinely different messages.

Someone who compared two pricing tiers repeatedly has a packaging question. Someone who detoured to your security page has a compliance question. Someone who read three case studies has a credibility question. Each needs a different ad, and a generic reminder that your product exists answers none of them. The diagnosis is the same one in where qualified leads stall on your pricing page.

C.What that requires

Building an audience on the sequence of behaviour rather than on a single page visit needs per-person history: what this individual did across several visits, and in what order. Pixel-based audiences are usually built on one page load, which is why the targeting is coarse.

Kissmetrics keeps behaviour as a person-level history and populations defined by behaviour rather than by a single event, so you can build an audience of people who compared two tiers and did not buy, or who visited security and left, and exclude people who came back on their own. Membership updates as events arrive, so the audience is current rather than last month's export.

It also makes the holdout easy to read, because the withheld group and the treated group can be followed to revenue on the same person records over the following months rather than compared on a platform-reported conversion count.