Home/Research

Measurement · forensics

The CPA spike was a measurement failure, not a performance failure

A store's reported cost per acquisition appeared to jump, so the budget was cut. Reported conversions had fallen from 158 to 44 a day while real orders held at 100 to 110 a day. Performance was fine, the tracking was not, and the budget went back.

Published 2026-08-12

The dashboard said the campaign had broken

A store's reported cost per acquisition appeared to jump. Nothing else in the account looked unusual. The budget was cut in response, which is what the number appeared to demand.

Every report agreed with every other report. That agreement felt like confirmation. It was not. The reports all drew on the same conversion feed, so they could not have disagreed. Consistency between views of one broken input is not evidence that the input is right.

The check that was missing: count the orders the shop actually received, in the shop's own records, over the same days.

What the forensic pass found

Reported conversions collapsed from 158 to 44 a day.

Over the same period, real orders held steady at 100 to 110 a day.

Two series describe the same events. One more than halved. The other did not move. Only one of them can be right about the world, and it is the one the warehouse fulfils against.

The performance was fine. The measurement was broken. The budget was put back.

A ratio breaks from either end

Cost per acquisition is a ratio: spend divided by conversions. The two halves are measured by different systems. Spend is counted by the advertising platform, which counts it carefully because it invoices from it. Conversions are counted by tracking that has to survive a browser, a consent prompt, a redirect and a confirmation page. The denominator is the fragile half.

When the denominator falls and the numerator holds, the ratio rises exactly as it would if performance had collapsed. The shape is identical. There is no signature in the curve that separates the two cases.

Nothing inside the ad account can tell them apart either, because the account holds only one of the two counts. The answer is always outside the platform that raised the alarm.

Where we were wrong

The budget had already been cut before anyone counted the orders. That is the real mistake in this story, and it is procedural rather than analytical. The forensic work was correct. It simply came second.

A cut driven by a broken denominator makes the real business worse while fixing nothing. Orders kept arriving while the spend behind them was reduced. The evidence that the campaign was healthy had been sitting in the shop's database the whole time, unread.

We now treat a move in a reported ratio as a question rather than an instruction. Cutting is an action, and an action can wait until both halves have been counted separately.

How to run the same check

Split the ratio and verify each half against the system that owns it. Spend against the advertising platform. Orders against the shop's own database, not against a reporting layer that sits downstream of the same tracking you are trying to test.

Put the two daily series side by side over the same window. If orders are flat and reported conversions fell, it is a tracking failure. If orders fell together with conversions, performance really did move and the ratio is telling the truth.

Mind the boundaries. Same days, same timezone, and one definition of an order held constant on both sides of the comparison, whether that is orders placed or orders confirmed.

Keep the check cheap enough to run before reacting. A costly check will be skipped on exactly the day it matters.

Limitations

This is one store over one period. It shows that a move in cost per acquisition can come from the denominator. It does not show how often that happens, in this account or anywhere else.

We are not publishing what broke the tracking. The claim here concerns the diagnosis, not the root cause, and a fix for one tracking fault does not generalise to others.

The check separates a tracking failure from a performance failure. It does not separate a tracking failure from an attribution shift, where the orders still arrive and are still counted but are credited to another channel. The order total stays flat in that case too, and telling the two apart needs channel-level work this check does not do.

We did not quantify what the period at a reduced budget cost us, and we cannot do so from the counts described here.

Questions and answers

Why did my cost per acquisition suddenly spike?

A cost per acquisition spike can come from either half of the ratio: spend going up or tracked conversions going down. When conversion tracking breaks, the denominator falls while spend holds, and the ratio rises exactly as it would if performance had collapsed. Compare tracked conversions with the real orders in the shop's own records before deciding that the campaign has broken.

How can I tell a tracking failure from a real drop in ad performance?

A tracking failure can be told apart from a real performance drop by putting two daily series side by side over the same window: reported conversions from the ad platform and orders from the shop's own database. If orders are flat while reported conversions fell, it is a tracking failure. If orders fell together with conversions, performance really moved and the cost per acquisition is telling the truth.

Why is the conversion count the weak part of cost per acquisition?

The conversion count is the weak part of cost per acquisition because it is measured by tracking that has to survive a browser, a consent prompt, a redirect and a confirmation page. Spend is counted by the advertising platform, which counts it carefully because it invoices from it. So the denominator of cost per acquisition is the fragile half of the ratio.

Should I cut ad budget as soon as cost per acquisition rises?

Ad budget should not be cut as soon as cost per acquisition rises, because a cut driven by a broken denominator makes the real business worse while fixing nothing. Orders can keep arriving while the spend behind them is reduced. Cutting is an action, and an action can wait until spend and orders have each been counted separately against the systems that own them.

What boundaries matter when comparing conversions with real orders?

When comparing conversions with real orders, use the same days, the same timezone and one definition of an order held constant on both sides, whether that is orders placed or orders confirmed. Count orders in the shop's own database, not in a reporting layer fed by the same tracking being tested. Keep the check cheap enough to run before reacting.

Can comparing orders and conversions detect an attribution shift?

Comparing total orders with tracked conversions cannot separate a tracking failure from an attribution shift. In an attribution shift the orders still arrive and are still counted, but they are credited to another channel, so the order total stays flat in that case too. Telling the two apart needs channel-level analysis that a simple orders-versus-conversions check does not do.

All research