Confessions of a media auditor: ‘Agencies often manipulate the numbers’

Excerpts have been lightly edited for clarity.

What’s a media audit pool?
Media auditors collect from media agencies the data of a particular client [or clients] they’re contracted to work with. That data is put into an anonymous pool, with other similar client data. Each client gets compared against that data pool, and people look for like-for-like comparisons to give an idea of how a client is performing compared to the rest of the market. Typically, you’d speak with the agency to prepare them for the results, and that’s when the discussions start and where it ultimately becomes a negotiation of what the results are, and that’s usually whatever reflects the agency’s performance in the best light.

What do you mean by negotiation?
If a client requests a media audit, they’re not told who else is in the pool, so it’s hard to tell if it’s genuinely a robust piece of analysis. Usually, it’s too small, with five or so clients in a pool, when it should really be up to 20 clients for digital because there are so many more data points than in traditional media. In reality, a typical digital pool would have 10 percent of the data on a client’s campaigns across the year because there are so many different variations of what you can buy from a digital provider: different placements, targeting and formats that can be bought 24 hours a day. So, trying to create like-for-like comparisons is really hard.

Why is this a big deal?
It’s an old practice that shouldn’t be happening anymore now that the [digital] media market is so complicated. These antiquated pooling processes have been applied to digital with catastrophic effects, with media auditors claiming they have a pool for digital, for programmatic. But that’s laughable because programmatic is an auction-based system, and anyone has access to buy auction-based inventory at the best possible prices. You don’t have to be an agency of scale. Ultimately, it’s the media auditors that keep these almost fictitious pools open, but it’s a shrinking black box of data that isn’t comparable anymore. And the agencies are just trying to give data that makes the performance look good.

Agencies fudge the numbers?
Yes. The only source to understand where the spend has gone [what channels and audiences] is the media agency. So you have to take their word that’s what they spent on those audiences.

What are the ramifications?
Clients will often write into their contract that they’ll pay a bonus based on a certain result versus a pool. I’ve seen it written into contracts that a client will pay a £100,000 ($136,000) bonus to their agency based on a pool’s results. So the agency is incentivized to provide the best possible audit result to gain its bonus. That sounds great in practice — of course, you should reward good performance, but the issue is that because of the bonus structure and the fact the pools are flawed, agencies often manipulate the numbers to show stronger performance. So clients are paying out large sums of money based on erroneous results. That hurts clients as they are paying out for improvements that are not really improvements.

Does that hurt the publishers, too?
It hurts media owners because they’re also having to offer better prices all the time to try and beat these pools, and those pools will never represent the whole media owner book.

Explain that.
It means media owners are pressured to offer the media agency better prices for that client so that the client gets the result versus the pool that they want. Media owners get frustrated with the media auditor because the latter is saying their average price on media inventory is much lower than what the media owner states it is. That means the media owner is under pressure to give the agency and client concerned better pricing to narrow the gap. Given you can see which media owner it is that’s contributing to the bad result, the media agency will phone them and tell them they’ve had a crap media audit and that they have to sort out the pricing on it.

What’s your advice to clients?
Focus your contract on outcomes and how the agency will improve your pricing based on looking within your agency group — your trusted agency should be able to open their data books up. There’s a reason you often hear the phrase “you never get a bad media audit” — it’s because there is all this “negotiation” and agencies manipulating the data. The client and their auditors need to wake up and realize this practice is no longer fit for purpose.