In part 1 last week, I set the foundation to help us better understand campaign lift reports and spent some time reviewing the hold out group methodology. This week we’re making things a bit more relevant by explaining how Chartable, Claritas, and Podsights run their lift reports.

It’s all About Style

Data, by its very definition, is objective. Data presentation, however, is subjective.

Every company working in data is motivated to prove the integrity of their offering along with presenting data in a unique fashion to differentiate themselves from their competitors.

That first part, proving the integrity of data, that’s a battle that companies constantly fight. Even if you’re certified with the IAB, MRC, or follow an open framework like oDL (Open Downloads), no two companies data will ever match 1:1.

That shouldn’t be a troubling statement.

In fact, not only is it ok for the data not to match directly, but you should prefer it. Methodology differences—what counts and what doesn’t—often account for discrepancies around 5%, and the lift partners know how to appropriately factor discrepancies like this across disparate data sources. If you want to run a campaign where data matches 1:1, then you’re stuck buying podcast ads from a single-stack provider, like Spotify.

And I know that Sounds Profitable readers are far too savvy to pay a company to grade their own homework.

That second part, the unique presentation of data, is what allows companies like Chartable, Claritas, and Podsights, to offer competitive podcast lift report products. While each of these companies use slightly different methodologies in their reporting, they all have key similarities.

Same Difference

Lift reports require a campaign attribution foundation to work. In podcasting, that requirement is the same minimum implementation:

Conversion Tag

At least one conversion tag needs to be placed on the website that the campaign is driving to. Without it, there’s no data to indicate the listener’s household ever got there. Claritas originally required the tag to be placed on the destination website a full two weeks before the campaign started, to gain an understanding for the traffic on the site before the campaign started. But as of now, all three vendors only require the tag be implemented before the ads start running.

Exposure Tag

Same idea here, but on the ads themselves. The podcast adserver will fire the vendor’s exposure tag when the episode containing the ad has been downloaded. This also needs to be placed in the adserver before the campaign starts.

I can hear some of you saying “what if I can’t get the tags live in time?”. Like most things, it’s a matter of degrees. If it takes a day or two to get the tags live, it’s not that big of a deal. And it’s certainly not worth chasing backfill data. So let it go and move on. Focus on what remains of that campaign. But if the campaign is further along when you decide to implement attribution tags, it may be wisest to wait until your next campaigns starts and save your attribution efforts for that one. While each of these partners can backfill data, there are legal implications to consider. And as things heat up in privacy management, expect fewer companies to be comfortable transferring or receiving raw data.

Now that you have a good grasp of the similarities between these vendors, let’s dig into the differences.

Device Graphs

Today, device graphs are a bit in flux. The changes made by Apple and Firefox with regards to blocking third party cookies, along with Apple’s move to require apps to ask users to opt in to sharing their identifier for advertising (IDFA) will force device graphs to drastically alter how they work.

A quick recap, to keep everything in context: The goal of a device graph in podcasting is to take an exposed listeners IP address, resolve it to a household, and then use the household as a focal point. That IP address resolves to the household, but so do other ID’s, like cookies and IDFA (for the time being) and eventually other identity based values. This is what enables an exposure, bound pretty heavily to requiring a household wifi IP address, to be matched to someone downloading a mobile app while connected over a cellular connection. Both sides resolve to the digital household, just in different ways.

Podsights uses a device graph maintained by TapAd, recently acquired in November by Experian for $280m. Experian has been no slouch on prioritizing their stake in the digital identity resolution space, so their purchase of TapAd signifies that TapAd is in good shape with the coming changes. While TapAd already had access to some of Experian’s demographic data, it’ll be interesting to see just how much this acquisition augments their offering.

Chartable started off working with TapAd, but they’ve confirmed that by the end of December 2020, they’re migrating over to Liveramp. While not a household name in podcasting yet, Liveramp was the first data partner to fully release a digital identity solution and has been at the forefront of that conversation. I expect to see Liveramp’s involvement in podcasting flourish in 2021, and partnering with Chartable is a great way to show they’re serious about it.

Claritas was already a big part of the device graph space before they bought the digital attribution company, Barometric. Claritas and Nielsen are the two names you’ll hear most often when discussing podcast demographic targeting. Podcast adservers Megaphone and Voxnest are integrated with Nielsen, while Art19 is integrated with Claritas. It makes sense that the Claritas attribution product would use the Claritas device graph, instead of working with an external partner.

While there are definitely device graphs out there with questionable datasets, all three of these vendors have chosen partners that will be proactive regarding the upcoming changes with user privacy that others may not be able to withstand.

Control Groups

It’s important to keep in mind what I mentioned earlier: data presentation is subjective.

Control groups are a great example of that. Remember, lift reports help us track a specific campaigns performance without having to tag every single campaign a brand is running on every single channel; digital or otherwise. Instead, control groups are created, consisting of people who have not been exposed to the campaign. The vendor then examines the rate at which the control group ends up converting on the website compared against those driven to the website post-campaign exposure.

In part 1, a big concern was overlap between the hold out-style control group and the campaign exposed group. That’s not a problem here as the control group isn’t a tag fired during the campaign at all, but something uniquely crafted from the datasets each of these vendors have cultivated.

Claritas creates 20 unique control groups per campaign before averaging them, from within their device graph. Their goal is to find a demographic, geographic, and time-based match for every impression in the exposed group. So a campaign with 1 million exposures would have 20 different control groups, each of the exact same size. The control group is made up of a sampling of the Claritas device graph, which spans much further than podcasting.

In contrast, Podsights creates 10 control groups from a randomized sample of podcast listeners before averaging them. While TapAd does have a robust multi-channel device graph like Claritas (though not demographic focused), Podsights specifically builds their control groups from podcast listeners only. After attributing the control groups to the conversion tag, Podsights looks for a stable window of time and uses that to aggregate a baseline. They do this to make sure the results of the lift testing are not being impacted by externalities that would lead to incorrect results. That concept of focusing on a specific time window is unique to Podsights, as both Claritas and Chartable focus on concurrently observed control groups, meaning they’re building and comparing the control groups in tandem with the exposed groups, throughout the whole campaign.

Chartable operates somewhere in-between the two. Similar to Claritas, they’re using geolocation and time of exposure as a factor to build their control group, but they don’t use any demographic information. At least not yet. But, just like Podsights, they’re focusing on only podcast listeners as a subset of their device graph. Chartable generates 4control groups, then averages them out before utilizing the results in their lift reports.

Keep in mind that these are all the default control group methodologies that each vendor offers. All three vendors have proven to be flexible in working with their clients to meet their unique needs. An example of a custom control group would be one created only with True Crime listeners for a campaign only running on True Crime inventory.

Impression Volume

The last major difference between the three is in the impression volume required to run a lift report. This is an important distinction as lift reports can be run either for an entire campaign, or broken down more granularly for specific strategies within the campaign. To compare the three vendors, I’m going to use a campaign that runs for 3 months, with 4 different targeting strategies.

Chartable comes in with the highest requirement, at 1 million impressions per campaign. Ideally, they also recommend that the 1 million impressions happen within a one month period. For the example campaign, that would be between 4 and 12 million impressions.

Podsights clocks in next at 250,000 impressions per lift report, but doesn’t require a specific timeline. So our example campaign would only need 1 million impressions total.

And Claritas requires the lowest impression volume at just 50,000 impressions per lift report, with no length requirements either. Which makes Claritas the most accessible at just 200,000 impressions for 4 lift reports in a 3 month period.

If you think about the size of the data set each company is working with, both their device graph and the podcast-specific view both Chartable and Podsights have, the impression minimums make a bit more sense as they are imposed specifically to make sure the lift reports have enough data for statistical significance.

Claritas, with a large footprint outside of just podcasting, has access to that pool to build many control groups from a relatively small number of exposures. Chartable and Podsights, on the other hand, have chosen to build out control groups only from podcast listeners they’ve directly encountered. While four control groups sounds low and twenty control groups sounds like overkill, I can comfortably say none of these vendors would be sticking with a methodology that they didn’t believe in and couldn’t defend.

Wrapping It Up

In part 1, I laid out the value of lift reports as a tool that measures directional effectiveness of a campaign through a path of least resistance. Instead of viewing each campaign as a silo or attempting to tag every single campaign run by a brand with a multi-touch attribution tag, lift reports help us understand how many conversions would not have happen without this specific campaign, while still taking into account other marketing efforts.

Each of these three vendors have truly proven themselves in the podcast data space. They all have their own angle for how they represent data and differences in their methodologies. But there’s no single best option. Nor will there ever be one.

Lift reports are a tool in our belt, used to get the most out of a campaign. If you tagged a campaign with all three vendors at the same time, I’d truly be shocked if the results weren’t directionally all pointing in the same way.

But when you use any tool, you need to be prepared to defend why you chose that particular tool. You need to be able to clearly and concisely explain the value of the tool. And, perhaps most importantly, you need to know how to use the tool correctly to get the best results.

Paying for a lift report and not being able to explain how they work shouldn’t fly anymore.

It’s no longer enough to loop in the account manager from the vendor you’re working with into a call hoping they can manage the setup on your behalf. Everyone involved in the process really needs to be able to speak to the value and understand how they got there and what to do next.

The more you know.

Special thanks to Matt Drengler of Podsights, Dave Zohrob of Chartable, and Omer Jilani of Claritas for walking me through their unique offerings.

Homework

The goal of Sounds Profitable is to educate and empower each of you. If we’ve had a chance to talk directly, you know that I am truly passionate about both adtech and podcasting. We learn through asking tough questions and discussing the answers. Armed with today’s new knowledge, I want to help you ask more questions. Please consider supporting Sounds Profitable through our Patreon.

And share this article on your social channels while asking your followers a question. Here are a few to try:

• Do you utilize campaign lift reports?
• Do you have a direct relationship with a vendor? If so, what made you pick them?
• Have you explored using different vendors depending on the campaigns requirements?
• What improvements would you be looking for from campaign lift reports to utilize them more?

Market Insights

Last week, Sounds Profitable kicked off the first episode of our Product Deepdive series, a completely free and on demand look at how podcast adtech actually works.

We’ll be premiering a new episode the third Thursday of every month at 4pm ET followed by a live Q&A. As of right now, we’ve booked one for every single month in 2021 except December 2021, so please reach out if your podcast adtech company is interested.

Our first episode offers a complete walkthrough of Podscribe for Advertiser, a market research tool designed to help advertisers find the right shows to advertise with. Definitely check it out and let us know your feedback.

Our next episode on January 21st, 2021 explores Triton Digital’s Podcast Metrics, with a specific focus on all the new features for advertisers.

New Sponsors!

It’s our goal to highlight the amazing people and companies that are helping Sounds Profitable keep the lights on. We’d like to personally thank our latest full sponsors:

• Amplifi Media is a digital audio agency which develops content and strategy to create engaging, viral audio-casts.
• LeadsRx is an impartial multi-touch attribution tool, supporting many different channels, including podcasts.

We’d also like to thank our latest individual sponsor Stephen Smyk SVP Podcast and Influencer Marketing at Veritone One.

We appreciate your support! If you’d like to learn more about sponsorship or advertising with us directly, please reach out!