Online Program Home
My Program

Abstract Details

Activity Number: 221 - Statistics in Marketing and Advertising: Saying it with Integrity, Accuracy, and Impact
Type: Invited
Date/Time: Monday, July 29, 2019 : 2:00 PM to 3:50 PM
Sponsor: Section on Statistics in Marketing
Abstract #300425
Title: Improved Estimation of View Through Lift from Randomized A/B Tests
Author(s): Kingshuk Roy Choudhury* and Anuvrat Singh
Companies: Amazon and Amazon
Keywords: A/B testing; view through lift; zero inflation

The efficacy of digital advertising is traditionally measured using the click through rate (CTR = # clicks/# ad impressions). The monetary value of advertising is subsequently measured using the conversion rate (CVR = # purchases/# clicks). The click based approach can’t capture cases where purchases were influenced by ads, but not made via clicks. To encompass these cases, we need an alternative measurement strategy, known as view through lift (VTL). This involves a randomized trial (A/B test) where one arm sees the ad and the other doesn’t. The lift is measured as the difference in key performance indicators (KPIs) between the arms. It appears similar to a traditional clinical trial and is commonly analysed using two sample t-tests, but the analysis raises some unique statistical issues. One issue is incomplete reach: not everyone in the treatment group gets to see an ad. A second issue is the highly skewed distribution of purchase values. In this talk, we describe how we can improve estimation efficiency and bias by addressing these issues. (Subject to clearance by Amazon Legal Dept.)

Authors who are presenting talks have a * after their name.

Back to the full JSM 2019 program