Can Machines Learn To Map Creative Videos to Marketing Campaigns?

Jarod Wang, Chirag Mandaviya; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2023, pp. 504-510

The demand for accurate estimation of marketing's incremental effect is rapidly increasing to enable marketers make informed decisions on their ad investment. The process of ad-mapping links an ad shown to consumers on the fixed marketing channels (Linear TV, Digital, Social) to the Amazon marketing creative video. Thus, an accurate ad-mapping, which is a special case of video copy detection, is a cornerstone of ensuring exposure of ad is linked to the correct creative and marketing campaign and hence precise marketing effect measurement. With each campaign having tens of creatives and each country (marketplace) having tens of marketing campaigns each week, the current process of human annotation of hundreds of creatives requires over 800+ team's hours annually. Moreover, this manual process causes significant challenges in onboarding new businesses and countries to measurement due to the absence of intelligent model based ad-mapping solution. To solve this problem, we built a machine learning (ML) model that leverages fingerprinting methodology and automatic language identification technology to match each creative to the marketing campaign. In the paper, we present the computing algorithm and implementation details with results from actual campaign dataset. Extensive validation and comparison studies conducted demonstrates improved mapping results with the new proposed method, achieving 87% F1 score and 82% accuracy. To our best knowledge, this is the first model that uses a fusion of visual, audio, language and metadata features for such ML based content mapping solution. The proposed method leads to 90% reduction on the time spent on ad-mapping compared to manual solutions.