Plots to Previews: Towards Automatic Movie Preview Retrieval Using Publicly Available Meta-Data

Bhagyashree Gaikwad, Ankita Sontakke, Manasi Patwardhan, Niranjan Pedanekar, Shirish Karande; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3205-3214

'Preview', a concept popularized by Netflix, is a contiguous scene of a movie or a TV show highlighting its story, characters, and tone, thus helping viewers to make quick viewing decisions. To create previews, one needs scene-level semantic annotations related to the story, characters, and tone. Soliciting such annotations is an involved exercise and these are expensive to generate automatically. Instead, we aim at creating previews by availing readily available scene meta-data, while avoiding dependency on semantic scene-level annotations. We hypothesize that movie scenes that best match publicly available IMDb plot summaries can make good previews. We use 51 movies from the MovieGraph dataset, and find that a match of the plot summaries with scene dialogues, available through subtitles, is adequate to create usable movie previews, without the need for other semantic annotations. We validate the hypothesis by comparing ratings for scenes selected by the proposed method to those for scenes selected randomly, obtained from regular viewers as well as an expert. We report that even with this 'minimalist' approach, we can select at least one good preview scene for 26 out of 51 movies, as agreed upon by a critical expert judgment. Error analysis of the scenes indicates that features related to the plot structure might be needed to further improve the results.