Due to the growth of video data on Internet, automatic video analysis has gained a lot of attention from academia as well as companies such as Facebook, Twitter and Google. Video analysis, unlike image analysis, requires incredibly high speeds of efficiency and increased confidence intervals to accuratley discern the images in the video. Creating these algorithms, these large tech companies have had to approach the problem from a creative and unintroduced algorithimic perspective.
Shot detection algorithms try to find the positions in the video, where one scene is replaced by another one with different visual content. Conventional approaches for shot detection usually involve the two steps of measuring the similarity of consecutive frames and then determining the shots’ boundaries. Two simple methods for measuring the similarity of frames are sum of absolute values of pixel-wise difference of frames and the difference between the histograms of frames. Through leveraging shot detection, Google is able to, in most cases with a 98% degree of certainty, categorize what is in the frame.