Τεχνικές μηχανικής μάθησης για διαχείριση γνώσης σε πολυμεσικά δεδομένα

Abstract

In this thesis we have proposed novel methods for video segmentation and representation that are based on machine learning techniques (classi cation, clustering). First, we considered support vector machines for video shot detection. Then, an improved spectral clustering algorithm was employed for video shot representation. The same algorithm in combination with a sequence alignment algorithm was employed for video scene segmentation. Movie segmentation into scenes and chapters was also implemented using temporally smoothed visual words histograms. Furthermore, the proposed techniques were also employed for video rushes summarization and event detection in video surveillance sequences. More speci cally, in order to perform video shot detection, we proposed in Chapter 2 a supervised learning methodology [11, 15]. In this way, we have avoided the use of thresholds and we were able to detect shot boundaries of videos with totally di erent 127 visual characteristics. Novel features have been de ned describing the variation between adjacent frames and the contextual information in a neighborhood of frames and became inputs to a SVM classi er which categorized transitions to normal, abrupt and gradual. In this way, all categories of video shot transitions were detected simultaneously. Numerical experiments that compare our algorithm with threshold dependent methods and another supervised learning methodology indicate that our algorithm provides superior results. In Chapter 3 a key-frame extraction algorithm [10, 14] has been presented that is based on the combination of spectral clustering approach and fast global k-means algorithm. We have also proposed a technique to estimate the number of the extracted key-frames. The extracted key-frames are unique, non-repetitive and summarize the video shot content, which is also indicated from the numerical experiments where appropriate quality measures were de ned and computed. In Chapter 4 we presented a novel video scene segmentation algorithm [9, 14] that employs the improved spectral clustering algorithm of Chapter 3 and a sequence alignment algorithm. Shots were rst clustered into groups based only on their visual similarity using the method presented in Chapter 3 and a label was assigned to each shot according to the group that it belonged to. Then, a sequence alignment algorithm was applied to detect when a change occurs to the pattern of shot labels, providing the nal scene segmentation result. Numerical experiments on TV-series and movies have shown that the proposed scene detection method accurately detects most of the scene boundaries, while preserving good tradeo between recall and precision. In Chapter 5 we presented a high-level movie segmentation algorithm [13]. In this approach, the movie shots were represented with local invariant descriptors instead of color histograms, resulting into a visual words histogram representation. Next the visual words histograms of shots were temporally smoothed (using a gaussian kernel) with respect to histograms of neighboring shots in order to preserve valuable contextual information. This semantic smoothing process at di erent time scales results in e#cient movie segmentation at di erent high-levels, such as scenes and chapters.