The study of motion is an important problem in computer vision and image processing. There are many ways in which modeling motion can be useful. For example, in image understanding, the fact that pixels associated with an object tend to move in a coherent way makes motion a very strong cue for object segmentation. In the domain of image compression, the ability to estimate motion allows sophisticated encoders to eliminate the temporal redundancies in the video sequence and achieve compression gains that would not be feasible otherwise. In computer vision, tasks such as object tracking or the construction of mosaics from video are, by definition, dependent on the ability to estimate motion. In video retrieval, the way in which objects move can have a big impact on the similarity of two video sequences, or their semantic categorization.
This project studies the design of motion models for various video analysis, understanding, and summarization tasks. These range from probabilistic models for motion segmentation that combine mixture models with priors that guarantee greater spatial coherence of the segmentation labels (as well as new expectation-maximization algorithms for learning the parameters of these models) to spatio-temporal motion models that augment affine constraints in the spatial domain with smoothness constraints in the temporal domain, providing better estimates of global motion parameters in the absence of strong object textured or when occlusion makes the estimation problematic.
These models have been shown to be useful in tasks such as video segmentation, video summarization with mosaics, estimation of dominant motion in a video sequence, and the extraction of semantic video descriptors.