Object and Scene Recognition with Bags of Features and Spatial Pyramids

Bag-of-features models, which represent images by distributions of salient local features contained in them, are among the most robust and powerful image descriptions currently used for object and scene recognition. In this talk, I will present fundamental techniques for designing effective bag-of-features models and their extensions by constructing discriminative visual codebooks and incorporating spatial relationships between local features.

The most basic operation in building a bag-of-features model is quantizing the local features, so that their distribution can be represented as a histogram of discrete “visual codewords.” I will introduce an information-theoretic approach to designing visual codebooks by minimizing the loss of discriminative information incurred when a continuous high-dimensional feature vector is mapped to a discrete codeword index. I will present experiments demonstrating the advantage of these codebooks for image classification, as well as an application of the same information-theoretic framework to image segmentation.

In the second part of the talk, I will describe an extension of a bag of features into a spatial pyramid, or a collection feature histograms computed at different levels of a hierarchical spatial decomposition of an image. The resulting method is simple and efficient, and it achieves state-of-the-art performance on difficult object and scene recognition tasks. It has already been adopted as a baseline for datasets containing hundreds of object categories, and has given rise to a winning recognition system in the international PASCAL Visual Object Classes Challenge.

Dr. Svetlana Lazebnik

Assistant Professor, University of North Carolina at Chapel Hill on October 5, 2007 at 12:30 PM in Engineering Building II, Room 1230

Dr. Svetlana Lazebnik has received the B.S. degree in computer science from DePaul University in Chicago, IL, in 2000 and the M.S. degree in computer science from the University of Illinois at Urbana-Champaign (UIUC) in 2002. She has completed her Ph.D. dissertation, entitled “Local, Semi-Local and Global Models for Texture, Object and Scene Recognition, at UIUC in the spring of 2006. This work was co-supervised by Prof, Jean Ponce at UIUC and Dr. Cordelia Schmid at INRIA Rhone-Alpes. After graduating, Dr. Lazebnik spent a year as a post-doctoral researcher at UIUC, and as of August 2007 she joined the Department of Computer Science at the University of North Carolina at Chapel Hill as an assistant professor. Her research interests include computer vision, object recognition, and machine learning.

The Department of Electrical and Computer Engineering hosts a regularly scheduled seminar series with preeminent and leading reseachers in the US and the world, to help promote North Carolina as a center of innovation and knowledge and to ensure safeguarding its place of leading research.