SVCL - Boosting for Feature Extraction and Selection

Home	People	Research	Publications	Demos

News	Jobs	Prospective Students	About	Internal

Boosting Algorithms for Simultaneous Feature Extraction and Selection

In this project the problem of simultaneous feature extraction and selection, for classifier design, is considered. A new framework is proposed, based on boosting algorithms that can either 1) select existing features or 2) assemble a combination of these features. This framework is simple and mathematically sound, derived from the statistical view of boosting and Taylor series approximations in functional space. Unlike classical boosting, which is limited to linear feature combinations, the new algorithms support more sophisticated combinations of weak learners, such as "sums of products" or "products of sums". This is shown to enable the design of fairly complex predictor structures with few weak learners in a fully automated manner, leading to faster and more accurate classifiers, based on more informative features. Extensive experiments on synthetic data, UCI datasets, object detection and scene recognition show that these predictors consistently lead to more accurate classifiers than classical boosting algorithms.

Key Idea:

In classic boosting, a classifier is trained based on a linear combination of weak learner. The key idea is to build a classifier that is able to use more complicated combinations of weak learner such as sum of product or product of sum. The proposed algorithms in this project are able to automatically build such complicated combinations of weak learners.

classic boosting: sum of weak learner

new method: sum of product of weak learner

Some example of feature combinations:

Some of the features learned for face detection are shown in the below picture. The learned features are rather intuitive, mostly detectors of faces parts, such as eyes, mouth, or ears, in certain relative positions. This enables the rejection of face-like false-positive with a few feature evaluations.

Related Publications:

	Boosting Algorithms for Simultaneous Feature Extraction and Selection Mohammad J. Saberian and Nuno Vasconcelos . Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, 2012. [ps] [pdf][code]