SVCL - Top-down Discriminant Saliency

Home People Research Publications Demos

News Jobs Prospective
Students About Internal

Top-down Discriminant Saliency

Biological vision systems rely on saliency mechanisms to cope with the complexity of visual perception. Rather than sequentially scanning all possible locations of a scene, saliency mechanisms make locations that merit further inspection "pop-out" from the background. This enables the efficient allocation of perceptual resources, and increases the robustness of recognition in highly cluttered environments.

In computer vision, saliency detectors have been widely used in the design of object recognition systems. In these applications, saliency is often justified as a pre-processing step that saves computation and improves robustness. However, most of these detectors are based solely on bottom-up processing, i.e. purely stimulus-driven, and do not tie the definition of saliency to the top-down goal of recognition. For example, saliency is frequently defined as the detection of edges, contours, corners, etc. In result, the detected locations do not co-occur with the objects of interest.

In this work, we introduce a computational definition of top-down saliency which equates saliency to discrimination. The salient attributes of a given visual class are defined as the features that enable best discrimination between that class and all other classes of recognition interest. An optimal (top-down) discriminant saliency detector is derived through combination of 1) efficient information theoretic methods for feature selection, 2) a decision-theoretic rule for the identification of salient locations, and 3) exploitation of well known statistical properties of natural images to guarantee computational efficiency. The resulting optimal detector has been applied to the problem of learning object detectors with weak supervised (unsegmented training examples). Experimental evaluation shows that it effectively acts as a focus-of-attention mechanism, capable of pruning away bottom-up interest points that are irrelevant for recognition. In particular, this focus-of-attention mechanism is shown to have good performance with respect to a number of desirable properties for recognition: 1) the ability to localize objects embedded in significant amounts of clutter, 2) the amount of information of relevance for the recognition task that is captured by the points, 3) the robustness of salient points to various geometric transformations and pose variability, and 4) the richness of the set of visual attributes that can be considered salient.

Selected Publications:

Discriminant saliency, the detection of suspicious coincidences, and applications to visual recognition
D. Gao, S. Han, and N. Vasconcelos
To appear in IEEE Trans. on Pattern Analysis and Machine Intelligence,
2009. � IEEE,

Discriminant Interest Points are Stable
D. Gao and N. Vasconcelos.
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Minneapolis, MN, 2007. � IEEE, [ps][pdf]

Integrated learning of saliency, complex features, and object detectors from cluttered scenes
D. Gao and N. Vasconcelos,
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
San Diego, CA, 2005. � IEEE, [ps][pdf] (A longer version is available [ps][pdf])

An Experimental Comparison of Three Guiding Principles for the Detection of Salient Image Locations: Stability, Complexity, and Discrimination
D. Gao and N. Vasconcelos,
The 3rd International Workshop on Attention and Performance in Computational Vision (WAPCV), San Diego, 2005. [ps] [pdf]

Discriminant Saliency for Visual Recognition from Cluttered Scenes
D. Gao and N. Vasconcelos,
Proceedings of Neural Information Processing Systems (NIPS) ,
Vancouver, Canada, 2004. [ps][pdf]

Demos/
Results:

Experimental results on object and texture databases.

Code:

The compiled binary for discriminant saliency detection is available here. Read the License conditions before download. See the README for the usage of the programs.
Installation of ImageMagick is needed for running the code. It can be downloaded free here.

Contact: Dashan Gao, Nuno Vasconcelos, Sunhyoung Han

Home	People	Research	Publications	Demos

News	Jobs	Prospective Students	About	Internal

Top-down Discriminant Saliency

Biological vision systems rely on saliency mechanisms to cope with the complexity of visual perception. Rather than sequentially scanning all possible locations of a scene, saliency mechanisms make locations that merit further inspection "pop-out" from the background. This enables the efficient allocation of perceptual resources, and increases the robustness of recognition in highly cluttered environments. In computer vision, saliency detectors have been widely used in the design of object recognition systems. In these applications, saliency is often justified as a pre-processing step that saves computation and improves robustness. However, most of these detectors are based solely on bottom-up processing, i.e. purely stimulus-driven, and do not tie the definition of saliency to the top-down goal of recognition. For example, saliency is frequently defined as the detection of edges, contours, corners, etc. In result, the detected locations do not co-occur with the objects of interest. In this work, we introduce a computational definition of top-down saliency which equates saliency to discrimination. The salient attributes of a given visual class are defined as the features that enable best discrimination between that class and all other classes of recognition interest. An optimal (top-down) discriminant saliency detector is derived through combination of 1) efficient information theoretic methods for feature selection, 2) a decision-theoretic rule for the identification of salient locations, and 3) exploitation of well known statistical properties of natural images to guarantee computational efficiency. The resulting optimal detector has been applied to the problem of learning object detectors with weak supervised (unsegmented training examples). Experimental evaluation shows that it effectively acts as a focus-of-attention mechanism, capable of pruning away bottom-up interest points that are irrelevant for recognition. In particular, this focus-of-attention mechanism is shown to have good performance with respect to a number of desirable properties for recognition: 1) the ability to localize objects embedded in significant amounts of clutter, 2) the amount of information of relevance for the recognition task that is captured by the points, 3) the robustness of salient points to various geometric transformations and pose variability, and 4) the richness of the set of visual attributes that can be considered salient.
Selected Publications:	Discriminant saliency, the detection of suspicious coincidences, and applications to visual recognition D. Gao, S. Han, and N. Vasconcelos To appear in IEEE Trans. on Pattern Analysis and Machine Intelligence, 2009. � IEEE, Discriminant Interest Points are Stable D. Gao and N. Vasconcelos. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Minneapolis, MN, 2007. � IEEE, [ps][pdf] Integrated learning of saliency, complex features, and object detectors from cluttered scenes D. Gao and N. Vasconcelos, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, 2005. � IEEE, [ps][pdf] (A longer version is available [ps][pdf]) An Experimental Comparison of Three Guiding Principles for the Detection of Salient Image Locations: Stability, Complexity, and Discrimination D. Gao and N. Vasconcelos, The 3rd International Workshop on Attention and Performance in Computational Vision (WAPCV), San Diego, 2005. [ps] [pdf] Discriminant Saliency for Visual Recognition from Cluttered Scenes D. Gao and N. Vasconcelos, Proceedings of Neural Information Processing Systems (NIPS) , Vancouver, Canada, 2004. [ps][pdf]
Demos/ Results:	Experimental results on object and texture databases.
Code:	The compiled binary for discriminant saliency detection is available here. Read the License conditions before download. See the README for the usage of the programs. Installation of ImageMagick is needed for running the code. It can be downloaded free here.
Contact:	Dashan Gao, Nuno Vasconcelos, Sunhyoung Han