Home People Research Publications Demos
         
News Jobs Prospective
Students 		About Internal

Dynamic Texture Models

One family of visual processes that has relevance for various applications of computer vision is that of, what could be loosely described as, visual processes composed of ensembles of particles subject to stochastic motion. The particles can be microscopic (e.g plumes of smoke), macroscopic (e.g. leaves blowing in the wind), or even objects (e.g. a human crowd or a traffic jam). The applications range from remote monitoring for the prevention of natural disasters (e.g. forest fires), to background subtraction in challenging environments (e.g. outdoor scenes with moving trees in the background), and to surveillance (e.g. traffic monitoring, crowd analysis and management). While traditional motion representations model the movement of individual particles (e.g. optical flow), which may be contrary to how these visual processes are perceived, recent efforts have advanced toward holistic modeling, by viewing video sequences derived from these visual processes as dynamic textures (Doretto et. al, IJCV 2003) or, more precisely, samples from a generative, stochastic, texture model defined over space and time.

The goal of this project is to develop a family of motion models that extends and complements the original dynamic texture model. These new models can solve challenging computer vision problems, such as motion segmentation and motion classification, and can be applied to interesting real-world problems, such as crowd and traffic monitoring.

Models:
Kernel Dynamic Textures
We introduce a kernelized dynamic texture, which has a non-linear observation function learned with kernel PCA. The new texture model can account for more complex patterns of motion, such as chaotic motion (e.g. boiling water and fire) and camera motion (e.g. panning and zooming), better than the original dynamic texture.
[demo coming soon]

 		    

Layered Dynamic Textures new!
One disadvantage of the dynamic texture is its inability to account for multiple co-occurring textures in a single video. We extend the dynamic texture to a multi-state (layered) dynamic texture that can model regions containing different dynamics.
[project][demo]

    

Mixtures of Dynamic Textures
We introduce the mixture of dynamic textures, which models a collection of video as samples from a set of dynamic textures. We use the model for video clustering and motion segmentation.
[project | demo]

 		    

Applications:
Pedestrian Crowd Counting new!
We estimate the size of moving crowds in a privacy preserving manner, i.e. without people models or tracking. The system first segments the crowd by its motion using a mixture of dynamic textures, extracts low-level features from each segment, and estimates the crowd count in each segment using a Gaussian process.
[project | demo | PETS2009 demo]

 		    

Motion Segmentation
The mixture of dynamic textures provides a natural framework for motion segmentation via clustering of video patches. The model is robust at segmenting a wide variety of visual processes (e.g. fire, water, smoke, trees, people, and cars), which are beyond the reach of the state-of-the-art in computer vision, e.g. optical flow.
[project | DTM demo | LDT demo]

 		    

Classification and Retrieval of Traffic Video
We classify traffic congestion in video by representing the video as a dynamic texture, and classifying it using an SVM with a probabilistic kernel (the KL kernel). The resulting classifier is robust to noise and lighting changes.
[project | demo]

 		    


Publications: Analysis of Crowded Scenes using Holistic Properties
A. B. Chan, M. Morrow, and N. Vasconcelos
In 11th IEEE Intl. Workshop on Performance Evaluation of Tracking and Surveillance (PETS 2009),
Miami, June 2009. © IEEE [pdf]

Layered Dynamic Textures
A. B. Chan and N. Vasconcelos.
IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Probabilistic Graphical Models in Computer Vision (TPAMI),
to appear 2009. � IEEE [ps][pdf]

Variational Layered Dynamic Textures
A. B. Chan and N. Vasconcelos.
In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR),
Miami, June 2009. � IEEE [pdf]

Derivations for the Layered Dynamic Texture and Temporally-Switching Layered Dynamic Texture
A. B. Chan and N. Vasconcelos.
Technical Report SVCL-TR-2009-01,
June 2009. [pdf]

Privacy Preserving Crowd Monitoring: Counting People without People Models or Tracking
A. B. Chan, Z. S. J. Liang, and N. Vasconcelos.
In, IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
June 2008. � IEEE [ps][pdf]

Modeling, clustering, and segmenting video with mixtures of dynamic textures
A. B. Chan and N. Vasconcelos.
IEEE Trans. on Pattern Analysis and Machine Intelligence,
Vol. 30(5), pp. 909-926, May 2008. [ps][pdf].

 Classifying Video with Kernel Dynamic Textures
A. B. Chan and N. Vasconcelos
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Minneapolis, May 2007. [ps][pdf]

Layered Dynamic Textures
A. B. Chan and N. Vasconcelos,
Proceedings of Neural Information Processing Systems 18 (NIPS),
pp. 203-210, Vancouver, December 2005. [ps][pdf]

Mixtures of Dynamic Textures
A. B. Chan and N. Vasconcelos,
In IEEE International Conference on Computer Vision, Proceedings
October 2005. � IEEE, [ps][pdf].

 The EM algorithm for mixtures of dynamic textures
A. B. Chan and N. Vasconcelos,
Technical Report SVCL-TR-2005-01, March 2005. [ps][pdf].

 Probabilistic Kernels for the Classification of Auto-regressive Visual Processes
A. B. Chan and N. Vasconcelos,
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition,
San Diego, June 2005. � IEEE, [ps][pdf] (A longer version is available [ps][pdf]).

Classification and Retrieval of Traffic Video using Auto-regressive Stochastic Processes
A. B. Chan and N. Vasconcelos,
Proceedings of 2005 IEEE Intelligent Vehicles Symposium,
Las Vegas, June 2005. � IEEE, [pdf].

 Efficient Computation of the KL Divergence between Dynamic Textures
A. B. Chan and N. Vasconcelos,
Technical Report SVCL-TR-2004-02, November 2004. [ps][pdf]
Databases: We have gathered several databases for evaluation and application of dynamic texture models.

database purpose link
Highway Traffic classification [tgz 60MB]
Motion Database classification [link Under Construction]
Highway Traffic clustering [zip 42MB]
Synthetic Video Textures   segmentation [zip 212MB]
Pedestrian Crowds segmentation   [zip 755MB | readme]

Links: Here are links to more resources on Dynamic Textures:
Contact: Antoni Chan, Nuno Vasconcelos





© SVCL