This project aims to characterize and quantify representation bias in video action recognition datasets. We measure the bias of popular action datasets towards object, scene and person representations, and collect a new dataset (Diving48) with explicit control of static biases, using it to evaluate the temporal modeling of action recognition models.

paper project

This project studies the problem of representation bias mitigation in vision datasets by resampling. We propose an adversarial resampling scheme that minimizes the bias of any given dataset, and demonstrate its effectiveness on reducing color bias in a synthetic Colored MNIST dataset, as well as static bias in action recognition. We show that representation debiasing is crucial in improving the generalization of models.

paper project

This project explores model debiasing as a means to improve transfer learning and generalization in video understanding. We quantify the dynamicness of video models by the difference between their predictions and those of a purely spatial model, and propose a dynamic representation learning (DRL) framework to improve transfer learning by optimizing the dynamic scores.

paper project

This project addresses model debiasing from an architectural perspective. We observe that video-language models suffer from static biases similar to those in video action recognition, preventing them from learning longer-term temporal dependencies. We propose a sparse architecture to improve the modeling efficiency of long clips, as well as a temporal expansion curriculum to facilitate learning beyond the static frames.

paper project