Action Recognition in Still Images


    We investigate the problem of action recognition in static images, for which problem bag-of-words image representations obtain promising results. These representations typically use only shape cues and ignore color information. Inspired by the recent success of color in image classification and object recognition, we investigate the potential of color for action recognition in static images.

    We perform a comprehensive evaluation of state-of-the-art color descriptors and fusion approaches for action recognition. Experiments were conducted on the three principal datasets for benchmarking of action recognition in still images: Willow, PASCAL VOC 2010 and Stanford-40. Our experiments demonstrate that incorporating color information considerably improves recognition performance. A descriptor based on color names is shown to outperform other pure color descriptors. Differently than in image classification, where early fusion yields the best results, our experiments demonstrate that late fusion of color and shape information outperforms other approaches on action recognition. Finally, we show that the different color-shape fusion approaches result in complementary information and combining them yields state-of-the-art performance.

Paper

    Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew D. Bagdanov, Antonio M. Lopez, Michael Felsberg, Coloring Action Recognition in Still Images , International Journal in Computer Vison (IJCV), 105(3):205:221, 2013.

    The article is partially based on earlier publications:

    1. Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew Bagdanov, Maria Vanrell, Antonio M. Lopez, Color Attributes for Object Detection, Proc. (CVPR), Providence, Rhode Island, USA, 2012.( webpage+code)

    2. Fahad Shahbaz Khan, Joost van de Weijer, Andrew Bagdanov, Maria Vanrell, Portmanteau Vocabularies for Multi-Cue Image Representation, Proc. (NIPS), Granada, Spain, 2011.( webpage+code)

    3. Fahad Shahbaz Khan, Joost van de Weijer, Maria Vanrell, Modulating Shape Features by Color Attention for Object Recognition , International Journal of Computer Vision (IJCV), vol 98(1), 49-64, 2012.( webpage+code)