Colour in Context
Research group
Computer Vision Center

Coloring Bag-of-Words based Object Recognition

Generally the bag-of-words based image representation follows a bottom-up paradigm. The subsequent stages of the process: feature detection, feature description, vocabulary construction and image representation are performed independent of the intentioned object classes to be detected. Within the bag-of-words framework the optimal fusion of different cues, such as shape, texture and color, still remains open to debate.

We proposed a novel method for recognizing object categories when using multiple cues by separating the shape and color cue. Color is used to guide attention by means of a top-down category-specific attention map. The color attention map is then further deployed to modulate the shape features by taking more features from regions within an image that are likely to contain an object instance. This procedure leads to a category-specific image histogram representation for each category. The proposed approach also combines the advantages of early and late fusion.

A brief, concise overview of our method is shown in the figure. In the standard bag-of-words image representation, the histogram over visual shape words, is constructed in a bottom-up fashion. In our approach we use top-down category-specific color attention to modulate the impact of the shape-words in the image during the histogram construction. Consequently, a separate histogram is constructed for the all categories, where the visual words relevant to each category (in this case flowers and butterflies) are accentuated.


Example code of color attention can be downloaded here.


Fahad Shahbaz Khan, Joost van de Weijer, Maria Vanrell Modulating Shape Features by Color Attention for Object Recognition , IJCV , accepted 20112.

Fahad Shahbaz Khan, Joost van de Weijer, Maria Vanrell Top-Down Color Attention for Object Recognition , Proc. ICCV09 , Kyoto, Japan, 2009.

  2008 Colour in context Group | Computer Vision Center. All rights reserved | Contact webmaster |  Last updated: Monday 11 May 2009     eXTReMe Tracker