Metrics details. We present a novel approach for contextual classification of image patches in complex visual scenes, based on the use of histograms of quantized features and probabilistic aspect models. Our approach uses context in two ways: 1 by using the fact that specific learned aspects correlate with the semantic classes, which resolves some cases of visual polysemy often present in patch-based representations, and 2 by formalizing the notion that scene context is image-specificβwhat an individual patch represents depends on what the rest of the patches in the same image are.
We demonstrate the validity of our approach on a man-made versus natural patch classification problem. Experiments on an image collection of complex scenes show that the proposed approach improves region discrimination, producing satisfactory results and outperforming two noncontextual methods. Furthermore, we also show that co-occurrence and traditional Markov random field spatial contextual information can be conveniently integrated for further improved patch classification.
Associating semantic class labels to image regions is a fundamental task in computer vision, useful in itself for image and video indexing and retrieval, and as an intermediate step for higher-level scene analysis [ 1 β 3 ]. While many image area classification approaches segment an image using all pixels [ 4 ] or by predefining a block-based image grid [ 1 , 3 ], in this work we consider local image patches characterized by viewpoint invariant descriptors [ 5 ].
This image representation, based on patches, robust with respect to partial occlusion, clutter, and changes in viewpoint and illumination, has shown its applicability in a number of vision tasks [ 2 , 6 β 9 ]. Local invariant regions do not cover the complete image, but they often occupy a considerable part of the scene and divide most of the scene into patches of salient content Figure 1.
In general, the constituent parts of a scene do not exist in isolation, and the visual contextβthe spatial dependencies between scene partsβcan be used to improve region classification [ 1 , 10 β 12 ]. Two image regions, indistinguishable from each other when analyzed independently, might be discriminated as belonging to the correct class with the help of context knowledge. Broadly speaking, there exists a continuum of contextual models for image region classification. On one end, one would find explicit models like Markov random fields MRFs , where spatial constraints are defined via local statistical dependencies between class region labels [ 10 , 13 ], and between observations and labels [ 1 ].
