The classical localization approaches only focus on the performance of features extracted from images but ignore contextual information hidden in the images. In this paper, it is annotated on the images and SVM model is used to classify different images for semantic localization. Supervised Latent Dirichlet Allocation (sLDA) model is introduced to obtain the annotations, and the standard SIFT algorithm is improved to extract feature descriptors. Two situations are designed for the acquisition of contextual annotations, which are to provide the accurate contextual annotations directly and to infer contextual information by sLDA model. The effect of contextual information in scene classification is simulated and verified.