Skip to main content
EURASIP Journal on Advances in Signal Processing
Download PDF
  • Research Article
  • Open access
  • Published:

Combining Global and Local Information for Knowledge-Assisted Image Analysis and Classification

EURASIP Journal on Advances in Signal Processing volume 2007, Article number: 045842 (2007) Cite this article

Abstract

A learning approach to knowledge-assisted image analysis and classification is proposed that combines global and local information with explicitly defined knowledge in the form of an ontology. The ontology specifies the domain of interest, its subdomains, the concepts related to each subdomain as well as contextual information. Support vector machines (SVMs) are employed in order to provide image classification to the ontology subdomains based on global image descriptions. In parallel, a segmentation algorithm is applied to segment the image into regions and SVMs are again employed, this time for performing an initial mapping between region low-level visual features and the concepts in the ontology. Then, a decision function, that receives as input the computed region-concept associations together with contextual information in the form of concept frequency of appearance, realizes image classification based on local information. A fusion mechanism subsequently combines the intermediate classification results, provided by the local- and global-level information processing, to decide on the final image classification. Once the image subdomain is selected, final region-concept association is performed using again SVMs and a genetic algorithm (GA) for optimizing the mapping between the image regions and the selected subdomain concepts taking into account contextual information in the form of spatial relations. Application of the proposed approach to images of the selected domain results in their classification (i.e., their assignment to one of the defined subdomains) and the generation of a fine granularity semantic representation of them (i.e., a segmentation map with semantic concepts attached to each segment). Experiments with images from the personal collection domain, as well as comparative evaluation with other approaches of the literature, demonstrate the performance of the proposed approach.

References

  1. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R: Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000,22(12):1349-1380. 10.1109/34.895972

    Article  Google Scholar 

  2. Bloehdorn S, Petridis K, Saathoff C, et al.: Semantic annotation of images and videos for multimedia analysis. Proceedings of the 2nd European Semantic Web Conference (ESWC '05), May-June 2005, Herakleion, Greece 592–607.

    Google Scholar 

  3. Barla A, Odone F, Verri A: Old fashioned state-of-the-art image classification. Proceedings of the 12th International Conference on Image Analysis and Processing (ICIAP '03), September 2003, Mantova, Italy 566–571.

    Google Scholar 

  4. Tang J, Zhang C-Y, Luo B: A graph and PNN-based approach to image classification. Proceedings of International Conference on Machine Learning and Cybernetics (ICMLC '05), August 2005, Guangzhou, China 8: 5122–5126.

    Google Scholar 

  5. Tanaka H, Sakano H, Ohtsuka S: Retrieval method for multi-category images. Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 2: 965–968.

    Article  Google Scholar 

  6. Ahmad I, Ibrahim MT: Image classification and retrieval using correlation. Proccedings of the 3rd Canadian Conference on Computer and Robot Vision (CRV '06), June 2006, Quebec City, Canada 60.

    Chapter  Google Scholar 

  7. Papadopoulos S, Mezaris V, Kompatsiaris I, Strintzis MG: A region-based approach to conceptual image classification. Proceedings of IEE International Conference on Visual Information Engineering (VIE '05), April 2005, Glasgow, UK 141–147.

    Chapter  Google Scholar 

  8. Tseng VS, Wang M-H, Su J-H: A new method for image classification by using multilevel association rules. Proceedings of the 21st International Conference on Data Engineering Workshops (ICDE '05), April 2005, Tokyo, Japan 1180.

    Chapter  Google Scholar 

  9. Blaschke T: Object-based contextual image classification built on image segmentation. Proceedings of IEEE Workshop on Advances in Techniques for Analysis of Remotely Sensed Data (WARSD '03), October 2003, Washington, DC, USA 113–119.

    Chapter  Google Scholar 

  10. Wang Z, Feng D, Chi Z: Region-based binary tree representation for image classification. Proceedings of International Conference on Neural Networks and Signal Processing (ICNNSP '03), December 2003, Nanjing, China 1: 232–235.

    Google Scholar 

  11. Dasiopoulou S, Mezaris V, Kompatsiaris I, Papastathis V-K, Strintzis MG: Knowledge-assisted semantic video object detection. IEEE Transactions on Circuits and Systems for Video Technology 2005,15(10):1210-1224.

    Article  Google Scholar 

  12. Hollink L, Little S, Hunter J: Evaluating the application of semantic inferencing rules to image annotation. Proceedings of the 3rd International Conference on Knowledge Capture (K-CAP '05), October 2005, Banff, Canada 91–98.

    Chapter  Google Scholar 

  13. Maillot N, Thonnat M, Hudelot C: Ontology based object learning and recognition : application to image retrieval. Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence (ICTAI '04), November 2004, Boca Raton, Fla, USA 620–625.

    Chapter  Google Scholar 

  14. Kompatsiaris I, Mezaris V, Strintzis MG: Multimedia content indexing and retrieval using an object ontology. In Multimedia Content and Semantic Web-Methods, Standards and Tools. John Wiley & Sons, New York, NY, USA; 2004:339-371.

    Google Scholar 

  15. Tansley R, Bird C, Hall W, Lewis P, Weal M: Automating the linking of content and concept. Proceedings of the 8th ACM International Multimedia Conference and Exhibition (MULTIMEDIA '00), October-November 2000, Los Angeles, Calif, USA 445–447.

    Chapter  Google Scholar 

  16. Kim KI, Jung K, Park SH, Kim HJ: Support vector machines for texture classification. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002,24(11):1542-1550. 10.1109/TPAMI.2002.1046177

    Article  Google Scholar 

  17. Chapelle O, Haffner P, Vapnik VN: Support vector machines for histogram-based image classification. IEEE Transactions on Neural Networks 1999,10(5):1055-1064. 10.1109/72.788646

    Article  Google Scholar 

  18. Mitchell M: An Introduction to Genetic Algorithms. MIT Press, Cambridge, Mass, USA; 1995.

    MATH  Google Scholar 

  19. Voisine N, Dasiopoulou S, Precioso F, Mezaris V, Kompatsiaris I, Strintzis MG: A genetic algorithm-based approach to knowledge-assisted video analysis. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 3: 441–444.

    Google Scholar 

  20. Papadopoulos GTh, Panagi P, Dasiopoulou S, Mezaris V, Kompatsiaris I: A learning approach to semantic image analysis. Proceedings of the 2nd International Mobile Multimedia Communications Conference (MobiMedia '06), September 2006, Alghero, Italy

    Google Scholar 

  21. MPEG-7 Visual Experimentation Model (XM) Version 10.0, ISO/IEC/JTC1/SC29/WG11, Doc. N4062, March 2001

  22. Adamek T, O'Connor N, Murphy N: Region-based segmentation of images using syntactic visual features. Proceedings of the 6th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS '05), April 2005, Montreux, Switzerland

    Google Scholar 

  23. Skiadopoulos S, Giannoukos C, Sarkas N, Vassiliadis P, Sellis T, Koubarakis M: Computing and managing cardinal direction relations. IEEE Transactions on Knowledge and Data Engineering 2005,17(12):1610-1623.

    Article  Google Scholar 

  24. Wang Y, Makedon F, Ford J, Shen L, Goldin D: Generating fuzzy semantic metadata describing spatial relations from images using the R-Histogram. Proceedings of the 4th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '04), June 2004, Tucson, Ariz, USA 202–211.

    Google Scholar 

  25. Staab S, Studer R (Eds): Handbook on Ontologies, International Handbooks on Information Systems. Springer, Berlin, Germany; 2004.

    Google Scholar 

  26. Tax DMJ, Duin RPW: Using two-class classifiers for multiclass classification. Proceedings of the 16th International Conference on Pattern Recognition (ICPR '02), August 2002, Quebec City, Canada 2: 124–127.

    Article  Google Scholar 

  27. Chang C-C, Lin C-J: LIBSVM: a library for support vector machines. 2001.https://doi.org/www.csie.ntu.edu.tw/~cjlin/libsvm/

    Google Scholar 

  28. Goldberg DE, Deb K: A comparative analysis of selection schemes used in genetic algorithms. In Foundations of Genetic Algorithms. Edited by: Rawlins G. Morgan Kaufmann Publishers, San Mateo, Calif, USA; 1991:69-93.

    Google Scholar 

  29. Wall M: GAlib: A C++ Genetic Algorithm Library. MIT, 2005, https://doi.org/lancet.mit.edu/ga/

  30. Flickr https://doi.org/www.flickr.com/

  31. Ren J, Shen Y, Ma S, Guo L: Applying multi-class SVMs into scene image classification. Proceedings of the 17th International Conference on Innovations in Applied Artificial Intelligence, May 2004, Ottawa, Canada 924–934.

    Chapter  Google Scholar 

  32. Spyrou E, Le Borgne H, Mailis T, Cooke E, Avrithis Y, O'Connor N: Fusing MPEG-7 visual descriptors for image classification. Proceedings of International Conference on Artificial Neural Networks (ICANN '05), September 2005, Warsaw, Poland 847–852.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Aristotle University of Thessaloniki, Thessaloniki, 54006, Greece

    G. Th. Papadopoulos & M. G. Strintzis

  2. Centre for Research and Technology Hellas (CERTH), Informatics and Telematics Institute, Thermi, 57001, Greece

    G. Th. Papadopoulos, V. Mezaris, I. Kompatsiaris & M. G. Strintzis

Authors
  1. G. Th. Papadopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. V. Mezaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I. Kompatsiaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. G. Strintzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to G. Th. Papadopoulos.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Papadopoulos, G.T., Mezaris, V., Kompatsiaris, I. et al. Combining Global and Local Information for Knowledge-Assisted Image Analysis and Classification. EURASIP J. Adv. Signal Process. 2007, 045842 (2007). https://doi.org/10.1155/2007/45842

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/45842

Keywords