Multimodal Image Retrieval Based on Query Weighting

Download Full Text
Ebrahim Naderi, Elham Nikookar, Ali Tayebi Babookani
Published Date:
December 05, 2015
Volume 5, Issue 2
1 - 6

content-based image retrieval, learning to rank, multimodal retrieval, query weighting, ranking model, resampling, text-based image retrieval
Ebrahim Naderi, Elham Nikookar, Ali Tayebi Babookani, "Multimodal Image Retrieval Based on Query Weighting". International Journal of Research in Computer Science, 5 (2): pp. 1-6, December 2015. Other Formats


Image retrieval methods try to increase the performance of retrieval by adjusting weights of relevant and non-relevant images. Adjusting weights of queries instead of images is studied in this paper to see whether it increases or decreases performance of retrieval system. Evaluation measure of the method is calculated for each query instead of each sample. In proposed method, textual and visual features of images are used to retrieve ranked results. To train visual module, query weighting and updating weights in each iteration are used to concentrate on the queries which did not provide acceptable results and also cause performance drop of final retrieval. Compared with other previous researches, the results of applying the proposed method to IAPR TC-12 dataset indicate high efficiency of proposed method. Image retrieval methods like this can be used in a lot of applications such as, gasometer, digit recognition using gasometer, digits image etc.

  1. R. Kiros, R. Salakhutdinov, R. Zemel, Multimodal neural language models, Proceedings of the 31st International Conference on Machine Learning (ICML-14), 2014.
  2. A. Tariq, H. Foroosh, Feature-Independent Context Estimation for Automatic Image Annotation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
  3. O. E. Demerdash, L. Kosseim and S. Bergler, CLaC at ImageCLEFphoto 2008, ImageCLEF 2008 Reports, 2008.
  4. Julia Vogel, Semantic Modeling of Natural Scenes for Content-Based Image Retrieval, International Journal of Computer Vision, Springer, Vol 72, pp. 133-157, 2007.
  5. D. Frossyniotis, A. Likas, A. Stafylopatis, A clustering method based on boosting, Pattern Reconition Letters, Vol. 25, pp. 641-654, Elsevier, 2004.
  6. Y. Freund, R. Schapire, A short introduction to boosting, Journal of Japanese Society for Artificial Intelligence, vol. 14(5), pp. 771-780, September, 1999.
  8. Sarin, W. Kameyama, Targeting Diversity in Photographic Retrieval Task with Commonsense Knowledge, Clef- Campaign, Kameyama, 2008.
  9. Andrew P. Berman and Linda G. Shapiro. A flexible image database system for content-based retrieval. Computer Vision and Image Understanding, Vol. 75, 1999.
  10. M. Grubinger, P. Clough, H. Müller, T. Deselaers, The IAPR TC-12 Benchmark: A New Evaluation Resource for Visual Information Systems, Proceedings of International Workshop OntoImage’2006 Language Resources for Content-Based Image Retrieval, pp. 13-23, 2006.
  11. A. Gupta, Y. Verma, C. Jawahar, Choosing Linguistics over Vision to Describe Images, AAAI, 2012.
  12. Y. Verma, C, Jawahar, Im2Text and Text2Im: Associating Images and Texts for Cross-Modal Retrieval, Proceedings of the British Machine Vision Conference. BMVA Press. 2014.
  13. J. Villena-Román, S. Lana-Serrano, J. González-Cristóbal, MIRACLE-GSI at ImageCLEFphoto 2008: Experiments on Semantic and Statistical Topic Expansion, Working Note for the ImageCLEFphoto task, 2008.
  14. M. Inoue,P. Grover, Effects of Visual Concept-Based Post-retrieval Clustering in ImageCLEFphoto 2008, Working Note for the ImageCLEFphoto task, Denmark, 2008.
  15. O. Demerdash, L. Kosseim,S. Bergler, CLaC at ImageCLEFPhoto 2008, Concordia University, CLEF 2008, 2008.
  16. S. Tollari, M. Detyniecki, M. Ferecatu, H. Glotin, P. Mulhem, M. Amini, A. Fakeri-Tabrizi, P. Gallinari, H Sahbi, Z Zhao, Consortium AVEIR at ImageCLEFphoto 2008: on the Fusion of Runs, Working Note for the ImageCLEFphoto 2008 task, 2008.
  17. K. Donald, G. Jones, Dublin City University at CLEF 2006: Experiments for the ImageCLEF Photo Collection Standard Ad Hoc Task, Lecture Notes in Computer Science, Springer, Vol. 4730/2007, pp. 633-637, 2007.
  18. K. Yamauchi, T. Nomura, K. Usui, Y. Kamoi, T. Takagi, Meiji University at ImageCLEF2008 Photo Retrieval Task: Evaluation of Image Retrieval Methods Integrating Different Media, Lecture Notes in Computer Science, Springer, Vol. 5706/2009, pp. 551-559, 2009.
  19. Y. Chang, H. Chen, Approaches of Using a Word-Image Ontology and an Annotated Image Corpus as Intermedia for Cross-Language Image Retrieval, Lecture Notes in Computer Science, Springer, Vol. 4730/2007, pp. 625-632, 2007.

    Sorry, there are no citation(s) for this manuscript yet.