Entropy-Guided Assessment of Image Retrieval Systems: Advancing Grouped Precision as an Evaluation Measure for Relevant Retrievability
DOI:
https://doi.org/10.31449/inf.v47i7.4661Abstract
The performance evaluation of Content Based Image Retrieval systems (CBIR), can be considered as a challenging and overriding problem even for human and expert users regarding the important numbers of CBIR systems proposed in the literature and applied to different image databases. The automatic measures widely used to assess CBIR systems are inspired from the general Text Retrieval (TR) domain such as precision and recall metrics. This paper proposes a new quantitative measure adapted to the CBIR particularity of relevant images grouping, which is based on the entropy of the returned relevant images. The proposed performance measure is easy to understand and to implement. A good discriminative power of the proposed measure is shown through a comparative study with the existing and well-known CBIR evaluation measures.References
Yong Rui and Thomas S. Huang (1999) Image retrieval: Current techniques, promising directions and open issues, Journal of Visual Communication and Image Representation, v.10, pp. 39–62.
Smeulders, Arnold W. M. and Worring, Marcel and Santini, Simone and Gupta, Amarnath and Jain, Ramesh (2000) Content-Based Image Retrieval at the End of the Early Years, IEEE Trans. Pattern Anal. Mach. Intell., IEEE Computer Society, v.22(12), pp. 1349–1380.
Datta, Ritendra and Joshi, Dhiraj and Li, Jia and Wang, James Z. (2008), Image Retrieval: Ideas, Influences, and Trends of the New Age, ACM Comput. Surv., ACM Publisher, v.40(2), pp. 5:1– 5:60.
url = http://trec.nist.gov/.
Wu, Shengli and McClean, Sally (2006) Information Retrieval Evaluation with Partial Relevance Judgment, Flexible and Efficient Information Handling, Springer Berlin Heidelberg, pp.86–93.
Yong Rui and Huang, Thomas S (2000) Optimizing learning in image retrieval, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, v.1, pp. 236–243.
Müller, Henning and Müller, Wolfgang and Squire, David McG. and Marchand-Maillet, Stephane and Pun, Thierry (2001) Performance Evaluation in Content-based Image Retrieval: Overview and Proposals, Pattern Recogn. Lett. Elsevier Science Inc., v.22(5), pp.593–601.
Cooper, William S. (1973) On selecting a measure of retrieval effectiveness, Journal of the American Society for Information Science, v.24(2), pp. 87-100.
Lewis, David D. (1995) Evaluating and Optimizing Autonomous Text Classification Systems, Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, Seattle, Washington, USA, pp.246–254.
Buckley, Chris and Voorhees, Ellen M. (2000) Evaluating Evaluation Measure Stability,Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieva, ACM, Athens, Greece, pp. 33–40.
David D. Lewis (1995) The TREC-4 Filtering Track, TREC, National Institute of Standards and Technology (NIST), Special Publication 500-236.
Zhao, Ying (2005) Criterion Functions for Document Clustering, phd thesis University of Minnesota USA.
Rosenberg, Andrew and Hirschberg, Julia (2007) V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pp.410–420.
Salton, G. (1971) The SMART Retrieval System—Experiments in Automatic DocumentProcessing, Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
C. J. van Rijsbergen (1979) Information Retrieval, Butterworth. [16] MPEG-7 (2000) Subjective evaluation of the MPEG-7 retrieval accuracy measure (ANMRR)./WG11, Doc. M6029.
Savvas A. Chatzichristofis and Chryssanthi Iakovidou and Yiannis S. Boutalis and Elli Angelopoulou (2014) Mean Normalized Retrieval Order (MNRO): a new content-based image retrieval performance measure, Multimedia Tools Appl., v.70(3), pp.1767–1798.
Harald Kosch and Paul Maier (2010) Content-Based Image Retrieval Systems - Reviewing and Benchmarking, JDIM, v.8(1), pp.54–64.
Gareth Loy and Jan-Olof Eklundh (2005) A Review of Benchmarking Content Based Image Retrieval.
Henning Müller, Antoine Geissbuhler, Stephane Marchand Maillet, Paul Clough (2004) Benchmarking image retrieval applications, Workshop on Visual Information Systems, pp.334–337.
Dubey, Shiv Ram (2020) A Decade Survey of Content Based Image Retrieval using Deep Learning, IEEE Transactions on Circuits and Systems for Video Technology
Xiwu Zhang and Lei Wang and Yan Su (2021) Visual place recognition: A survey from deep learning perspective, Pattern Recognition, v.113(2), pp. 107760.
Alistair Moffat and Justin Zobel (2008) Rank-biased precision for measurement of retrieval ef- fectiveness, ACM Trans. Inf. Syst., v.27, pp.2:1- 2:27.
Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
