Simultaneous image classification and annotation via fusing multimodal heterogeneous image features

Abstract

The increased prevalence of digital imagery drives the development and use of automated image indexing. Current techniques rely upon a single image feature type and consider image classification and annotation separately. The approach presented in this research strives to improve automation of image cataloguing by developing a model that uses multimodal heterogeneous image features in identifying classifications and annotations of images. The model in this research uses the correlation between classifications and annotations to enhance the predictive power of image labeling techniques. The main contributions of this work include (i) development of a unified framework for joint classification and annotation; (ii) use of heterogeneous image fusion with multiple image feature types; (iii) creation of an efficient algorithm with a theoretical proof of convergence. Using the NUS-WIDE image database, we compare the method presented in this research to competing methods in both classifications and annotations. Results show that the proposed model is competitive to competing models in predicting classifications and is superior in predicting annotations.