Context-Based Image Semantic Similarity for Prosthetic Knowledge

Textual information is concept-based information which is used for image representation, like captions, tags or comments. It can convey more concept-related meaning than low-level features. In this work, we will analyze the text connected to images (metadata, comments, tags, etc.) to extract a set of concepts, which can characterize the semantic context of the given image. We propose a context-based image similarity scheme for prosthetic knowledge by evaluating image similarity using the associated groups of concepts. The evaluation can be used in combination with different measures such as WordNet, Wikipedia, and other basic distance metrics to build the group distance comparison. Among semantic measures, web-based proximity measures (e.g. MC, Jaccard, Dice), which exploit statistical data provided by search engines, are particularly effective for similarity evaluation between concepts. Experiments are conducted on tagged images from Flickr repository. The results show that the proposed approach is adequate to measure the image concept similarity and the relationships among images with respect to human evaluation. The proposed methodology is able to reflect the collective notion of semantic similarity.