Image collection exploration

Last updated

Image collection exploration is a mechanism to explore large digital image repositories. The huge amount of digital images produced every day through different devices such as mobile phones bring forth challenges for the storage, indexing and access to these repositories. Content-based image retrieval (CBIR) has been the traditional paradigm to index and retrieve images. However, this paradigm suffers of the well known semantic gap problem. Image collection exploration consists of a set of computational methods to represent, summarize, visualize and navigate image repositories in an efficient, effective and intuitive way. [1]

Contents

Summarization

Automatic summarization consists in finding a set of images from a larger image collection that represents such collection. [2] Different methods based on clustering have been proposed to select these image prototypes (summary). The summarization process addresses the problem of selecting a representative set of images of a search query or in some cases, the overview of an image collection.

Visualization

Image collection visualization is the process of visualize a set of images using a visualization metaphor, in which an image similarity function is used to represent image relations in a visualization layout. [3] Information visualization is an active area that investigates new ways to visualize information by using visualization metaphors. Particularly, new ways of visualizing image collections are being investigated, which propose conventional [4] and unconventional [5] visualization metaphors. If the images are sorted according to their similarities, a hierarchic image browsing approach similar to cartographic services such as Google Maps can be used. picsbuffet [6] is an online demo of such an approach. [7]

Interaction

Image collection interaction consists in offering users mechanisms to feedback image search systems. [8] In this interaction process, the system learns from user feedback to retrieve results more precise and relevant to the user.

Related Research Articles

Information retrieval (IR) is the activity of obtaining information system resources that are relevant to an information need from a collection of those resources. Searches can be based on full-text or other content-based indexing. Information retrieval is the science of searching for information in a document, searching for documents themselves, and also searching for the metadata that describes data, and for databases of texts, images or sounds.

Integrated development environment Software application used to develop software

An integrated development environment (IDE) is a software application that provides comprehensive facilities to computer programmers for software development. An IDE normally consists of at least a source code editor, build automation tools and a debugger. Some IDEs, such as NetBeans and Eclipse, contain the necessary compiler, interpreter, or both; others, such as SharpDevelop and Lazarus, do not.

Web crawler Software which systematically browses the World Wide Web

A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing.

An image retrieval system is a computer system for browsing, searching and retrieving images from a large database of digital images. Most traditional and common methods of image retrieval utilize some method of adding metadata such as captioning, keywords, title or descriptions to the images so that retrieval can be performed over the annotation words. Manual image annotation is time-consuming, laborious and expensive; to address this, there has been a large amount of research done on automatic image annotation. Additionally, the increase in social web applications and the semantic web have inspired the development of several web-based image annotation tools.

Visualization (graphics) set of techniques for creating images, diagrams, or animations to communicate a message

Visualization or visualisation is any technique for creating images, diagrams, or animations to communicate a message. Visualization through visual imagery has been an effective way to communicate both abstract and concrete ideas since the dawn of humanity. Examples from history include cave paintings, Egyptian hieroglyphs, Greek geometry, and Leonardo da Vinci's revolutionary methods of technical drawing for engineering and scientific purposes.

Content-based image retrieval method of image retrieval

Content-based image retrieval, also known as query by image content (QBIC) and content-based visual information retrieval (CBVIR), is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching for digital images in large databases. Content-based image retrieval is opposed to traditional concept-based approaches.

In statistics and related fields, a similarity measure or similarity function is a real-valued function that quantifies the similarity between two objects. Although no single definition of a similarity measure exists, usually such measures are in some sense the inverse of distance metrics: they take on large values for similar objects and either zero or a negative value for very dissimilar objects.

Tag cloud hitype of visual representation for text data

A tag cloud is a novelty visual representation of text data, typically used to depict keyword metadata (tags) on websites, or to visualize free form text. Tags are usually single words, and the importance of each tag is shown with font size or color. This format is useful for quickly perceiving the most prominent terms to determine its relative prominence. Bigger term means greater weight. When used as website navigation aids, the terms are hyperlinked to items associated with the tag.

Quartz Composer programming language

Quartz Composer is a node-based visual programming language provided as part of the Xcode development environment in macOS for processing and rendering graphical data.

Orange (software) component-based data mining and machine learning software suite

Orange is an open-source data visualization, machine learning and data mining toolkit. It features a visual programming front-end for explorative data analysis and interactive data visualization.

Geovisualization or geovisualisation, refers to a set of tools and techniques supporting the analysis of geospatial data through the use of interactive visualization.

Interaction technique

An interaction technique, user interface technique or input technique is a combination of hardware and software elements that provides a way for computer users to accomplish a single task. For example, one can go back to the previously visited page on a Web browser by either clicking a button, pressing a key, performing a mouse gesture or uttering a speech command. It is a widely used term in human-computer interaction. In particular, the term "new interaction technique" is frequently used to introduce a novel user interface design idea.

The UCSC Genome Browser is an on-line, and downloadable, genome browser hosted by the University of California, Santa Cruz (UCSC). It is an interactive website offering access to genome sequence data from a variety of vertebrate and invertebrate species and major model organisms, integrated with a large collection of aligned annotations. The Browser is a graphical viewer optimized to support fast interactive performance and is an open-source, web-based tool suite built on top of a MySQL database for rapid visualization, examination, and querying of the data at many levels. The Genome Browser Database, browsing tools, downloadable data files, and documentation can all be found on the UCSC Genome Bioinformatics website.

MeVisLab cross-platform application framework for medical image processing and scientific visualization

MeVisLab is a cross-platform application framework for medical image processing and scientific visualization. It includes advanced algorithms for image registration, segmentation, and quantitative morphological and functional image analysis. An IDE for graphical programming and rapid user interface prototyping is available.

Distinguishable interfaces use computer graphic principles to automatically generate easily distinguishable appearance for computer data.

Line integral convolution

In scientific visualization, line integral convolution (LIC) is a technique to visualize a vector field, like a fluid motion, such as the wind movement in a tornado. LIC has been proposed by Brian Cabral and Leith Leedom. Compared to other integration-based techniques that compute field lines of the input vector field, LIC has the advantage that all structural features of the vector field are displayed, without the need to adapt the start and end points of field lines to the specific vector field. LIC is a method from the texture advection family.

Pipeline Pilot is a desktop software program sold by Dassault Systèmes for processing and analyzing data. Originally used in the natural sciences, the product's basic ETL and analytics capabilities have been broadened. The product is now used for data science, ETL, reporting, prediction and analytics in a number of sectors. The main feature of the product is the ability to design data workflows using a graphical user interface. The program is an example of visual and dataflow programming. It has use in a variety of settings, such as cheminformatics and QSAR, Next Generation Sequencing, image analysis, and text analytics.

DeepDream is a computer vision program created by Google engineer Alexander Mordvintsev which uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia, thus creating a dream-like hallucinogenic appearance in the deliberately over-processed images.

BisQue is a free, open source web-based platform for the exchange and exploration of large, complex datasets. It is being developed at the Vision Research Lab at the University of California, Santa Barbara. BisQue specifically supports large scale, multi-dimensional multimodal-images and image analysis. Metadata is stored as arbitrarily nested and linked tag/value pairs, allowing for domain-specific data organization. Image analysis modules can be added to perform complex analysis tasks on compute clusters. Analysis results are stored within the database for further querying and processing. The data and analysis provenance is maintained for reproducibility of results. BisQue can be easily deployed in cloud computing environments or on computer clusters for scalability. BisQue has been integrated into the NSF Cyberinfrastructure project CyVerse. The user interacts with BisQue via any modern web browser.

Social navigation is a form of social computing introduced by Dourish and Chalmers in 1994. They defined it as when "movement from one item to another is provoked as an artifact of the activity of another or a group of others". According to later research in 2002, "social navigation exploits the knowledge and experience of peer users of information resources" to guide users in the information space. With all of the digital information available both on the World Wide Web and from other sources, it is becoming increasingly difficult to navigate and search efficiently. Studying others' navigational trails and understanding their behavior can help improve one's own search strategy by helping them to make more informed decisions based on the actions of others. "The idea of social navigation is to aid users to navigate information spaces through making the collective, aggregated, or individual actions of others visible and useful as a basis for making decisions on where to go next and what to choose."

References

  1. Camargo, Jorge E.; Caicedo, Juan C.; Gonzalez, Fabio A. (2013). "A kernel-based framework for image collection exploration". Journal of Visual Languages & Computing. 24 (1): 53–57. doi:10.1016/j.jvlc.2012.10.008.
  2. Yang, Chunlei; Shen, Jialie; Peng, Jinye; Fan, Jianping (2013). "Image collection summarization via dictionary learning for sparse representation". Pattern Recognition. 46 (3): 948–961. doi:10.1016/j.patcog.2012.07.011.
  3. Nguyen, G.P.; Worring, M. (2008). "Interactive access to large image collections using similarity-based visualization". Journal of Visual Languages & Computing. 19 (2): 203–224. doi:10.1016/j.jvlc.2006.09.002.
  4. Chaoli Wang, John P Reese, Huan Zhang, Jun Tao, Yi Gu, Jun Ma, and Robert J. Nemiroff. "Similarity-based visualization of large image collections". Information Visualization 1473871613498519, first published on August 6, 2013 doi : 10.1177/1473871613498519
  5. Marco Porta. 2006. Browsing large collections of images through unconventional visualization techniques. In Proceedings of the working conference on Advanced visual interfaces (AVI '06). ACM, New York, NY, USA, 440–444. doi : 10.1145/1133265.1133354
  6. picsbuffet
  7. Barthel, K.U.; Hezel, N.; Mackowiak, R. (2015). "ImageMap - Visually Browsing Millions of Images". Lecture Notes in Computer Science, Springer. Lecture Notes in Computer Science. 8936: 287–290. doi:10.1007/978-3-319-14442-9_30. ISBN   978-3-319-14441-2.
  8. Camargo, J.E.; Caicedo, J.C.; Chavarro, A.M.; Gonzaléz, F.A., "A kernel-based strategy for exploratory image collection search," Content-Based Multimedia Indexing (CBMI), 2010 International Workshop on , vol., no., pp.1,6, 23–25 June 2010. doi : 10.1109/CBMI.2010.5529893