También disponible en Español


The digital magazine of InfoVis.net

Visual Search Engines
by Sergi Mínguez y Juan C. Dürsteler [message nº 198]

Internet has become a key player in our daily lives and a primary source of information. With nearly 70% of web users using a search engine as their entry point, the way the cascade of information they provide is visually presented becomes more relevant. Here we summarise the main treats of visual search engines and a list of the more conspicuous ones.
SearchmeMadonna.jpg (245686 bytes)
Searchme results for the query "Madonna" under the topic "images"
Source: as can be seen in the searchme web site
Click on the image to enlarge it

Most of the existing contents is published, referenced or illustrated in Internet. Thus, it is normal that people consider Internet as their primary tool to find the resources they need. To some extent if it's not in the Web it doesn't exists in the real world.

Furthermore, Social media is growing fast providing new Internet applications like YouTube, Facebook, or Flicker, that are introducing a large amount of new data in an exponential way. Images, videos, books, news, music and any kind of multimedia is available online with a wealth of interaction interfaces. 

The most important companies developing search engines such as Yahoo, Google or Microsoft do know that they do not have the capacity to search within this type of data. Therefore, they are starting to improve their engines and modify their current interfaces to allow for new functions.

We can identify some examples like:

  • Yahoo Alpha: Provides a new interface where the traditional list is related to other kinds of contents ( music, video, images, etc. ) using an internal categorization.

  • Yahoo Monkey Search: SearchMonkey is an open platform that enables developers  to use structured data to enhance Yahoo! Search results including reviews, photos and deep links.

  • Google Universal Search: Announced in May 2007 and now part of Googles's search experience it blends the traditional listings with news, video and images.

Despite this improvement, the new interfaces inherit the same old and basic problems without providing significant enhancement in the interaction. The explosion of multimedia and the new information needs have only increased the need for more complex, visually rich interfaces capable of searching among related multimedia content. For a long while now some Visual browsers have appeared to be trying to overcome the limitations of traditional search engine presentation and interaction.

Until now, Visual Search Engines have been considered more artistic than useful online applications. We need to introduce two concepts in order to understand why visual search engines are needed.

  • Query intention: Classic Information Retrieval defines the query intention as an "information need". However, the need behind web search is often not only informational. Andrei roder introduced in 2002 the article A Taxonomy of Web Search to identify different needs of web searches and discuss how global search engines evolved to deal with this web-specific needs. In the article he states that there exists three different categories to describe different behaviours in web search:

    • Navigational. The immediate intent is to reach a particular site that the user has in mind, either since they visited it in the past or because they assume that such a site exists.

    • Informational. The intent is to acquire some information assumed to be present on one or more web pages. No further interaction is predicted, except reading the contents.

    • Transactional. The intent is to perform some web-mediated activity. The interaction constitutes the transaction defining these queries, and they are the most difficult to evaluate.

    In order to determine the relative relevance of the various types of queries Broder made a survey of AltaVista users and an analysis of the query log at AltaVista which concluded that, in 2002, 50% of the queries were informational, 30% transactional and only 20% navigational. Lately, in 2008, Bernard J. Jansen et al.  implemented a system for the automatic classification of a large set of queries (over 1,5 million) from a Web search engine into Broder's taxonomy. The results pointed out that 80.6% of the queries were Informational, 10.2% Navigational and only a 9.2% were Transactional.

    As we can see acknowledgement of the intention of the user is necessary to understand the need behind the query and from this to provide proper results for the searcher. Therefore, Visual Search Engines can benefit from dynamical queries to redefine the query terms in real time providing a richer and more complex search.

  • Exploratory Search: In 2006 G. Marchionini published a work more focused on interaction with search engines based on Exploratory Search. In his study he distinguishes three different main tasks users carry out while querying. He defines them as follows:

    • Lookup is the most basic kind of search task and has been the focus of development for database management systems and much of what Web search engines support. Return discrete and well-structured objects such as numbers, names, short statements, or specific files of text or in other formats.

    • Searching to learn is increasingly viable as more primary materials go online. It involves multiple iterations and what it returns is sets of objects that require from the user cognitive processing and interpretation.

    • Searches to investigate involve several iterations taking place sometimes over long periods of time. They can return results that have been critically evaluated before integrating them into knowledge bases.

    Finally Marchionini claims that an exploratory search must be a combination of the two last task categories where people should be engaged by concepts as knowledge acquisition, comparison, synthesis, investigation or socialization.

From what we have seen before, conventional search engines only provide lookup search returning discrete and well-structured objects basically for informational purposes. Only more exploratory applications, like Visual Search Engines, appear to be able to properly support searching to learn or searches that support investigation. It's not clear at this moment whether those type of search engines can change the ratio of informational/navigational/transactional queries or not.

Some examples of these tools are

Kartoo09.gif (153185 bytes) Co-founded in France in 2001, the company launched in 2002, KartOO  a meta search engine implemented in Flash that provides a visual interface to explore the results. Once you enter a new search you come up with a "map" instead of a list. This map visualises the results as entities and relevant words as relationships. Web sites are represented with a low resolution snapshot of the site and the URL. Moreover, the size is used as a visual attribute to map the relevance of the site in the query (see also issue number 97).

Relevant words are also used to relate different results displayed in the map, and furthermore, to add the possibility to redefine the search adding the word to the initial query. The same relational information produces level curves using the landscape metaphor you can find in cartographical maps. The tool uses enclosure as a way to enable the user to see what the relations are between objects more clearly than with links or patches.

Rich interaction engages the user in the search process providing exploration. Thus, it appears to be a good solution for informational tasks. The image shows the map generated in response to the query Madonna. The home of the artist has the biggest size, with Wikipedia and MTV following. As relevant words we could find news, music, ringtones or premieres.

Notice that relevant words show the intention of the user more than displaying the results hidden behind them. For example, if the user continues the search adding music it is because he is carrying out an informational search, however if the user adds downloads, this is because the intention of the user is to perform a transactional task. 

Ujiko09.gif (145925 bytes) Kartoo has other interesting projects, like Ujiko. The display shows the results within an oval menu with an open center section where the user can flag specific results as relevant. The open section center is used to add information to the results. Coloured blocks help the user to identify nominal clusters.

Another interesting feature of this search engine is that the more you use it, the more functions it is able to offer. Each time you visit a new site, you are gaining one point of expertise. With every 10 points, you move to the next level achieving new functionalities. 

This innovative feature uses the well known scheme of role games to engage the user in the application providing a progressive discovery of its possibilities and easing the learning curve. 

Grokker09.gif (90767 bytes) Grokker provides a set of intuitive ways to navigate through large sets of results. One example is Grokker's visual map, based on the balloon tree visual metaphor. In it a set of nested circles represent categories for which size represents the amount of content within each circle. The user can click on the circles to explore its contents and also has the possibility to zoom in and out until he or she reaches the individual documents. See also issue number 138

Another interesting feature is result refinement through the use of filters. Grokker allows you to dynamically refine results without having to reframe queries. In particular you can filter by date, author, publication, country, etc, as well as by keyword inclusion/ exclusion. Furthermore you can customise the results display with different levels of detail by adding or excluding more information. 

Hence, Grokker appears to be a good alternative to navigate through large amounts of data. As it is implemented in JavaScript it's faster than other flash applications. On the contrary, JavaScript's limitation in visualisation causes it to not use visual information in the results display. In conclusion, we could consider visualisation of the clustering as the main contribution of grokker to visual search engines.

simploos09.gif (175510 bytes) Simploos is another Visual Search Engine developed in Mexico and presented in 2008. It is implemented in Flash and it introduces an interesting idea. Simploos displays the results as high resolution pre-visualisation of the web sites. This feature allows the user to perform fast comparisons avoiding the loading of the page and without performing a click.

Nevertheless, even though they use cache to store the images, the mechanism used to obtain the pre-visualisation is considerably slow. Another important feature to consider is the navigation throughout the pre-visualisation. The user is able to scroll right-left or bottom-up with only the movement of the mouse in a quite intuitive way once you get used to it.

Therefore, despite Simploos being only a meta search engine that uses Yahoo or Google depending on users choice, its contribution is powerful. It shows that pre-visualising the real content of the site, more than with just a snapshot, gives important information that could accelerate the final searching process.

quintura.gif (50110 bytes) Quintura is another visual search engine developed in JavaScript. Presented in 2008 too, it follows the same idea introduced by KartOO. It uses relevant words ( tags ) to refine the queries and entice people to explore more than search. 

The interaction with the cloud of tags presents another innovative feature. When the user hovers over a tag he is capable of seeing a previsualisation of the search. This fast task allows him or her to navigate more intuitively seeing which could be the next step before choosing the final path.

Save or share the query is another feature that is introduced with this application. However, due to the necessity to replace the results easily, Quintura doesn't use any visual approach to display the results. Therefore, the only important feature introduced with this software ( apart of a not very intuitive interface for kids ) is a fast navigational cloud of tags.

oSkope.gif (274627 bytes) oSkope is a free online service developed by oSkope media gmbh n Zurich and Berlin. It is implemented in flash and it uses information obtained from different kinds of search engines: Yahoo, Ebay, Youtube, Flickr, Fotolia or Amazon.

As a purely visual search engine its most interesting feature is its approach to display query results on the screen. It uses fast pre-visualisations distributed in different layouts. The results could be represented in a grid, in a stack, like a pile or as a list. Another innovative feature is that the user is able to represent the results in a scatter plot where the data is arranged by date or ranking or even price as with books, for example.

Furthermore, when the element is focused pre-visualisations are provided displaying more information to allow different degrees of resolution in the images. The user can drag and drop the elements easily building groups with the results and creating visual arrangements. However, despite the user having a folder to store all found results, the arrangements cannot be saved or used for further rankings.

Therefore, we consider that the dynamical management of the results is the best feature introduced by oSkope. The search is only available for visual content referred to; images, videos or books and probably the most interesting application is with eBay data. oSkope allows users to choose among all eBay categories displaying results by important attributes as price or duration.

News Map
newsmap.gif (216027 bytes) Another interesting application of Visual Search engines is to obtain an overview of the most important news of the day. News Map (see also issue number 160) is one of the first visual applications (with permission of the now extinct and excellent newsmaps.com) that takes advantage of information visualization to let readers obtain a fast overview in the news of the day. It visually reflects the constant change of the Google News aggregator.

A treemap visualization algorithm helps user to divide information into quickly recognizable categories like world, nation, business, technology, sports, etc. Every category is mapped using a nominal color palette and size is used in News Map to represent the relevance of every essay. Moreover, different buttons are placed in the visualization to help the user to filter information by category, date or country. 

searchme.gif (182326 bytes) Searchme is probably, the most featured visual search engine. Under the words "It lets you see what you're searching for" this new search engine uses a dynamic flash interface to manage all kind of query results. Initially, without typing any query, relevant webs on the news, sports or some others featured categories are displayed to explore them just to test the flexibility of the application. 

The same horizontal scroll recently introduced for Apple is used to navigate through different websites. These results are presented in a good resolution to provide a detailed overview of the content. Moreover, after a few seconds, a new tool is available to zoom further into the image. Furthermore, the same technology is available also for images and videos permitting the reproduction in real time.

Once a query is entered, the content is displayed with reasonable speed. Once the page is loaded, the words in the query are highlighted into the page to obtain a fast visual overview. Furthermore, other subcategories appeared next to the search field to filter the results. For instance, we can see how music, colleges & universities, news, fitness or tickets appear to redefine the query adding the intention of the user into the original search. 

Finally the user is allowed to store all results in customised subcategories called stacks. The ease of creating customised folders turns searchme into a good application to store or share any interesting content found throughout the web.

The set of Visual Search Engines introduced above gather important features that could be summarised as follows:

  • Previsualisations as visual representation of the results.

  • Rich interaction to navigate and arrange the resulting contents.

  • Use of filters to dynamically redefine the retrieved information.

  • Tags or customised categories to relate entities and to add intention to initial query.

  • Maps to organise the results through relevant attributes.

All of these characteristics introduce important improvements over common search engines providing a new way of interacting to motivate exploration and engagement. But, furthermore, they are proposing new features not covered by current search engines like tag suggestion, visual previsualisation or use of filters. These last features were available in other software applications more focused on multimedia like: YouTube, Flickr or Facebook. 

In any case it seems that Visual Search Engines should help in this quest for efficiency when searching the web.

Since 70% of web searchers use a search engine as their entry point to Internet, search engines are forced to provide access to all content, websites, images, videos , news, books, multimedia at the end of the day. Some of these engines have been around for some time now, like KartOO and Grokker. But still visual search is not part of the common internet search experience. Becoming so would be a good impulse for infomation visualisation in general. We are not as far as we think from that moment (possibly).

Sergi Mínguez is currently working as software developer at the Fundació Barcelona Media and as a professor in programming at the University Pompeu Fabra (UPF), combining his work with finishing a master's degree at the same university.

His interests in design and digital art combined with his studies in Computer Science led him to discover Information Visualization and make it his job. He has collaborated in the WebSite Exploration Tool (WET) project presented in previous articles, in TangibleSQL, application developed on reacTable to display database query results, and he is finishing his Master thesis supervised by Ricardo Baeza Yates devoted to the development of a novel interface to perform exploratory search.

Links of this issue:

http://www.searchme.com   SearchMe website
http://au.alpha.yahoo.com/   Yahoo! Alpha website
http://developer.yahoo.com/searchmonkey/   Search Monkey website
http://searchengineland.com/google-universal-search-2008-edition-13256   Article about Google Universal Search
http://research.yahoo.com/bouncer_user/17   Andrei Broder personal page
http://www.kartoo.com   KartOO website
http://www.infovis.net/printMag.php?num=97&lang=2   Issue number 97 about KartOO
http://www.infovis.net/printMag.php?num=168&lang=2   Issue number 168 about The Landscape Metaphor
http://www.ujiko.com/v2a/flash.php?langue=en   Ujiko website
http://www.infovis.net/printMag.php?num=138&lang=2   Issue number 138 about Grokker, or Visual Navigation
http://www.quintura.com/   Quintura website
http://www.oskope.com   oSkope website
http://newsmap.jp   Newsmap website
http://www.infovis.net/printMag.php?num=160&lang=2   Issue number 160 about Newsmap
http://www.barcelonamedia.org/   Barcelona Media website
http://www.upf.edu/   Universitat Pompeu Fabra website
http://www.infovis.net/printMag.php?num=193&lang=2   Issue number 189 about Eight Years and WET!
http://www.infovis.net/printMag.php?num=189&lang=2   Issue number 189 about Reactable
© Copyright InfoVis.net 2000-2014