Last night’s Debate: The Mighty Metas!
Last night’s Debate: The Mighty Metas!
Every Tuesday on AltSearchEngines, CEOs from prominent Alternative Search Engines discuss their area of search. After the debate, readers are encouraged to post questions or comments for the participants. You can still post comments today.
Last night, the topic was Meta search with search engines CrossEngine , (Juan Sosa) GoshMe , (Rafael Costa) and Srchr (Matt). Here’s the recap of the debate:
Question #1 What is a Meta Search Engine? Why should someone use a Meta search engine instead of one of the major search engines like Google, Yahoo!, or MSN?
Srchr:
I think the best definition of a meta-search engine is simply a search engine that returns the results of other search engines. I would say that the results should come back unfiltered and unbiased, i.e. if there is an algorithm applied to the results then it’s not really a meta-search engine. As for why one should use a meta-search as opposed to a major, think wisdom of crowds: more results from a diverse set of independent sources aggregated in the right way will yield the best answer. For Srchr specifically, the way we think of it is not “why should someone use meta search instead of G/Y/M”, it’s “someone should use G/Y/M all at once.”
GoshMe:
Basically a Meta-Search Engine is a search-tool that makes use of search engines in order to reach broader and more relevant results to a keyword-targeted search. Some may think that those billion results from Google are enough. But the fact is that no Search Engine is even close of having all the information available on the Internet. If you think about meta-searching general engines such as Google, Yahoo and MSN, you’ll be reaching more of the surface web (information that can be found via link-trough crawling). If you’re meta-searching vertical databases you can reach not only more of the surface web, but also and specially information of the so called Deep Web, information that can’t be seen by General Engines –extremely higher quality information thought to be several times bigger than the Surface Web. So meta-searching follows the wise practice of using more then one single source. Moreover, Meta-Search Engines can use comparative algorithms so they can merge the different ranks of each search engine into a more relevant single page of results.
CrossEngine:
1. Because Folk wisdom asserts that more heads are better than one.
2. Little overlap occurs among the top results of regular engines. This illustrative Screencast by Gregg R. Notes from SearchEngineShowdown clearly demonstrates this situation.
Link Editor’s note: Any technical glitches are mine, and mine alone!
3. In specific fields such as Video and Images, people is uploading their clips or pictures to different services (not only Youtube or Flickr), but they don’t upload the same content to more than one or two services (why would they?). Directly querying these different providers is the only way (as of now) to discover unique multimedia content; for this case, is very clear the more sources you can query, the better.
Question #2 Some Meta Search Engines search 2-4 major engines; some search dozens of sources, and some hundreds or thousands! What’s the guiding rationale?
Is it simply to have more sources than anybody else?
Srchr:
The guiding rationally should be to give the user what they want, and this is where the concept of “search aggregation” really starts to make sense. The vision behind Srchr was to pioneer the concept of search aggregation, what essentially amounts to an RSS reader for OpenSearch (parameterized RSS) queries. If our user wants 1, 10, 100 sources, they can have that, and our interface supports that level of intuitive customization as simply as any traditional RSS reader.
GoshMe:
It depends on the purpose of the Meta-Engine. In GoshMe’s case the aim is to gather more relevant information, and for that we need as much sources as possible. Some specialists believe that Google has not even 10% of the information that is available online. The only way of getting close to reach all information of both the Surface and Deep Web is by putting all the Major Engines and specially the Vertical Search Engines (more than half a million databases) together. The point here is not flooding user’s head with tens of billions of results, but giving just one – the best result. We believe that we can only be sure we will be providing the best results on the Internet if we’re able to retrieve the Internet itself. Seems like an impossible task.But meta-searching together with targeted Information Retrieval algorithms can make it reality. We’re now meta-searching about 2,000 sources and we’re really close of reaching full scalability. We expect to reach 100,000 sources in 2008.
CrossEngine:
The same way search engines strive to index as many web pages as they can, the more information sources meta search engines can provide access to, the better, which is a challenging endeavor, given meta search services should be able to guarantee trustfulness of their search sources and provide an uncluttered UI.
Question #3 There are many ways to present the results of a Meta search query. Is there a presentation of results that you think is best, and, why do you think that?
Srchr:
As a search aggregator, our mandate is to be true to the source content and so we try to display it as simply as possible making it clear where the results came from while still allowing the user to 1) compare results from different sources and 2) drill down for more info without leaving the page. We think this is the right way to do things because our “special sauce” is not in the algorithm that searches the web, it’s in the platform that allows users to manage their own experience. Our long-term goal is to add value by helping our users create the best experience; we’re not in the business of improving search algorithms by interleaving results from different sources. We also believe in social features and have built them into the application.
GoshMe:
We believe in a new perspective on meta-search: instead of presenting web-documents as results, as a conventional search engine do, GoshMe presents a list of databases (Vertical and also General Search Engines) relevant to the query, i.e., instead of ranking web-documents we rank Search Engines. We do that for many reasons. I’ll list a few. 1) The selection of the source is a very important step in filtering the search; 2) By not by-passing engines’pages of results this model allows a fairly use of their services; 3) Every Search Engine, particularly verticals, provide a unique search experience.Many bring special features, such as targeted layout, filters, section of reviews, and last but definitely not the least, every vertical is a whole specialized repertory, most times as a consequence of a cooperative environment with professionals and experts.
CrossEngine:
Our approach at CrossEngine.com is to provide means for users to find useful web content with as less clicks as possible, in an uncluttered page layout. Giving personalization and display options is also a good measure.
Results mashup is also a valid approach (as in srchr), I just think they need to provide a more uncluttered UI.
Question #4 Meta Search is broad, running the query “horizontally” across many sources. Vertical Search is deep, running the query “vertically” down a narrow index.
Is this an “either or” choice for the user? Is one better? Can they be combined?
Srchr:
We think that search should always include results from diverse content, and we’ve laid out our site to facilitate that as much as possible. As a search aggregator, our users are searching through their own pre-qualified trusted sources (in addition to the default set we recommend), and this can lead to serendipitous results that other metas or even Google can’t provide. For example, say I wanted to always see what Charles has to say whenever I’m doing a web search…I can see Google, Yahoo, Google Blogs, and Read/WriteWeb results all on one page (http://www.srchr.com/?PageID=189496, the perfect web search :). For shopping queries, maybe you’d want to add an image search to the mix, or even a social, blog or video search to get other perspectives (http://www.srchr.com/?PageID=189499). Suggesting what sources to look at for a given query (or deciphering intent as some might say) is a very interesting problem and one where we’d like to add value in the future.
GoshMe:
For sure they can. Meta-search together with verticalization is an extremely powerful combination. In fact that’s what GoshMe is about. No engine is the best to everything and for every single query there is at least one vertical engine, among more than 500,000 possibilities, that is highly appropriate and would make the search experience unique. The problem is how users can know what are the best sources to every single search. So GoshMe makes use of some meta-search techniques together with exclusive IR algorithms in order to answer this question.
CrossEngine:
They are combined already, in industry-specific topics like health and travel, specialized search tools are querying thousands of documents at different levels, across different domains.
Question #5 Finally, the “Top 100″ list of alternative search engines has always had numerous Meta Search engines - too many according to some. Are there ways that Meta search engines can work together in order to create even better search engines?
Srchr:
Sharing expertise is definitely a good thing… for instance, relating to my last point above, suggesting sources is an area where GoshMe has done some work. If they were to come up with an API that would allow us to get suggested sources for a given query and provide those to our users, that’s something we’d take a look at. Otherwise, I think that all alternative search engines can help each other by adopting and promoting search standards like OpenSearch.
CrossEngine:
There are search tools clearly complementary, some others not at all, they’re just competition. As per CrossEngine.com, we’re willing to work together with complementary tools.
test
