Transcript of last night’s Debate!
Our guests last night were Liz Cohen from Answers.com and Dmitri Soubbotin of SenseBot. Their search query results go beyond the “snippets” that you see in some search engines. Tonight they will discuss their individual approaches to search results and related topics. Please feel free to post any questions or comments in the comment section. Liz and Dmitri will respond if/when they are able (very busy and different time zones!).
1) Which specific characteristics separate your approach from the other search engines?
Liz: On Answers.com you can enter a keyword or question and get actual facts and information on over 4 million topics (covering health, technology, entertainment, science, etc.). Let’s say you want to learn more about search engine optimization. As an alternative to major search engines, when you click the search button on Answers.com, you’ll get information directly instead of only links. In this case the results include the definition of the concept, facts, articles and related blogs about search engine optimization while staying on the same Answers.com page.
Ironically, on other search engines, doing a search for search engine optimization will be massively manipulated by SEO firms, and full of commercial links that are, for all intents and purposes, identical-looking to the “content”. While Answers.com is ad-supported, the ads are clearly marked from the reliable content. You could say that what we’re striving for at Answers.com is to not only get you answers, but to do it as directly as possible, as the first result after you enter your search.
Dmitri: The key feature that separates SenseBot from other engines is that it delivers a summary in response to a search query instead of a collection of links to Web pages. In its current form, SenseBot takes results returned by a major Web search engine (Google or Yahoo) and summarizes them, preparing a concise digest on the topic of your query. The most significant aspects of the search results are blended together in the summary, using multi-document summarization techniques.
There are search engines that provide summaries of individual pages; or fuller extracts of relevant content from a page. The difference of SenseBot is that all top relevant pages are mixed together and analyzed as a whole repository of information about the user’s query. The summary itself becomes the main result of the search. This approach is unique to SenseBot and is our intellectual property.
In general, any type of search where the user is trying to understand a concept or a particular area of knowledge; or get a digest on a person or event or activity; or quickly identify the best source of information on the topic, will benefit greatly from SenseBot. (Our Samples page contains many types of queries to give you an idea of the possibilities). The more diverse are the views on the topic, the higher is the benefit of seeing their summary rather than studying each view separately.
2) Fuller answers vs. snippets: what are the pros and cons?
Liz: Snippets are great if you’re looking to search - if you’re looking to take that next step from getting a bunch of choices and guessing which will have the best answer. A site providing snippets provides breadth but makes you work past your initial search entry. It’s what we like to call the “Click-Back-Click-Back Syndrome.” Receiving a full answer upon entering your search term makes the search process only one step long. Answers.com provides a full, all-in-one page so that after you click, you get your answer and more – with reliable sources, at that!
Dmitri: Snippets work relatively well for simple, usually single-term queries, and help the user to identify which pages to navigate to. Quite often, though, a snippet would send you to a page based on its title, only to be disappointed by a shallow or irrelevant content.
Fuller answers may serve two purposes (which are not mutually exclusive): to give the user a better idea of the result page (i.e. whether it is really related to what he is trying to find); and to provide an answer to the query without the need for clicking through the result pages. The former purpose is usually addressed by showing more content, in the form of complete sentences, and more relevant to the query. Some engines also allow to view a summary of an individual page.
The latter purpose is addressed by perceiving the user’s search query more as a question than a straight search directive; and providing an answer to the question. For example, when the user searches for “net neutrality”, a “snippets” search engine would just show the links to pages that discuss the concept. The problem is that the net neutrality subject is not easy to grasp, and most pages about it are highly biased. So it would take time and effort to go through a lot of pages before you can form your own opinion on the subject. Unlike that, a “fuller answers” system would attempt to provide a definition of net neutrality and related issues right on the search results page. That’s what SenseBot does, building a summary of definitions and different views on the topic out of the top relevant pages. The pages that are not heavy on relevant content will not even make it into the summary - SenseBot often drops about half of the sources as having little value for the query.
3) Quality of results: what do you do to ensure it and where are the difficulties?
Liz: Answers.com prides itself on its over 180 quality reference titles, which mostly include licensed content from reliable sources such as Britannica, Barron’s AMG, and Houghton Mifflin. Along with traditional encyclopedias, we’re also proud to provide answers that include user-contributed information, such as Wikipedia, blogs and our very own Q&A site, WikiAnswers. Of course, we can’t always guarantee absolute quality when it comes to user-contributed sources, but we do feel that the wiki community as a whole is interested in maintaining and sharing reliable content.
Dmitri: We have seen very encouraging results from SenseBot searches, some of which are posted on the Samples page. We employ sophisticated methods of multi-document summarization and are backed by years of commercial application of Natural Language Processing technologies. Still, it would be unrealistic to expect SenseBot to match a summary prepared by a human expert. It is unlikely that you will be able to take a SenseBot summary and hand it to your boss, presenting it as a result of your hard work (but if you succeed, please let us know).
However, we believe that even less than perfect summaries may be of great help for Internet users, and would be preferable to numerous links to Web pages. We want to enable the users to grasp 80% of the idea in a few seconds rather than spending a long time sifting through often irrelevant pages.
4) How do you position your engine vis-à-vis Google and other major players? How do you view your search technology with regard to other modern search technologies - as a competition or potentially integration?
Liz: We view Google, Yahoo and Ask as friends, not competitors. If you think about it we’re essentially a content suite with a search interface. But that means that in a sense we “fuel” these search engine; we give them great stuff to spider and present to users. If you search on Google, you’ll notice on the top right corner there is a definition link for what you searched. That link points to Answers.com, where you get direct definitions. We are also big fans of Wikipedia and see their community-based content as a great complement to our more traditional material. Answers.com includes Wikipedia as one of many reference sources because we value its vast, up-to-date coverage of millions of topics.
Dmitri: I am actually envisioning in a couple of years every reputable search engine to have a SenseBot-like feature for a summary – in addition, of course, to conventional search features. So I am viewing other search engines as potential partners rather than competitors.
We also believe that our summaries can get even more relevant and readable if SenseBot is integrated with a specialized search engine targeting a particular subject field, or a Web directory where users navigate to a specific area, thus narrowing down the subject of their query. The advent of Semantic Web and tools like Yahoo! Mindset would also allow SenseBot to get a more accurate identification of the type of Web pages, thus improving the quality of generated summaries.
5) How might your project benefit by association with another alt engine - or are you content to “go it alone?”
Liz: Collaboration can be fun for the user and valuable to the search engine! If there’s a feature that enhances the user experience and it makes business sense as well, we’d be delighted to consider it. Obviously, it can’t conflict with our basis business propositions or values, but if it’s complementary, we’re interested. Answers.com holds many partnerships (such as with news sites, like NYTimes.com and CBSNews.com) so we’re open to creative prospects.
Dmitri: Some alts have very interesting technologies that may augment what SenseBot is doing, mainly in the areas of understanding the query and page content. Another potential area is specialized search engines – medical, legal, etc. – where the user intent can be well understood and the summary get even more coherent and focused on the user’s need. We are open to cooperation in these and other areas.
Charles: I encourage our readers to try Answers.com and SenseBot.net for themselves and see first hand what you have read about tonight. The comment section is now open!
7 Responses to “Transcript of last night’s Debate!”
test
June 27th, 2007 at 2:11 am Answers.com is all in one. It has a full answer and if you’re looking for a snippet you can use their 1-click Answers or AnswerTips for bloggers!
June 27th, 2007 at 7:26 am Thanks Charles and Dmitri for this cool opportunity. If anyone has anymore questions about Answers.com, feel free to get in touch with me: lizc at answers.com.Liz
Chief BlogWatcher
Answers.com
June 27th, 2007 at 3:03 pm I would answer by saying, sure, Answers.com doesn’t have spam in their results for SEO. That’s because they always just hit the usual suspects - Columbia Encyclopedia, Wikipedia. Factbites on the other hand is a genuine search engine that determines what sites you see based on what you search for. And unlike Google, even if the page is commercial, it will extract the most descriptive and accurate content it can find from the page. See http://www.factbites.com/topics/Search-Engine-Optimizers Factbites is a means of finding new authoritative sources while Answers is a means of looking up the most standard ones.Of course, Answers has a use. But if you want to find new sources, sources that are particularly good at the topic you’re after, not just jack-of-all-trades encyclopedias, then Factbites is for you.
June 28th, 2007 at 6:17 am Two things I’d like to mention:1. - When you click on the Factbites link above, it seems you are getting results with the words search engine optimizers in them. When looking it up on Answers.com, you’re getting the actual definition and more (http://www.answers.com/search%20engine%20optimizer)- which is a big difference between the two right there.
2. - Based on Luke’s point about content, because I didn’t specify this enough in the above debate.
The answers and topic pages you get on Answers.com are comprised from over 180 reference titles. A number of our content sources aren’t the ‘typical’ big-names; they’re smaller sources with deeper knowledge of a certain subject.
These sources include licensed published works from all corners of the ’search’ spectrum: guidebooks for gardening and cooking; encyclopedias covering small business, sports, zoology and literature; dictionaries of medicine, technology, music and poker; glossaries on Harry Potter, wine and aromatherapy. We include translations, pronunciations and even citations information (for all the students out there).
That’s just to name a very few. I must mention the dozens of original content entries our editorial staff provides to our answers, and the answers provided by users through our Q&A function, WikiAnswers (www.wikianswers.com).
We’re serious when we say Answers.com covers over 4 million topics covering ranges of categories under the sun (oh - we have astronomy lexicons as well).
And the very best part? Everything on Answers.com is free, always. That’s our contribution to alternative search.
Again, feel free to contact me with any questions (lizc at answers.com).
Liz
Chief BlogWatcher
Answers.com
June 28th, 2007 at 6:54 am […] According to Dmitri Soubbotin of SenseBot all top relevant pages are mixed together and analyzed as a whole repository of information about the user’s query. […]
June 28th, 2007 at 12:18 pm I think there is a commonality between Answers, factbites, and SenseBot, in the fact that we are all trying to serve content to the user as opposed to just serving links. There are also differences - I think that Answers.com’s primary goal is to provide the users with reliable content related to the query; whereas factbites and SenseBot are primarily analyzing the content of web pages and trying to bring the most relevant pieces back to the users. The key difference of SenseBot, the “big idea” if you will, is that we offer a new concept of a Web search result - an overall summary on the topic of a query, as opposed to individual web pages. The summary itself becomes the main result, which we believe will satisfy a great percentage of web searches.
July 10th, 2007 at 10:34 am […] summarization techniques to present a summary that becomes the main link for the result. In this Q/A session featured on AltSearchEngines, Dmitri Soubbotin of SenseBot details on several of SenseBot’s […]