Comments on: To Crawl or not to Crawl? The Alts Speak Out! http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/ The most wonderful search engines you've never seen! Mon, 13 Oct 2008 17:07:04 +0000 http://wordpress.org/?v=2.2 By: Alt Search Engines » Blog Archive » Guest Author: Crawling and Indexing the Web http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6868 Alt Search Engines » Blog Archive » Guest Author: Crawling and Indexing the Web Thu, 06 Sep 2007 16:06:50 +0000 http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6868 [...] Today we are very fortunate to have another blogger’s perspective on the post that we did on alternative search engines, asking which of the ”Alts” crawled the web and why:“To crawl or not to crawl? The Alts speak out!”  [...] […] Today we are very fortunate to have another blogger’s perspective on the post that we did on alternative search engines, asking which of the ”Alts” crawled the web and why:“To crawl or not to crawl? The Alts speak out!”  […]

]]> By: Tom Dibaja http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6160 Tom Dibaja Tue, 28 Aug 2007 16:47:35 +0000 http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6160 Surprising words from Migoa.... surely they ought to know that we - Properazzi - have a proprietary crawler. Not least because we're the world's largest real estate search engine and we're both based in Barcelona! :) Plus, there's at least half-a-dozen other real estate sites in Europe with crawlers... Surprising words from Migoa…. surely they ought to know that we - Properazzi - have a proprietary crawler. Not least because we’re the world’s largest real estate search engine and we’re both based in Barcelona! :)

Plus, there’s at least half-a-dozen other real estate sites in Europe with crawlers…

]]> By: Oli http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6126 Oli Mon, 27 Aug 2007 20:33:40 +0000 http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6126 If they'd all crawl, some small sites may get the majority of their traffic from spiders ;-) Anyway, IMO it makes more sense for a small SE to just try to directly work on Alexa's indices/data repositories than crawling _yet again_ and collecting all the data all others are collecting as well (what a waste of bandwidth). With more and more SEs coming out, there must be a market for crawled data, particularly pre-extracted data (e.g. extracted text from pdf, OCRed text, speech-to-text on audio files, other "strange" document formats, etc. which is harder to collect than plain vanilla HTML). If they’d all crawl, some small sites may get the majority of their traffic from spiders ;-)

Anyway, IMO it makes more sense for a small SE to just try to directly work on Alexa’s indices/data repositories than crawling _yet again_ and collecting all the data all others are collecting as well (what a waste of bandwidth). With more and more SEs coming out, there must be a market for crawled data, particularly pre-extracted data (e.g. extracted text from pdf, OCRed text, speech-to-text on audio files, other “strange” document formats, etc. which is harder to collect than plain vanilla HTML).

]]> By: Tommy Chieng http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6103 Tommy Chieng Mon, 27 Aug 2007 05:12:55 +0000 http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6103 Thanks for visiting my blog >> http://www.crispnetworks.com Your blog looks interesting. Definitely a source for me to know more about alternate search engines Cheers. Thanks for visiting my blog >> http://www.crispnetworks.com

Your blog looks interesting. Definitely a source for me to know more about alternate search engines

Cheers.

]]> By: Yakov http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6102 Yakov Mon, 27 Aug 2007 04:17:35 +0000 http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6102 To crawl or not to crawl is more a business question that anything else. For those search engines that want to create the most long-term value for their shareholders, it is a 'must have' to be independent from the others, i.e. to crawl and index the Web. To crawl or not to crawl is more a business question that anything else. For those search engines that want to create the most long-term value for their shareholders, it is a ‘must have’ to be independent from the others, i.e. to crawl and index the Web.

]]> By: University Update - Open Source - To Crawl or not to Crawl? The Alts Speak Out! http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6097 University Update - Open Source - To Crawl or not to Crawl? The Alts Speak Out! Sun, 26 Aug 2007 23:47:26 +0000 http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6097 [...] To Crawl or not to Crawl? The Alts Speak Out! » This Summary is from an article posted at Alt Search Engines on Sunday, August 26, 2007 This [...] […] To Crawl or not to Crawl? The Alts Speak Out! » This Summary is from an article posted at Alt Search Engines on Sunday, August 26, 2007 This […]

]]> By: University Update - Yahoo - To Crawl or not to Crawl? The Alts Speak Out! http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6091 University Update - Yahoo - To Crawl or not to Crawl? The Alts Speak Out! Sun, 26 Aug 2007 21:46:47 +0000 http://altsearchengines.com/2007/08/26/to-crawl-or-not-to-crawl-the-alts-speak-out/#comment-6091 [...] To Crawl or not to Crawl? The Alts Speak Out! » This Summary is from an article posted at Alt Search Engines on Sunday, August 26, 2007 This [...] […] To Crawl or not to Crawl? The Alts Speak Out! » This Summary is from an article posted at Alt Search Engines on Sunday, August 26, 2007 This […]

]]>