To get started on this month’s Top 100 list, I started with the April Top 100 and added every search engine that we covered this past month - more than 1 per day - resulting in 142 alternative search engines! How to make it 100? By eliminating 42 of them. It’s not an easy job, but you can see the results below in the attached files.

To see the Top 100 according to your votes, check out The Search Race!

Allth.at
Answers.com
Audiobaba
Bessed
blinkx
Blogdigger
BlogDimension
Bookmach
BooRah
Buddy Fetch
ChaCha
ClipBlast!
Clueray
Cluuz
Cognition
Collarity
Congoo
Dabble
Deligio
Ecocho.com.au
eeggi
Eurekster
EveryZing
Exalead
Factbites
Faroo
FindSounds
FreeMobile411
GameSkoot
GenieKnows (local)
Gigablast
GoLexa
googlUpon
GoPubMed
GoshMe
Grayboxx
Guitarati
Hakia
Healthline
HealthPricer
Helia
HowDoYa
Icerocket
iMedix
indeed
Intelways
iSeek
itzbig
ixquick
Jixperts
Jobfox
JobisJob
KartOO
KoolTorch
Kosmix
Krugle
Lexxe
Liveplasma
Mahalo
Matchpoint
Medioh.tv
Megaglobe
Metaverse Ink
Midomi
Midomi mobile
Mojeek
Molu
Mozbot
MP3Realm
Mugr
Netvue
Netzwellan
Nofoodhere
Noza
Nsyght
Omgili
Onkosh
OrganizedWisdom
PageLens
PeekYou
Picsearch
Pixsy
Pluggd
PropertyOwl
Quick.as
Quintura
Quintura Kids
RedZee
Retrevo
RSS Micro
Search the Tail
Search.com
Searchme
SearchMedica
Searchpickr
SearchRadar
Seeqpod
SenseBot
ShoppingVale
Sidekiq
Silobreaker
SkreemR
Slifter
Snipp.tv
Snooth
SomethingSimpler
Sphere
Spock
Sproose
Sputtr
Streamzy
Summize Twitter Search
Surf Canyon
Swamii
Swotti
Tayait
TheAdDatabase
TheFind.com
TheFindGreen
Travelgrove
True Knowledge
Turboscout
TWERQ
Twing
Twingly
UpTake
VeoSearch
Vinquire
WASALive
WhatsOpen.com
WhoisLike.it
Wikiwix
Wink
Wundrbar
Yelp
Youlicit
Yureekster
ZabaSearch
Zoomf
ZoomProspector
Zuula

The Top 100 Alternative Search Engines, May 2008 (.pdf)

The Top 100 Alternative Search Engines, May 2008 (.xls)

The other part of my job is to select one (1) search engine from this list of 100 search engines to be the Search Engine of the Month for May 2008.

(In February it was ChaCha; in March it was EveryZing; and in April it was Sputtr.)

And for May, the Search Engine of the Month is…FAROO!

During the course of the week in San Francisco, I had the opportunity to hear CEO Małgorzata (Gosia) Garbe describe the unique approach that FAROO brings to Search.

I have joined the community by downloading FAROO on my PC - want to join us?

Idee (in English below)
Die Erfassung des gesamten, exponentiell wachsenden Web auf einem zentralen System ist auf Dauer nicht erfolgversprechend. Trotz ständig höherer Investition im Hardwarebereich wird die Vollständigkeit und Aktualität der so realisierten Suchindizes zurückgehen.

Beim Konzept der verteilten Peer-to-Peer-Suchmaschine stellen die Anwender, die für das Wachstum der Inhalte des Internets sorgen, auch deren Auffindbarkeit sicher. Damit werden die Suchergebnisse aktueller, umfassender und kostengünstiger realisiert.

Der Suchende ist selbst Teil der Suchmaschine. Die Architektur ist dezentral wie das Internet selbst. Es steht keine zentrale Institution mehr zwischen Informationsanbieter und Suchendem.

Mit dem Wachstum der Informationen steigen die Anforderungen an die Relevanz der Suchergebnisse. Diese wird durch eine vollautomatische Bewertung von Inhalten durch die Nutzer der Peer-to-Peer-Suchmaschine sichergestellt.

Dezentrale Architektur

* P2P-Web-Suchmaschine
* Veteilter Index
* Verteilter Crawler
* Verteiltes Ranking
* Skaliert mit dem Wachstum des Internet

Schutz der Privatsphäre

* Privacy durch Verschlüsselung
* Zensur-Resistenz
* gegen Wissens-Monopole

überlegene Ergebnisqualität

* Ranking auf Basis des Nutzerverhaltens aller User
* Personalisierung (Nutzungsprofil basierte Relevanz)
* Effektiver Spamfilter
* Integration von Web-Suche und Desktop-Suche

Geringste Kosten

* keine Hardware
* keine Betriebskosten

The Idea
The collection of the whole, exponentially growing Web on a single central system is not successful on the long run. Despite constantly increasing investments in hardware the completeness and freshness of the search index will steadily decrease. With the concept of a distributed peer-to-peer search engine the same users, who provide the content of the internet assure also, that it is findable too. Thus the search results may be provided more up-to-date, more comprehensive and more cost-efficient.

The searcher becomes a part of FAROO. The architecture is decentralized like the Internet itself. There stands no more central institution between the information source and the searcher. With the tremendous growth of information the relevance of search results becomes more important. This is assured by an fully automatic ranking of web page content by the users of the peer-to-peer search engine.

Decentralized architecture

* Peer-to-peer web search engine
* distributed Index
* distributed Crawler
* distributed Ranking
* Scales with the growth of the internet

Privacy protection

* Privacy protection and censor resistance through encryption of search queries and results
* No search logs
* Alternative to information monopoly

Superior result quality

* Ranking based on anonymized user behavior
(democratic or social ranking)
* Personalization
(Ranking based on detection of personal preferences)
* Effective spam filter
* Integration of web search and desktop search

Lowest costs

* no hardware costs
* no operational costs

Distributed crawling
When an user opens a page with the browser, it will be automatically inserted into the distributed index of the p2p network. he additional network load and the site submission of a traditional crawler is omitted. Assuming a wide spread of FAROO this enables an almost complete index, updated in real time.

Distributed index
FAROO requires no central index server. Every FAROO user is part of a distributed index.
Every FAROO user is responsible for certain words. If an other FAROO user visits a web page, which contains one of these words, he stores the URL of the web page to the list of the assigned user. There are always multiple users assigned to one word, in order to preserve the information, when a user leaves the network. Then an other FAROO-user takes his place.

Distributed ranking: PeerRank
PeerRank is a newly developed ranking algorithm. FAROO takes the user behavior when viewing a page into consideration for ranking. This is done automatically, without requiring a manual rating from the user. Thus for the first time the user as audience of a page also decides about its ranking. With previous ranking methods only the site owners determined the page rank, which were based on the linking of web pages among each other

Putting the ranking on a much wider base leads to a democratized, user centric ranking, while resistant against manipulation. For the first time the users decide themselves, which results are most important. In the respect this is a kind of social search, but omitting the existing drawbacks. Not only people known to the user are involved into the ranking, but all FAROO users. No registration is required, and no user profile or search history is stored at a central server.

Personalized ranking: PersonalRank
The personalized ranking of pages is based upon the areas of interest of the searching user. In order to determine the user focus are besides the visited web pages also the content of local documents analyzed.

If somebody is looking for a car and he has a pdf-brochure from VW on his desktop, then the car results, in which also the term VW occurs are higher ranked. If somebody has a lot of addresses from New York City within its documents and he is looking for a pizzeria, then the pizzerias from New York City are ranked higher. This analysis is done at the users computer. No personal information is transferred outside from his computer. Besides this the user may at anytime disable analysis and personalization. At present no ranking is activated.

Technology Break Through
Until now no peer-to-peer web search didn’t exist, that could be a serious alternative to the current legacy search engines. The idea was investigated since a couple of years mainly in the academic area and sometimes contradictory findings about the feasibility were obtained as well as first prototypes were built. The necessary combination of scaling of users, documents and words during indexation, ranking and search, as well as Boolean queries, speed, ranking, and completeness of index and results was neither as theoretic concept nor as functional implementation sufficiently solved.

By combining radical new distributed index structures, indexation and search techniques, the usage of recent developments from various fields and their enhancement in many respects we were the first to succeed in developing an integrated conception and implementing it in a working distributed web search engine (p2p web search engine) for the mass market.

Effective Search

* Supports Boolean queries: AND PHRASE NOT NEAR.
* Guarantees top-1000 results also for Boolean queries.
* Guarantees a response time comparable to common search engines.
* No intersection of posting lists required.
* Complete results also for rare combinations of frequent words.
* Incremental search (result 1..10, 11..20).
* Domain collapsing: only one result per domain / further results of this Domain.

Peer Ranking and Personalization

* PeerRank: Ranking by a fully automated, anonymized behavior analysis of all users across all visited web pages.
* PersonalRank:fully automated user preference based ranking: implicit social search without user action or loss of privacy.

Instant and Incremental Crawling

* Instant indexing, updating and ranking: No crawler required.
* Incremental indexing, updating and ranking: Deletion and updating are supported.
* Sparse and traffic saving indexing, updating and ranking.

Loadbalancing and Churn

* Instant load balancing for sudden query peaks.
* Persistent information despite churn.

Scaling

* Organic scaling: Performance and load grows equally with number of users.
* Scales for a virtually unlimited number of users, documents, words, updates and queries.
* Scales for frequent words and frequent queries.

Compression and Privacy

* Highly effective distributed data base compression: similar to jpg, divx,mp3 or mpeg only data relevant to the user are stored.
* Compressed index and transfer.
* Encrypted index and encrypted queries for full search privacy.
* Standard http-protocol, with nat and firewall traversal.

Research & Development

* 5 years research & development.
* Using latest academic publications in p2p overlay protocols, synchronization, cryptography, distributed indexes, index compression, distributed queries, stemming, ranking (1500+ papers)
* 100+ own inventions.

This entry was posted on Thursday, May 1st, 2008 at 10:57 pm and is filed under Top 100. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Sphere: Related Content