A very clever ‘find’ engine! Clever Search Engine
From Benjy Boak: While having a lively and seriously interesting chat with Charles Knight of AltSearchEngines about the search engine world in general, the proposition of a guest post arose and I now find myself trying to provide something that might somehow succinctly summarize something of what my nine years in the search engine business has led me to conclude from the standpoint of a designer and developer.The concept of me being able to be succinct and demonstrating a decent level of brevity on such wide subject matter is something those who know me well will find highly amusing. That said we will give it a go.
The art of brevity and an ability to be succinct is actually a good place to start. It was my inability to often achieve both through a somewhat dyslexic tendency that first lead me into the search engine world. Nine years ago, as we all know, the search engine world was built on a very different landscape. There were not many search engines to choose from (certainly not enough to have a Top 100 from) and I often found I either could not locate what I was really looking for at all or I simply knew there was more appropriate material out there but I simply was not sure how to get to it. The search engines of those days, while a definite help, was far from intelligent and as far as assistance in anything that involved complex search often brought the computing term ‘dumb and terminal’ to my mind though not necessarily in the way it is conventionally used!
I thought the way forward then was to have an intelligent search engine, one that ‘knew’ and ‘understood’ what it had indexed. They say knowledge is power so it seemed logical. If you are in a library and you cannot find what you are looking for on the computer you go to a librarian. You want his knowledge to help you get what you want quicker rather than the dumb terminal with green text and a totally inflexible approach. It is not a uniquely eureka moment is it? Many have thought this for many years, yet, where is it?
I have seen many great search engines, especially from the Top 100 list at AltSearchEngines, but I have to say I don’t see my vision of the perfect intelligent search engine out there yet. (Which is good as I have spent so long on developing our own. It would be a bummer to be beaten to it!) I would like to add to that last sentence that not for one second am I arrogant enough to think Refract Speech and I are right on this, but when developing your own product you have to go with your own designs and hope you capture the imagination of the searching public. (If you are involved in a search engine project and my comment above has raised your blood pressure then it means you are striving with your baby for it to be the best thing out there. When one of us is able to come up with something that moves people away from the big G and the others in terms of decent market share, the rest of us will see what it was that was needed.
This leads me on to the following point that Charles discussed with me, and that was the small % that the majority of search engines share compared to the major search engines. Forgive me if I am misquoting you Charles but I remember a figure of 1.7% being mentioned. (The major search engines having 98.3% of the US search market.)
For my own thoughts on how to grow this figure I wonder if perhaps it is a simple case of back to basics. Without wanting to step on a certain search engine company’s previous strap line, perhaps the problem is that we still call them search engines. Indicating perhaps a focus on the searching element when the focus from a user’s perspective is a find engine!
People talk of producing the next Google killer. Why would anyone want to try and kill Google? I am a search engine designer and I hope that when my product is ready it will compete with Google, but I don’t want to kill them! It is folly to think they do not have a great product. Don’t try and take them on their own ground. Get out there and design your own totally proprietary system from the ground up. Don’t think about Google or your designs are always going to have a base in something similar. Start with a clean sheet of paper.
As Google and the others grow by adding value it is going to be more difficult to compete with them on their own terms. Work on your own terms and if your product is good enough to attract users then it will; and if some experienced professional with lots of traditional search engine development knowledge tells you you are on the wrong track, keep going! You are actually on the right track - that’s true innovation.
I genuinely feel it is about making the process of finding quicker and more relevant that is important to the user, not how it does it. We should not be so obsessed with the search process. That should be complex and back room. After all we call vehicles cars, not drive engines. Perhaps a clumsy analogy but hopefully I make a point.
There was discussion about combining resources and working collectively together. That’s a concept that I like. On the one hand I would be concerned about the ability of a large number of essentially competing businesses to agree and move quickly in such a resource sharing situation. On the other it offers perfect economies of scale and an ability to index billions of pages for less cost and making the playing field a little more level and increasing relevance for the user.
That however would be my final thought; it is all about the user, we all know that, build it right and as someone once said, “they will come”…if they don’t keep trying until one of us gets it right and takes search to the next level.
I leave this article with the phrase where I came in on, the Intelligent Search Engine development road that I started on nine years ago, and I am afraid is something I still largely feel, and that is.. “it is a fundamental flaw of all current search engine design that a person’s ability to use a computer has a real and direct effect on the quality of the information they receive. It is our mission that this stops now.” Anyone interested in seeing if our search engine lives up to what I talk of can register for our private beta at www.cleversearchengine.com.
