Om Shanti Shanti…

Charles Knight, Nitin Karandikar and I (Kaila Colbin of VortexDNA) have been at it again. It all started when someone (and I won’t identify who, other than to say his name rhymes with Shmarles Shmight) laid down the gauntlet and said algorithms are essentially as useful as tits on a bull. His bold position sparked a flurry of emails amongst the three of us, culminating in Nitin’s public defence of his own viewpoint on this blog two weeks ago. Charles responded last week, and now it’s my job to get them to play nicely with each other.

Nitin says algorithm. Charles says interface. In this piece, I’ll take a look at how the two intersect, and give you my take on which has the possibility of being a true Google-killer.

We’ve set this up as a three-part debate, but in reality this is just the beginning. Charles, Nitin and I have three distinct opinions; no doubt each of you has your own, and we’d really like to hear it. Please have a read, and then let us know in the comments where you agree with us, where you disagree with us, and where you suspect we’ve all been smoking a bit of wacky baccy.

Let’s start with the algorithm.

Nitin compares Alexa rankings and Google Trends for Powerset, Mahalo, Quintura and Hakia to provide a bit of context to his proclamation that algos rule.

Despite the fact that Powerset hasn’t even launched, they have the second highest traffic in the group of four. Nitin could argue that this demonstrates the strength of a good algorithm, but, again, they still haven’t launched! So nobody visiting their site has any idea of how good their algorithm is. What this demonstrates is the power of good PR.

Mahalo has the most traffic, which doesn’t bode well for Nitin’s argument. An algorithm is an automatic rules-based system for processing information (I looked it up just now to make sure I’m not making more of a fool of myself than usual or necessary). This definition means that Mahalo’s strategy, which uses humans to create SERPs, isn’t relying on an algorithm.

Nitin has a two-fold rationale for championing the algorithm:

If a search engine does not provide a high degree of relevance in results, then even the greatest UI in the world will not help capture market share; conversely, a bare-bones functional UI on top of very good search results would, in my opinion, make a big difference.

An improved algorithm - be it semantic parsing of the query or index, or social-based approaches like Ant Colony Optimization, or something entirely new - is a feature that has to be “baked-in” into the architecture; I imagine it is truly difficult to add it in later without a big impact on existing users (e.g. the process of tweaking Google’s search algorithms seems to be excruciatingly complex). On the other hand, a superior UI on top can be added in later, as Quintura has proved by leveraging search results from Google and other major search engines.

Nitin compares ‘the greatest UI in the world’ with ‘a bare-bones functional UI’. While it may or may not be true that a good algorithm can compensate for a lousy interface, a bare-bones functional UI is not necessarily a lousy interface, as Google has so deftly proved. However, that’s not to say that it isn’t possible to improve on their relatively blank slate.

His second point is that a good algorithm is its own barrier to competition, so a search engine with a good algorithm is more likely to gain market advantage without being copied—an excellent point, and a vital ingredient for any startup looking to displace the currently entrenched establishment. I would posit, though, that an appropriately and effectively developed UI could also be difficult to replicate. Read on; my reasons for thinking this way will all be revealed.

And now for the interface…

At this stage, Charles is probably thinking that I’m on his side or something. Fear not, dear readers; he is soon to be disillusioned.

He will be disillusioned because I take exception to something he says right at the beginning of his piece:

We are all very familiar with the two pillars of this dichotomy.  Nitin calls it Beauty or Brains. (Tricky, since Kaila has both.)  It’s also been called steak or sizzle.  Cake or the icing on the cake. 

Obviously, I don’t mind that he calls me beautiful and brainy—thanks, Charles! J It’s the rest of the paragraph that I have trouble with. Quite frankly, if I smelled sizzle that didn’t deliver steak, I’d be mightily disappointed. And icing with no cake would just make me nauseous. If you want superficiality that is unsupported by substance, look no further than my beloved Miss Teen South Carolina.

Charles and Nitin both fall for the trap of describing the interface as something completely distinct from the algorithm: a skin, a coat of paint, a false mustache and a pair of glasses. While some people may prefer a blue Google to a red one, a coat of paint will never be more than a band-aid. It will never be a Google-killer.

The Genius of the AND

The discussion thus far succumbs to what Jim Collins and Jerry Porras call the ‘Tyranny of the OR’ in the book Built to Last. They suggest that there is a better way:

Instead of being oppressed by the “Tyranny of the OR,” highly visionary companies liberate themselves with the “Genius of the AND”—the ability to embrace both extremes of a number of dimensions at the same time. Instead of choosing between A OR B, they figure out a way to have both A AND B.

…We’re not talking about mere balance here. “Balance” implies going to the midpoint, fifty-fifty, half and half. A visionary company doesn’t seek balance between short-term and long-term, for example. It seeks to do very well in the short-term and very well in the long-term. A visionary company doesn’t simply balance between idealism and profitability; it seeks to be highly idealistic and highly profitable. A visionary company doesn’t simply balance between preserving a tightly held core ideology and stimulating vigorous change and movement; it does both to an extreme. In short, a highly visionary company doesn’t want to blend yin and yang into a gray, indistinguishable circle that is neither highly yin nor highly yang; it aims to be distinctly yin and distinctly yang—both at the same time, all the time.

So how would the ‘Genius of the AND’ play out for a search engine? Charles points out the challenge for a small company:

…a start up company has limited resources, and they have decide which side of this fork in the road to take.  Just imagine that Nitin works for me.  He can only work on a new algorithm or a new UI, he can’t do both.

Imagine that you were trying to build a house, and someone told you, “You can invent a new way of making bricks, or you can focus on choosing the curtains. Which one do you want to do?” You’re pretty likely to reject both options and work on the structure, aren’t you?

Yes, it is the interface, but it’s not the pretty, sit-on-top interface that Charles describes; it’s the interface that dictates how a human being can dive into the richness of the available data, interact with it, and extract meaning from it.

Quintura is a good example of what I’m talking about. Their interface has a direct and dramatic impact on how you search and what results you obtain. The user navigates differently, searches differently, thinksdifferently with Quintura’s interface than with Google’s.
It’s the same concept as a hardware abstraction layer, which is the bit that sits between your computer and the software that runs on it. HALs are not pretty fluff. HALs are the reason your apps can take advantage of your quad-core processor and 4GB of RAM.

An interface that is integrated with the algorithm empowers users to access the full depth of the available results. The Google-killer will be an engine that leverages both the richness of the back end and the endless creativity of the people who use it.

That is quite definitely enough of my opinion! Especially since many of you reading this have first-hand experience building search engines, and have profound insight into what tactic has the best potential for success. So let’s hear from you—what do you think?

This entry was posted on Thursday, September 27th, 2007 at 6:01 am and is filed under Guest Authors. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Sphere: Related Content