Dear MSN and Yahoo
So, you want to beat Google, huh?
Well, you have your work cut out for you. It's not impossible to get me to switch, though. I've switched search engines twice before.
It might look hopeless right now. I do admit Google has a special place in my heart. When I reinstalled Windows XP on my main system yesterday, what was the first third-party app I loaded? That's right. The Google toolbar. It +is+ that important to my life.
The other day I started counting how many times in a day I use Google. I lost count at more than 50.
OK, some history of search engines and the Web (my wife says I need to make my point in fewer words, so I cut out a whole bunch of engines that simply didn't matter).
First, there was Yahoo. It was a directory kept by people. It was awesome. I went there and found cool stuff. One problem. Soon I started publishing. Soon I had the number one NetMeeting Web site anywhere.
One problem: Yahoo wouldn't list it. They still don't to this day. I was getting 300,000 visits a month at one point. Yet Yahoo never listed it (even now it's not listed). I'll never forgive them of that.
So, I switched to Alta Vista. This engine was a computer. No humans. A little piece of software visited the Web and kept updating. This was freaking awesome. My NetMeeting site got listed within a month of it starting up. (And is still listed there today — it's the #3 link if you search for Netmeeting).
But, AltaVista had a problem. They were easily spammed. Hey, search on NetMeeting and you'll see a sex site even today. Why? Because their algorithm was flawed. It tried to figure out the importance of a page by looking at the content of that page. That simply doesn't work, since webmasters can figure out tricks to “fool” the engine.
Along came Google. All the “biggies” counted them out. (Remember that MSN and Yahoo!) They had a new approach. They built a new algorithm: one that looked at status on the web.
Why do I say “human?” because that's what they built. A search engine that values individuals as much as corporations.
Hey, search Google for “NEC Tablet” and you'll see my site, along with a couple of other weblogs, along with NEC's official site.
Google links my NetMeeting site very highly, even today.
So, here's my advice to MSN and Yahoo:
1) Build an algorithm that reflects human relationships and status.
2) Do something better than Google. Make it 10 times faster, for instance. Have a WAY better interface. Or, add community into the mix (read on for more about that).
3) Pay attention to folks who publish on the web. They are the ones who'll decide whether or not your search engine is awesome or not. Look at my experiences above. I'd never recommend Yahoo to my friends until my sites get high placement — if they deserve it. 300,000 visitors? That is deserving of at least a link on the top 20 pages.
4) Pay attention to advanced users. They are the ones who are like my dad. He uses a lot of quotes and plus signs to do very scientific searches. The search audience is getting a lot more sophisticated about using search engines. My dad took two years to switch to Google after I had switched. Why? Because he learned all of Alta Vista's advanced tricks and he learned to trust Alta Vista.
So, what would be my plan if I was in charge of MSN or Yahoo?
1) Do the fundamentals as good as Google. Spider as many pages. Display results as fast, or faster. Have the same quality or better. Provably so (all this alone is VERY hard to do).
2) Make a better user interface. Google's “Search Toolbar” is awesome. If you aren't using it yet, you are really missing out. But, can Yahoo and MSN do better? Of course they can!
3) Build in humans. Hey Yahoo is great at this. Yahoo's main directories are all done by humans. But, they didn't have a good spider and a good algorithm, so they lost. Why not mix the two? Of course, I think Google is already thinking of this, which is my theory of why Google bought Blogger.
How about when you do a search, along the top you can display real humans' view of that search term? For instance, why not have Dave Winer and Don Box do their own versions of the search term “SOAP?” You know, if you do a good job of getting humans involved, I'd switch search engines.
Well, until you offer me something radical, I'm not likely to switch.
PS: do an RSS search engine. That's all about humans.