Rich Skrenta points to a web article by database experts Michael Stonebraker and David Dewitt, as they exude a fusillade of hate on Google's Map / Reduce computing model.
Skrenta dubs their screed: "The sound of disruption".
Basically they mis-understand the purpose of the thing. One thing Map / Reduce is great at is processing log files. Databases aren't so hot when you have 100M things a day or more to look at.
As I wrote earlier, Google is making efforts to get college kids to learn to think in map / reduce ways. Now they are offering free access to scientific datasets in mapreduce clusters to certain universities.
The upshot is that the web requires parallel processing. No one has really extracted a lot of knowledge out of the terabytes of web usage data that flow by every day.
But the data is out there, and paradigms like map / reduce are how it's gonna be dissected. So if you want to work in the consumer web, with billions of users doing stuff every day, leaving data tracks, you should spend some time learning map / reduce.
Not to get too geeky, but to collect these in a single place, here are some key papers to read if you want to understand the google architecture:
And a bit of bonus inspiration - the story of how an New York Times blogger converted 70 years of archives (over 11 million articles) to PDF in under a day using Hadoop on Amazon EC2.
¶ 1/18/2008
So you want to work for Google?
Well, Google wants you to work for them, if you are a halfway smart computer science grad. To see this, do a search on any relevant computer science topic, and look at the ads.
Google is going out of its way to train you for free on their tools before you even get there. Check out this blog post at Google Code and the free tools posted here and at Google Code for Educators.
You can now learn how to use MapReduce and BigTable (actually Hadoop - the open source version of Google's internal parallel computing infrastructure) on your own laptop using VMware, while read the courseware that Google interns used this summer.
If you can master Hadoop, and you can solve some classic programming puzzles that Googlers like to ask in interviews, you too can work at the Googleplex.
If you are not good at interviews, perhaps you can start your own company using Hadoop on rented hosted machines, and just wait for Google to buy you...
In any case, going the step beyond by learning Hadoop won't hurt your chances, and it might prove useful in the future.
¶ 9/13/2007
