Greg Linden puts some estimates up for how many servers (and how much memory) Google has in their world-wide supercomputer cluster. Basically he thinks they have around 450,000 servers and perhaps as much as 4 Petabytes of total RAM.

I think that's right on for the server number and high on the memory - my estimate would be 500k servers, and just under 2 petabytes.

Greg also writes that Google is adding 100k servers a quarter now - but I think that's a typo - because in an earlier post he estimates it would be about 25k new servers per *quarter* - and even that's probably high. I don't think 100k/quarter is realistic.

People were wondering if all of Google's $300M+ quarterly capital expenditures (CAPEX) were going into more servers. But now we know now from Henry Blodget, that a very large chunk of the CAPEX is accounted for in Real Estate, not hardware.

So I think Google might be at 500k total servers now, and yet they are still in a significant server crunch overall - they constantly need to reclaim & consolidate boxes just to keep up with their own demands.

I base the total server number on extrapolation from when they tacitly acknowledged that they had over 100k servers, and on the number of world wide datacenters people know about. The fact that Greg's estimate is in the same ballpark of what I thought is also re-assuring.

As for the total memory, I'd bet the average memory per server is not 8GB - perhaps 4GB would be the baseline. New boxes coming in will probably be 8GB. And of course, every "box" is a dual CPU system.

So the total memory in the Google cluster - across all datacenters - would be more like 2 petabytes, IMHO.

The interesting question for Greg would be: how many boxes will the new 34,000 sq ft. datacenter in The Dalles, Oregon hold? Remember that Google has custom power supplies to minimize power per server. These dual cpu 8GB Xeon boxes don't take 300W when Google builds them.

My guess is that they are building the datacenter to dissipate as much as 200W per sq ft - being Google (aka Design LLC), cutting edge and all. So probably 60k more boxes in Oregon coming in time for the Christmas rush. That alone shows how hard it is to add hundreds of thousands of boxes, however...

¶ 6/27/2006