By Jeff Stibel
Google's stock is now down more than 50% year to date but the Google guys don't seem to be concerned. Here's why -- and why it is critically important to your business:
Most people think the reason is because Google dominates search. But Google is building a new secret weapon that has more to do with the brain than search. The effort is called Map reduce, a simple yet powerful software program that enables Google to use the Internet to think.
MapReduce does what our brains do all the time: It categorizes (Maps) key pieces of information, distributes it across its server farm of PCs, and then eliminates (Reduces) irrelevant data (computers--unlike MapReduce and the brain--soak in everything). Google now uses MapReduce for over 10,000 programs, ranging from the processing of satellite imagery, language processing and responding to popular queries. It is now processing roughly 100,000 functions daily and digesting 20 petabytes of data each day.
Does this sound like the perfect computer? Think again. This is not even your typical computer: one that is stable, logical, and failsafe. Instead, it is error prone, strapped together with Velcro (literally) and unreliable. Or as one Senior Vice President at Google recently said, "Nobody builds servers as unreliable as we do." But it is the same paradox that makes the brain work, wherein its seeming imperfections are what make MapReduce (and the brain) so powerful.
As the inventors of MapReduce noted in a recent paper, "It has been used across a wide range of domains within Google including: large-scale machine learning problems; clustering problems...; extracting data to produce reports of popular queries; extracting properties of Web pages for new experiments and products...; processing of satellite imagery data; language model processing for statistical machine translation, and; large-scale graph computation." Or in other words, the tasks Google performs are similar to the functions performed by the brain: learning, categorization, vision and language.
If all this sounds a bit more like human thought than computing, it should. We often fail to see just how powerful Google really is and how much it behaves like a brain, if for no other reason than we interact with Google through its homepage. As Vint Cerf, Google's Chief Internet Evangelist, points out, "While it presents itself as a web interface to most people, Google could just as well present itself as a programmable interface, which means that you can start writing software that gets information through the eyes, so to speak, of Google. That creates a vocabulary, if you like, that programmable systems can use in order to take advantage of what Google is capable of doing with its gigantic database."
Even Google's harshest critics no longer dismiss the value of MapReduce or the power of the computing cloud. In a recent New York Times article, Bill Gates "acknowledged that MapReduce was a significant technology, but he asserted that Microsoft was building its own parallel processing software..." In a somewhat circuitous compliment, Gates said: "They did MapReduce; but we have this thing, called Dryad, that's better."
What's happening is that MapReduce is opening the door to the analysis of vast amounts of information--from terabytes of data on the voting habits of Americans, to the fluctuations of billions of individual airline fares, to scores of terabytes of health data. This will change the landscape of virtually everything we do. "The biggest challenge of the Petrabyte Age won't be storing all the data," Wired magazine noted recently, "but figuring out how to make sense of it." Making sense of it: That is where the brain behind the Internet is now heading.