Nuclear Rooster

13Mar/090

MapReduce: Powerful Elegance

I've just started playing around with Hadoop, Apache's implementation of Google's MapReduce paradigm. I can feel the power that MapReduce brings to the table, but I have had trouble trying to describe it to other people. Here's a nice quote from google's whitepaper:

Most such calculations are conceptually straightforward. However, ... the issues of how to parallelize the computation, distribute the data, and handle failures conspire to obscure the original simple computation with large amounts of complex code...

So MapReduce doesn't help you solve really difficult problems, it effectively manages simple problems with a teeny dataset, or a ginormous one. Its as if it was equally effective for one pointy-haired boss to manage a team of 5 as it was a team of 5000. Combine that with Cloud Computing's ability to reduce the cost of acquiring new processing power, and you have quite a bit of power at your fingertips.

Comments (0) Trackbacks (0)

No comments yet.


Leave a comment


No trackbacks yet.