|
Federation is coming to Hadoop. It sounds like it will solve the problems with having a single name node. Will federation also benefit MapR? |
|
Federation does get rid of having a single name node which allows Hadoop clusters to have more files by a small factor, but it doesn't solve the problems of having a single name node. The problems with a single name node are scalability and reliability. With federation, the problem of scalability is eased a little bit, but definitely not solved because the low performance inherent with sending all meta-data requests through one or a few hosts still remain. The problem of reliability just get worse with federation because the previous single point of failure is multiplied. Looking only at hardware reliability, if you have ten name nodes, the mean time between hardware failures for your cluster is likely to decrease to a few months down from the current level of a few years. Of course, human fallibility normally increases these failure rates significantly. MapR addresses both hardware and human failure modes and also provides much higher scalability than Apache Hadoop. |