MapR Filesystem APIs

Question

For the MapR filesystem, do we keep using HDFS File APIs, like SequenceFile.Reader/Writer etc or do you provide your own packages/classes for file read/write. If you are having your own APIs, can you please point us to some tutorials which has usage of this APIs.
Next question is about NFS mount point: What are the main advantages of mounting MapR DFS as a NFS mount point. One thing, we get is that we can use the standard File APIs to work with this mount point and that falls under easy to use category. Thats a good feature to have. For an experienced developer who is used to work with HDFS APIs and dont mind doing coding with those APIs - Does this NFS mount point provide any other attraction. Also, comign to performace - do we get better performace while using NFS mount point vs standard HDFS APIs.
Installation: Whats the recommended procedure for the whole cluster installation. Do we have to run "configure.sh" on each of the node or is there anyway wherein we can give list of nodes to the installer and it goes around installating on each node.

Accepted Answer

MapR is fully compatible with Apache Hadoop. See discussion to question 'Is MapR wire compatible or API compatible with Hadoop 0.20.2?' which should address your first question.

The NFS mount is an extra feature that will give you a convenient way to get your data in and out of the filesystem and a way to view and manipulate your data (files) via standard filesystem utilities. If using the Hadoop API method is working for you I think you should continue to do so. As for your question on performance, see discussion on question 'Which is more performant: using MapR over NFS or via HDFS?' which addresses the same thing.

Accepted Answer

1

Regarding installation..

Yes, you'll need to run configure.sh on each node.

The documentation for installation is located here. If you have keyless ssh setup between your cluster nodes, you can wrap the installation steps in a script and execute it from a single node.

link

answered 06 Jul '11, 03:33

Santosh
261●3●5
accept rate: 45%

Accepted Answer

Regarding this specific question:

Installation: Whats the recommended procedure for the whole cluster installation. Do we have to run "configure.sh" on each of the node or is there anyway wherein we can give list of nodes to the installer and it goes around installating on each node.

What I typically do is leverage passwordless SSH and a utility such as "parallel-ssh" (or pssh for short). This makes it relatively easy and painless to install, configure and start MapR.

Here is some example output from one of my lab machines:

root@n206:~# cat /usr/bin/pssh
echo @ $@
parallel-ssh -O StrictHostKeyChecking=no -h /root/nodes -p 10 -i -t 9999999 $@
root@n206:~# cat /root/nodes
10.10.1.200
10.10.1.204
10.10.1.201
10.10.1.202
10.10.1.203
root@n206:~# pssh date
@ date
[1] 09:02:29 [SUCCESS] 10.10.1.202
Thu Jul  7 09:01:17 PDT 2011
[2] 09:02:29 [SUCCESS] 10.10.1.200
Thu Jul  7 09:01:35 PDT 2011
[3] 09:02:29 [SUCCESS] 10.10.1.201
Thu Jul  7 09:01:28 PDT 2011
[4] 09:02:29 [SUCCESS] 10.10.1.204
Thu Jul  7 09:01:23 PDT 2011
[5] 09:02:29 [SUCCESS] 10.10.1.203
Thu Jul  7 09:01:27 PDT 2011
root@n206:~# dpkg -l | grep pssh
ii  pssh                                 2.1.1-1                                         Parallel versions of SSH-based tools
root@n206:~#

If you use the pssh utility as shown above then you can install/setup/start the MapR software as follows:

Install software: pssh dpkg/rpm -i <packages> -OR- pssh yum/apt-get install <packages>
Configure software: pssh /opt/mapr/server/configure.sh -C <CLDB nodes> -Z <ZooKeeper nodes>
Configure storage: pssh /opt/mapr/server/disksetup -F <disks file>
Start ZooKeeper: pssh /etc/init.d/mapr-zookeeper start
Start MapR: pssh /etc/init.d/mapr-warden start

MapR Filesystem APIs

Follow this question

Related questions