I setup mapr-oozie on one of my cluster nodes. When the user root runs the oozie example code, it finishes successfully. However, when the other user (mapr) runs it, it fails generating the following error.
java.io.IOException: Open failed for file: /var/mapr/cluster/mapred/jobTracker/staging/mapr/.staging/job_201204241610_0001/job.split, error: Permission denied (13)
at com.mapr.fs.MapRClient.open(MapRClient.java:189)
at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:323)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:460)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:336)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:368)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
It seems that the job executes as the user mapr, but it cannot read its job staging files which are created by root and accessible only by the user root. By setting the hadoop.proxyuser.oozie.groups property, the user root should impersonate the user mapr, however it doesn't seem to work here.
I tried the following steps:
(1) Create the user mapr on all the nodes
[root] groupadd -g 600 mapr
[root] useradd -u 600 -g 600 mapr
(2) Edit /opt/mapr/hadoop/hadoop-0.20.2/conf/mapred-site.xml on all the nodes
<property>
<name>hadoop.proxyuser.root.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.root.groups</name>
<value>mapr,root</value>
</property>
(3) Unzip oozie examples and run it as the user root --> it finishes successfully
[root] cd /opt/mapr/oozie/oozie-3.0.0/
[root] unzip oozie-examples.tar.gz
[root] hadoop fs -put examples examples
[root] oozie job --oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -DjobTracker=<jobtracker node>:9001 -run
(4) Run it as the user mapr --> it fails
[mapr] cd /opt/mapr/oozie/oozie-3.0.0/
[mapr] hadoop fs -put examples examples
[mapr] oozie job --oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -DjobTracker=<jobtracker node>:9001 -run
Any comments are greatly appreciated.
asked
24 Apr '12, 02:33
nagix
56●9●9●13
accept rate:
0%