I setup mapr-oozie on one of my cluster nodes. When the user root runs the oozie example code, it finishes successfully. However, when the other user (mapr) runs it, it fails generating the following error.

java.io.IOException: Open failed for file: /var/mapr/cluster/mapred/jobTracker/staging/mapr/.staging/job_201204241610_0001/job.split, error: Permission denied (13)
    at com.mapr.fs.MapRClient.open(MapRClient.java:189)
    at com.mapr.fs.MapRFileSystem.open(MapRFileSystem.java:323)
    at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:460)
    at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:336)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:368)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:327)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1109)
    at org.apache.hadoop.mapred.Child.main(Child.java:264)

It seems that the job executes as the user mapr, but it cannot read its job staging files which are created by root and accessible only by the user root. By setting the hadoop.proxyuser.oozie.groups property, the user root should impersonate the user mapr, however it doesn't seem to work here.

I tried the following steps:

(1) Create the user mapr on all the nodes

[root] groupadd -g 600 mapr
[root] useradd -u 600 -g 600 mapr

(2) Edit /opt/mapr/hadoop/hadoop-0.20.2/conf/mapred-site.xml on all the nodes

<property>
  <name>hadoop.proxyuser.root.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.root.groups</name>
  <value>mapr,root</value>
</property>

(3) Unzip oozie examples and run it as the user root --> it finishes successfully

[root] cd /opt/mapr/oozie/oozie-3.0.0/
[root] unzip oozie-examples.tar.gz
[root] hadoop fs -put examples examples
[root] oozie job --oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -DjobTracker=<jobtracker node>:9001 -run

(4) Run it as the user mapr --> it fails

[mapr] cd /opt/mapr/oozie/oozie-3.0.0/
[mapr] hadoop fs -put examples examples
[mapr] oozie job --oozie http://localhost:11000/oozie -config examples/apps/map-reduce/job.properties -DjobTracker=<jobtracker node>:9001 -run

Any comments are greatly appreciated.

asked 24 Apr '12, 02:33

nagix's gravatar image

nagix
569913
accept rate: 0%


nagix,

Thanks for the feedback and analysis. You have correctly pointed to the reason behind the error.

This is a known issue which we have fixed in our upcoming v2.0 release.

link

answered 24 Apr '12, 03:53

Nabeel's gravatar image

Nabeel ♦♦
2.2k147
accept rate: 24%

Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or __italic__
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×8

Asked: 24 Apr '12, 02:33

Seen: 482 times

Last updated: 24 Apr '12, 03:53

powered by OSQA