I'm having a peculiar problem with a permission denied error.
I'm trying to run a map-reduce job over a set of input files. I have checked that I can view the files on every node with the user I am submitting the job as. Yet, the map tasks can't get started due to the following:
Is the Child process getting run as a different user? I'm able to catch some of the Child java processes on the tasktracker nodes using 'ps' and they appear to be getting executed as my user.
That said, my user id's and groups come from openldap, could that be a problem for map reduce? As I said, I can view the files just fine in either the shell or using 'hadoop fs -cat'
asked 20 Oct '11, 14:52
If I get this right all tasks are failing with permission denied? All mapreduce task run as user who submitted job. maprfs supports unix uid/gid and permission checks are based on uid/gid. The most common issue we see is that a username is present on all nodes but uid is different and hence task fail to access a file.
user->uid/gid mapping on all nodes must be consistent.