|
curious whether it's technically alright to have a symbolic link to a directory outside an nfs share and use with hadoop jobs. |
|
Symbolic links that point to outside of MapR cannot be interpreted by the Hadoop Java client. They can be interpreted with NFS, as long as all hosts have the same symbolic link target. For example, a link to /bin/sh from inside MapR works on all machines, since /bin/sh exists on all machines. |
|
You can definitely have the link, but if the target of that link does not exist on all machines that need it then your hadoop tasks may be mightily confused. Remember that the target of a symbolic link will be resolved symbolically on each host. That means that you have to be able to reach the target by the same path on every machine where you try to read or write that data. Usually, this requirement makes it a really bad idea to use symbolic links that point outside of the distributed file system. |
