|
I tried giving the complete local path as input in a map-reduce java program. It is running successfully in the eclipse,but not running in the cluster. So how can i directly give a local file as input inside a map-reduce java program,so that i can run it in the hdfs cluster. |
|
Is this Apache Hadoop or MapR that you are asking about? |
|
Using local input usually only makes sense when you run a job in a standalone mode, e.g: hadoop jar /opt/mapr/hadoop/hadoop-0.20.2/hadoop-0.20.2-dev-examples.jar wordcount -Dmapred.job.tracker=local -Dmapreduce.jobtracker.staging.root.dir=/tmp -Dfs.default.name=file:/// /abs/path/in /abs/path/out for basic debugging. Which is maybe what you are doing in your IDE. When you run a job in a Hadoop cluster all task tracker nodes need to be able to access the input path. That is why you would and should normally use MapR-FS. An alternative that in a way defeats Hadoop's purpose of pushing computation to the data includes having the same mount point to your NAS/SAN on every node. |
