|
Client trying to run PigMix on small cluster and get the follwoing. What could be the reason? ERROR 2999: Unexpected internal error. Failed to create DataStorage java.lang.RuntimeException: Failed to create DataStorage at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75) at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:213) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:133) at org.apache.pig.impl.PigContext.connect(PigContext.java:183) at org.apache.pig.PigServer.<init>(PigServer.java:226) at org.apache.pig.PigServer.<init>(PigServer.java:215) at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55) at org.apache.pig.Main.run(Main.java:452) at org.apache.pig.Main.main(Main.java:107) Caused by: java.io.IOException: No FileSystem for scheme: maprfs at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1375) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95) at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72) |
|
In order to have this working correctly, MapR have modifyed pigmix, pointing to correct configurations and correct class paths. Ask MapR support if you need one. |
|
This error also occurs if a pig script is executed inside java. The pig executable in the mapr package is set up with the correct classpaths, but while running pig scripts in java, the classpath needs to be set up manually. Here's how the appropriate java properties and classpath need to be setup for mapr: /usr/lib/jvm/java-6-sun/bin/java -Djava.library.path=/opt/mapr/lib -cp /opt/mapr/pig/pig-0.9.0/pigperf.jar:/opt/mapr/pig/pig-0.9.0/conf/pig.properties:/opt/mapr/hadoop/hadoop-0.20.2/lib/hadoop-0.20.2-dev-core.jar:/opt/mapr/pig/pig-0.9.0/pig-0.9.0-core.jar:/opt/mapr/hadoop/hadoop-0.20.2/conf:/opt/mapr/hadoop/hadoop-0.20.2/lib/maprfs-0.1.jar:/opt/mapr/zookeeper/zookeeper-3.3.2/zookeeper-3.3.2.jar:/opt/mapr/pig/pig-0.9.0/lib/* org.apache.pig.Main yourpigscript.pig |
|
This sounds like you are using the ordinary hadoop distribution on the client side. You may need to install the MapR client distribution on the machine running the program from. Can you include more details about which nodes are running what software? |
|
Working Pigmix script can be downloaded from http://package.mapr.com/tools/pigmix2.tar.gz It have all class paths defined correctly. |
