Using bixo

Question

Hi, I am trying to run bixo on a client machine. I have mapr-core installed on to it. I also exported the following library export LD_LIBRARY_PATH=/opt/mapr/lib

When I run the crawl example, I get the following exception:

11/12/02 12:27:08 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 0 time(s).
11/12/02 12:27:09 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 1 time(s).
11/12/02 12:27:10 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 2 time(s).
11/12/02 12:27:11 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 3 time(s).
11/12/02 12:27:12 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 4 time(s).
11/12/02 12:27:13 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 5 time(s).
11/12/02 12:27:14 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 6 time(s).
11/12/02 12:27:15 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 7 time(s).
11/12/02 12:27:16 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 8 time(s).
11/12/02 12:27:17 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9001. Already tried 9 time(s).
11/12/02 12:27:17 INFO ipc.RPC: FailoverProxy: Server on localhost/127.0.0.1:9001 is lost due to java.net.SocketException: Call to localhost/127.0.0.1:9001 failed on socket exception in call getClusterStatus
11/12/02 12:27:17 INFO ipc.RPC: Searching for the Active Server ...
11/12/02 12:27:17 INFO ipc.RPC: Attempt# 1 . Trying to connect Server atlocalhost/127.0.0.1:9001

It looks like it doesn't find the DFS and the jobtracker. DO I need to modify any configuration? I also set up hive on the same client and it can launch mapred jobs successfully. Thanks.

Answer 1

0

Kas,

How did you tell Bixo where the cluster is?

Do you have paths that look like hdfs://host:port/path or maprfs://host:port/path ? If so, these need to be just maprfs:///path

Make sure you don't mention a host name in the file path.

link

answered 02 Dec '11, 12:35

TedDunning ♦♦
2.4k●3●15
accept rate: 28%

My core-site.xml looks like:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
<property>
  <name>fs.default.name</name>
  <value>maprfs:///</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri's scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri's authority is used to
  determine the host, port, etc. for a filesystem.</description>
</property>
</configuration>

(02 Dec '11, 12:37) Kas

Sure. The fact that hive works well indicates that core-site probably isn't the problem.

How do you tell Bixo where to get and put files? What do you tell it?

(02 Dec '11, 12:42) TedDunning ♦♦

I am using mapR-core as oppossed to mapR-client. Do you know if they might impact the working of Bixo?

(05 Dec '11, 16:15) Kas

mapr-core is a superset of mapr-client. Other then that I assume you run configure.sh on the node you are trying to run Bixo on, so that node can become part of the cluster.

(05 Dec '11, 19:41) yufeldman ♦♦

Using bixo

Follow this question

Related questions