Configure.sh error on ZK nodes

Question

Hello I realize this may be something simple but I'm at a dead end on Google and the forums. We have a simple mapr setup deployed on Ubuntu server 10.4 LTS. At the initial configuration everything went well. I was able to execute all steps in the config document. However after loading data and submitting jobs we've found that the task tracker wasn't starting. Upon trying to reconfigure I am getting this error:

root@MAPR1:/etc/apache2# /opt/mapr/server/configure.sh -C mapr1,mapr2,mapr3 -Z mapr1,mapr2,mapr3
Node setup configuration:  cldb fileserver jobtracker nfs tasktracker webserver zookeeper
Log can be found at:  /opt/mapr/logs/prerequisitecheck.log
ERROR: Can not find matching ZK node IP based on provided input: mapr1:5181,mapr2:5181,mapr3:5181

I can ping all hosts, both via name and ip, I can resolve forwards and backwards (name => ip ip => name) and this error only occurs on nodes that have the mapr-zookeeper packages installed. All other nodes configure fine, I've tried this with zookeeper running and not running. Any insight into what may be causing this?

Contents of prerequitecheck.log
INFO:Version of java:
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
INFO:
INFO:OS and Java versions are MapR compatible in the current machine. Go Ahead with MapR installation
INFO:Using 7222 port for CLDB mapr1
INFO:Using 7222 port for CLDB mapr2
INFO:Using 7222 port for CLDB mapr3
INFO:Using 5181 port for ZooKeeper mapr1
INFO:Using 5181 port for ZooKeeper mapr2
INFO:Using 5181 port for ZooKeeper mapr3
INFO:
INFO:Node install STARTED
INFO:-----------------------
INFO:Contructing ClusterConfFile: cldb node list: 127.0.1.1:7222 192.168.106.67:7222 192.168.106.66:7222
INFO:Contructing ClusterConfFile: Done
INFO:Updating file client config
ERROR: Can not find matching ZK node IP based on provided input: mapr1:5181,mapr2:5181,mapr3:5181

Accepted Answer

2

I don't think you provided output from "gethostip -d maprN", or if the first output is the one from that command then for some reason it shows local ip: 127.0.0.1 for the node you are currently on. When you used IPs it figured out your node, but when you were giving names - they did not match (case sensitivity as you can see: MAPRN versus maprN)

link

answered 16 Sep '11, 11:12

yufeldman ♦♦
1.9k●2●7
accept rate: 25%

The first output is from the gethost ip command, executed on each node. I'm not sure why it worked today but I assure you the command gave me the same error yesterday with both name and IP. While I am glad that it is now working, I'm a little puzzled as to why it "just worked" nothing in the logs really lead me to believe that anything changed in the way I entered the command.

I noted your post about 127.x.x.x in another thread. How would I get around this as it's a basic part of linux networking? Always use IP's? I prefer to use names so I can change things around on the back end of things without having to recode scripts.

(16 Sep '11, 11:33) tskyers

You can surely use names - just make sure you use names that match either to "cat /opt/mapr/hostname" or resolvable to non-local IP. Also double check your /etc/hosts - it may have entries for all the hosts except the current one. I feel that you run "gethostip -d MAPR3.lab.net" you get your global IP and not 127.0.0.1 for mapr3 (same for others)

(16 Sep '11, 11:46) yufeldman ♦♦

Thank you for your quick and helpful responses!

(17 Sep '11, 06:36) tskyers

Answer 2

1

Could you please check following:

 1. run "gethostip -d mapr1" (repeat same for mapr2 and mapr3)
 2. run: hostname --fqdn 
 3. cat /opt/mapr/hostname

On a particular node

Results from #2 and #3 should match
result of #1 should be match at least one of ifconfig IPs

Please let us know

link

answered 15 Sep '11, 22:14

yufeldman ♦♦
1.9k●2●7
accept rate: 25%

Answer 3

I've run the commands here is the output from each. I've included a non ZK node (mapr4) as a control.

Output from mapr1.lab.net ..

      127.0.1.1
      192.168.106.67
      192.168.106.66
      192.168.106.74

Output from mapr2.lab.net ..

      192.168.106.65
      127.0.1.1
      192.168.106.66
      192.168.106.74

Output from mapr3.lab.net ..

      192.168.106.65
      192.168.106.67
      127.0.1.1
      192.168.106.74

Output from mapr4.lab.net ..

      192.168.106.65
      192.168.106.67
      192.168.106.66
      127.0.1.1

hostname -fqdn

Output from mapr1.lab.net ..

      MAPR1.lab.net

Output from mapr2.lab.net ..

      MAPR2.lab.net

Output from mapr3.lab.net ..

      MAPR3.lab.net

Output from mapr4.lab.net ..

      MAPR4.lab.net

cat /opt/mapr/hostname

Output from mapr1.lab.net ..

      MAPR1.lab.net

Output from mapr2.lab.net ..

      MAPR2.lab.net

Output from mapr3.lab.net ..

      MAPR3.lab.net

Output from mapr4.lab.net ..

      MAPR4.lab.net

I've tried to do the configure.sh via ip as well (and for strange and unimaginable reason it worked today!!)

root@MAPR1:/opt/mapr/logs# /opt/mapr/server/configure.sh -C 192.168.106.65,192.168.106.66,192.168.106.67 -Z 192.168.106.65,192.168.106.66,192.168.106.67
Node setup configuration:  cldb fileserver jobtracker nfs tasktracker webserver zookeeper
Log can be found at:  /opt/mapr/logs/prerequisitecheck.log
root@MAPR1:/opt/mapr/logs#

root@MAPR1:/opt/mapr/logs#  /opt/mapr/server/configure.sh -C mapr1,mapr2,mapr3 -Z mapr1,mapr2,mapr3
Node setup configuration:  cldb fileserver jobtracker nfs tasktracker webserver zookeeper
Log can be found at:  /opt/mapr/logs/prerequisitecheck.log
ERROR: Can not find matching ZK node IP based on provided input: mapr1:5181,mapr2:5181,mapr3:5181

Very very interesting. I'm curious as to the problem ... is it the 127.x.x.x?

Configure.sh error on ZK nodes

Follow this question

Related questions