Installing Pseudo- Distributed HBase on Ubuntu

HBase run modes: Standalone and Distributed

Standalone mode: By default HBase runs in standalone mode. In standalone mode, HBase does not use HDFS. 

Distributed mode: Distributed mode can be subdivided into distributed but all daemons run on a single node is pseudo-distributed— and fully-distributed where the daemons are spread across all nodes in the cluster.

Hadoop version support matrix

HBase-0.92.x HBase-0.94.x HBase-0.95
Hadoop-0.20.205

S

X

X

Hadoop-0.22.x

S

X

X

Hadoop-1.0.0-1.0.2[a]

S

S

X

Hadoop-1.0.3+

S

S

S

Hadoop-1.1.x

NT

S

S

Hadoop-0.23.x

X

S

NT

Hadoop-2.x

X

S

S

[a] HBase requires hadoop 1.0.3 at a minimum; there is an issue where we cannot find KerberosUtil compiling against earlier versions of Hadoop.

Where

S = supported and tested,
X = not supported,
NT = it should run, but not tested enough.

Pseudo- Distributed Installation

The hbase-0.94.8 installation is done in below versions of Linux, Java and Hadoop respectively.

UBUNTU 13.4

JAVA 1.7.0_25

HADOOP 1.1.2

I have hduser as a dedicated hadoop system user. I had installed my Hadoop in /home/hduser/hadoop folder. Now I am going to install hbase in /usr/lib/hbase folder.

  • Download hbase<version>.tar.gz stable version from here
  • Enter into the directory where the stable version is downloaded. By default it downloads in “Downloads” directory
$ cd Downloads/
  • Unzip the tar file.
$ tar -xvf hbase-0.94.8.tar.gz
  • Create directory
$ sudo mkdir /usr/lib/hbase
  • move  hbase-0.94.8 to hbase
$ mv hbase-0.94.8 /usr/lib/hbase/hbase-0.94.8
  • Open your hbase/conf/hbase-env.sh and modify these lines
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_25

export HBASE_REGIONSERVERS=/usr/lib/hbase/hbase-0.94.8/conf/regionservers

export HBASE_MANAGES_ZK=true
  • Set the HBASE_HOME path in bashrc file

To open bashrc file use this command

$ gedit ~/.bashrc

In bashrc file append the below 2 statements

export HBASE_HOME=/usr/lib/hbase/hbase-0.94.8

export PATH=$PATH:$HBASE_HOME/bin

 

  • Update hbase-site.xml in HBASE_HOME/conf folder with required properties.

    hbase-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>

<property>

<name>hbase.rootdir</name>

<value>hdfs://localhost:9000/hbase</value>

</property>

<property>

<name>hbase.cluster.distributed</name>

<value>true</value>

</property>

<property>

<name>hbase.zookeeper.quorum</name>

<value>localhost</value>

</property>

<property>

<name>dfs.replication</name>

<value>1</value>

</property>

<property>

<name>hbase.zookeeper.property.clientPort</name>

<value>2181</value>

</property>

<property>

<name>hbase.zookeeper.property.dataDir</name>

<value>/home/hduser/hbase/zookeeper</value>

</property>

</configuration>
  • Now check Hadoop version support matrix. If Hadoop is not supported your hbase version then you will get some exception. To fix this simply copy hadoop-core-*.jar from your HADOOP_HOME and commons-collections-*.jar from HADOOP_HOME/lib folder into your HBASE_HOME/lib folder.
  • Extra steps

In /etc/hosts there are two entries:127.0.0.1 and 127.0.1.1.Change the second entry 127.0.1.1 to 127.0.0.1  otherwise it gives error: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing

  • To start Hbase [ First start hadoop ]
hduser@archana:~$ start-hbase.sh

localhost: starting zookeeper, logging to /usr/lib/hbase/hbase-0.94.8/bin/../logs/hbase-hduser-zookeeper-archana.out
 starting master, logging to /usr/lib/hbase/hbase-0.94.8/logs/hbase-hduser-master-archana.out
 localhost: starting regionserver, logging to /usr/lib/hbase/hbase-0.94.8/bin/../logs/hbase-hduser-regionserver-archana.out

jps command list down all currently running processes

hduser@archana:~$ jps

 4334 HQuorumPeer
 2882 SecondaryNameNode
 4867 Jps
 3207 TaskTracker
 2460 NameNode
 4671 HRegionServer
 4411 HMaster
 2977 JobTracker
 2668 DataNode

Hbase Shell

hduser@archana:~$ hbase shell

HBase Shell; enter 'help<RETURN>' for list of supported commands.
 Type "exit<RETURN>" to leave the HBase Shell
 Version 0.94.8, r1485407, Wed May 22 20:53:13 UTC 2013

hbase(main):001:0> create 't1','c1'
  • To stop HBase
HBASE_PATH$ bin/stop-hbase.sh

stopping hbase...............

To use the web interfaces

http://localhost:60010 for master
http://localhost:60030 for region server

  • Reference :

http://hbase.apache.org/book/standalone_dist.html

http://hbase.apache.org/book/standalone_dist.html#confirm

Note:The information provided here is best of my knowledge and experience if at all any modifications are to be made please help me with ur valuable suggestion which are always welcome…. :)

Advertisements

24 Responses to Installing Pseudo- Distributed HBase on Ubuntu

  1. Pingback: Connecting Hbase using Java | Big Data World

  2. Pingback: Install Hadoop in just a few steps – Extreme Computing | Ankita Kulkarni's Blog

  3. Prasanna says:

    Thanks a lot Archana….great help on your blog….i have installed hadoop, hbase, pig, hive based on your instructions….they are precise and a blessing to ‘not so technically good’ guys like me….

    Like

  4. I got the error after run “start-hbase.sh”.
    “/usr/lib/hbase/hbase-0.94.14/bin/../conf/hbase-env.sh: line 29: export: `/usr/lib/jvm/java-7-oracle’: not a valid identifier”.
    what should I fix ?

    Like

  5. Hi, I’m getting Error when i trying to create table and list the table.

    ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 times

    Like

  6. Haider Ali says:

    I am getting this error in my log file. Can anyone help me.

    2014-04-29 15:49:11,360 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /127.0.0.1:60020 could not be reached after 1 tries, giving up.
    2014-04-29 15:49:11,361 WARN org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of -ROOT-,,0.70236052 to serverName=localhost,60020,1398800937177, load=(requests=0, regions=0, usedHeap=0, maxHeap=0), trying to assign elsewhere instead; retry=0
    org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy interface org.apache.hadoop.hbase.ipc.HRegionInterface to /127.0.0.1:60020 after attempts=1
    at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:355)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:969)
    at org.apache.hadoop.hbase.master.ServerManager.getServerConnection(ServerManager.java:620)
    at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:555)
    at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1043)
    at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:858)
    at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:838)
    at org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:92)
    at org.apache.hadoop.hbase.master.AssignmentManager.addToRITandCallClose(AssignmentManager.java:422)
    at org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:348)
    at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:315)
    at org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransitionAndBlockUntilAssigned(AssignmentManager.java:279)
    at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:428)
    at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:388)
    at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:282)
    Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
    at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
    at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
    at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
    … 14 more
    2014-04-29 15:49:11,384 WARN org.apache.hadoop.hbase.master.AssignmentManager: Unable to find a viable location to assign region -ROOT-,,0.70236052

    thanks

    Like

  7. Johng840 says:

    I really appreciate this post. I’ve been looking all over for this! Thank goodness I found it on Bing. You’ve made my day! Thanks again! daaeaefkkgag

    Like

  8. Agung says:

    Hi Archana

    I have following this post. I finished the installation of hbase and hbase request password to start it. I have no idea what is the password. this is command in my terminal:

    indi60@arrasyid:/usr/lib/hbase/hbase-0.98.5-hadoop2/bin$ start-hbase.sh
    /usr/lib/hbase/hbase-0.98.5-hadoop2/bin/hbase: line 386: /usr/lib/jvm/jdk.1.7.0/bin/java: cannot execute binary file
    /usr/lib/hbase/hbase-0.98.5-hadoop2/bin/hbase: line 386: /usr/lib/jvm/jdk.1.7.0/bin/java: Success
    /usr/lib/hbase/hbase-0.98.5-hadoop2/bin/hbase: line 386: /usr/lib/jvm/jdk.1.7.0/bin/java: cannot execute binary file
    /usr/lib/hbase/hbase-0.98.5-hadoop2/bin/hbase: line 386: /usr/lib/jvm/jdk.1.7.0/bin/java: Success
    starting master, logging to /usr/lib/hbase/hbase-0.98.5-hadoop2/logs/hbase-indi60-master-arrasyid.out
    /usr/lib/hbase/hbase-0.98.5-hadoop2/bin/hbase: line 386: /usr/lib/jvm/jdk.1.7.0/bin/java: cannot execute binary file
    /usr/lib/hbase/hbase-0.98.5-hadoop2/bin/hbase: line 386: /usr/lib/jvm/jdk.1.7.0/bin/java: Success
    indi60@localhost’s password:
    localhost: Permission denied, please try again.

    do you have any suggestion? thanks!

    Like

  9. Nice Blog, thanks for sharing this kind of information.

    Liked by 1 person

  10. Nice Blog, thanks for sharing this kind of information.

    Liked by 1 person

  11. chaitanya says:

    Dear Archana,
    I have installed HBase successfully. but my hmaster is not starting.
    when I do jps HRegiomServer and HQuorumPeer are working
    please help me

    Like

  12. krishan23 says:

    The web interface is not working for me… I followed all the steps here… what could be the problem???

    Like

  13. Srinivasan V says:

    When starting HBASE in standalone Ubuntu, i am getting the following errors

    Error: Could not find or load main class org.apache.hadoop.hbase.util.HBaseConfTool
    Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
    Error: Could not find or load main class org.apache.hadoop.hbase.zookeeper.ZKServerTool
    starting master, logging to /usr/local/hbase/logs/hbase-ubuntu-master-ubuntu-VirtualBox.out
    Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
    starting regionserver, logging to /usr/local/hbase/logs/hbase-ubuntu-1-regionserver-ubuntu-VirtualBox.out

    Please help me.

    Like

  14. Jennings says:

    ERROR: Can’t get master address from ZooKeeper; znode data == null

    Like

  15. Praveen says:

    I’m getting following error after running $ start-hbase.sh

    please help

    Error: Could not find or load main class org.apache.hadoop.hbase.util.HBaseConfTool
    Error: Could not find or load main class org.apache.hadoop.hbase.zookeeper.ZKServerTool
    mkdir: cannot create directory ‘/usr/local/hbase/logs’: Permission denied
    starting master, logging to /usr/local/hbase/logs/hbase-medgini-master-ubuntu.out
    /usr/local/hbase/bin/hbase-daemon.sh: line 189: /usr/local/hbase/logs/hbase-medgini-master-ubuntu.out: No such file or directory
    head: cannot open ‘/usr/local/hbase/logs/hbase-medgini-master-ubuntu.out’ for reading: No such file or directory
    mkdir: cannot create directory ‘/usr/local/hbase/logs’: Permission denied
    starting regionserver, logging to /usr/local/hbase/logs/hbase-medgini-1-regionserver-ubuntu.out
    /usr/local/hbase/bin/hbase-daemon.sh: line 189: /usr/local/hbase/logs/hbase-medgini-1-regionserver-ubuntu.out: No such file or directory
    head: cannot open ‘/usr/local/hbase/logs/hbase-medgini-1-regionserver-ubuntu.out’ for reading: No such file or directory

    Like

  16. Hello. I wanted to ask 1 thing…is this a wordpress internet website as we are preparing to be shifting more than to WP. Furthermore did you make this template yourself? Thanks.

    Like

  17. Debashis says:

    Worked miraculously!! 🙂 Many many thanks 🙂

    Like

  18. Koushik Paul says:

    worked like a charm, without any error

    Thanks bro !

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: