设为首页 加入收藏

TOP

HBase实战（2）：搭建分布式集群HBASE
2019-04-14 13:49:25 【大 中 小】 浏览:45次
Tags：HBase 实战 搭建 分布式 集群 HBASE
版权声明：王家林大咖2018年新书《SPARK大数据商业实战三部曲》清华大学出版，微信公众号：从零起步学习人工智能 https://blog.csdn.net/duan_zhihua/article/details/80664859

搭建分布式集群HBASE

1，单机版本的安装请参考https://blog.csdn.net/duan_zhihua/article/details/80622166

2，ZOOKEPER集群的安装请参考https://blog.csdn.net/duan_zhihua/article/details/70885469

2，分布式集群HBASE安装：

  • 配置vim hbase-env.sh
root@master:/usr/local/hbase-1.2.0/conf# vi hbase-env.sh
export JAVA_HOME=/usr/local/jdk1.8.0_60
export HBASE_MANAGES_ZK=false
  • 配置vim hbase-site.xml
  •  <configuration>  
     
        <property>  
            <name>hbase.rootdir</name>  
            <value>hdfs://master:9000/hbase</value>  
        </property>  
        
        <property>  
            <name>hbase.cluster.distributed</name>  
            <value>true</value>  
        </property>  
          
        <property>  
            <name>hbase.zookeeper.quorum</name>  
            <value>192.168.189.1:2181,192.168.189.2:2181,192.168.189.3:2181</value>  
        </property>  
    </configuration>
    配置regionservers
  • root@master:/usr/local/hbase-1.2.0/conf# vi regionservers

master
worker1
worker2
worker3
~
~
    从Master复制hbase目录到从节点worker1 worker2 worker3中
  • scp -r /usr/local/hbase-1.2.0 worker1:/usr/local/hbase-1.2.0
scp -r /usr/local/hbase-1.2.0 worker2:/usr/local/hbase-1.2.0
scp -r /usr/local/hbase-1.2.0 worker3:/usr/local/hbase-1.2.0
    依次启动Hadoop、zookeeper、hbase
  • 启动hadoop
  • root@master:~# /usr/local/hadoop-2.6.0/sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Starting namenodes on [master]
master: starting namenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-namenode-master.out
worker3: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-datanode-worker3.out
worker1: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-datanode-worker1.out
worker2: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-datanode-worker2.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-root-secondarynamenode-master.out
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-resourcemanager-master.out
worker2: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-nodemanager-worker2.out
worker1: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-nodemanager-worker1.out
worker3: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-root-nodemanager-worker3.out
root@master:~# hadoop dfsadmin -safemode leave 
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Safe mode is OFF
root@master:~# 
root@master:~# jps
2722 SecondaryNameNode
3204 Jps
2893 ResourceManager
2542 NameNode
root@master:~#
启动zookeeper

root@master:~# zkServer.sh start 
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@master:~# zkServer.sh  status  
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
root@master:~# jps
2722 SecondaryNameNode
3385 Jps
3324 QuorumPeerMain
2542 NameNode
root@master:~# 
root@worker1:/usr/local# zkServer.sh start 
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@worker1:/usr/local# zkServer.sh  status  
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: leader
root@worker1:/usr/local# jps
2800 Jps
2530 NodeManager
2741 QuorumPeerMain
2399 DataNode
root@worker2:~# zkServer.sh start 
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
root@worker2:~# zkServer.sh  status  
JMX enabled by default
Using config: /usr/local/zookeeper-3.4.6/bin/../conf/zoo.cfg
Mode: follower
root@worker2:~# jps
2707 Jps
2654 QuorumPeerMain
2287 DataNode

启动HBASE
root@master:~# start-hbase.sh
starting master, logging to /usr/local/hbase-1.2.0/logs/hbase-root-master-master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker1: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker1.out
worker2: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker2.out
worker3: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker3.out
master: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-master.out
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
root@master:~# jps
4002 Jps
2722 SecondaryNameNode
3687 HRegionServer
3324 QuorumPeerMain
3532 HMaster
2542 NameNode
root@master:~# 
root@worker1:/usr/local# jps
2530 NodeManager
2741 QuorumPeerMain
3035 Jps
2878 HRegionServer
2399 DataNode
root@worker1:/usr/local# 
root@worker2:~# jps
2707 Jps
2654 QuorumPeerMain
2287 DataNode
root@worker2:~# jps
2792 HRegionServer
3005 Jps
2654 QuorumPeerMain
2287 DataNode
root@worker2:~# 

root@worker3:~# jps
2711 HRegionServer
2344 DataNode
2921 Jps
2479 NodeManager
root@worker3:~#

在hbase-site.xml中加入web的配置：

<property>
        <name>hbase.master.info.port</name>
        <value>60010</value>
</property>

hbase-site.xml复制到worker节点

root@master:/usr/local/hbase-1.2.0/conf# scp -r /usr/local/hbase-1.2.0/conf/hbase-site.xml worker1:/usr/local/hbase-1.2.0/conf/hbase-site.xml
hbase-site.xml                                                                                                                        100%  603     0.6KB/s   00:00    
root@master:/usr/local/hbase-1.2.0/conf# scp -r /usr/local/hbase-1.2.0/conf/hbase-site.xml worker2:/usr/local/hbase-1.2.0/conf/hbase-site.xml
hbase-site.xml                                                                                                                        100%  603     0.6KB/s   00:00    
root@master:/usr/local/hbase-1.2.0/conf# scp -r /usr/local/hbase-1.2.0/conf/hbase-site.xml worker3:/usr/local/hbase-1.2.0/conf/hbase-site.xml
hbase-site.xml                                                                                                                        100%  603     0.6KB/s   00:00    
root@master:/usr/local/hbase-1.2.0/conf#

重启hbase。

root@master:~# start-hbase.sh
starting master, logging to /usr/local/hbase-1.2.0/logs/hbase-root-master-master.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker1: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker1.out
master: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-master.out
worker3: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker3.out
worker2: starting regionserver, logging to /usr/local/hbase-1.2.0/bin/../logs/hbase-root-regionserver-worker2.out
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker1: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
master: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
worker2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
root@master:~#

打开HBASE的web页面地址http://192.168.189.1:60010/master-status，部署成功！



进入hbase shell

root@master:~# hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hbase-1.2.0/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/alluxio-1.7.0-hadoop-2.6/client/alluxio-1.7.0-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.2.0, r25b281972df2f5b15c426c8963cbf77dd853a5ad, Thu Feb 18 23:01:49 CST 2016

hbase(main):001:0>

在hbase中分布式集群中操作：


在web页面检查新建的表
【大 中 小】【打印】 【繁体】【投稿】【收藏】 【推荐】【举报】【评论】 【关闭】 【返回顶部】
上一篇：java连接不上hbase无异常信息 下一篇：[OOM] hbase堆外内存溢出底层定位

最新文章

愚公移山  
Hbase架构   Hbase Region的拆..
5 hbase-shell +   hbase的j..
Hbase   MapReduce例子
愚公移山  

热门文章

GeoMesa-HBase部署实践
Oracle GoldenGate 同步数据至Hbase
Flink读写系列之-读HBase并写入HBase
传统的行存储和（HBase）列存储的区..
flink实战--读写Hbase

Hot 文章

GeoMesa-HBase部署实践
Oracle GoldenGate 同步数据至Hbase
Flink读写系列之-读HBase并写入HBase
传统的行存储和（HBase）列存储的区..
flink实战--读写Hbase

Python

C 语言

C++基础

大数据基础

linux编程基础

C/C++面试题目