一. HBase环境搭建
1. 编辑/opt/hbase-1.3.1/conf/hbase-env.sh,如下所示:
export JAVA_HOME=/usr/local/jdk1.8.0_141
export HBASE_HOME=/opt/hbase-1.3.1
export HBASE_LOG_DIR=/home/ssw/Data/hbase-logs
export HBASE_MANAGES_ZK=true
2. 编辑/opt/hbase-1.3.1/conf/hbase-site.xml,如下所示:
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://master:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.master</name>
<value>master:60000</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>master,slave1,slave2</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/ssw/Data/zookeeper</value>
</property>
</configuration>
3. 编辑/opt/hbase-1.3.1/conf/regionservers,如下所示:
Slave1
Slave2
4. 启动HBase集群
/opt/hbase-1.3.1/bin/start-hbase.sh
5.查看集群是否启动成功
Master:HQuorumPeer;HMaster。Slave:HQuorumPeer;HRegionServer。
二. Hive环境搭建
1. 安装MySQL
wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm
sudo yum install mysql-server
sudo chown -R root:root /var/lib/mysql
service mysqld restart
$ mysql -u root
mysql > use mysql;
mysql > update user set password=password('root') where user='root';
mysql > exit;
service mysqld restart
2.为Hive建立相应的MySQL账户,并赋予足够的权限,如下所示:
create user 'hive' identified by 'mysql';
grant all privileges on *.* to 'hive'@'%' with grant option;
flush privileges;
service mysqld restart
3.建立Hive专用元数据库,用创建的hive账号登录,如下所示:
mysql -uhive -p'mysql'
mysql> create database hive;
4.加载MySQL驱动包
将mysql-connector-java-5.1.44-bin.jar放$HIVE_HOME$/lib目录下面。
5. 编辑/etc/profile,如下所示:
export HIVE_HOME=/opt/hive-2.3.0
export PATH=$PATH:$HIVE_HOME/bin
6. 编辑hive-config.sh,如下所示:
export JAVA_HOME=/usr/local/jdk1.8.0_141
export HIVE_HOME=/opt/hive-2.3.0
export HADOOP_HOME=/opt/hadoop-2.7.3
7. 编辑hive-site.xml,如下所示:
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hivecreateDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>mysql</value>
</property>
8. 编辑hive-env.sh,如下所示:
export HOME==/opt/hadoop-2.7.3
export HIVE_CONF_DIR=/opt/hive-2.3.0/conf
export HIVE_AUX_JARS_PATH=/opt/hive-2.3.0/lib
9.验证Hive配置是否成功,如下所示:
hive> create table test_table (id int, name string, no int);
mysql -uhive -p'mysql'; show databases; use hive; show tables; select * from TBLS;
说明:Hive将元数据库存储在MySQL中,元数据包括表的名字,表的列和分区及其属性,表的属性(是否为外部表等),表的数据所在目录。
三. Kylin环境搭建
四. 遇到的问题
1.create directory /tmp/hive/XXX. Name node is in safe mode.
解析:hdfs dfsadmin -safemode leave
2.FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException:
Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.
解析:用hive -hiveconf hive.root.logger=DEBUG,console排查问题。
3.org.apache.hadoop.hive.metastore.api.MetaException: Version information not found in metastore.
参考文献:
[1]Kylin安装配置实验:http://blog.csdn.net/wzy0623/article/details/51283352
[2] Access denied for user 'hive'@'localhost' :http://blog.csdn.net/adminyou/article/details/48328417
[3]Hive完全分布式集群安装过程(元数据库:MySQL):http://www.aboutyun.com/thread-6902-1-1.html
[4]Hive常见问题汇总:http://blog.csdn.net/freedomboy319/article/details/44828337