hive 结合mysql快速安装 - Hive

系统使用centos7.1
官方下载hadoop,hive,jdk的安装包,解压在新建用户hive的目录下,安装mariadb-server(mysql-server)
[hive@localhost ~]$ ll
total 544900
drwxrwxr-x 9 hive hive 4096 Sep 10 07:31 apache-hive-2.1.0-bin
-rw-rw-r-- 1 hive hive 149599799 Jun 21 01:26 apache-hive-2.1.0-bin.tar.gz
drwxrwxr-x 9 hive hive 149 Sep 9 10:45 apache-tomcat-7.0.70
-rw-r--r-- 1 hive hive 8924465 Sep 9 10:29 apache-tomcat-7.0.70.tar.gz
drwxr-xr-x 9 root root 139 Aug 18 01:49 hadoop-2.7.3
-rw-r--r-- 1 root root 214092195 Aug 25 19:25 hadoop-2.7.3.tar.gz
lrwxrwxrwx 1 hive hive 12 Sep 9 08:28 hadoop-last -> hadoop-2.7.3
lrwxrwxrwx 1 hive hive 21 Sep 9 09:08 hive-last -> apache-hive-2.1.0-bin
drwxr-xr-x 8 hive hive 4096 Jun 23 01:56 jdk1.8.0_102
-rw-r--r-- 1 root root 181435897 Sep 9 06:57 jdk-8u102-linux-x64.tar.gz
drwxr-xr-x 4 hive hive 143 May 4 11:11 mysql-connector-java-5.1.39
-rw-r--r-- 1 hive hive 3899019 Sep 10 07:18 mysql-connector-java-5.1.39.tar.gz
-rw-rw-r-- 1 hive hive 11183 Sep 9 10:18 wc.txt

修改用户的环境
[hive@localhost ~]$ cat .bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

# User specific environment and startup programs
export JAVA_HOME=/home/hive/jdk1.8.0_102
export HADOOP_HOME=/home/hive/hadoop-last
export HIVE_HOME=/home/hive/hive-last

PATH=$PATH:$HOME/.local/bin:$HOME/bin:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME/bin:$HIVE_HOME/conf

export PATH

启动mysql数据库,为hadoop/hive添加一个用户
[root@localhost ~]# systemctl start mariadb
[root@localhost ~]# mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 3
Server version: 5.5.50-MariaDB MariaDB Server

Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> CREATE USER 'hadoop'@'localhost' IDENTIFIED BY 'hadoop';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'hadoop'@'localhost' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> EXIT
Bye

MariaDB [(none)]> CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> exit
Bye

进入$HIVE_HOME/conf,去掉以下三个文件的后辍.template
-rw-r--r-- 1 hive hive 2378 Sep 10 06:24 hive-env.sh
-rw-r--r-- 1 hive hive 2299 Jun 3 10:43 hive-exec-log4j2.properties
-rw-r--r-- 1 hive hive 2950 Sep 10 06:25 hive-log4j2.properties
重命令hive-default.xml.template为hive-site.xml
-rw-r--r-- 1 hive hive 225729 Sep 10 03:21 hive-site.xml
添加hive-env.sh两行
export HADOOP_HOME=/home/hive/hadoop-last
export HIVE_CONF_DIR=/home/hive/hive-last/conf
修改hive-site.xml
默认hive.metastore.warehouse.dir是/user/hive/warehouse
hive.exec.scratchdir是/tmp/hive
创建这个目录

如果是使用内置的derby存储metadata
[hive@localhost hive-last]$ schematool -dbType derby -initSchema
which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hive/.local/bin:/home/hive/bin:/home/hive/jdk1.8.0_102/bin:/home/hive/hadoop-last/bin:/home/hive/hive-last/bin:/home/hive/hive-last/conf)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hive/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hive/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.derby.sql
Initialization script completed
schemaTool completed
修改
chmod a+rw /tmp/hive/
下面修改为使用mysql

hive-site.xml
480: javax.jdo.option.ConnectionPassword
481- hive
498: javax.jdo.option.ConnectionURL
499- jdbc:mysql://localhost:3306/hivecreateDatabaseIfNotExist=true
932: javax.jdo.option.ConnectionDriverName
933- com.mysql.jdbc.Driver
960: javax.jdo.option.ConnectionUserName
961- hive

下载mysql-connector
[hive@localhost mysql-connector-java-5.1.39]$ cp mysql-connector-java-5.1.39-bin.jar ~/hive-last/lib/ -v
‘mysql-connector-java-5.1.39-bin.jar’ -> ‘/home/hive/hive-last/lib/mysql-connector-java-5.1.39-bin.jar’

[hive@localhost hive-last]$ schematool -dbType mysql -initSchema
which: no hbase in (/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/hive/.local/bin:/home/hive/bin:/home/hive/jdk1.8.0_102/bin:/home/hive/hadoop-last/bin:/home/hive/hive-last/bin:/home/hive/hive-last/conf)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hive/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hive/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://localhost:3306/hivecreateDatabaseIfNotExist=true
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: hive
Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.mysql.sql
Initialization script completed
schemaTool completed

修改hive-site.xml的system变量,解决变量无法识别的问题,使用绝对路径
:%s#${system:java.io.tmpdir}#/tmp/javaiotmp#
:%s#${system:user.name}#hive#