1003-Hive的安装 - Hive - 程序员开发

TOP

1003-Hive的安装

2019-03-24 01:05:10 【大中小】浏览:67次

1、上传Hive安装包

2、解压Hive到app目录下

tar -zxvf hive-0.12.0.tar.gz -C app/

3、Hive项目结构

[hadoop@cloud04 ~]$ cd app/hive-0.12.0/
[hadoop@cloud04 hive-0.12.0]$ ll
total 228
drwxr-xr-x. 3 hadoop hadoop 4096 Oct 9 2013 bin
drwxr-xr-x. 2 hadoop hadoop 4096 Apr 27 02:53 conf
drwxr-xr-x. 6 hadoop hadoop 4096 Apr 27 02:53 docs
drwxr-xr-x. 4 hadoop hadoop 4096 Oct 9 2013 examples
drwxr-xr-x. 7 hadoop hadoop 4096 Oct 9 2013 hcatalog
drwxr-xr-x. 4 hadoop hadoop 4096 Apr 27 02:53 lib
-rw-rw-r--. 1 hadoop hadoop 23828 Oct 9 2013 LICENSE
-rw-rw-r--. 1 hadoop hadoop 1559 Oct 9 2013 NOTICE
-rw-rw-r--. 1 hadoop hadoop 3832 Oct 9 2013 README.txt
-rw-rw-r--. 1 hadoop hadoop 166087 Oct 9 2013 RELEASE_NOTES.txt
drwxr-xr-x. 3 hadoop hadoop 4096 Oct 9 2013 scripts
drwxrwxr-x. 28 hadoop hadoop 4096 Apr 27 02:54 src

Hive配置文件介绍

hive-site.xml hive的配置文件

hive-env.sh hive的运行环境文件

hive-default.xml.template 默认模板

hive-env.sh.template hive-env.sh默认配置

hive-exec-log4j.properties.template exec默认配置

hive-log4j.properties.template log默认配置

4、设置环境变量

export JAVA_HOME=/home/hadoop/app/jdk1.7.0_65
export PATH=$PATH:$JAVA_HOME/bin

export HADOOP_HOME=/home/hadoop/app/hadoop-2.4.1
export HIVE_HOME=/home/hadoop/app/hive-0.12.0
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin

#使之生效

source /etc/profile

5、验证

#比较./hive 和 ./bin/hive 区别

#进入bin后，执行hive

[hadoop@cloud04 bin]$ ./hive

hive> create databases hive01;

hive> create table tlb01 (id int ,name string);

hive> show databases;
OK
default
hive01
Time taken: 3.45 seconds, Fetched: 2 row(s)
hive> use hive01;
OK
Time taken: 0.033 seconds
hive> show tables;
OK
tlb01
Time taken: 0.249 seconds, Fetched: 1 row(s)

[hadoop@cloud04 bin]$ ll
total 40
-rwxr-xr-x. 1 hadoop hadoop 881 Oct 9 2013 beeline
-rw-rw-r--. 1 hadoop hadoop 363 Apr 27 05:51 derby.log
drwxr-xr-x. 3 hadoop hadoop 4096 Apr 27 02:53 ext
-rwxr-xr-x. 1 hadoop hadoop 5691 Oct 9 2013 hive
-rwxr-xr-x. 1 hadoop hadoop 1900 Oct 9 2013 hive-config.sh
-rwxr-xr-x. 1 hadoop hadoop 885 Oct 9 2013 hiveserver2
drwxrwxr-x. 5 hadoop hadoop 4096 Apr 27 05:51 metastore_db
-rwxr-xr-x. 1 hadoop hadoop 832 Oct 9 2013 metatool
-rwxr-xr-x. 1 hadoop hadoop 884 Oct 9 2013 schematool

#在bin目录下执行命令 ./bin/hive

[hadoop@cloud04 bin]$ ./bin/hive

[hadoop@cloud04 hive-0.12.0]$ ./bin/hive
hive> show databases;
OK
default
Time taken: 3.646 seconds, Fetched: 1 row(s)

发现没有刚才创建的数据库hive01

总结：对比发现，元数据仅存在执行命令的目录，在不同的目录数据不能互相访问，要解决该问题，引入MySQL存储元数据。

6、查看元数据在hdfs上存储位置

[hadoop@cloud01 app]$ hadoop fs -ls /user/
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2015-04-27 03:50 /user/hive
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2015-04-27 03:50 /user/hive/warehouse
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive/warehouse
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2015-04-27 04:16 /user/hive/warehouse/hive01.db
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive/warehouse/hive01.db/
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2015-04-27 04:16 /user/hive/warehouse/hive01.db/tlb01
[hadoop@cloud01 app]$ hadoop fs -ls /user/hive/warehouse/hive01.db/tlb01

其中： /user/hive/warehouse/数据库名称/表名，例如：/user/hive/warehouse/hive01.db/tlb01 中hive01.db为数据库表名，tlb01位表名

7、常见错误

7.1 使用默认Derby存储元数据

错误1：

hive> create database hive02;

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

方案1：

[hadoop@cloud04 bin]$./hive

[hadoop@cloud04 bin]$ more derby.log

java.sql.SQLException: Another instance of Derby may have already booted the database /home/hadoop/app/hive-0.12.0/bin/metastore_db.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:254)
at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:305)

[hadoop@cloud04 bin]$ ps -ef | grep hive
hadoop 4228 2373 0 04:26 pts/0 00:00:13 /home/hadoop/app/jdk1.7.0_65/bin/java -Xmx256m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/home/hadoop/app/hadoop-2.4.1/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/home/hadoop/app/hadoop-2.4.1 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,console -Djava.library.path=/home/hadoop/app/hadoop-2.4.1/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xmx512m -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.util.RunJar /home/hadoop/app/hive-0.12.0/lib/hive-cli-0.12.0.jar org.apache.hadoop.hive.cli.CliDriver
hadoop 4862 4655 0 05:37 pts/2 00:00:00 grep hive
[hadoop@cloud04 bin]$ kill -9 4228

hive> show databases;
OK
default
hive01
Time taken: 4.016 seconds, Fetched: 2 row(s)

发现确实是因为这个原因，该问题解决。总之，无论遇到任何问题，不要着急，要通过日志可以快速定位并给予解决。

参考文章：

http://blog.chinaunix.net/uid-451-id-3143781.html


【大中小】【打印】【繁体】【投稿】【收藏】【推荐】【举报】【评论】【关闭】【返回顶部】

上一篇：hive中使用-e方式执行查询	下一篇：Hive 升级小结