Hive Server 2 安装部署测试(二)
pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2013-09-17 14:59:23,000 INFO metastore.ObjectStore (ObjectStore.java:setConf(205)) - Initialized ObjectStore
2013-09-17 14:59:23,909 INFO metastore.HiveMetaStore (HiveMetaStore.java:logInfo(452)) - 0: get_databases: default
2013-09-17 14:59:23,912 INFO HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(238)) - ugi=hadoop/test84.hadoop@DIANPING.COM ip=unknown-ip-addr cmd=get_databases: default
2013-09-17 14:59:23,933 INFO service.AbstractService (AbstractService.java:start(104)) - Service:ThriftCLIService is started.
2013-09-17 14:59:23,948 INFO service.AbstractService (AbstractService.java:start(104)) - Service:HiveServer2 is started.
2013-09-17 14:59:24,025 INFO security.UserGroupInformation (UserGroupInformation.java:loginUserFromKeytab(633)) - Login successful for user hadoop/test84.hadoop@DIANPING.COM using keytab file /etc/hadoop.keytab
2013-09-17 14:59:24,047 INFO thrift.ThriftCLIService (ThriftCLIService.java:run(435)) - ThriftCLIService listening on test84.hadoop/10.1.77.84:10000
可以看到在HiveServer2已经变成一个CompisiteService了,它包含了一组service,包括OperationManager,SessionManager,CLIService,ThriftCLIService。并且在初始化的时候会建立HiveMetaStore连接,并调用get_databases命令来测试。最后启动thrift server,监听在test84.hadoop/10.1.77.84:10000端口上
1. Beeline访问hive server 2
Beeline是hive 0.11引入的新的交互式CLI,它基于SQLLine,可以作为Hive JDBC Client端访问Hive Server 2,启动一个beeline就是维护了一个session。
由于采用了kerberos认证方式,所以需要在本地有kerberos ticket,并且在connection url中指定hive server 2的service principal,此处为principal=hadoop/test84.hadoop@DIANPING.COM,另外用户名和密码可以不用填写,之后的语句会以当前ticket cache中principal的用户身份来执行。
[plain]
-dpsh-3.2$ bin/beeline
Beeline version 0.11.0 by Apache Hive
beeline> !connect jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM
scan complete in 2ms
Connecting to jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM
Enter username for jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM:
Enter password for jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM:
Connected to: Hive (version 0.11.0)
Driver: Hive (version 0.11.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://test84.hadoop:10000/default> select count(1) from abc;
+------+
| _c0 |
+------+
| 0 |
+------+
1 row selected (29.277 seconds)
0: jdbc:hive2://test84.hadoop:10000/default> !q
Closing: org.apache.hive.jdbc.HiveConnection
thrift client和server会建立一个session handler,有唯一的HandleIdentifier,由SessionManager Service管理,这也就是Hive server 2支持concurrency的方式。每次操作(会有不同的opType,比如EXECUTE_STATEMEN)会生成独立的OperationHandle,也有各自的HandleIdentifier。用户在beeline中输入"!q"会销毁该session,并且销毁相应的资源
ps : 用下来有点不太爽的是执行mapreduce job时候没有执行过程信息,如果是一个执行时间很长的语句,会等很久而没有任何信息反馈。
2. JDBC方式
hive server 1的driver classname是org.apache.hadoop.hive.jdbc.HiveDriver,Hive Server 2的是org.apache.hive.jdbc.HiveDriver,这两个容易混淆。
另外可以在connectionUrl中指定HiveCo