大数据集群环境搭建之一 hadoop-ha高可用安装 - 数据库编程

TOP

大数据集群环境搭建之一 hadoop-ha高可用安装(四)

2019-10-10 18:15:10 【大中小】浏览:256次

h start journalnode ============ starting journalnode, logging to /export/servers/hadoop-2.8.5/logs/hadoop-root-journalnode-jiang01.out 命令执行成功 ============= jiang02 : hadoop-daemon.sh start journalnode ============ starting journalnode, logging to /export/servers/hadoop-2.8.5/logs/hadoop-root-journalnode-jiang02.out 命令执行成功 ============= jiang03 : hadoop-daemon.sh start journalnode ============ starting journalnode, logging to /export/servers/hadoop-2.8.5/logs/hadoop-root-journalnode-jiang03.out 命令执行成功 [root@jiang01 servers]# 启动journalnode

先选取一个namenode（jiang01）节点进行格式化

[root@jiang01 servers]# hadoop namenode -format

View Code

格式化zkfc，只能在nameonde节点进行

主节点上面启动 dfs文件系统：

[root@jiang01 dfs]# start-dfs.sh

jiang002启动yarm

[root@jiang02 mapreduce]# start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /export/servers/hadoop-2.8.5/logs/yarn-root-resourcemanager-jiang02.out
jiang03: starting nodemanager, logging to /export/servers/hadoop-2.8.5/logs/yarn-root-nodemanager-jiang03.out
jiang01: starting nodemanager, logging to /export/servers/hadoop-2.8.5/logs/yarn-root-nodemanager-jiang01.out
jiang02: starting nodemanager, logging to /export/servers/hadoop-2.8.5/logs/yarn-root-nodemanager-jiang02.out
[root@jiang02 mapreduce]#

View Code

jiang03启动：resourcemanager

[root@jiang03 hadoopDatas]#  yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /export/servers/hadoop-2.8.5/logs/yarn-root-resourcemanager-jiang03.out

View Code

hadoop wordcount程序启动：

1 cd /export/servers/hadoop-2.8.5/share/hadoop/mapreduce/

2 生成数据文件：

touch word.txt
echo "hello world" >> word.txt
echo "hello hadoop" >> word.txt
echo "hello hive" >> word.txt

3 创建hadoop 文件目录

hdfs dfs -mkdir -p /work/data/input

4 向hadoop上传数据文件

hdfs dfs -put ./word.txt /work/data/input

5 计算例子

hadoop jar hadoop-mapreduce-examples-2.8.5.jar wordcount /work/data/input /work/data/output

6 查看结果：

[root@jiang01 mapreduce]# hadoop jar hadoop-mapreduce-examples-2.8.5.jar wordcount /work/data/input /work/data/output
19/10/09 11:44:48 INFO input.FileInputFormat: Total input files to process : 1
19/10/09 11:44:48 INFO mapreduce.JobSubmitter: number of splits:1
19/10/09 11:44:48 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1570635804389_0001
19/10/09 11:44:48 INFO impl.YarnClientImpl: Submitted application application_1570635804389_0001
19/10/09 11:44:48 INFO mapreduce.Job: The url to track the job: http://jiang02:8088/proxy/application_1570635804389_0001/
19/10/09 11:44:48 INFO mapreduce.Job: Running job: job_1570635804389_0001
19/10/09 11:45:00 INFO mapreduce.Job: Job job_1570635804389_0001 running in uber mode : false
19/10/09 11:45:00 INFO mapreduce.Job:  map 0% reduce 0%
19/10/09 11:45:11 INFO mapreduce.Job:  map 100% reduce 0%
19/10/09 11:45:20 INFO mapreduce.Job:  map 100% reduce 100%
19/10/09 11:45:20 INFO mapreduce.Job: Job job_1570635804389_0001 completed successfully
19/10/09 11:45:21 INFO mapreduce.Job: Counters: 49
        File System Counters
                FILE: Number of bytes read=54
                FILE: Number of bytes writte

首页上一页 1 2 3 4 5 下一页尾页 4/5/5
【大中小】【打印】【繁体】【投稿】【收藏】【推荐】【举报】【评论】【关闭】【返回顶部】

上一篇：PostgreSQL 常用函数	下一篇：SQLyog连接MySQL8.0报2058错误的..