Hadoop的HA高可用（可行）
一、集群的规划
Zookeeper集群
192.168.116.121 | 192.168.116.122 | 192.168.116.123 |
hsiehchou121 | hsiehchou122 | hsiehchou123 |
Hadoop集群
192.168.116.121 | 192.168.116.122 | 192.168.116.123 | 192.168.116.124 |
hsiehchou121 | hsiehchou122 | hsiehchou123 | hsiehchou124 |
NameNode1 | NameNode2 | DataNode1 | DataNode2 |
ResourceManager1 | ResourceManager2 | NodeManager1 | NodeManager2 |
Journalnode | Journalnode |
二、准备工作
1、安装JDK
2、配置环境变量
3、配置免密码登录
4、配置主机名
三、配置Zookeeper（在192.168.116.121安装）
在主节点（hsiehchou121）上配置ZooKeeper
1、配置/root/hd/zookeeper-3.4.10/conf/zoo.cfg文件
2、在/root/training/zookeeper-3.4.6/tmp目录下创建一个myid的空文件
echo 1 > /root/hd/zookeeper-3.4.10/tmp/myid
3、将配置好的zookeeper拷贝到其他节点，同时修改各自的myid文件
scp -r /root/hd/zookeeper-3.4.10/ hsiehchou122:/root/hd
scp -r /root/hd/zookeeper-3.4.10/ hsiehchou123:/root/hd
四、安装Hadoop集群（在hsiehchou121上安装）
1、修改hadoo-env.sh
2、修改core-site.xml
3、修改hdfs-site.xml（配置这个nameservice中有几个namenode）
4、修改mapred-site.xml
5、修改yarn-site.xml
6、修改slaves
hsiehchou123
hsiehchou124
7、将配置好的hadoop拷贝到其他节点
scp -r /root/hd/hadoop-2.8.4/ root@hsiehchou122:/root/hd/
scp -r /root/hd/hadoop-2.8.4/ root@hsiehchou123:/root/hd/
scp -r /root/hd/hadoop-2.8.4/ root@hsiehchou124:/root/hd/
五、启动Zookeeper集群
1、格式化zookeeper
[root@hsiehchou121 hadoop-2.8.4]# hdfs zkfc -formatZK
2、启动Zookeeper集群
[root@hsiehchou121 hadoop-2.8.4]# zkServer.sh start
[root@hsiehchou122 hadoop-2.8.4]# zkServer.sh start
[root@hsiehchou123 hadoop-2.8.4]# zkServer.sh start
六、在和hsiehchou122上启动journalnode
hadoop-daemon.sh start journalnode
七、格式化HDFS（在hsiehchou121上执行）
1. 格式化zookeeper
[root@hsiehchou121 hadoop-2.8.4]# hdfs zkfc -formatZK
2、启动hdfs
1）在各个JournalNode节点上，输入以下命令启动journalnode服务
[root@hsiehchou121 hadoop-2.8.4]# sbin/hadoop-daemon.sh start journalnode
2）在[nn1]上，对其进行格式化，并启动
[root@hsiehchou121 hadoop-2.8.4]# bin/hdfs namenode -format
[root@hsiehchou121 hadoop-2.8.4]# sbin/hadoop-daemon.sh start namenode
3）在[nn2]上，同步nn1的元数据信息
[root@hsiehchou121 hadoop-2.8.4]# bin/hdfs namenode -bootstrapStandby
八、在hsiehchou121上启动Hadoop集群
[root@hsiehchou121 hadoop-2.8.4]# start-all.sh
日志
This script is Deprecated. Instead use start-dfs.sh and start-yar
Starting namenodes on [hsiehchou121 hsiehchou122]
hsiehchou121: starting namenode, logging to /root/hd/hadoop-2.8.4-hsiehchou121.out
hsiehchou122: starting namenode, logging to /root/hd/hadoop-2.8.4-hsiehchou122.out
hsiehchou124: starting datanode, logging to /root/hd/hadoop-2.8.4-hsiehchou124.out
hsiehchou123: starting datanode, logging to /root/hd/hadoop-2.8.4-hsiehchou123.out
Starting journal nodes [hsiehchou121 hsiehchou122 ]
hsiehchou121: starting journalnode, logging to /root/hd/hadoop-2.alnode-hsiehchou121.out
hsiehchou122: starting journalnode, logging to /root/hd/hadoop-2.alnode-hsiehchou122.out
Starting ZK Failover Controllers on NN hosts [hsiehchou121 hsiehc
hsiehchou121: starting zkfc, logging to /root/hd/hadoop-2.8.4/logou121.out
hsiehchou122: starting zkfc, logging to /root/hd/hadoop-2.8.4/logou122.out
starting yarn daemons
starting resourcemanager, logging to /root/hd/hadoop-2.8.4/logs/ysiehchou121.out
hsiehchou123: starting nodemanager, logging to /root/hd/hadoop-2.ager-hsiehchou123.out
hsiehchou124: starting nodemanager, logging to /root/hd/hadoop-2.ager-hsiehchou124.outhsiehchou122上的ResourceManager需要单独启动
命令
[root@hsiehchou121 hadoop-2.8.4]# ./sbin/yarn-daemon.sh start resourcemanager
