一、官方链接

官方搭建文档:https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/ch_Getting_Ready.html离线搭建yum源,包下载列表:https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/hdp_25_repositories.html适合系统:RedHat 7 CentOS 7 Oracle Linux 7Ambari 2.4.2.0Base URL http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0Repo File http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repoTarball md5 | asc http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari-2.4.2.0-centos7.tar.gzHDP-2.5.3.0HDP Version Definition File (VDF) http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/HDP-2.5.3.0-37.xmlBase URL http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0Repo File http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/hdp.repoTarball md5 | asc http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/HDP-2.5.3.0-centos7-rpm.tar.gzHDP-UTILS 1.1.0.21Base URL http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7Tarball md5 | asc http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz


二、 环境准备

1、主机172.19.80.2 SY-001.hadoop172.19.80.3 SY-002.hadoop172.19.80.4 SY-003.hadoop172.19.80.5 ambari-mysql2、安装基础环境 (在主机SY-001.hadoop,SY-002.hadoop,SY-003.hadoop部署)(1)安装基本包$ yum install openssh wget vim openssh-clients openssl gcc openssh-server mysql-connector-odbc -y$ systemctl enable sshd && systemctl start sshd(2)安装mysql jdbc 驱动$ yum install mysql-connector-java -y(3)安装JDK版本要求:Oracle JDK 1.8 64-bit (minimum JDK 1.8.0_77) (default)Oracle JDK 1.7 64-bit (minimum JDK 1.7_67)OpenJDK 8 64-bit (not supported on SLES)OpenJDK 7 64-bit (not supported on SLES)$ mkdir /usr/local/src/Ambari/$ wget http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz?AuthParam=1480914946_a78f7ba2e70d2f0abed0b0af73c74214$ mkdir -p /opt/java/ && tar xf jdk-8u112-linux-x64.tar.gz -C /opt/java$ vim /etc/profile添加下面内容:export JAVA_HOME=/opt/java/jdk1.8.0_112export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarPATH=$PATH:$HOME/bin:$JAVA_HOME/bin$ source /etc/profile(4)配置SSH免密码登录主节点里root用户登录执行如下步骤$ ssh-keygen$ cd ~/.ssh/$ cat id_rsa.pub >> authorized_keys$ chmod 600 ~/.ssh/authorized_keys把各个主机上公钥id_rsa.pub文件内容放到各个主机authorized_keys中(5)开启NTP服务$ yum -y install ntp$ systemctl is-enabled ntpd$ systemctl enable ntpd$ systemctl start ntpd(6)检查DNS和NSCDambari在安装时需要配置全域名,所以需要检查DNS。为了减轻DNS的负担, 建议在节点里用 Name Service Caching Daemon (NSCD)$ cat /etc/hosts172.19.80.2 SY-001.hadoop172.19.80.3 SY-002.hadoop172.19.80.4 SY-003.hadoop172.19.80.5 ambari-mysql(7)每台节点里配置FQDN,如下以主节点为例$ cat /etc/sysconfig/networkNETWORKING=yesHOSTNAME=SY-001.hadoop(8)关闭防火墙和SELinux$ systemctl disable firewalld$ systemctl stop firewalld临时关闭,不用重启机器:$ setenforce 0修改配置文件让机器重启也生效:$ cat /etc/sysconfig/selinuxSELINUX=disabled


三、搭建数据库(在主机ambari-mysql上部署)

Ambari安装会将安装等信息写入数据库,建议使用自己安装的Mysql数据库,也可以不安装而使用默认数据库PostgreSQL版本要求:- PostgreSQL 8- PostgreSQL 9.1.13+,9.3, 9.4***- MariaDB 10*- MySQL 5.6- Oracle 11gr2- Oracle 12c**按你系统来选择源:http://downloads.mariadb.org/mariadb/repositories/Here is your custom MariaDB YUM repository entry for CentOS. Copy and paste it into a file under /etc/yum.repos.d/ (we suggest naming the file MariaDB.repo or something similar).# MariaDB 10.1 CentOS repository list - created 2016-12-09 06:03 UTC# http://downloads.mariadb.org/mariadb/repositories/[mariadb]name = MariaDBbaseurl = http://yum.mariadb.org/10.1/centos7-amd64gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDBgpgcheck=1After the file is in place, install MariaDB with:$ sudo yum install MariaDB-server MariaDB-clientIf you haven't already accepted the MariaDB GPG key, you will be prompted to do so. See "Installing MariaDB with yum" for detailed information.安装完成后创建ambari数据库及用户,登录root用户执行下面语句:create database ambari character set utf8 ; CREATE USER 'ambari'@'%'IDENTIFIED BY 'Ambari-123';GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';FLUSH PRIVILEGES;如果要安装Hive,再创建Hive数据库和用户 再执行下面的语句:create database hive character set utf8 ; CREATE USER 'hive'@'%'IDENTIFIED BY 'Hive-123';GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';FLUSH PRIVILEGES;如果要安装Oozie,再创建Oozie数据库和用户 再执行下面的语句:create database oozie character set utf8 ; CREATE USER 'oozie'@'%'IDENTIFIED BY 'Oozie-123';GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';FLUSH PRIVILEGES;


四、制作本地源 (在SY-001.hadoop部署)

下载 Ambari HDP HDP-UTILS 源包(文件很大,需要很长下载时间,请耐心等待。。。)$ cd /usr/local/src/Ambari/$ wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari-2.4.2.0-centos7.tar.gz$ wget http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/HDP-2.5.3.0-centos7-rpm.tar.gz$ wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz安装本地源制作相关工具$ yum install yum-utils createrepo -y安装httpd服务$ yum install httpd -y$ systemctl enable httpd && systemctl start httpd$ cd /var/www/html/ && mkdir ambari$ cd /usr/local/src/Ambari/$ tar xf ambari-2.4.2.0-centos7.tar.gz -C /var/www/html/ambari/$ tar xf HDP-2.5.3.0-centos7-rpm.tar.gz -C /var/www/html/ambari/$ tar xf HDP-UTILS-1.1.0.21-centos7.tar.gz -C /var/www/html/ambari/验证httd网站是否可用,可以使用links 命令或者浏览器直接访问下面的地址:$ links http://172.19.80.2/ambari/结果是看到刚才解压的目录配置ambari、HDP、HDP-UTILS的本地源首先下载上面资源列表中的相应repo文件,修改其中的URL为本地的地址,相关配置如下:$ cd /etc/yum.repos.d/$ cat ambari.repo#VERSION_NUMBER=2.4.2.0-136[Updates-ambari-2.4.2.0]name=ambari-2.4.2.0 - Updatesbaseurl=http://172.19.80.2/ambari/AMBARI-2.4.2.0/centos7/2.4.2.0-136gpgcheck=1gpgkey=http://172.19.80.2/ambari/AMBARI-2.4.2.0/centos7/2.4.2.0-136/RPM-GPG-KEY/RPM-GPG-KEY-Jenkinsenabled=1priority=1$ cat hdp.repo#VERSION_NUMBER=2.5.3.0[HDP-2.5.3.0]name=HDP Version - HDP-2.5.3.0baseurl=http://172.19.80.2/ambari/HDP/centos7/2.x/updates/2.5.3.0gpgcheck=1gpgkey=http://172.19.80.2/ambari/HDP/centos7/2.x/updates/2.5.3.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkinsenabled=1priority=1[HDP-UTILS-1.1.0.21]name=HDP Utils Version - HDP-UTILS-1.1.0.21baseurl=http://172.19.80.2/ambari/HDP-UTILS-1.1.0.21/repos/centos7gpgcheck=1gpgkey=http://172.19.80.2/ambari/HDP/centos7/2.x/updates/2.5.0.0/RPM-GPG-KEY/RPM-GPG-KEY-Jenkinsenabled=1priority=1$ yum clean all$ yum makecache$ yum repolist


五、安装Ambari (在SY-001.hadoop部署)

安装Ambari$ yum install ambari-server -y配置Ambari$ ambari-server setup下面是配置执行流程,按照提示操作(1) 提示是否自定义设置。输入:yCustomize user account for ambari-server daemon [y/n] (n)? y(2)ambari-server 账号。输入:ambariEnter user account for ambari-server daemon (root):ambariAdjusting ambari-server permissions and ownership...(3)设置JDK。输入:3Checking JDK...Do you want to change Oracle JDK [y/n] (n)? y[] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7[] Custom JDK==============================================================================Enter choice (1): 3(4)如果上面选择3自定义JDK,则需要设置JAVA_HOME。输入:/opt/java/jdk1.8.0_112WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.Path to JAVA_HOME: /opt/java/jdk1.8.0_112Validating JDK on Ambari Server...done.Completing setup...(5)数据库配置。选择:yConfiguring database...Enter advanced database configuration [y/n] (n)? y(6)选择数据库类型。输入:3Configuring database...==============================================================================Choose one of the following options:[1] - PostgreSQL (Embedded)[2] - Oracle[3] - MySQL[4] - PostgreSQL[5] - Microsoft SQL Server (Tech Preview)[6] - SQL Anywhere==============================================================================Enter choice (3): 3(7)设置数据库的具体配置信息,根据实际情况输入,如果和括号内相同,则可以直接回车。Hostname (localhost):Port ():Database name (ambari):Username (ambari):Enter Database Password (Ambari-123):(8)将Ambari数据库脚本导入到数据库如果使用自己定义的数据库,必须在启动Ambari服务之前导入Ambari的sql脚本用Ambari用户(上面设置的用户)登录mysqlmysql -u ambari -puse ambarisource /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql(9)启动Amabri$ ambari-server start(10)成功启动后在浏览器输入Ambari地址:http://sy-001.hadoop:8080/


六、使用Ambari界面安装Hadoop、Hbase、Log Search服务

1、出现登录界面,默认管理员账户登录, 账户:admin 密码:admin

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_ambari

2、登录成功后出现下面的界面,至此Ambari的安装成功

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_centos_02

3、安装安装HDP 2.5.3 配置集群(点击上面登录成功页面的Launch Install Wizard 按钮进行集群配置)
设置集群名称

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_运维_03

4、置HDP安装源(选择HDP2.5 ,并且设置Advanced Repository Options 的信息,本次使用本地源,所以修改对用系统的安装源为本地源地址)

123

选择RedHat7,填写下面源链接HDP-2.5: http://172.19.80.2/ambari/HDP/centos7/2.x/updates/2.5.3.0HDP-UTILS-1.1.0.21: http://172.19.80.2/ambari/HDP-UTILS-1.1.0.21/repos/centos7

 

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_ambari_04

5、设置集群机器(下载SY-001.hadoop机器上的id_rsa私钥到本地)

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_ambari_05

6、Host确认(确认前面配置集群中hosts列表 中的机器是否都可用,也可以移除相关机器,集群中机器Success后进行下一步操作)

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_ambari_06

7、选择要安装的服务

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_java_07

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_centos_08

8、各个服务Master配置

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_ambari_09

9、服务的Slaves 和 Clients节配置

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_数据库_10

10、服务的客制化配置

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_数据库_11

11、显示配置信息

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_java_12

12、开始安装(安装各个服务,并且完成安装后会启动相关服务,安装过程比较长,如果中途出现错误,请根据具体提示或者log进行操作)

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_数据库_13

13、全部安装成功界面如下

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_ambari_14

14、安装完成

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_centos_15

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_数据库_16



七、Hadooop开启高可用(现在安装好Hadoop是不支高可用的,下面准备开启Hadoop高可用)

1、开启高可用开关

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_运维_17

2、设置高可用名称(如果hbase是启动的话请关闭在开启HA高可用)

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_java_18

3、选择服务安装在那台主机上

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_centos_19

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_java_20

4、按照提示操作,最后显示如下表示HA安装成功

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_运维_21

5、Hbase高可用会显示如下图

使用 Ambari 安装 hadoop spark ambari安装hadoop(centos7)_运维_22