准备环境

1、虚拟机安装三台centos6.5系统的机器

2、修改(setup或者vim /etc/network)主机名分别为hadoop01/hadoop02/hadoop03

a、setup设置

ntp离线安装包centOS7 centos离线包本地安装_hadoop


b、vim /etc/sysconfig/network-scripts/ifcfg-eth0

ntp离线安装包centOS7 centos离线包本地安装_hadoop_02


配置完后ifup eth0 (开启你设置的网卡 不一定是eth0)

c、ifconfig查看是否配置成功

3、关闭防火墙及selinux

a、首先关闭防火墙 service iptables stop

b、设置防火墙开机自动关闭 chkconfig iptables off

c、关闭selinux vim /etc/sysconfig/selinux

ntp离线安装包centOS7 centos离线包本地安装_hadoop_03


需重启机器

4、将三台机器名、IP写入到/etc/hsots文件中

ntp离线安装包centOS7 centos离线包本地安装_ntp离线安装包centOS7_04


5、免密登录

ssh 配置免密码登录

ssh-keygen -t rsa
 ssh-copy-id -i hadoop02/03

6、本地源配置
将CentOS-6.5-x86_64-bin-DVD1.iso上传至主机/opt/iso目录
在/var/www/html/目录下创建 mkdir iso 文件夹
将iso镜像挂在到/var/www/html/iso目录下

mount -o loop /opt/iso/CentOS-6.5-x86_64-bin-DVD1.iso /var/www/html/iso/

启动httpd服务 service httpd start

网址访问验证镜像是否挂载成功 http://主机ip/iso

ntp离线安装包centOS7 centos离线包本地安装_centos_05


进入/etc/yum.repos.d/目录下将原有repo文件都移入bak文件夹

ntp离线安装包centOS7 centos离线包本地安装_cloudera_06


vim centos.repo如下

ntp离线安装包centOS7 centos离线包本地安装_ntp离线安装包centOS7_07


验证iso本地源是否配置成功,显示如下就成功

ntp离线安装包centOS7 centos离线包本地安装_ntp离线安装包centOS7_08


yum install createrepo安装createrepo(createrepo就在刚配置的本地软件库中,所以下载安装非常快)

以上是配置本地软件的本地源安装,接下来是配置本地cloudera安装包访问

进入cd /var/www/html/目录创建cm文件夹 mkdir cm

将cloudera相关rpm文件都移到cm中

ntp离线安装包centOS7 centos离线包本地安装_ntp离线安装包centOS7_09


网址访问是否成功

ntp离线安装包centOS7 centos离线包本地安装_hadoop_10


进入/var/www/html/cm/目录下执行

createrepo . 命令生产repodata文件(createrepo后面有一个点)

配置yum.repos.d文件夹

ntp离线安装包centOS7 centos离线包本地安装_hadoop_11


执行两条命令

yum clean all

yum makecache

执行yum list | grep cloudera命令查看cloudera本地源是否配置成功

ntp离线安装包centOS7 centos离线包本地安装_hadoop_12


将配置好的本地源配置文件centos.repo文件拷贝到hadoop02 hadoop03上

ntp离线安装包centOS7 centos离线包本地安装_cloudera_13


注:在hadoop02 hadoop03 /etc/yum.repos.d/目录下创建bak文件夹,将原有repo文件全部移入到bak中再做以上操作

cdh本地源如上一样操作,在此不做赘述

7、jdk安装(所有机器)

rpm -qa | grep jdk命令查看系统是否有自带的openjdk

ntp离线安装包centOS7 centos离线包本地安装_centos_14


rpm -e –nodeps命令卸载自带jdk

ntp离线安装包centOS7 centos离线包本地安装_vim_15


yum list | grep cloudera查看jdk

ntp离线安装包centOS7 centos离线包本地安装_cloudera_16


yum install oracle-j2sdk1.7.x86_64安装jdk(1.7版本)

设置java环境变量

vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
export PATH=$JAVA_HOME/bin:$PATH
立即生效source /etc/profile

查看是否配置成功

ntp离线安装包centOS7 centos离线包本地安装_cloudera_17


hadoop01&02照以上操作一遍

8、mysql安装(yum方式)rpm方式也可以

yum list installed | grep mysql查看系统是否有自带mysql

yum -y remove mysql-libs.x86_64命令卸载自带mysql

ntp离线安装包centOS7 centos离线包本地安装_centos_18


查看本地源mysql yum list | grep mysql

使用yum -y install mysql-server mysql mysql-devel 命令安装mysql-server、mysql、mysql-devel

rpm -qi mysql-server命令查看刚安装的mysql版本信息

ntp离线安装包centOS7 centos离线包本地安装_hadoop_19


启动MySQL服务 service mysqld start

使用命令设置mysql数据库密码 mysqladmin -uroot password ‘jiahemk(所设置的密码)’

进入mysql

ntp离线安装包centOS7 centos离线包本地安装_vim_20


创建数据

create database scm DEFAULT CHARACTER SET utf8;
create database hive DEFAULT CHARACTER SET utf8;
create database hue DEFAULT CHARACTER SET utf8;
create database rman DEFAULT CHARACTER SET utf8;
create database oozie DEFAULT CHARACTER SET utf8;
grant all privileges on *.* to root@'%' identified by "hduap001"with grant option;
flush privileges;

9、ntp时间同步配置

启动ntp服务 service ntpd start

主机上手动同步一下当前时间 date -s “2017-7-29 22:45:00”

修改时区

vim /etc/sysconfig/clock修改成”Asia/Shanghai”

ntp离线安装包centOS7 centos离线包本地安装_centos_21


更改/etc/ntp.conf文件如下

vim /etc/ntp.conf

ntp离线安装包centOS7 centos离线包本地安装_hadoop_22


重启主机上的ntpd服务 service ntpd restart

修改hadoop01&o2的 ntp.conf文件如下

ntp离线安装包centOS7 centos离线包本地安装_centos_23


启动hadoop01&02ntp服务,然后执行同步

ntp离线安装包centOS7 centos离线包本地安装_centos_24


节点设置定时同步

crontab -e(半小时同步一次)

ntp离线安装包centOS7 centos离线包本地安装_cloudera_25


crontab -l查看时钟

安装CM

1、yum list | grep cloudera 查看cloudera包

ntp离线安装包centOS7 centos离线包本地安装_hadoop_26


yum install 安装 cloudera-server 和 cloudera-agent、cloudera-manager-server-db-2.x86_64

拷贝CDH文件到/opt/cloudera/parcel-repo目录

ntp离线安装包centOS7 centos离线包本地安装_centos_27


拷贝mysql-connector-java-5.1.32-bin.jar到以下目录

cp mysql-connector-java-5.1.32-bin.jar /usr/share/cmf/lib/
cp mysql-connector-java-5.1.32-bin.jar /usr/share/java/

主机启动cloudera-server 和clooudera-agent

/etc/init.d/cloudera-scm-server start或者service cloudera-scm-server start
Starting cloudera-scm-server: [ OK ]
/etc/init.d/cloudera-scm-agent start或者service cloudera-scm-agent start
Starting cloudera-scm-agent: [ OK ]

日志文件位置:

tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
 tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log

报错cloudera-scm-server dead but pid file exists
查看 service cloudera-scm-server status出现以下信息
cloudera-scm-server dead but pid file exists
删除cloudera-scm-server.pid

rm -rf /var/run/cloudera-scm-server.pid

启动service cloudera-scm-server-db start

重新启动service cloudera-scm-server start

web访问http://主机IP:7180(虚拟机配置需等待很长时间)

检查主机时提示

ntp离线安装包centOS7 centos离线包本地安装_vim_28


解决方法:

交换分区设置

sysctl vm.swappiness=10 临时修改

永久修改,/etc/sysctl.conf添加vm.swappiness=10

echo ‘vm.swappiness=10’>> /etc/sysctl.conf(建议系统安装完后就进行此操作)

透明度设置

echo never > /sys/kernel/mm/transparent_hugepage/defrag(临时)

永久生效在/etc/rc.local中添加

if test -f /sys/kernel/mm/redhat_transparent_hugepage/enabled; then

echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

fi

调准文件描述符大小

vim /etc/security/limits.conf

添加

* soft nofile 65535

* hard nofile 65535

* soft nproc 65535

mysql更改root密码

进入mysql 使用命令 use mysql;选择mysql数据库

UPDATE user SET Password=PASSWORD(‘newpassword’) where USER=’root’;