基础环境配置
基本按照官网文档来配置。因为以下配置都是在root下进行,所以没有提权sudo。
1. IP设置
修改网络信息
vim /etc/sysconfig/network-scripts/ifcfg-enp5s0
修改以下选项
ONBOOT=yes开机自动启动
IPADDR:IP地址
* GATEWAY:默认网关
* DNS1,DNS2:DNS服务器
重启网络
systemctl restart network
测试网络
ping baidu.com
如果要修改ech0，显示ech0网卡
vim /etc/default/grub
GRUB_CMD_LINE_LINUX=""中插入"net.ifnames=0 biosdevname=0"
更新配置
grub2-mkconfig -o /boot/grub2/grub.cfg
重启服务器
reboot
查看到eth0网卡
ifconfig -a
2. 主机名及内网映射
查看主机名
hostname
uname –n
修改主机名
hostnamectl set-hostname cdh-master
或者
vim /etc/hostname
查看ip
ip addr
ifconfig
配置内网映射
vim /etc/hosts
添加
192.168.100.45 cdh-master
192.168.100.46 cdh-slave01
192.168.100.47 cdh-slave02
如果有免密
scp -r /etc/hosts root@cdh-slave01:/etc/
scp -r /etc/hosts root@cdh-slave02:/etc/
3. 关闭防火墙
7.0以后默认firewalld,但运维安装了iptables
查看状态
systemctl status firewalld
systemctl status iptables
停止
systemctl stop firewalld
systemctl stop iptables
禁止开机启动
systemctl disable firewalld
systemctl disable iptables
4. 禁用Selinux(重启生效)
查看Selinux状态
sestatus/getenforce
vim /etc/sysconfig/selinux
或者
vim /etc/selinux/config
修改配置项
SELINUX=disabled
临时关闭
setenforce 0
shell方式：
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
5. 配置ntp时间同步服务
cdh-master设置为ntp server，cdh-slaves作为ntp client
cdh-master同步外网时间，cdh-slaves同步cdh-master时间
安装
yum install ntp
配置cdh-master：
vim /etc/ntp.conf
# 允许内网其他机器同步时间，如果不添加该约束默认允许所有IP访问本机同步服务
# IP为本局域网内的子网IP
restrict 192.168.100.0 mask 255.255.255.0 nomodify notrap
#配置和上游标准时间同步
server 210.72.145.44 # 中国国家授时中心
server 0.cn.pool.ntp.org
server 1.cn.pool.ntp.org
server 2.cn.pool.ntp.org
server 3.cn.pool.ntp.org
#如果外部时间服务器不可用，NTP Server以本地时间作为时间服务器
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
配置cdh-slaves
#配置和上游标准时间同步
Server 192.168.100.45
#配置允许上游时间服务器主动修改本机的时间
restrict 192.168.100.45 nomodify notrap noquery
其他
同步时间（先同步再启动 ntpd 服务，能够加速同步）
ntpdate ntp1.aliyun.com
从节点同步主节点
ntpdate 192.168.100.45
设置完成启动ntp服务器，5-10分钟会同步成功
systemctl start ntpd
开机启动
systemctl enable ntpd
重启ntp服务器
systemctl restart ntpd
查看服务
systemctl status ntpd
查看当前时间和NTP服务的同步状态
timedatectl
查看时间同步状态
ntpstat
查看同步情况
# * 表示同步成功的上层NTP服务器， + 表示已经连接成功，可以作为下一个提供时间更新的候选者
ntpdc -np
ntpq -p
设置硬件时间与系统时间同步
hwclock --systohc
6. 配置SSH免密登录
一般设置
1.ssh-keygen -t rsa （三次回车）
2.发送到其他节点
ssh-copy-id cdh-master
ssh-copy-id cdh-slave01
ssh-copy-id cdh-slave02
3.验证
ssh cdh-master
ssh cdh-slave01
ssh cdh-slave02
exit
其他方式设置
1. 设置config
vim /etc/ssh/sshd_config
PermitRootLogin yes # 允许root用户登录
PasswordAuthentication no # 不使用密码登录
systemctl restart sshd.service
2. 只在cdh-master服务器
ssh-keygen -t rsa
3. 生成authorized_keys
cat /root/.ssh/id_rsa.pub >> authorized_keys
4. 下载 私匙id_rsa 和 authorized_keys 文件,并上传到其他节点~/.ssh目录下
chmod 700 id_rsa
mkdir /root/.ssh
5. 验证
ssh cdh-slave01
ssh cdh-slave01
exit
7. 安装python2.7
Centos7.3自带python2.7，附上安装步骤
yum install centos-release-scl
yum install scl-utils
yum install python27
source /opt/rh/python27/enable
python --version
8. 安装JDK
必须安装在 /usr/java/
* 因为是通过跳板机登录，不能直接使用sftp，所以使用rzsz上传
* yum -y install lrzsz
1.检查自带JDK
java –version
rpm –qa | grep java
2.卸载
rpm –e –-odeps xx
3.解压
mkdir /usr/java
tar -zxvf jdk-8u211inux-x64.tar.gz -C /usr/java/
4.配置环境变量
vim /etc/profile
添加
export JAVA_HOME=/usr/java/jdk1.8.0_211
export CLASSPATH=$JAVA_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin
5.生效
source /etc/profile
9. jdbc
上传jdbc到/usr/share/java/目录下，重命名为mysql-connector-java.jar
也可以在线下载解压
wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.47.tar.gz
tar zxvf mysql-connector-java-5.1.47.tar.gz
mkdir -p /usr/share/java/
cp /jars/mysql-connector-java-8.0.16-bin.jar /usr/share/java/mysql-connector-java.jar
scp /usr/share/java/mysql-connector-java.jar cdh-slaves01:/usr/share/java
10. 禁用透明大页面压缩
查看
cat /sys/kernel/mm/transparent_hugepage/defrag
修改
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled
将命令写入开机启动
vim /etc/rc.local
11. 设置swapping交换区
查看
cat /proc/sys/vm/swappiness
修改
echo "vm.swappiness = 1" >> /etc/sysctl.conf
生效
sysctl -p
12. 禁用tuned系统调优服务
查看tuned状态
systemctl status tuned
查看活动的服务
tuned-adm list
关闭tuned服务
tuned-adm off
确定没有活动的服务（No current active profile）
tuned-adm list
关闭并禁止开启启动
systemctl stop tuned
systemctl disable tuned
13. 安装MySQL(只在master，离线方式)
因为mysql安装和设置稍麻烦，所以放到最后。
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
5.7 授权
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456';
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '123456';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456';
FLUSH PRIVILEGES;
8.0 授权
create user 'scm'@'%' identified by '123456';
create user 'amon'@'%' identified by '123456';
create user 'rman'@'%' identified by '123456';
create user 'hue'@'%' identified by '123456';
create user 'hive'@'%' identified by '123456';
create user 'sentry'@'%' identified by '123456';
create user 'nav'@'%' identified by '123456';
create user 'navms'@'%' identified by '123456';
create user 'oozie'@'%' identified by '123456';
grant all on scm.* to 'scm'@'%';
grant all on amon.* to 'amon'@'%';
grant all on rman.* to 'rman'@'%';
grant all on hue.* to 'hue'@'%';
grant all on metastore.* to 'hive'@'%';
grant all on sentry.* to 'sentry'@'%';
grant all on nav.* to 'nav'@'%';
grant all on navms.* to 'navms'@'%';
grant all on oozie.* to 'oozie'@'%';
FLUSH PRIVILEGES;
查看授权是否正确，必须验证
show grants for 'scm'@'%';
show grants for 'amon'@'%';
show grants for 'rman'@'%';
show grants for 'hue'@'%';
show grants for 'hive'@'%';
show grants for 'sentry'@'%';
show grants for 'nav'@'%';
show grants for 'navms'@'%';
show grants for 'oozie'@'%';
