一、案例概述

本案例采用四层模式实现,主要分为前端反向代理、web层、数据库缓存层和数据库层。

  • 前端反向代理采用主备模式
  • web层采用群集模式
  • 数据库缓存层采用主备模式
  • 数据库层采用主从模式

大型网站架构之百万PV网站架构案例
由于实验条件限制,本次实验共打开四台虚拟机,此处实验将前端代理层、数据库缓存层、数据库层服务搭建在前两台虚拟服务器上,web层采用群集模式,用于单独放置两台虚拟机。故本次实验实际模型为了模拟实际环境,服务搭建按照如下拓扑搭建。
大型网站架构之百万PV网站架构案例

二、实验环境

主机名 操作系统 IP地址 用途
server1 centosx84_64 192.168.144.112 前端反向代理Nginx、Redis缓存主机、MySQL主数据库
server2 centosx84_64 192.168.144.111 前端反向代理备Nginx、Redis备缓存主机、MySQL备数据库
web1 centosx84_64 192.168.144.113 Web服务tomcat
web2 centosx84_64 192.168.144.114 Web服务tomcat

三、实验部署

3.1、master和slave设置keepalive与Nginx反向代理

  • 配置yum源

rpm -ivh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm

  • 安装keepalive软件和Nginx

yum install -y keepalived nginx

3.1.1配置keepalive

  • 配置主从的keepalive,两台配置基本相同,配置文件不同处已做说明

vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
    route_id NGINX_HA    //主从不同
}

vrrp_script nginx {
    script "/opt/shell/nginx.sh"    //配置带动Nginx启动脚本
    interval 2   //每隔2s响应
}

vrrp_instance VI_1 {
    state MASTER    //备为BACKUP
    interface ens33 //注意主机网卡端口名称
    virtual_router_id 51   //从需不同
    priority 100    //从需比主低
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
}

track_script {
    nginx        //调度Nginx启动脚本的函数名
}

virtual_ipaddress {
    192.168.144.188    //设置虚拟IP
    }
}

3.1.2创建keepalive带动Nginx启动脚本

mkdir -p /opt/shell
vim /opt/shell/nginx.sh

#!/bin/bash
k=`ps -ef | grep keepalived | grep -v grep | wc -l`
if [ $k -gt 0 ];then
    /bin/systemctl start nginx.service
else
/bin/systemctl stop nginx.service
fi

chmod +x /opt/shell/nginx.sh

3.1.3配置Nginx前端调度功能

vim /etc/nginx/nginx.conf

  • 在include 上面一行新增
upstream tomcat_pool {
                server 192.168.144.113:8080;
                server 192.168.144.114:8080;
                ip_hash;        //会话稳固功能,否则无法通过vip地址登陆
        }
        server {
                listen 80;
                server_name 192.168.144.188;  //虚拟IP
                location / {
                        proxy_pass http://tomcat_pool;
                        proxy_set_header X-Real-IP $remote_addr;
                }
        }
  • 配置完成后,检查Nginx配置文件语法

nginx -t -c /etc/nginx/nginx.conf

  • 关闭防火墙和SELinux,准备启动keepalive,随后会通过配置文件带动脚本,启动Nginx,此处需要注意,若要停止Nginx,则需要先关闭keepalive,然后才可以。

  • 查看Nginx是否启动

netstat -ntap | grep nginx

3.2、部署两台web服务器

3.2.1两台web服务器部署tomcat,部署步骤完全相同,为了试验区分,需要在首页内容做区别

tar xf apache-tomcat-8.5.23.tar.gz
tar xf jdk-8u144-linux-x64.tar.gz
mv jdk1.8.0_144/ /usr/local/java
mv apache-tomcat-8.5.23/ tomcat8

  • 解压完成后,配置jdk环境变量,让jdk的各种命令为系统识别

vim /etc/profile

export JAVA_HOME=/usr/local/java
export JRE_HOME=/usr/local/java/jre
export PATH=$PATH:/usr/local/java/bin
export CLASSPATH=./:/usr/local/java/lib:/usr/local/java/jre/lib

source /etc/profile

  • 配置tomcat启动与关闭命令为系统识别

    ln -s /usr/local/tomcat8/bin/startup.sh /usr/bin/tomcatup
    ln -s /usr/local/tomcat8/bin/shutdown.sh /usr/bin/tomcatdown

  • 启动服务,查看端口状态,使用自身网页测试,观察服务是否能够提供。

tomcatup
netstat -anpt | grep 8080

http://192.168.144.113:8080/ //测试默认测试页是否正常显示
http://192.168.144.114:8080/

  • 为了试验区分两台web服务器,修改首页内容

vim /usr/local/tomcat8/webapps/ROOT/index.jsp

<h1>Server 129!!</h1>    //注意,web2需要首页内容不同
  • 输入调度器地址,也就是虚拟地址,测试两台节点的调度情况。

http://192.168.175.188/

大型网站架构之百万PV网站架构案例

3.2.2搭建会员商城

  • 首先在tomcat配置文件中添加支持商城模块

cd /usr/local/tomcat8/conf/
vim server.xml

  • 跳到行尾,在Host name下新增 在148行位置

    <Context path="" docBase="SLSaleSystem" reloadable="true" debug="0"></Context>
    //日志调试信息debug为0表示信息越少,docBase指定访问目录
  • 将会员商城软件包解压缩

tar zxvf SLSaleSystem.tar.gz -C /usr/local/tomcat8/webapps/

cd /usr/local/tomcat8/webapps/SLSaleSystem/WEB-INF/classes

vim jdbc.properties //修改数据库IP地址是VRRP的虚拟IP,以及授权的用户名root和密码abc123。

driverClassName=com.mysql.jdbc.Driver
url=jdbc\:mysql\://192.168.144.188\:3306/slsaledb?useUnicode\=true&characterEncoding\=UTF-8  //该成我们设定的虚拟IP
uname=root
password=123456
minIdle=10
maxIdle=50
initialSize=5
maxActive=100
maxWait=100
removeAbandonedTimeout=180
removeAbandoned=true
  • 客户端测试

http://192.168.144.113:8080/ //默认的用户名admin 密码:123456
http://192.168.144.114:8080/

大型网站架构之百万PV网站架构案例

http://192.168.175.188 //输入虚拟地址测试登录,并且关闭主再测试登录

大型网站架构之百万PV网站架构案例

3.3、部署MySQL以及主从

3.3.1安装mysql

  • 在实际生产环境中,此处应当搭建MySQL服务,但由于实验条件限制,本实验使用mariadb代替MySQL。

yum install -y mariadb-server mariadb

systemctl start mariadb.service
systemctl enable mariadb.service

netstat -anpt | grep 3306

mysql_secure_installation //

Set root password? [Y/n] y         //设置MySQL管理员账户的密码,我选择密码为abc123
New password: 
Re-enter new password: 
Password updated successfully!
Reloading privilege tables..
 ... Success!

By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] n        //删除匿名用户?
 ... skipping.

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] n   //拒绝root用户远程登录?
 ... skipping.

By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] n  //删除test数据库?
 ... skipping.

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] y    //重新加载数据库的所有表?
 ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!

mysql -uroot -p //进入数据库

3.3.2导入数据库,授权

mysql -u root -p < slsaledb-2014-4-10.sql
mysql -uroot -p

show databases;

GRANT all ON slsaledb.* TO 'root'@'%' IDENTIFIED BY 'abc123';    //授予slsaledb数据库所有表所有权限给root用户在任意网段登录,密码为abc123

flush privileges; 

3.3.3 MySQL主从配置

  • mysql主服务器下。

vim /etc/my.cnf

[mysqld]下添加

binlog-ignore-db=mysql,information_schema       //二进制日志格式
character_set_server=utf8
log_bin=mysql_bin  //开启二进制日志
server_id=1  
log_slave_updates=true
sync_binlog=1   //同步日志

systemctl restart mariadb

netstat -anpt | grep 3306

mysql -u root -p

show master status; //记录日志文件名称和 位置值

+------------------+----------+--------------+--------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB         |
+------------------+----------+--------------+--------------------------+
| mysql_bin.000001 |      626 |              | mysql,information_schema |
+------------------+----------+--------------+--------------------------+

grant replication slave on *.* to 'rep'@'192.168.144.%' identified by '123456';  //授予主从状态

flush privileges;
  • MySQL从服务器下

vim /etc/my.cnf

[mysqld]下添加

server_id=2

systemctl restart mariadb

netstat -anpt | grep 3306
mysql -u root -p

change master to master_host='192.168.144.112',master_user='rep',master_password='123456',master_log_file='mysql_bin.000001',master_log_pos=2626;

start slave;

show slave status;

            Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
  • 测试主从状态是否可行,方可进行下一步骤。

3.4部署Redis数据库缓存层

3.4.1安装redis

  • 很对主从redis数据库缓存,首先需要安装redis软件

yum install -y epel-release //安装扩展源
yum install redis -y

3.4.2设置主从关系

  • 配置主Redis

vim /etc/redis.conf

bind 0.0.0.0   //将监听网址修改成任意网段

systemctl start redis.service
netstat -anpt | grep 6379

  • 测试本身安装完成能否使用

redis-cli -h 192.168.144.112 -p 6379 //测试连接

192.168.144.112:6379> set name test //设置name 值是test

192.168.144.112:6379> get name //获取name值

  • 配置从Redis

vim /etc/redis.conf

bind 0.0.0.0   //61行,修改监听地址
...
slaveof 192.168.114.112  6379 //266行下添加主服务器的IP,不是虚拟IP
  • 两台redis服务器服务重启

systemctl restart redis.service

  • 从服务器上进入redis,发现已经复制完成

redis-cli -h 192.168.144.111 -p 6379
192.168.144.111:6379> get name
"test"

  • 至此,redis主从配置完成。

3.4.3配置商城项目中连接redis的参数

  • web节点中配置商城项目,指定Redis虚拟IP

vim /usr/local/tomcat8/webapps/SLSaleSystem/WEB-INF/classes/applicationContext-mybatis.xml

      <!--redis 配置 开始-->

         <constructor-arg value="192.168.144.188"/>    //47行
         <constructor-arg value="6379"/>              //48行

3.4.4测试缓存效果

redis-cli -h 192.168.144.188 -p 6379

192.168.175.188:6379> info

keyspace_hits:1  或者 keyspace_misses:2//关注这个值,命中数和未命中数

登录商城,然后反复点击需要数据库参与的操作页面,再回来检查keyspace_hits或者keyspace_misses: 值变化。