三、LVS 持续性连接

LVS的持久连接:

持久连接即是不考虑LVS的转发方法,确保所有来自同一个用户的连接转发到同一个RealServer上。

 

lvs持久连接适用于大部分调度算法。当某一种请求需要定向到一个real  server 时,就要用到持久连接
一般应用到:ssl(http.https等)、ftp。
-p  //表示此连接为持久连接
N  //表示维持此持久连接的时间。默认6分钟。当超过这个时间后,如果网页还没有关掉,仍处于激活状态,重新复位时间为2分钟。


持久连接的类型:

1.PCC(persistent client connector,持久用户连接)同一个用户所有的请求在超时范围之内都被定位到同一个RealServer上,这个时候在指定端口的时候使用的是0端口,就是所有的请求都转发出去。

[root@localhost ~]# ipvsadm -A -t 10.40.0.51:0 -s rr -p

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:0 -r 192.168.1.11 -g

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:0 -r 192.168.1.12 -g

[root@localhost ~]# ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.40.0.51:0 rr persistent 360

  -> 192.168.1.11:0               Route   1      0          0         

  -> 192.168.1.12:0               Route   1      0          0         


2.PPC(persistent port connector)用户的所有请求在超时范围内按照端口定位到不同的RS上。,只对一个服务进行持久链接。


[root@localhost ~]# ipvsadm -A -t 10.40.0.51:22 -s rr -p

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:22 -r 192.168.1.11:22 -g

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:22 -r 192.168.1.12:22 -g

[root@localhost ~]# ipvsadm -A -t 10.40.0.51:80 -s rr -p

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:80 -r 192.168.1.12:80 -g

[root@localhost ~]# ipvsadm -a -t 10.40.0.51:80 -r 192.168.1.11:80 -g

[root@localhost ~]# ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.40.0.51:22 rr persistent 360

  -> 192.168.1.11:22              Route   1      0          0         

  -> 192.168.1.12:22              Route   1      0          0         

TCP  10.40.0.51:80 rr persistent 360

  -> 192.168.1.11:80              Route   1      0          0         

  -> 192.168.1.12:80              Route   1      0          0         


3.防火墙标记:把相关联的端口在防火墙上打上同样的标记,用户在访问两个相关联的服务的时候,就会定位到同一个RealServer上。
4.FTP connection:由于ftp使用的是两个端口号,所以需要单独列出来。



参考文献:

http://www.360doc.com/content/17/0314/14/29704676_636776330.shtml


http://blog.itpub.net/124805/viewspace-1047686/

https证书及CA创建过程

假设ca放在192.168.1.15上,httpd服务器是192.168.1.11、192.168.1.12

linux集群之LVS入门和企业级实战(续二)_lvs

1、创建CA,并生成自签证书PEM(windwos系统的格式通常是DER)

[root@ 192.168.1.15 CA]# (umask 077; openssl genrsa 2048 >private/cakey.pem)

Generating RSA private key, 2048 bit long modulus

........+++

........................................+++


e is 65537 (0x10001)

[root@ 192.168.1.15 CA]# openssl req -new -x509 -in private/cakey.pem -out cacert1.pem -days 3650

Generating a 2048 bit RSA private key

........................................+++

....................................................+++

writing new private key to 'privkey.pem'

Enter PEM pass phrase:

140021996894024:error:28069065:lib(40):UI_set_result:result too small:ui_lib.c:869:You must type in 4 to 1024 characters

140021996894024:error:0906406D:PEM routines:PEM_def_callback:problems getting password:pem_lib.c:111:

140021996894024:error:0907E06F:PEM routines:DO_PK8PKEY:read key:pem_pk8.c:130:

[root@localhost CA]# openssl req -new -x509 -key private/cakey.pem -out cacert1.pem -days 3650

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [XX]:CN

State or Province Name (full name) []:HENAN

Locality Name (eg, city) [Default City]:ZZ

Organization Name (eg, company) [Default Company Ltd]:MAGEEDU

Organizational Unit Name (eg, section) []:TECH

Common Name (eg, your name or your server's hostname) []:ca.magedu.com  

Email Address []:ca@magedu.com

[root@ 192.168.1.15 CA]# touch index.txt

[root@ 192.168.1.15 CA]# echo “01”>serial


2、创建服务器的证书签发请求CSR

[root@192.168.1.12 ~]# cd /etc/httpd/conf/&& mkdir ssl&& cd ssl &&(umask 077;openssl genrsa 1024 >httpd.key)

Generating RSA private key, 1024 bit long modulus

................++++++

......................................++++++

e is 65537 (0x10001)

[root@192.168.1.12 /etc/httpd/conf/ssl]# openssl req -new -key httpd.key -out httpd.csr

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [XX]:CN

State or Province Name (full name) []:HENAN

Locality Name (eg, city) [Default City]:ZZ

Organization Name (eg, company) [Default Company Ltd]:MAGEEDU

Organizational Unit Name (eg, section) []:TECH

Common Name (eg, your name or your server's hostname) []:test.magedu.com

Email Address []:


Please enter the following 'extra' attributes

to be sent with your certificate request

A challenge password []:

An optional company name []:

[root@192.168.1.12/etc/httpd/conf/ssl]$ls

httpd.csr  httpd.key


3、CA颁发给服务器证书

[root@ 192.168.1.15 CA]# scp -r 192.168.1.12:/etc/httpd/conf/ssl/httpd.csr /root/

root@192.168.1.12's password: 

httpd.csr                                     100%  647     0.6KB/s   00:00

[root@ 192.168.1.15 CA]# openssl ca -in /root/httpd.csr -out /root/httpd.crt -days 265

Using configuration from /etc/pki/tls/openssl.cnf

unable to load number from /etc/pki/CA/serial

error while loading serial number

140408525461320:error:0D066096:asn1 encoding routines:a2i_ASN1_INTEGER:short line:f_int.c:215:

[root@ 192.168.1.15 CA]# ls

cacert1.pem  certs  httpd.csr  newcerts  privkey.pem

cacert.pem   crl    index.txt  private   serial

[root@ 192.168.1.15 CA]# echo "01">serial

[root@192.168.1.15 CA]# cat serial 

01

[root@localhost CA]# openssl ca -in /root/httpd.csr -out /root/httpd.crt -days 265

Using configuration from /etc/pki/tls/openssl.cnf

Check that the request matches the signature

Signature ok

Certificate Details:

        Serial Number: 1 (0x1)

        Validity

            Not Before: Mar 14 12:42:02 2017 GMT

            Not After : Dec  4 12:42:02 2017 GMT

        Subject:

            countryName               = CN

            stateOrProvinceName       = HENAN

            organizationName          = MAGEEDU

            organizationalUnitName    = TECH

            commonName                = test.magedu.com

        X509v3 extensions:

            X509v3 Basic Constraints: 

                CA:FALSE

            Netscape Comment: 

                OpenSSL Generated Certificate

            X509v3 Subject Key Identifier: 

                A8:9F:C2:4C:1D:64:A1:A3:3E:07:C5:2D:81:73:7E:4B:55:F8:CD:A0

            X509v3 Authority Key Identifier: 

                keyid:A3:7A:FD:8A:E0:F0:FF:E0:5F:F1:BE:F8:11:BA:7A:BD:53:8E:58:38


Certificate is to be certified until Dec  4 12:42:02 2017 GMT (265 days)

Sign the certificate? [y/n]:y



1 out of 1 certificate requests certified, commit? [y/n]y

Write out database with 1 new entries

Data Base Updated


4、http的服务器安装配置证书、安装mod_ssl模块。

[root@192.168.1.12 ~]# scp  192.168.1.15:/root/httpd.crt  /etc/httpd/conf/ssl/

root@192.168.1.15s password: 

httpd.crt                                     100% 3066     3.0KB/s   00:00   

[root@192.168.1.12 /etc/httpd/conf/ssl]# yum install mod_ssl



5、修改/etc/httpd/conf.d/ssl.conf配置文件。

#   Server Certificate:

# Point SSLCertificateFile at a PEM encoded certificate.  If

# the certificate is encrypted, then you will be prompted for a

# pass phrase.  Note that a kill -HUP will prompt again.  A new

certificate can be generated using the genkey(1) command.

#SSLCertificateFile /etc/pki/tls/certs/localhost.crt

SSLCertificateFile /etc/httpd/conf/ssl/httpd.crt



#   Server Private Key:

#   If the key is not combined with the certificate, use this

#   directive to point at the key file.  Keep in mind that if

#   you've both a RSA and a DSA private key you can configure

#   both in parallel (to also allow the use of DSA ciphers, etc.)

#SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

SSLCertificateKeyFile /etc/httpd/conf/ssl/httpd.key


# General setup for the virtual host, inherited from global configuration

DocumentRoot "/var/www/html"

#ServerName www.example.com:443



6、对于多台实现负载均衡功能的httpd服务器,可以直接将ssl.conf和证书复制到其他httpd服务器上即可。

[root@192.168.1.12 /etc/httpd/conf/ssl]# scp -rp /etc/httpd/conf/ssl 192.168.1.11:/etc/httpd/conf/

The authenticity of host '192.168.1.11 (192.168.1.11)' can't be established.

RSA key fingerprint is ed:f5:8b:3a:70:4e:d6:7d:16:59:aa:52:14:34:5d:1f.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added '192.168.1.11' (RSA) to the list of known hosts.

root@192.168.1.11's password: 

httpd.crt                                     100% 3066     3.0KB/s   00:00    

httpd.csr                                     100%  647     0.6KB/s   00:00    

httpd.key                                     100%  891     0.9KB/s   00:00    

[root@192.168.1.12 /etc/httpd/conf/ssl]# scp -rp /etc/httpd/conf.d/ssl.conf 192.168.1.11:/etc/httpd/conf.d/

root@192.168.1.11's password: 

ssl.conf                                      100% 9568     9.3KB/s   00:00    


[root@192.168.1.12 /etc/httpd/conf/ssl]# 


[root@192.168.1.12 /etc/httpd/conf/ssl]# netstat -tunlp |grep [80,443]

tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1344/sshd           

tcp        0      0 :::80                       :::*                        LISTEN      3300/httpd          

tcp        0      0 :::22                       :::*                        LISTEN      1344/sshd           

tcp        0      0 :::443                      :::*                        LISTEN      3300/httpd          


linux集群之LVS入门和企业级实战(续二)_lvs_02

参考文献:

证书颁发及文件名后缀介绍


常见ipvs的持久链接,方式为防火墙标记:

[root@localhost ~]# iptables -t mangle -A PREROUTING -d 10.40.0.51 -p tcp --dport 80 -j MARK --set-mark 9

[root@localhost ~]# iptables -t mangle -A PREROUTING -d 10.40.0.51 -p tcp --dport 443 -j MARK --set-mark 9

 

[root@localhost ~]# iptables -L -t mangle

Chain PREROUTING (policy ACCEPT)

target     prot opt source               destination         

MARK       tcp  --  anywhere             10.40.0.51          tcp dpt:http MARK set 0x9 

MARK       tcp  --  anywhere             10.40.0.51          tcp dpt:http MARK set 0x9 

MARK       tcp  --  anywhere             10.40.0.51          tcp dpt:https MARK set 0x9 


Chain INPUT (policy ACCEPT)

target     prot opt source               destination         


Chain FORWARD (policy ACCEPT)

target     prot opt source               destination         


Chain OUTPUT (policy ACCEPT)

target     prot opt source               destination         


Chain POSTROUTING (policy ACCEPT)

target     prot opt source               destination         


[root@localhost ~]# 


[root@localhost ~]# iptables -t mangle -D PREROUTING  2

[root@localhost ~]# 


[root@localhost ~]# ipvsadm -A -f 9 -s rr -p

[root@localhost ~]# ipvsadm -a -f 9 -r 192.168.1.11 -g

[root@localhost ~]# ipvsadm -a -f 9 -r 192.168.1.12 -g


[root@localhost ~]# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

FWM  9 rr persistent 360

  -> 192.168.1.11:0               Route   1      0          0         


  -> 192.168.1.12:0               Route   1      0          0     


[root@localhost ~]# ipvsadm -Lc

IPVS connection entries

pro expire state       source             virtual            destination

TCP 00:58  SYN_RECV    10.40.0.208:call-logging 10.40.0.51:http    192.168.1.12:http

TCP 00:17  SYN_RECV    10.40.0.208:ms-v-worlds 10.40.0.51:http    192.168.1.12:http

TCP 00:18  SYN_RECV    10.40.0.208:ncr_ccl 10.40.0.51:http    192.168.1.12:http

TCP 00:58  SYN_RECV    10.40.0.208:vytalvaultpipe 10.40.0.51:http    192.168.1.12:http

TCP 00:54  SYN_RECV    10.40.0.208:pclemultimedia 10.40.0.51:http    192.168.1.12:http

IP  05:51  NONE        10.40.0.208:0      0.0.0.9:0          192.168.1.12:0

TCP 00:17  SYN_RECV    10.40.0.208:ema-sent-lm 10.40.0.51:http    192.168.1.12:http

[root@localhost ~]# 


LVS在director服务器的防火墙工作流程

 NAT :

进入:PREROUTING-OUTPUT-POSTROUTING

返回:PREROUTING-FORWARD-POSTROUTING