一、环境:



系统环境:centos 5.8 x86_64



应用软件:所以应用软件都放在/usr/local/src目录下







pcre-8.12.tar.gz 
  
 
  

             apache-tomcat-7.0.27.tar.gz


















各应用软件官方站点:


nginx: http://www.nginx.org 
  
 
  

    nginx_ajp: https://github.com/yaoweibin/nginx_ajp_module 
  
 
  

    pcre: http://pcre.org/ 
  
 
  

    apache-tomcat: http://tomcat.apache.org 
  
 
  

    jdk: http://www.oracle.com/technetwork/java/javase/downloads/index.html


所需服务器:



192.168.8.10   -->安装nginx,作为分发器



 server2:jvm1服务器 192.168.8.11   -->安装apache-tomcat、jdk



 server3:jvm2服务器 192.168.8.12   -->安装apache-tomcat、jdk






二、安装nginx+ajp



1. 1.建立运行用户 
2. groupadd www 
3. useradd -g www -s /sbin/nologin -M www 
4. cd /usr/local/src 
5. 2.安装pcre 
6. tar zxf pcre-8.12.tar.gz 
7. cd pcre-8.12 
8. ./configure 
9. make ; make install 
10. 3.安装nginx_ajp和nginx 
11. unzip yaoweibin-nginx_ajp_module-v0.2.5-17-ge971b84.zip 
12. tar zxf nginx-1.2.2.tar.gz 
13. cd nginx-1.2.2 
14. patch -p1 < ../yaoweibin-nginx_ajp_module-e971b84/ajp.patch 
15. ./configure --prefix=/usr/local/nginx --user=www --group=www  --with-http_stub_status_module  --with-http_ssl_module --without-mail_pop3_module --without-mail_imap_module --without-http_uwsgi_module --without-http_scgi_module  --without-http_memcached_module --without-http_autoindex_module --without-http_ssi_module --add-module=../yaoweibin-nginx_ajp_module-e971b84 
16. make ; make install

三、安装tomcat、jdk



1. 1.安装tomcat 
2. cd /usr/local/src 
3. tar zxf apache-tomcat-7.0.27.tar.gz 
4. mv apache-tomcat-7.0.27 /usr/local/tomcat 
5. 2.安装jdk 
6. tar zxf jdk-7u4-linux-x64.gz 
7. mv jdk1.7.0_04 /usr/local/jdk 
8. 3.设置java、tomcat运行环境变量 
9. vi /etc/profile 
10. 在文件最后加入如下内容: 
11. #Set java & tomcat Environment
12. JAVA_HOME=/usr/local/jdk 
13. export
14. JRE_HOME=/usr/local/jdk/jre 
15. export
16. CLASSPATH=/usr=/usr/local/tomcat/common/lib/:/usr/local/jdk/lib:/usr/local/jdk/jre/lib 
17. export
18. PATH=$PATH:/usr/local/tomcat/bin/:/usr/local/apache/bin:/usr/local/jdk/bin:/usr/local/jdk/jre/bin 
19. export
20. TOMCAT_HOME=/usr/local/tomcat 
21. export
22. 让环境变量立即生效: 
23. source /etc/profile

四、安装apr、native



1. (1)安装apr 
2. wget http://archive.apache.org/dist/apr/apr-1.4.5.tar.gz
3. tar zxf apr-1.4.5.tar.gz  
4. cd apr-1.4.5 
5. ./configure --prefix=/usr/local/apr 
6. make 
7. make install 
8. (2)安装apr-iconv 
9. wget http://archive.apache.org/dist/apr/apr-iconv-1.2.1.tar.gz
10. tar -zxf apr-iconv-1.2.1.tar.gz 
11. cd apr-iconv-1.2.1 
12. ./configure --prefix=/usr/local/apr-iconv --with-apr=/usr/local/apr  
13. make 
14. make install 
15. (3)安装apr-util 
16. wget http://archive.apache.org/dist/apr/apr-util-1.3.10.tar.gz
17. tar zxf apr-util-1.3.10.tar.gz 
18. cd apr-util-1.3.10 
19. ./configure --prefix=/usr/local/apr-util  --with-apr=/usr/local/apr --with-apr-iconv=/usr/local/apr-iconv/bin/apriconv  
20. make 
21. make install 
22. (4)安装tomcat-native
23. 安装好tomcat7之后,tomcat-native已经在tomcat的bin目录下了,可以直接使用 
24. cd /usr/local/tomcat/bin    
25. tar zxf tomcat-native.tar.gz  
26. cd tomcat-native-1.1.23-src/jni/native
27. ./configure --with-apr=/usr/local/apr --with-java-home=/usr/local/jdk  
28. make   
29. make install
30. 
31. (5)设置 apr 的环境变量:
32. vi /etc/profile
33. 在文件后面添加以下内容:
34. export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/apr/lib
35. 
36. 让配置生效: source /etc/profile

PS:对于为什么要安装apr和native,是apache-tomcat-7.0.27这个版本需要这些包的支持,不然启动会出现如下错误:






cat /usr/local/tomcat/logs/catalina.out 
 
  
Jul 27, 2012 7:19:34 PM org.apache.catalina.core.AprLifecycleListener init


INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib




安装上面几个包即可解决!!!!!!!!!!!




四、配置nginx



在192.168.8.10服务器上操作


五、配置tomcat集群

所以环境都搭建好,下面开始配置! 
  
 
   
  
1. 1.创建 nginx 配置文件 
2. cd /usr/local/nginx/conf/ 
3. mv nginx.conf nginx.conf.bak 
4. vi nginx.conf 
5. 加入如下内容: 
6. user  www www; 
7. worker_processes 4; 
8. worker_cpu_affinity 0001 0010 0100 1000; 
9. error_log  /usr/local/nginx/logs/nginx_error.log  crit; 
10. pid        /usr/local/nginx/nginx.pid; 
11. worker_rlimit_nofile 65535; 
12. events          
13. {       
14.   use epoll; 
15.   worker_connections 65535; 
16.   } 
17. http 
18. { 
19.   include       mime.types; 
20.   default_type  application/octet-stream; 
21.   charset  utf-8; 
22.   server_names_hash_bucket_size 128; 
23.   client_header_buffer_size 2k; 
24.   large_client_header_buffers 4 4k; 
25.   client_max_body_size 8m; 
26.   sendfile on; 
27.   tcp_nopush     on; 
28.   keepalive_timeout 60; 
29.   open_file_cache max=204800 inactive=20s; 
30.   open_file_cache_min_uses 1; 
31.   open_file_cache_valid 30s; 
32.   tcp_nodelay on; 
33.   gzip on; 
34.   gzip_min_length  1k; 
35.   gzip_buffers     4 16k; 
36.   gzip_http_version 1.0; 
37.   gzip_comp_level 2; 
38.   gzip_types       text/plain application/x-javascript text/css application/xml; 
39.   gzip_vary on; 
40.     upstream tomcats { 
41.             server 192.168.8.11:8009 srun_id=jvm1; 
42.             server 192.168.8.12:8009 srun_id=jvm2; 
43. 
44.         #jvm_route $cookie_JSESSIONID reverse;
45.             keepalive 10; 
46.         } 
47. server 
48.   { 
49.     listen       80 default_server; 
50.     server_name  _; 
51.     index   index.htm index.html; 
52.     root  /data/www/; 
53.     location / { 
54.         ajp_pass tomcats; 
55.     } 
56.     location /status 
57.     { 
58.         stub_status on; 
59.     } 
60.     location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|css)$ 
61.     { 
62.       expires      30d; 
63.     } 
64.    } 
65. '$remote_addr - $remote_user [$time_local] "$request" '
66. '$status $body_bytes_sent "$http_referer" '
67. '"$http_user_agent" $http_x_forwarded_for'; 
68.     access_log  /usr/local/nginx/logs/access.log  access; 
69. }



在11、12服务器操作



修改server.xml配置文件:



1. 192.168.8.11服务器: 
2. 1.在/usr/local/tomcat/conf/server.xml 中修改Enginx节点信息,大概在103行: 
3. 把<Engine name="Catalina" defaultHost="localhost"> 
4. 修改成: 
5. <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1"> 
6. 2.加入Cluster节点信息,大概在110行左右处加入如下内容: 
7. <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> 
8. <Manager className="org.apache.catalina.ha.session.DeltaManager"
9. "false"
10. "true"/>   
11. "org.apache.catalina.tribes.group.GroupChannel">   
12. "org.apache.catalina.tribes.membership.McastService"
13. "228.0.0.4"
14. "45564"
15. "500"
16. "3000"/>   
17. "org.apache.catalina.tribes.transport.nio.NioReceiver"
18. "192.168.8.11"
19. "4000"
20. "100"
21. "5000"
22. "6"/>   
23. "org.apache.catalina.tribes.transport.ReplicationTransmitter">   
24. "org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>   
25.     </Sender>   
26. "org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>   
27. "org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>   
28.   </Channel>   
29. "org.apache.catalina.ha.tcp.ReplicationValve"
30. ""/>   
31. "org.apache.catalina.ha.session.JvmRouteBinderValve"/>   
32. "org.apache.catalina.ha.deploy.FarmWarDeployer"
33. "/tmp/war-temp/"
34. "/tmp/war-deploy/"
35. "/tmp/war-listen/"
36. "false"/> 
37. "org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>   
38. "org.apache.catalina.ha.session.ClusterSessionListener"/>   
39. </Cluster> 
40. 192.168.8.12服务器: 
41. 1.在/usr/local/tomcat/conf/server.xml 中修改Enginx节点信息,大概在103行: 
42. 把<Engine name="Catalina" defaultHost="localhost"> 
43. 修改成: 
44. <Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm2"> 
45. 2.加入Cluster节点信息,大概在110行左右处加入如下内容: 
46. <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="8"> 
47. <Manager className="org.apache.catalina.ha.session.DeltaManager"
48. "false"
49. "true"/>   
50. "org.apache.catalina.tribes.group.GroupChannel">   
51. "org.apache.catalina.tribes.membership.McastService"
52. "228.0.0.4"
53. "45564"
54. "500"
55. "3000"/>   
56. "org.apache.catalina.tribes.transport.nio.NioReceiver"
57. "192.168.8.12"
58. "4000"
59. "100"
60. "5000"
61. "6"/>   
62. "org.apache.catalina.tribes.transport.ReplicationTransmitter">   
63. "org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>   
64.     </Sender>   
65. "org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>   
66. "org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>   
67.   </Channel>   
68. "org.apache.catalina.ha.tcp.ReplicationValve"
69. ""/>   
70. "org.apache.catalina.ha.session.JvmRouteBinderValve"/>   
71. "org.apache.catalina.ha.deploy.FarmWarDeployer"
72. "/tmp/war-temp/"
73. "/tmp/war-deploy/"
74. "/tmp/war-listen/"
75. "false"/>   
76. "org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>   
77. "org.apache.catalina.ha.session.ClusterSessionListener"/>   
78. </Cluster> 
79. 3.测试目录负载均衡
80. 在11、12服务器上建立测试目录,观察负载均衡情况。 
81. mkdir -p /data/www/ 
82. cat /data/www/test.jsp 
83. <% 
84. "Welcome tomcat_cluster session repication"); 
85. %>

1. 4.启动各服务器服务 
2. 10服务器: 
3. /usr/local/nginx/sbin/nginx 
4. 11、12服务器 
5. startup.sh


  1. 在浏览器上输入:192.168.8.10/test.jsp,然后分别在11、12服务器查看输出日志:
  2. 11服务器:

nginx alias 设置下载响应头 nginx ajp_运维

12服务器:


nginx alias 设置下载响应头 nginx ajp_网络_02


5.测试负载session

在/data/www目录下创建admin目录,再在admin目录下创建WEB-INF目录

mkdir -p /data/www/admin/WEB-INF 
1. cd /data/www/admin/WEB-INF 
2. vi web.xml 
3. 加入如下内容: 
4. <web-app xmlns="http://java.sun.com/xml/ns/j2ee"
5. xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
6. xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd" version="2.4"> 
7. <display-name>TomcatDemo</display-name> 
8. <distributable/> 
9. </web-app> 
10. PS: <distributable/>这个必须加进去,不然也实现不了复制! 
11. 在admin目录下创建测试代码: 
12. cat index.jsp 
13. <%@ page contentType="text/html; charset=GBK"
14. <%@ page import="java.util.*"
15. <html><head><title>Cluster App Test</title></head> 
16. <body> 
17. Server Info: 
18. <% 
19. out.println(request.getLocalAddr() + " : " + request.getLocalPort()+"<br>");%> 
20. <% 
21. out.println("<br> ID " + session.getId()+"<br>"); 
22. // 如果有新的 Session 属性设置
23. String dataName = request.getParameter("dataName"); 
24. if (dataName != null
25. String dataValue = request.getParameter("dataValue"); 
26. session.setAttribute(dataName, dataValue); 
27. } 
28. out.print("<b>Session 列表</b>"); 
29. Enumeration e = session.getAttributeNames(); 
30. while
31. String name = (String)e.nextElement(); 
32. String value = session.getAttribute(name).toString(); 
33. out.println( name + " = " + value+"<br>"); 
34. System.out.println( name + " = "
35. } 
36. %> 
37. <form action="index.jsp" method="POST"> 
38. 名称:<input type=text size=20 name="dataName"> 
39. <br> 
40. 值:<input type=text size=20 name="dataValue"> 
41. <br> 
42. <input type=submit> 
43. </form> 
44. </body> 
45. </html>

然后分别在11、12服务器上查看日志输出:



11服务器:



nginx alias 设置下载响应头 nginx ajp_运维_03



12服务器:



nginx alias 设置下载响应头 nginx ajp_运维_04



浏览器结果:



nginx alias 设置下载响应头 nginx ajp_nginx alias 设置下载响应头_05

nginx alias 设置下载响应头 nginx ajp_网络_06

总结:

1.本文采用的是通过 nginx_ajp模块来实现tomcat负载均衡,nginx_ajp_module 是一个 Nginx 的第三方模块,用来使用 Ajp 协议连接后端的 Tomcat ,相当于 Apache 上的 mod_jk 模块。也可以通过nginx-upstream-jvm-route 这个模块来实现,方法类似!

2.对于session共享,还可以通过memcached来保存session!后面有时间会一一介绍!

3.至于前面的nginx调度器我这里只用一台,没有实现高可用,大家可以用nginx+keepalived来实现调度器的高可用,实现也比较简单。


转载于:https://blog.51cto.com/opsmysql/945204