最近有业务需求要在高并发下保持web应用的高性能,具体的业务中是在nginx来反向代理tomcat集群的基础上,做了三次实验:一种情况是使用nginx的proxy_cache来做缓存,经测试,性能最佳;第二种是使用nginx来做动静分离,经测试,性能次于第一种,第三种是使用nginx的proxy_cache来做缓存,同时在nginx的前端使用keepalived来实现nginx的failover切换,即HA。
参考博文一:nginx实现动静分离:http://quenlang.blog.51cto.com/4813803/1570477
参考博文二:nginx使用proxy_cache做缓存:http://quenlang.blog.51cto.com/4813803/1570352
测试工具使用apache的httpd-tools工具,首先在hadoop1上安装该工具,该工具提供ab命令
[root@hadoop1 ~]# yum install httpd-tools -y
ab命令的参数如下:
-n:请求总数
-c:每次访问的并发数
如:ab -n 50000 -c 10000 http://192.168.0.101/shop/index.html
1、首先测试html静态页面
测试nginx实现web缓存高并发访问下的性能,共5000个请求,每次并发1000,用时0.583秒,IO吞吐量为330.699M/s
[root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.101/shop/index.html Server Software: nginx/1.7.6 Server Hostname: 192.168.0.101 Server Port: 80 Document Path: /shop/index.html Document Length: 192 bytes Concurrency Level: 1000 Time taken for tests: 0.583 seconds Complete requests: 5000 Failed requests: 4200 (Connect: 0, Receive: 0, Length: 4200, Exceptions: 0) Write errors: 0 Non-2xx responses: 819 Total transferred: 197305366 bytes HTML transferred: 195812742 bytes Requests per second: 8581.54 [#/sec] (mean) Time per request: 116.529 [ms] (mean) Time per request: 0.117 [ms] (mean, across all concurrent requests) Transfer rate: 330699.98 [Kbytes/sec] received
测试nginx实现动静分离高并发访问下的性能,共5000个请求,每次并发1000,用时3.106秒,IO吞吐量为74.187M/s
[root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.100/shop/index.html Server Software: nginx/1.7.6 Server Hostname: 192.168.0.100 Server Port: 80 Document Path: /shop/index.html Document Length: 46428 bytes Concurrency Level: 1000 Time taken for tests: 3.106 seconds Complete requests: 5000 Failed requests: 0 Write errors: 0 Total transferred: 235926480 bytes HTML transferred: 234298640 bytes Requests per second: 1609.99 [#/sec] (mean) Time per request: 621.122 [ms] (mean) Time per request: 0.621 [ms] (mean, across all concurrent requests) Transfer rate: 74187.39 [Kbytes/sec] received
测试不通过nginx代理,直接高并发访问后端tomcat时的性能,共5000个请求,每次并发1000,用时3.532秒,IO吞吐量为65.928M/s
[root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.103:8080/shop/index.html Server Software: Apache-Coyote/1.1 Server Hostname: 192.168.0.103 Server Port: 8080 Document Path: /shop/index.html Document Length: 46428 bytes Concurrency Level: 1000 Time taken for tests: 3.532 seconds Complete requests: 5000 Failed requests: 0 Write errors: 0 Total transferred: 238450924 bytes HTML transferred: 236978956 bytes Requests per second: 1415.62 [#/sec] (mean) Time per request: 706.405 [ms] (mean) Time per request: 0.706 [ms] (mean, across all concurrent requests) Transfer rate: 65928.82 [Kbytes/sec] received
可见,针对html静态页面而言,nginx作为web缓存时的性能要远远优于nginx作为动静分离时的性能,而nginx作为动静分离时的性能略优于不使用nginx代理而直接访问后端tomcat的性能。
2、接着测试图片
测试nginx实现web缓存高并发访问下的性能,共5000个请求,每次并发1000,用时2.155秒,IO吞吐量为935.765M/s
[root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.101/shop/QQ.jpg Server Software: nginx/1.7.6 Server Hostname: 192.168.0.101 Server Port: 80 Document Path: /shop/QQ.jpg Document Length: 192 bytes Concurrency Level: 1000 Time taken for tests: 2.155 seconds Complete requests: 5000 Failed requests: 3919 (Connect: 0, Receive: 0, Length: 3919, Exceptions: 0) Write errors: 0 Non-2xx responses: 1081 Total transferred: 2064708785 bytes HTML transferred: 2063553225 bytes Requests per second: 2320.48 [#/sec] (mean) Time per request: 430.945 [ms] (mean) Time per request: 0.431 [ms] (mean, across all concurrent requests) Transfer rate: 935765.00 [Kbytes/sec] received
测试nginx实现动静分离高并发访问下的性能,共5000个请求,每次并发1000,用时24.867秒,IO吞吐量为96.387M/s
[root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.100/shop/QQ.jpg Server Software: nginx/1.7.6 Server Hostname: 192.168.0.100 Server Port: 80 Document Path: /shop/QQ.jpg Document Length: 525861 bytes Concurrency Level: 1000 Time taken for tests: 24.867 seconds Complete requests: 5000 Failed requests: 371 (Connect: 0, Receive: 0, Length: 371, Exceptions: 0) Write errors: 0 Non-2xx responses: 352 Total transferred: 2454418691 bytes HTML transferred: 2453209149 bytes Requests per second: 201.07 [#/sec] (mean) Time per request: 4973.454 [ms] (mean) Time per request: 4.973 [ms] (mean, across all concurrent requests) Transfer rate: 96387.47 [Kbytes/sec] received
测试不通过nginx代理,直接高并发访问后端tomcat时的性能,共5000个请求,每次并发1000,结果发现多次测试都无法成功,也就是说直接
访问后端,后端的tomcat是无法承担起这么高的并发访问压力
[root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.104:8080/shop/QQ.jpg Benchmarking 192.168.0.104 (be patient) Completed 500 requests Completed 1000 requests Completed 1500 requests Completed 2000 requests apr_socket_recv: Connection reset by peer (104) Total of 2027 requests completed [root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.104:8080/shop/QQ.jpg Benchmarking 192.168.0.104 (be patient) Completed 500 requests Completed 1000 requests Completed 1500 requests Completed 2000 requests Completed 2500 requests Completed 3000 requests Completed 3500 requests apr_socket_recv: Connection reset by peer (104) Total of 3756 requests completed [root@hadoop1 ~]# ab -n 5000 -c 1000 http://192.168.0.104:8080/shop/QQ.jpg Benchmarking 192.168.0.104 (be patient) Completed 500 requests Completed 1000 requests apr_socket_recv: Connection reset by peer (104) Total of 1198 requests completed
可见,针对图片而言,nginx作为web缓存时的性能要远远优于nginx作为动静分离时的性能,而nginx作为动静分离时的性能又要大大优于不使用nginx代理而直接访问后端tomcat的性能。
3、测试jsp动态程序
测试nginx实现web缓存高并发访问下的性能,共2600个请求,每次并发1000,用时34.936秒,IO吞吐量为0.263M/s,整个过程,tomcat后台日志中无错误信息抛出
[root@hadoop1 ~]# ab -n 2600 -c 1000 -k http://192.168.0.101/shop/admin/login.jsp Server Software: nginx/1.7.6 Server Hostname: 192.168.0.101 Server Port: 80 Document Path: /shop/admin/login.jsp Document Length: 192 bytes Concurrency Level: 1000 Time taken for tests: 34.936 seconds Complete requests: 2600 Failed requests: 1568 (Connect: 0, Receive: 0, Length: 1568, Exceptions: 0) Write errors: 0 Non-2xx responses: 1033 Keep-Alive requests: 1567 Total transferred: 9421232 bytes HTML transferred: 8762752 bytes Requests per second: 74.42 [#/sec] (mean) Time per request: 13437.110 [ms] (mean) Time per request: 13.437 [ms] (mean, across all concurrent requests) Transfer rate: 263.35 [Kbytes/sec] received
测试nginx实现动静分离高并发访问下的性能,共2600个请求,每次并发1000,用时58.147秒,IO吞吐量为0.248M/s,这时tomcat的后台日志中抛出大量应用程序的错误日志,说明应用已经开始出错
[root@hadoop1 ~]# ab -n 2600 -c 1000 -k http://192.168.0.100/shop/admin/login.jsp Server Software: nginx/1.7.6 Server Hostname: 192.168.0.100 Server Port: 80 Document Path: /shop/admin/login.jsp Document Length: 5462 bytes Concurrency Level: 1000 Time taken for tests: 58.147 seconds Complete requests: 2600 Failed requests: 32 (Connect: 0, Receive: 0, Length: 32, Exceptions: 0) Write errors: 0 Non-2xx responses: 32 Keep-Alive requests: 2568 Total transferred: 14818864 bytes HTML transferred: 14032560 bytes Requests per second: 44.71 [#/sec] (mean) Time per request: 22364.327 [ms] (mean) Time per request: 22.364 [ms] (mean, across all concurrent requests) Transfer rate: 248.88 [Kbytes/sec] received
测试不通过nginx代理,直接高并发访问后端tomcat时的性能,共2600个请求,每次并发1000,结果发现多次测试都无法成功,也就是说直接访问后端,后端的tomcat是无法承担起这么高的并发访问压力
[root@hadoop1 ~]# ab -n 2600 -c 1000 -k http://192.168.0.102:8080/shop/admin/login.jsp This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.0.102 (be patient) apr_socket_recv: Connection reset by peer (104) Total of 100 requests completed [root@hadoop1 ~]# ab -n 2600 -c 1000 -k http://192.168.0.102:8080/shop/admin/login.jsp This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.0.102 (be patient) apr_socket_recv: Connection reset by peer (104) Total of 57 requests completed [root@hadoop1 ~]# ab -n 2600 -c 1000 -k http://192.168.0.102:8080/shop/admin/login.jsp This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking 192.168.0.102 (be patient) apr_socket_recv: Connection reset by peer (104)
可见,针对jsp动态程序而言,尽管nginx的配置没有缓存动态程序,但nginx作为web缓存时的性能还是要远远优于nginx作为动静分离时的性能,而nginx作为动静分离时的性能又要大大优于不使用nginx代理而直接访问后端tomcat的性能。
5、结论
测试结果显示,尽管nginx动静分离可以提高web的性能,确实如此,相比于直接访问tomcat来说,动静分离能够大幅度提高站点的并发访问性能。但是这种架构跟nginx反向代理tomcat集群的同时使用自身proxy_cache模块来作为web站点的缓存服务器的情形相比,显然后者对web站点高并发访问压力下性能的提升更加明显和高效,因此,建议再使用nginx的proxy_cache来缓存web的静态文件以实现web站点性能的提升。