fail2ban 防止WEB攻击

1. 简介

fail2ban简单来说就是通过不断读取设定的日志文件，并通过正则校验每条日志是否符合规则。一旦符合，则提取日志中的IP地址与时间戳，然后写入到fail2ban的数据库中。写库的同时进行计数，如果该IP在设定的时间间隔内被匹配的次数超过阈值，则调用iptables，将其的请求reject。

Linux 系统生产环境配置指南  ​​https://blog.51cto.com/waringid/5782872​​

Linux 系统防火墙 Firewall-cmd 日常操作指南 ​https://blog.51cto.com/waringid/5806004​​

利用 ipset 封禁大量 IP ​https://blog.51cto.com/waringid/5806004​​

前期的准备操作请参考“​​Linux 系统生产环境配置指南​​”。Fail2ban 结构如下：

/etc/fail2ban ## fail2ban 服务配置目录

/etc/fail2ban/action.d ## iptables 、mail 等动作文件目录

/etc/fail2ban/filter.d ## 条件匹配文件目录，过滤日志关键内容

/etc/fail2ban/jail.conf ## fail2ban 防护配置文件

/etc/fail2ban/fail2ban.conf ## fail2ban 配置文件，定义日志级别、日志、sock 文件位置等

2. 检查防火墙

#如果您已经安装iptables建议先关闭
service iptables stop
#查看Firewalld状态
firewall-cmd --state
#启动firewalld
systemctl start firewalld
#设置开机启动
systemctl enable firewalld.service
#放行22端口
firewall-cmd --zone=public --add-port=80/tcp --permanent
#重载配置
firewall-cmd --reload
#查看已放行端口
firewall-cmd --zone=public --list-ports

3. 安装配置

3.1. 安装

yum -y install fail2ban

安装成功后fail2ban配置文件位于/etc/fail2ban，其中jail.conf为主配置文件，相关的匹配规则位于filter.d目录，其它目录/文件一般很少用到，如果需要详细了解可自行搜索。

3.2. 配置jail

cp jail.conf jail.local
vim jail.local

banaction = firewallcmd-ipset
action = %(action_mwl)s
#将默认的执行动作由iptables改为firewallcmd-ipset
[waf-blockip]
enabled = true
filter = waf-blockip
port = http,https
logpath = /tmp/2019*_waf.log
blocktype = DROP
action          = %(action_mwl)s
datapattern = %Y-%m-%d %H:%M:%S
bantime = 120
maxretry = 1
findtime = 1
 
[nginx-cc]
enabled = true
filter = nginx-cc
port = http,https
logpath = /tmp/cc.log
blocktype = DROP
action          = %(action_mwl)s
bantime = 120
maxretry = 1
findtime = 1

以上每行内容的大致意义如下：

[nginxx-cc]：定义jail名称

• enabled：是否启用该jail，默认的所有规则都没有该项，需要手动添加
• port：指定封禁的端口，默认为0:65535，也就是所有端口，但可以在jail中设定
• filter：指定过滤器名称
• logpath：日志路径
• action：达到阈值后的动作
• maxretry：阈值
• findtime：时间间隔
• bantime：封禁时长
• ignoreip：忽略的IP

注意

logpath与action可以有多行，如action中的设定：调用iptables-multiport封禁目标IP访问的多个端口，调用sendmail发送告警邮件

findtime不是检查日志的时间间隔，日志的检查是实时的。因为fail2ban自带数据库，所以可以在设定的时间内统计匹配次数

ignoreip添加后端服务器的IP或CDN的IP

3.3. 配置filter

vim waf-blockip.conf
[Definition]
failregex = \"<HOST>\",\"rule_tag\"
ignoreregex =

日志内容如下所示：

{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 11:59:45","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_agent":"ApacheBench\/2.3"}
{"local_time":"2019-06-18 12:00:13","client_ip":"192.168.13.215","rule_tag":"-","attack_method":"CC_Attack","server_name":"status.myj.com.cn","req_url":"\/","req_data":"-","user_age

vim nginx-cc.conf
[Definition]
failregex =  <HOST> - - \[.*\] .*GET \/\?=*
#failregex = <HOST> - - \[.*\] \".*(403|400).*\"      #匹配403或404的错误
ignoreregex =

47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
50.195.104.171 - - [13/Jun/2019:18:18:47 +0800] "GET /?=13253 HTTP/1.1" 504 905"https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; U; NetBSD amd64; en-US; rv:1.9.2.15) Gecko/20110308 Namoroka/3.6.15"
47.52.195.189 - - [13/Jun/2019:18:18:47 +0800] "GET /?=18566 HTTP/1.1" 403 552 "https://www.google.com/search?q=120.79.35.33/" "Mozilla/5.0 (X11; Linux x86_64)AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10 Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24"
192.168.13.43 - - [06/Jun/2019:18:25:58 +0800] "GET /ngx_status HTTP/1.1" 301 16"-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Ge

4. 常用配置

#查看被ban IP，其中ssh-iptables为名称，比如上面的[ssh-iptables]和[nginx-dir-scan]
fail2ban-client status ssh-iptables
#添加白名单
fail2ban-client set ssh-iptables addignoreip IP地址
#删除白名单
fail2ban-client set ssh-iptables delignoreip IP地址
#查看状态
fail2ban-client status
#查看日志
tail /var/log/fail2ban.log
#查看规则
firewall-cmd --direct --get-all-rules
ipset list

4.1. 定时清理日志

vi /root/del_waf_log.sh
#! /bin/bash
cat /dev/null > /tmp/`date +"%Y-%m-%d"`-waf.log
chmod +x /root/del_waf_log.sh
crontab -e
30 3 * * 0 sh /root/del_waf_log.sh

5. 内核优化

针对TIME_WAIT的问题，可以通过调整内核的方式进行优化。
#修改超时时间为30秒，某些情况可以进一步降低该值
net.ipv4.tcp_fin_timeout = 30
#将keepalive的发送频率降低为20分钟一次
net.ipv4.tcp_keepalive_time = 1200
#开启SYN Cookies，当SYN队列溢出时启用Cookies
net.ipv4.tcp_syncookies = 1
#开启TIME-WAIT sockets重用功能
net.ipv4.tcp_tw_reuse = 1
#开启TIME-WAIT sockets快速回收功能
net.ipv4.tcp_tw_recycle = 1
#加大SYN队列
net.ipv4.tcp_max_syn_backlog = 8192
#最大TIME_WAIT保持数，超过将全部清除
net.ipv4.tcp_max_tw_buckets = 5000

6. 过滤规则

6.1. nginx的验证

fail2ban-regex /tmp/2019-07-10_waf.log /etc/fail2ban/filter.d/waf-blockip.conf
failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (password mismatch|was not found in ".*"), client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$
^ \[error\] \d+#\d+: \*\d+ no user/password was provided for basic authentication, client: <HOST>, server: \S+, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"\s*$

6.2. 检查fail2ban启动问题

fail2ban-client -x start

6.3. modsecurity 日志过滤

利用 modsecurity 结合 nginx 并整合到 微服务网关实现 web 安全防护。

kong 网关结合 modsecurity 输出的日志格式如下

112.119.33.91 - - [18/Dec/2020:09:21:58 +0800] "GET /socket.io/?__sails_io_sdk_version=0.13.8&__sails_io_sdk_platform=browser&__sails_io_sdk_language=javascript&EIO=3&transport=polling&t=NPpS75x&sid=o6ZI8VoIHqBUgvFMAACb HTTP/2.0" 200 4 "https://dashboard.waringid.me/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
2020/12/18 09:23:10 [error] 25142#0: *659 [client 113.78.65.148] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `5' ) [file "/usr/local/owasp-modsecurity-crs-3.2.0/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "79"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 5)"] [data ""] [severity "2"] [ver ""] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "172.19.0.5"] [uri "/.env"] [unique_id "1608254590"] [ref ""], client: 113.78.65.148, server: kong, request: "GET /.env HTTP/1.1", host: "dashboard.waringid.me"
113.78.65.148 - - [18/Dec/2020:09:23:10 +0800] "GET /.env HTTP/1.1" 403 150 "-" "curl/7.29.0"
51.159.23.43 - - [18/Dec/2020:09:28:46 +0800] "GET / HTTP/1.1" 404 48 "-" "-"
89.97.157.10 - - [18/Dec/2020:09:43:52 +0800] "GET / HTTP/1.1" 404 48 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36"

注意2种不同的日志格式类型，2中不同日志类型匹配不同的正则式。

同时注意不同日志时间格式也不一样。