Awstats分析Nginx日志

原创

liangxiaowei66 2013-11-12 12:04:35 ©著作权

文章标签 日志 Awstats 分析Nginx 文章分类 Nginx 服务器

©著作权归作者所有：来自51CTO博客作者liangxiaowei66的原创作品，请联系作者获取转载授权，否则将追究法律责任

nginx日志格式设定,去掉如下行#号

#vim /usr/local/nginx/conf/nginx.conf

log_format access '$remote_addr -$remote_user [$time_local] "$request" '

'$status $body_bytes_sent"$http_referer" '

'"$http_user_agent"$http_x_forwarded_for';

access_log log/access.log main;

参数说明：

$remote_addr 与$http_x_forwarded_for 用以记录客户端的ip地址；
$remote_user ：用来记录客户端用户名称；
$time_local ：用来记录访问时间与时区；
$request ：用来记录请求的url与http协议；
$status ：用来记录请求状态；成功是200，
$body_bytes_s ent ：记录发送给客户端文件主体内容大小；
$http_referer ：用来记录从那个页面链接访问过来的；
$http_user_agent ：记录客户毒啊浏览器的相关信息；

nginx日志切割脚本

#vim /usr/local/nginx/nginx_log.sh

#!/bin/bash

logs_path="/usr/local/nginx/logs/" #设置日志文件存放目录

pid_path="/usr/local/nginx/logs/nginx.pid" #设置pid文件

mv${logs_path}access.log ${logs_path}access_$(date -d "yesterday" +"%Y%m%d").log

#重命名日志文件，将access.log切割为昨天的日志文件，因为要在每天零点切割，所有日志文件格式是前一天的日期，如access_20130728.log

kill-USR1 `cat${pid_path}` #向nginx主进程发信号重新打开日志

安装和配置awstats

wget http://awstats.sourceforge.net/files/awstats-7.0.tar.gz

tar -xvf awstats-7.0.tar.gz

mv awstats-7.0 /usr/local/awstats

chown -R root:root /usr/local/awstats

chmod -R =rwX /usr/local/awstats

chmod +x /usr/local/awstats/tools/*.pl

chmod +x /usr/local/awstats/wwwroot/cgi-bin/*.pl

执行 tools 目录中的 awstats_configure.pl 配置向导，创建一个新的统计

cd /usr/local/awstats/tools

./awstats_configure.pl

将会有如下一些提示：

-----> Running OS detected: Linux, BSD orUnix

----->Check for web server install

Enterfull config file path of your Web server.

Example:/etc/httpd/httpd.conf

Example:/usr/local/apache2/conf/httpd.conf

Example:c:\Program files\apache group\apache\conf\httpd.conf Config file path ('none' to skip web server setup):

>none #这里添none并回车，因为我们没有使用apache

回车之后下一个选项：

Yourweb server config file(s) could not be found.

Youwill need to setup your web server manually to declare AWStats

script as a CGI, if you want tobuild reports dynamically. See AWStats setup documentation (file docs/index.html)

----->Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf' File awstats.model.conf updated.

----->Need to create a new config file ? Do you want me to build anew AWStats config/profilefile (required if first install) [y/N] ? y #这里选Y，创建一个新的配置文件

-----> Define config file name to create

What is the name of your web site or profile analysis ?

Example: www.mysite.com

Example: demo

Yourweb site, virtual server or profile name:

>nginx #这里输入你要分析的域名，或是随便一个你易记的配置名并回车

----->Define config file path In which directory do you plan to store your config file(s) ?

Default: /etc/awstats

Directorypath to store config file(s) (Enter fordefault):

> #直接回车，定义你的配置文件存放的路径，使用默认路径/etc/awstats

----->Create config file '/etc/awstats/awstats.nginx.conf'

Configfile /etc/awstats/awstats.nginx.conf created.

-----> Add updateprocess inside a scheduler Sorry, configure.pl does not support automatic addto cron yet.

You can do it manually by adding the following command to yourcron: /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=nginx

Or if you have several config files and prefer having only onecommand: /usr/local/awstats/tools/awstats_updateall.pl now

Press ENTER to continue...#按回车继续

A SIMPLE config file has been created: /etc/awstats/awstats.nginx.conf You should have a lookinside to check and change manually main parameters.

Youcan then manually update your statistics for'yuyuanchun.com' withcommand: > perl awstats.pl -update -config=nginx

Youcan also build static report pages for'nginx' with command:> perl awstats.pl -output=pagetype -config=nginx

PressENTER to finish... #回车完成配置文件的创建

默认会生成一个名为awstats.nginx.conf配置文件在/etc/awstats/目录下，修改该配置文件的日志位置

vim /etc/awstats/awstats.nginx.conf

LogFile="/usr/local/nginx/logs/access_%YYYY-0%MM-0%DD-24.log"

这里是对应上面Nginx日志切割所生成的目录存放位置，注意awstats的年月日格式，-24表示昨天的日志，-0表示当前的

分析的执行顺序是：

Nginx 产生日志 –> 日志切割 –> Nginx 继续产生日志 –> 另存切割日志 –> 交由Awstats统计 –> 生成结果

日志说明

# "LogFile" contains the web, ftp or mail server logfile to analyze.

#You can also use tags inthis filename if you need a dynamic file name

#depending on date or time (Replacement is made by AWStats at the beginning

# ofits execution). This is available tags :

# %YYYY-n is replaced with 4 digits year we were n hours ago

# %YY-n is replaced with 2 digits year we were n hours ago

# %MM-n is replaced with 2 digits month we were n hours ago

# %MO-n is replaced with 3 letters month we were n hours ago

# %DD-n is replaced with day we were n hours ago

# %HH-n is replaced with hour we were n hours ago

# %NS-n is replaced with number of seconds at 00:00 since 1970

# %WM-n is replaced with the week number in month (1-5)

# %Wm-n is replaced with the week number in month (0-4)

# %WY-n is replaced with the week number in year (01-52)

# %Wy-n is replaced with the week number in year (00-51)

# %DW-n is replaced with the day number in week (1-7, 1=sunday)

# use n=24if you need (1-7, 1=monday)

# %Dw-n is replaced with the day number in week (0-6, 0=sunday)

# use n=24if you need (0-6, 0=monday)

# Use 0for n if you need current year, month, day, hour...

n表示时间间隔，我这里就是分析距离当前今天24小时前（昨日）的日志，改成48就是前两天的日志

创建一个awstats用于记录数据的目录

mkdir -p /var/lib/awstats

然后运行awstats的wwwroot目录中的awatsts.pl来测试一下

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=nginx

出现报错：

Fromdata in log file "/usr/local/nginx/logs/access_20131111.log"...

Error:Couldn't open server log file "/usr/local/nginx/logs/access_20131111.log" : 没有那个文件或目录

解决办法：

touch /usr/local/nginx/logs/access_20131111.log

再次执行

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=nginx

如果看到类似下面的提示就说明配置文件都正确了

Create/Updatedatabase for config "/etc/awstats/awstats.nginx.conf" by AWStats version 7.0 (build 1.964) From data

in log file "/home/wwwlogs/access.log_20130727"... Phase 1 : First bypass old records, searching new record...

Directaccess after last parsed record (after line 43260) Jumped lines in file: 43260 Found 43260 already parsed records.

Parsedlines in file: 0

Found0 dropped records,

Found0 comments,

Found0 blank records,

Found0 corrupted records,

Found0 old records,

Found0new qualified records

Nginx 对 Perl 支持并不好，所以这里利用 awstats 的工具将统计的结果生成静态文件

首先在 webroot 目录下创建一个文件夹。例：/home/www/awstats

mkdir -p /home/www/awstats

生成静态文件，

/usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=nginx -lang=cn-dir=/home/www/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

运行脚本如果出现生成一堆网页类似为正确

Launchupdate process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=nginx -update -configdir= ......

Buildkeywords page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=nginx -staticlinks

-lang=cn-output=keywords Build errors404 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl"

-config=nginx-staticlinks -lang=cn -output=errors404 20 files built.

Main HTML page is'awstats.nginx.html'.

/usr/local/awstats/tools/awstats_buildstaticpages.pl Awstats静态页面生成工具
-update -config=nginx 更新配置项
-lang=cn 语言为中文
-dir=/home/www/awstats 统计结果输出目录
-awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl Awstats 日志更新程序路径

修改nginx配置文件，添加日志分析结果站点

#vim /usr/local/nginx/conf/nginx.conf

在最末尾添加如下行

server

{

listen 8080;

server_name localhost;

index index.html index.htmindex.php default.html default.htm default.php;

root /home/www/awstats;

location ~ ^/awstats/ { #html静态页面目录

root /home/www/awstats;

index index.html;

access_log off;

}

location ^~ /icon/ { #图标目录

root /usr/local/awstats/wwwroot;

index index.html;

access_log off;

}

autoindex on; #开启目录浏览功能

access_log off;

}

重新加载nginx

ln -s /usr/local/nginx/sbin/nginx /usr/bin/

nginx –t #检查nginx配置文件是否有错误

nginx –s reload #加载

添加定时任务，每晚切割，分析nginx日志

# vim /usr/local/nginx/awstats.sh

#!/bin/bash

/bin/bash /usr/local/nginx/nginx_log.sh

/usr/local/awstats/wwwroot/cgi-bin/awstats.pl-update -config=nginx

/usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=nginx -lang=cn-dir=/home/www/awstats/ -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

# chmod +x /usr/local/nginx/awstats.sh

# crontab -e

01 00 * * * /usr/local/nginx/awstats.sh

这样访问http://192.168.233.210:8080就可以看到统计结果静态页面了。