Awstat配置手册

原创

technology1209 2012-12-22 14:23:32 博主文章分类：Linux ©著作权

文章标签 Apache AWStats 文章分类 运维

©著作权归作者所有：来自51CTO博客作者technology1209的原创作品，请联系作者获取转载授权，否则将追究法律责任

AWStats分析Tomcat\Apache\IIS\nginx 的访问日志

Apache+AWStats安装使用[原创]

AWStats安装使用

1、事先安装好Apache

2、安装awstats的命令(将awstats安装在/data/soft/awstats目录)
mkdir -p /var/lib/awstats
cd /data/soft
wget http://jaist.dl.sourceforge.net/project/awstats/AWStats/7.0/awstats-7.0.tar.gz
tar zxvf awstats-7.0.tar.gz
mv awstats-7.0 awstats
cd awstats/tools/
perl awstats_configure.pl

3、Perl脚本awstats_configure.pl安装过程(以下内容引用AWStats英文使用说明)

(1)
-----> Running OS detected: Linux, BSD or Unix
Warning: AWStats standard directory on Linux OS is '/usr/local/awstats'.
If you want to use standard directory, you should first move all content
of AWStats distribution from current directory:
/data/soft/awstats
to standard directory:
/usr/local/awstats
And then, run configure.pl from this location.
Do you want to continue setup from this NON standard directory [yN] ? y

这时选择y回车。

(2)
-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):
> /etc/httpd/conf/httpd.conf

第一次使用请输入Apache的httpd.conf路径，例如/etc/httpd/conf/httpd.conf
以后如果再使用perl awstats_configure.pl生成配置文件，则可以输入none跳过。

(3)
-----> Check and complete web server config file '/etc/httpd/conf/httpd.conf'
Add 'Alias /awstatsclasses "/data/soft/awstats/wwwroot/classes/"'
Add 'Alias /awstatscss "/data/soft/awstats/wwwroot/css/"'
Add 'Alias /awstatsicons "/data/soft/awstats/wwwroot/icon/"'
Add 'ScriptAlias /awstats/ "/data/soft/awstats/wwwroot/cgi-bin/"'
Add '<Directory>' directive
AWStats directives added to Apache config file.

-----> Update model config file '/data/soft/awstats/wwwroot/cgi-bin/awstats.model.conf'
File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y

创建一个新的配置文件，选择y

(4)
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
>xqhan

输入站点名称，例如xqhan

(5)
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
>

-----> Create config file '/etc/awstats/awstats.xqhan.conf'
Config file /etc/awstats/awstats.xqhan.conf created.

-----> Restart Web server with '/sbin/service httpd restart'
httpd: Could not reliably determine the server's fully qualified domain name, using 172.16.34.237 for ServerName
停止 httpd：[确定]
启动 httpd：[确定]

输入AWStats配置文件存放路径，一般直接回车则使用默认路径/etc/awstats

(6)
-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/data/soft/awstats/wwwroot/cgi-bin/awstats.pl -update -config=xqhan
Or if you have several config files and prefer having only one command:
/data/soft/awstats/tools/awstats_updateall.pl now
Press ENTER to continue...

按回车键继续

(7)
A SIMPLE config file has been created: /etc/awstats/awstats.xqhan.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'xqhan' with command:
> perl awstats.pl -update -config=xqhan
You can also read your statistics for 'xqhan' with URL:
> http://localhost/awstats/awstats.pl?config=xqhan

Press ENTER to finish...

按回车键结束

4、修改awstats.xqhan.conf配置
vi /etc/awstats/awstats.xqhan.conf

按?，在之后输入要搜索的内容LogFile="
然后按Ins键，找到LogFile="/var/log/httpd/access_log"
改为要分析的Apache日志路径与文件名。

(1)多日志合并分析(例：其中一台服务器11月24日-11月26日的日志ex121124.log-ex121126.log)
LogFile="/data/soft/awstats/tools/logresolvemerge.pl /var/log/httpd/iis/ex121124.log /var/log/httpd/iis/ex121125.log /var/log/httpd/iis/ex121126.log|"
或
LogFile="/data/soft/awstats/tools/logresolvemerge.pl /var/log/httpd/iis/ex12112*.log|"

(2)分析使用gzip压缩过的日志文件
LogFile="gzip -d </var/log/httpd/apache/access.log.gz|"

5、更新分析报告
perl /data/soft/awstats/wwwroot/cgi-bin/awstats.pl -config=xqhan -update

如果出现以下错误提示
(1)
This means each line in your web server log file need to have "combined log format" like this:
111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] "GET / HTTP/1.1" 200 1234 "http://www.fromserver.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
And this is an example of records AWStats found in your log file (the record number 50 in your log):
2012-11-23 16:00:09 W3SVC1244844107 10.0.2.71 GET /js/jquery.cookie.js - 80 - 116.24.227.156 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727) 200 0 0
Setup ('/etc/awstats/awstats.xqhan.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory)

删除掉这些行的日志即可：

(2)
Error: AWStats database directory defined in config file by 'DirData' parameter (/var/lib/awstats) does not exist or is not writable.
Setup ('/etc/awstats/awstats.xqhan.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).、

创建mkdir -p /var/lib/awstats目录即可

[root@CentOS5-C iis]# perl /data/soft/awstats/wwwroot/cgi-bin/awstats.pl -config=xqhan -update
Create/Update database for config "/etc/awstats/awstats.xqhan.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/data/soft/awstats/tools/logresolvemerge.pl /var/log/httpd/iis/ex121124.log /var/log/httpd/iis/ex121125.log /var/log/httpd/iis/ex121126.log|"...
Phase 1 : First bypass old records, searching new record...
Direct access to last remembered record has fallen on another record.
So searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 1885088
Found 0 dropped records,
Found 0 comments,
Found 0 blank records,
Found 945799 corrupted records,
Found 268267 old records,
Found 671022 new qualified records.

6、查看分析报告
http://172.16.34.237/awstats/awstats.pl?config=apache

这里的apache对应/etc/awstats/awstats.apache.conf中的apache

注：
修改日志文件格式。因为IIS默认记录的日志并不符合awstats程序中的格式2，所以需要自定义日志格式，如下：
LogFormat="date time cs-method cs-uri-stem cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc-status sc-bytes"
分析iis日志需要修改etc/awstats/awstats.xqhan.conf

==================================================================================================
nginx
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" $request_time';

log_format main '$remote_addr - $remote_user [$time_local] '
            '"$request" $status $bytes_sent '
            '"$http_referer" "$http_user_agent" '
            '"$gzip_ratio" "$http_x_forwarded_for" ';


log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                       '$status $body_bytes_sent Referer:"$http_referer" '
                       'UA:"$http_user_agent" "$http_x_forwarded_for" ';

日志格式说明：
$remote_addr 远程请求使用的IP地址
$remote_user 远端登录名
$time_local 时间，用普通日志时间格式(标准英语格式)
$request 请求的第一行
$status 状态。
$body_bytes_sent 请求返回的字节数，包括请求头的数据
$http_referer 请求头Referer的内容
$http_user_agent 请求头User-Agent的内容
$request_time 处理完请求所花时间，以秒为单位

apache
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T " combined
CustomLog log/access_log combined

日志格式说明：
%h 请求使用的IP地址
%l 远端登录名(由identd而来，如果支持的话)，除非IdentityCheck设为"On"，否则将得到一个"-"。
%u 远程用户名(根据验证信息而来；如果返回status(%s)为401，可能是假的)
%t 时间，用普通日志时间格式(标准英语格式)
%r 请求的第一行
%s 状态。对于内部重定向的请求，这个状态指的是原始请求的状态，---%>s则指的是最后请求的状态。
%b 以CLF格式显示的除HTTP头以外传送的字节数，也就是当没有字节传送时显示'-'而不是0。
\"%{Referer}i\" 发送到服务器的请求头Referer的内容。
\"%{User-Agent}i\" 发送到服务器的请求头User-Agent的内容。
%T 处理完请求所花时间，以秒为单位。
%I 接收的字节数，包括请求头的数据，并且不能为零。要使用这个指令你必须启用mod_logio模块。
%O 发送的字节数，包括请求头的数据，并且不能为零。要使用这个指令你必须启用mod_logio模块。

iis
GET /seek/p_w_picpaths/ip.gif - 200 Mozilla/5.0+(X11;+U;+Linux+2.4.2-2+i686;+en-US;+0.7)

连接时间            2007-09-21 01:10:51
IP地址              10.152.8.17 - 10.152.8.2
端口                80
请求动作            GET /seek/p_w_picpaths/ip.gif - 200
返回结果            - 200 （用数字表示，如页面不存在则以404返回）
浏览器类型          Mozilla/5.0+
系统等相关信息      X11;+U;+Linux+2.4.2-2+i686;+en-US;+0.7

日志实例

[root@CentOS5-C httpd]# tail -n 1 iis/ex121124.log
2012-11-24 15:59:58 W3SVC1244844107 10.0.2.71 GET /js/jquery-1.4.1.js - 80 - 14.145.137.144 Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1) 200 0 0
[root@CentOS5-C httpd]# tail -n 1 nginx/access.2012-11-24
42.51.134.75 - - [24/Nov/2012:23:59:59 +0100] "POST /user/login/doLogin HTTP/1.1"200 5137 Referer:"http://www.xqhan.com/user/login?ADTAG=guangzhou_vivid_login"UA:"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
[root@CentOS5-C httpd]# tail -n 1 apache/api-access_log
113.108.89.16 - - [27/Nov/2012:03:00:02 +0100] "GET /caibei HTTP/1.0" 200 20374539 "http://cb.qq.com/shop/tuan/spider" "-"

============================================

172.16.34.80 - - [27/Nov/2012:12:45:24 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)" "-"
172.16.34.80 - - [27/Nov/2012:12:45:24 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)" "-"
172.16.34.80 - - [27/Nov/2012:12:45:25 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)" "-"
172.16.34.80 - - [27/Nov/2012:12:45:25 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)" "-"
[root@CentOS5-D logs]# cat access2.log
172.16.34.80 - - [27/Nov/2012:12:47:53 +0800] "GET / HTTP/1.1" 304 158 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)" "-" "-"
172.16.34.80 - - [27/Nov/2012:12:47:53 +0800] "GET / HTTP/1.1" 304 158 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)" "-" "-"
172.16.34.80 - - [27/Nov/2012:12:47:54 +0800] "GET / HTTP/1.1" 304 158 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.3)" "-" "-"
172.16.34.80 - - [27/Nov/2012:12:47:54 +0800] "GET / HTTP/1.1" 304 158 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR

angzhou_vivid_login""Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
42.51.134.69 - - [24/Nov/2012:00:00:56 +0100] "POST /user/login/doLogin HTTP/1.1"200 5139 "http://www.xqhan.com/user/login?ADTAG=guangzhou_vivid_login""Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
42.51.134.84 - - [24/Nov/2012:00:00:56 +0100] "POST /user/login/doLogin HTTP/1.1"200 5138 "http://www.xqhan.com/user/login?ADTAG=guangzhou_vivid_login""Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
42.51.1