使用awstats监控Tomcat日志

原创

sunzhandong 2015-12-01 17:09:32 ©著作权

文章标签 awstats 日志监控 tomcat awstats 文章分类 运维

©著作权归作者所有：来自51CTO博客作者sunzhandong的原创作品，请联系作者获取转载授权，否则将追究法律责任

awstats 是由perl语言编写的，所以要首先准备好awstats的运行环境。

# yum install –y perl

1、下载安装awstats

awstats install 文件夹

# cd /opt
# wget http://www.awstats.org/files/awstats.tar.gz
# tar xvf awstats.tar.gz 
# mv awstats/usr/local/awstats

2、创建一个tomcot统计

[root@portal-1 tools]cd /user/local/awstats/tools
[root@portal-1 tools]# perl awstats_configure.pl 
----- AWStats awstats_configure 1.0 (build 1.9) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).
-----> Running OS detected: Linux, BSD or Unix
-----> Check for web ser

ver install
Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path ('none' to skip web server setup):

> none    //tomcat 设置为空
Your web server config file(s) could not be found.
You will need to setup your web server manually to declare AWStats
script as a CGI, if you want to build reports dynamically.
See AWStats setup documentation (file docs/index.html)
-----> Update model config file '/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf'
  File awstats.model.conf updated.
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.ejoywifi.com

-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
>

-----> Create config file '/etc/awstats/awstats.www.ejoywifi.com.conf'
 Config file /etc/awstats/awstats.www.ejoywifi.com.conf created.
-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your cron:
/usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.ejoywifi.com
Or if you have several config files and prefer having only one command:
/usr/local/awstats/tools/awstats_updateall.pl now
Press ENTER to continue... 
A SIMPLE config file has been created: /etc/awstats/awstats.www.ejoywifi.com.conf
You should have a look inside to check and change manually main parameters.
You can then manually update your statistics for 'www.ejoywifi.com' with command:
> perl awstats.pl -update -config=www.ejoywifi.com
You can also build static report pages for 'www.ejoywifi.com' with command:
> perl awstats.pl -output=pagetype -config=www.ejoywifi.com
Press ENTER to finish…

3、tomcat7 logs 设置

配置tomcat日志文件格式

vi /data/portalCms/web/tomcat7_web/conf/server.xml
 
pattern="%h %u %t &quot;%r&quot; %s %b %{Referer}i &quot;%{User-Agent}i&quot;;" />

本服务器tomcat /data/portalCms/web/tomcat7_web

/data/portalCms/web/tomcat7_web/logs/web_access_log.2015-09-06.txt

/data/portalCms/web/tomcat7_web/logs/web_access_log.%YYYY-%MM-%DD.txt

tomcat输出的日志格式

10.238.38.190 - [21/Oct/2015:15:39:09 +0800] "GET /favicon.ico HTTP/1.1" 200 310 http://10.248.2.10/awstats/cgi-bin/awstats.pl?config=www.ejoywifi.com

"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36"

4、修改awstats配置文件

[root@portal-1 awstats]# vi /etc/awstats/awstats.www.ejoywifi.com.conf

#LogFile="/var/log/httpd/mylog.log"

设置tomcat日志路径
LogFile=“/data/portalCms/web/tomcat7_web/logs/web_access_log.%YYYY-%MM-%DD.txt"
设置日志格式通配
LogFormat =“%host %logname %time1 %methodurl %code %bytesd %referer %uaquot”
设置图标调用，默认为 /icon，显示有问题
DirIcons="./icon"

awstats logformat:

# %host Client hostname or IP address (or Sender host for mail log)

# %host_r Receiver hostname or IP address (for mail log)

# %lognamequot Authenticated login/user with format: "john"

# %logname Authenticated login/user with format: john

# %time1 Date and time with format: [dd/mon/yyyy:hh:mm:ss +0000] or [dd/mon/yyyy:hh:mm:ss]

# %time2 Date and time with format: yyyy-mm-dd hh:mm:ss

# %time3 Date and time with format: Mon dd hh:mm:ss or Mon dd hh:mm:ss yyyy

# %time4 Date and time with unix timestamp format: dddddddddd

# %methodurl Method and URL with format: "GET /index.html HTTP/x.x"

# %methodurlnoprot Method and URL with format: "GET /index.html"

# %method Method with format: GET

# %url URL only with format: /index.html

# %query Query string (used by URLWithQuery option)

# %code Return code status (with format for web log: 999)

# %bytesd Size of document in bytes

# %refererquot Referer page with format: "http://from.com/from.htm"

# %referer Referer page with format: http://from.com/from.htm

# %uabracket User agent with format: [Mozilla/4.0 (compatible, ...)]

# %uaquot User agent with format: "Mozilla/4.0 (compatible, ...)"

# %ua User agent with format: Mozilla/4.0_(compatible...)

# %gzipin mod_gzip compression input bytes: In:XXX

# %gzipout mod_gzip compression output bytes & ratio: Out:YYY:ZZpct.

# %gzipratio mod_gzip compression ratio: ZZpct.

# %deflateratio mod_deflate compression ratio with format: (ZZ)

# %email EMail sender (for mail log)

# %email_r EMail receiver (for mail log)

# %virtualname Web sever virtual hostname. Use this tag when same log

# contains data of several virtual web servers. AWStats

# will discard records not in SiteDomain nor HostAliases

# %cluster If log file is provided from several computers (merged by

# logresolvemerge.pl), use this to define cluster id field.

# %extraX Another field that you plan to use for building a

# personalized report with ExtraSection feature (See later).

# If your log format has some fields not included in this list, use:

# %other Means another not used field

# %otherquot Means another not used double quoted field

5、测试执行读取日志更新

[root@portal-1 cgi-bin]# perl awstats.pl -update -config=www.ejoywifi.com
Error: AWStats database directory defined in config file by 'DirData' parameter (/var/lib/awstats) does not exist or is not writable.
Setup ('/etc/awstats/awstats.www.ejoywifi.com.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).
#用来存放awstats的数据文件
[root@portal-1 /]# mkdir -m 755 /var/lib/awstats

手动生成awstats日志分析数据库信息
perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.ejoywifi.com
[root@portal-1 conf]# perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.ejoywifi.com
Create/Update database for config "/etc/awstats/awstats.www.ejoywifi.com.conf" by AWStats version 7.0 (build 1.975)
From data in log file "/data/portalCms/web/tomcat7_web/logs/web_access_log.2015-10-13.txt"...
Phase 1 : First bypass old records, searching new record...
Direct access after last parsed record (after line 1202)
Jumped lines in file: 1202
 Found 1202 already parsed records.
Parsed lines in file: 4
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 4 new qualified records.

6、创建awstats日志报告静态html目录

mkdir -p /data/portalCms/web/www/static/awstats/
cp -R /usr/local/awstats/wwwroot/css /data/portalCms/web/www/static/awstats/
cp -R /usr/local/awstats/wwwroot/icon /data/portalCms/web/www/static/awstats/

7、手动生成html静态页面报告

[root@portal-1 /]# /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.ejoywifi.com -lang=cn -dir=/data/portalCms/web/www/static/awstats/wifiportal -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl

Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -update -configdir=
Build main page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output
Build alldomains page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=alldomains
Build allhosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=allhosts
Build lasthosts page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=lasthosts
Build unknownip page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=unknownip
Build allrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=allrobots
Build lastrobots page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=lastrobots
Build session page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=session
Build urldetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=urldetail
Build urlentry page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=urlentry
Build urlexit page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=urlexit
Build osdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=osdetail
Build unknownos page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=unknownos
Build browserdetail page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=browserdetail
Build unknownbrowser page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=unknownbrowser
Build refererse page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=refererse
Build refererpages page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=refererpages
Build keyphrases page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=keyphrases
Build keywords page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=keywords
Build errors404 page: "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.ejoywifi.com -staticlinks -lang=cn -output=errors404
20 files built.
Main HTML page is 'awstats.www.ejoywifi.com.html'.
[root@portal-1 /]#

8、放在crontab计划任务中

在上面我们只是手动生成了html报告文件，这里让它自动完成这些操作.

编辑/etc/crontab文件，在最后添加以下两行，每天23:58分重新生成一次统计页面

# crontab –e

58 23 * * * /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.ejoywifi.com -lang=cn -dir=/data/portalCms/web/www/static/awstats -awstatsprog=/usr/local/awstats/wwwroot/cgi-bin/awstats.pl > /dev/null 2&>1

9、访问静态页面

http://10.248.2.10:8080/static/awstats/awstats.www.ejoywifi.com.html

10、Tomcat7 开启CGI，并配置awstats日志监控

1. 修改<Tomcat_HOME>\conf\web.xml配置文件，取消 cgi servlet 和对应的 mapping 注释，让Tomcat支持cgi程序执行：

    <servlet>
    <servlet-name>cgi</servlet-name>
    <servlet-class>org.apache.catalina.servlets.CGIServlet</servlet-class>
    <init-param>
        <param-name>debug</param-name>
        <param-value>0</param-value>
    </init-param>
    <init-param>
        <param-name>cgiPathPrefix</param-name>
        <param-value>WEB-INF/cgi</param-value>
    </init-param>
    <init-param>
        <param-name>passShellEnvironment</param-name>
        <param-value>true</param-value>
    </init-param>
 <load-on-startup>5</load-on-startup>
</servlet>
 
<servlet-mapping>
    <servlet-name>cgi</servlet-name>
    <url-pattern>/cgi-bin/*</url-pattern>
</servlet-mapping>

11. 修改<Tomcat_HOME>\conf\context.xml配置文件，在Context上添加 privileged="true" 属性：

</Context>

12. 测试CGI是否成功，在<Tomcat_HOME>\webapps\ROOT\WEB-INF\cgi\下创建cgi脚本文件 test.cgi：

# filename: test.cgi

print "Content-type: text/html\n\n";

print "Hello, world!\n"

启动Tomcat，并访问：http://localhost:8080/cgi-bin/test.cgi，如果能看到 Hello, world! 页面，说明cgi开启成功。

13. a. 在<Tomcat_HOME>/webapps/下创建 awstats目录以及相应的WEB-INF目录

b. 将<AWSTATS_HOME>user/local/awststs/wwwroot/目录下 css/,icon/,js/目录拷贝到 <Tomcat_HOME>/webapps/awstats/ 目录下

c. 将<AWSTAS_HOME>user/local/awststs/wwwroot/cgi-bin/ 下的所有文件拷贝到 <TOMCAT_HOME>/webapps/awstats/WEB-INF/cgi 目录下

14. 拷贝ejoywifi配置文件至cig cp /etc/awstats/awstats.www.ejoywifi.com.conf /data/portalCms/web/tomcat7_web/webapps/awstats/WEB-INF/cgi