Using AWSTATS on downloaded log files
I'm working on analyzing downloaded web server logs and it took a while to find a suitable tool for it. I tried a couple of installables but the reports and information where just not up to par. If only Google analytics would accept these files then I would have been set. Anyways, I finally settled on using AWSTATS, I already had experience with it before and from the alternatives, it was as closest to a full featured analytical tool that is free I could come close to.
Installation is straightforward and I will not go in too much details on it. Let's just assume ActivePerl and AWSTATS have already been installed successfully. Let's then start with my setup, I used WinXP with no web servers installed. ActivePerl 5.10.1 build 1007 and AWSTATS 6.95 (build 1.943) was installed. When the CLI popped up during AWSTATS installation, I did not configure it for a web server.
After installing both, I then copied my log files onto the cgi-bin folder for easier typing when using CLI, although the log location manually entered in the log file, but it did not matter in the end. I did not need to configure anything else as I already have perl and awstats, the only thing needed to do was do a quick modifications on the profile config file.
I updated the LogFile paramater and pointed it to the location of the log file, it took a while for me to figure out that I needed to include the logs filename and not the folder where the logs where located. I then double checked if the default settings for LogType, LogFormat, LogSeparator, SiteDomain,HostAliases, DNSLookup, DirData,DirCgi,DirIcons and AllowToUpdateStatsFromBrowser where correct. I changed SiteDomain to my name to match the config file which was a big mistake. Make sure the value of the SiteDomain is the same with what is in the log files to avoid dropping all records, if you have a different value with what you have on the log files, the records will be dropped during parsing and your report will be empty. I also changed DirIcons but later on thought it was of no use coz I was gonna be generating static reports instead of viewing it from a site but it turns out I was correct as this is the source location for the icons used when generating static reports, the value should be based on where you will output your generated report. The rest, you can leave as default.
I was then able to parse all the logs and generate the report. to parse the logs use awstats.pl -config=<name of profile> -update. Do not be alarmed if you see corrupted records during parsing, it is normal to have such as long as it is around less than 5% of the total, any more than that and there probably is something wrong with the webserver, application or the logging settings. After parsing all the logs, it was now time to generate the reports, use the command awstats_buildstaticpages.pl -config=<name of profile> -month=<month to generate report on>, if you do not include the month parameter, report for the current month will be generated. I added -buildpdf=<location of htmldoc> and a PDF was generated after the static html reports You need to download and install HTMLDoc first.
My logfiles were also on separate files by date and it was a hassle to udpate the profile to process each log file, so I used logresolvemerge.pl to combine all the logs and process them once after.
That's it! I now had the report I needed. Good luck to everyone and I hope this helps.