[Linux] Monitoring web server - beginner's questions

[Danny Rathjens]

Steven Benmosh wrote:
> I am using Ubuntu 7.10 Server on a web site, and am experiencing 
> slowness. I would like to find out two things:
> 
> 1. Is my server really that overloaded, or is it the result of some 
> malicious action (some guy constantly looking at my web pages from one 
> or two addresses).

The only way to know is to look. top for cpu and ram usage totals and per process.
sar and iostat from the sysstat package for all sorts of info like swapping, load,
device usage, etc. reports over time so you can see when spikes occurred (sar is
like mrtg without graphics for nearly every stat; which I resolve with some custom
scripts to send all this other type of data into mrtg. heh)

You can also enable ExtendedStatus on apache to get detailed status from apache itself
per child process.  With that info you can fine tune apache settings to handle your
traffic as per http://httpd.apache.org/docs/2.2/misc/perf-tuning.html
And, as you surmised, you can use apache log analyzers to know what your web traffic is.

> 2. What are the statistics (how many views for each page, most popular 
> pages, geo-location or at least IP of page views, etc.)

I asked the bot in #apache@chat.freenode.net to list all of the log analyzing software.
I use awstats myself.

<fajita> analysis is ask me about one of the following: analog awstats webalizer visitors webtrax http-analyze awffull sawmill summary hitbox webtrends
<dkr> analog
<fajita> analog is http://www.analog.cx
<dkr> awstats
<fajita> awstats is http://awstats.sourceforge.net/ - However there are serious vulnerabilities in versions < 6.5
<dkr> webalizer
<fajita> webalizer is http://webalizer.org/
<dkr> visitors
<fajita> visitors is a very fast web log analyzer for Linux, Windows, and other Unix-like operating systems. Find it at http://www.hping.org/visitors/
<dkr> webtrax
<fajita> webtrax is a very mature log analysis tool, available at http://www.multicians.org/thvv/webtrax-help.html
<dkr> http-analyze
<fajita> http-analyze is yet another open-source log file analyzer, available in binary form for many platforms at http://http-analyze.org/
<dkr> awffull
<fajita> awffull is http://www.stedee.id.au/awffull
<dkr> sawmill
<fajita> sawmill is quite well laid out, has easy to read and well linked pages. It's available from http://www.sawmill.net/
<dkr> summary
<fajita> summary is a web server log analysis and traffic monitoring tool. Available at http://www.summary.net/ or not free.
<dkr> hitbox
<fajita> hitbox is a business-grade web analytics service http://www.hitbox.com/
<dkr> webtrends
<fajita> webtrends is nice too, if you have a boatload of money and nothing much to do with it. http://www.webtrends.com/