[frisco]

Hi-
I'm looking for recommendations for reporting software for different server aspects.

Right now we're using awstats (http://awstats.sourceforge.net/) for understanding web server logs. Anyone else have a better suggestion? Anything out there OSS that tracks specific user paths? For reference, our main website generates about 1 million log entries a day. Our secondary sites generate another couple million.

I'm thinking of using linkchecker (http://linkchecker.sourceforge.net/) across all our websites to look for broken links. Anything better out there i can use?

Right now i use the StatsView::Graph (http://search.cpan.org/author/ABURLISON/StatsView-1.4/Graph/Graph.pm) perl module to generate graphs out of Solaris sar output, and i think i could work it to also interpret *BSD *stat output. Is there anything else i should consider for generating these? Main reason i'm using that perl module is it already interprets the Solaris sar info, as opposed to me having to do that and then feed the results to a different graphing program.


All of these tools need to produce output that others can interpret, like the web developers for the weblogs and linkchecker, and our dba's for the sar/*stat output, so something that does html output, or something that can fit into html output, is ideal.

[molotov]

I wrote a perl script to split logs up based on search path and run awstats on each subsite, but it was for an old job of mine. I can try to get the script if you want. Just a warning, it is slow as all hell.

[frisco]

I wrote a perl script to split logs up based on search path and run awstats on each subsite, but it was for an old job of mine. I can try to get the script if you want. Just a warning, it is slow as all hell.

Thanks, i ended up writing something similar, a perl script which runs multiple streams of awstats, passing each one data depending on whether the logfile matches a particular pattern. The web folks wanted to split out not just some specific paths but also whether a hit is internal or external. Takes about 25 minutes to run through our logs, currently opening 9 simultaneous streams (solaris 10, v880). Don't know how well it'll scale, though...

Also i've been using checkbot (http://degraaff.org/checkbot/) instead of linkchecker since the former is perl but the latter is python, and i've perl on more systems. I don't like it too much, but the developers don't seem to mind so i'm settling - it has already helped us identify some problems, so i guess it's ok.