Back to business
Posted by Pepijn Oomen
Christmas holidays are over, and although I do not go back to work before tomorrow, I had to get started today anyways. The load on one of the back-end servers was way too high, but we still are missing a proper tool to identify the real culprit. It turned out that most of the problems were caused by just two IP addresses which were very busy with collection all pages of sites hosted at us. After blacklisting those things became much more manageable.
But still, a good realtime analysis tool would be helpful. We run ntop in promiscuous mode right behind the firewall, but this does not really show the type of traffic we are after. Spread looks kinda promising, but I just discovered that our internal firewall then needs a kernel change to enable multicast routing. I am not putting a management machine directly in the DMZ.
Update
Just had another look at apachetop and the author mentions in the ChangeLog that he did have spread support, but removed it:
remove mod_log_spread code; I'm not happy with including this since I have no idea how it works, no idea how the new filecode breaks it, and I haven't had chance to test it. This will be re-introduced when I can test it.
Then it struck me that spread/apachetop might be the right combination for our environment/requirements. I do not need the mod_log_spread code in apachetop, just use mod_log_spread/spread to distribute the logs to our management server and use spreadlogd combined with apachetop to do the analysis in real time :)
Now, if we only could get that through a web-interface?
