Advanced Apache Configuration for High Traffic Sites

Created by: spiderr, Last modification: 30 Dec 2006 (11:26 UTC) by xing

httpd.conf VirtualHost settings

Install cronolog. It's cool, fast, and simple. The following VirtualHost block is designed to make efficient log files by removing the junk and the bots. This allows log analysers, like our favorite wusage, to go a hell of a lot faster. You can also analyse the bots separately from the real people.

<VirtualHost *:80>
    ServerName www
    ErrorLog logs
# Do not log images and junk - adjust to your site since file sharing abuses can be detected this way
SetEnvIf Request_URI "(\.gif|\.png|\.jpg|\.css|\.js|jscompressor.php|bitversion.txt|\.ico|ie7-load\.htc)$" nolog
# Pull out all the bots and log them to a different place
SetEnvIf User-Agent ".*(Accoona|almaden|Bot|Baiduspider|ichiro|JemmaTheTourist|Netcraft|bot|oogle||Yahoo|Slurp|slurp|herlock|cfetch|Crawler|ia_archiver|Teoma|LinkWalker|grub|sohu|nusearch||voyager|*" NOLOG=nolog
    SetEnvIf User
-Agent ".*(Accoona|almaden|Bot|Baiduspider|ichiro|JemmaTheTourist|Netcraft|bot|oogle||Yahoo|Slurp|slurp|herlock|cfetch|Crawler|ia_archiver|Teoma|LinkWalker|grub|sohu|nusearch||voyager|*" BOT=bot
# Set up the log format to use mod_cookie
LogFormat "%h %l %u %t \"%r\" %>s %b %v \"%{Referer}i\" \"%{User-agent}i\" \"%{Cookie}n\""  combinedcookie
# CustomLog pipes through cronolog and neatly organizes your log files by year. Having all _access and _error for 1 year in one directory is a good balance
CustomLog "|/usr/local/sbin/cronolog --symlink=/var/log/httpd/bitweaver/access_log /var/log/httpd/bitweaver/%Y/%Y-%m-%d-access_log" combinedcookie env=!nolog
"|/usr/local/sbin/cronolog --symlink=/var/log/httpd/bots/bot_log /var/log/httpd/bitweaver/bots/%Y/%Y-%m-%d_bot_access_log" combinedcookie env=bot

"1 months"
CookieName  mod_usertrack
    CookieTracking on


php.ini configuration

Log all of your php data to a file!

Log errors to specified file.
error_log = /var/log/httpd/php_log