Advanced Apache Configuration for High Traffic Sites

Created by: spiderr, Last modification: 30 Dec 2006 (11:26 UTC) by xing

httpd.conf VirtualHost settings

Install cronolog. It's cool, fast, and simple. The following VirtualHost block is designed to make efficient log files by removing the junk and the bots. This allows log analysers, like our favorite wusage, to go a hell of a lot faster. You can also analyse the bots separately from the real people.


<?php
<VirtualHost *:80>
    
ServerAdmin root@bitweaver.org
    ServerName www
.bitweaver.org
    DocumentRoot 
/home/bitweaver/live
    ErrorLog logs
/bitweaver/error_log
# Do not log images and junk - adjust to your site since file sharing abuses can be detected this way
    
SetEnvIf Request_URI "(\.gif|\.png|\.jpg|\.css|\.js|jscompressor.php|bitversion.txt|\.ico|ie7-load\.htc)$" nolog
    
# Pull out all the bots and log them to a different place
    
SetEnvIf User-Agent ".*(Accoona|almaden|Bot|Baiduspider|ichiro|JemmaTheTourist|Netcraft|bot|oogle|dir.com|Yahoo|Slurp|slurp|herlock|cfetch|Crawler|ia_archiver|Teoma|LinkWalker|grub|sohu|nusearch|updated.com|voyager|www.walhello.com).*" NOLOG=nolog
    SetEnvIf User
-Agent ".*(Accoona|almaden|Bot|Baiduspider|ichiro|JemmaTheTourist|Netcraft|bot|oogle|dir.com|Yahoo|Slurp|slurp|herlock|cfetch|Crawler|ia_archiver|Teoma|LinkWalker|grub|sohu|nusearch|updated.com|voyager|www.walhello.com).*" BOT=bot
    
# Set up the log format to use mod_cookie
    
LogFormat "%h %l %u %t \"%r\" %>s %b %v \"%{Referer}i\" \"%{User-agent}i\" \"%{Cookie}n\""  combinedcookie
    
# CustomLog pipes through cronolog and neatly organizes your log files by year. Having all _access and _error for 1 year in one directory is a good balance
    
CustomLog "|/usr/local/sbin/cronolog --symlink=/var/log/httpd/bitweaver/access_log /var/log/httpd/bitweaver/%Y/%Y-%m-%d-access_log" combinedcookie env=!nolog
    CustomLog 
"|/usr/local/sbin/cronolog --symlink=/var/log/httpd/bots/bot_log /var/log/httpd/bitweaver/bots/%Y/%Y-%m-%d_bot_access_log" combinedcookie env=bot

    CookieDomain 
.bitweaver.org
    CookieExpires 
"1 months"
    
CookieName  mod_usertrack
    CookieTracking on

</VirtualHost>
?>


php.ini configuration

Log all of your php data to a file!


<?php
Log errors to specified file.
error_log = /var/log/httpd/php_log
?>