One of the servers I manage runs CentOS 5.0 with Apache 2.2 and PHP 5.1.6, and uses PHP’s APC 3.0.14 (Alternative PHP Cache). We process the logs using AWStats every 15 minutes, and rotate the logs on a daily basis. However, after two or three days of working correctly, Apache would crash and we’d get the following in the log files: “seg fault or similar nasty error detected in the parent process”
After some amount of searching on the Internet, the only thing I could come up with was that there was a bug in the PCRE extension in older versions of PHP that would cause the same message to occur in the error logs. One of the posts indicated that this particular message is also a symptom for lots of other memory-type bugs in Apache, so I can only assume that it’s the use of APC which is causing the error.
The interesting thing is that if Apache is reloaded or reloaded with “graceful” a few minutes later, then there aren’t any segfaults. Again, I can only guess that the memory issue causing the segfault is due to issues with the memory caching over time.
The script that processes the logs with AWStats, and also rolls the logs on a daily basis is as follows:
#!/bin/bash if [ `date +%H%M` == 0600 ] then mydate=`date +%Y%m%d` cd /var/log/httpd mv mylog.log mylog.log.$mydate /sbin/service httpd reload /var/www/awstats/cgi-bin/awstats.pl -update -config=myconfig -LogFile=mylog.log.$mydate > /dev/null bzip2 mylog.log.$mydate else /var/www/awstats/cgi-bin/awstats.pl -update -config=myconfig > /dev/null fi
A quick summary of what this is: if it’s 6am in the morning, the current log file is renamed with the current day’s date (eg mylog.log.20070921), processed using AWStats, and then compressed with bzip2. At all other times during the day, just AWStats is called.
The error message that I got in the log files was as follows:
[Fri Sep 21 06:00:03 2007] [notice] SIGHUP received. ?Attempting to restart [Fri Sep 21 06:00:03 2007] [notice] seg fault or similar nasty error detected in the parent process
After the attempt to reload Apache and getting this error message, Apache failed to start up again. Calling “service httpd start” got it running again.
The solution I have found is to stop and start Apache. There may be some other way of sorting this memory issue with APC out but I was unable to find a solution for it. So the script now looks like this:
#!/bin/bash if [ `date +%H%M` == 0600 ] then mydate=`date +%Y%m%d` cd /var/log/httpd mv mylog.log mylog.log.$mydate /sbin/service httpd stop /sbin/service httpd start /var/www/awstats/cgi-bin/awstats.pl -update -config=myconfig -LogFile=mylog.log.$mydate > /dev/null bzip2 mylog.log.$mydate else /var/www/awstats/cgi-bin/awstats.pl -update -config=myconfig > /dev/null fi
Since changing it, I have had no further segfaulting issues.
