Thanks for visiting this documentation, I truly believe you've made the right decision, as this contains all I have to offer in terms of learning how to use this powerful tool in the spirit of the way it was originally designed, and intended to be used.
My name is Richard Forth and I am the author of apache2buddy.pl. In this document I offer my own personal insights into history, new features, usage, best practice, and tips and tricks to get the best experience with apache2buddy.pl for yourself and your customers - be they yourself, internal reporting lines or external customers or peers.
This document is aimed at System Administrators who work with apache webservers on a regular basis, and have a need to tune and troubleshoot apache installations at a variety of levels of technical ability. Ideally you will have 18 months' minimum experience working with Linux systems. This document assumes you are already technical at the intermediate to advanced level.
I hope this document delivers all the knowledge it promises whilst keeping you engaged and entertained.
Be aware that this is a living document that will be updated regularly, if you find a bullet point above that has not been covered in as much detail as you like, or is missing after a few weeks, raise an issue here and I will make sure to cover that topic in more depth / add relevant sections.
I havent added a navigation bar yet but there are plans to add navigation and support for mobile devices but that's quite a lot of CSS and jquery heavy lifting I need to do, for now the most important thing is content. Once I have completed the main bulk of my brain-dump on apache2buddy.pl, I will begin the polishing process.
If you're new to apache2buddy.pl, in order to really understand the evolution of the script that we know and love today, we have to first go back and meet the grandfather of apache tuning, apachebuddy.pl.
apachebuddy.pl was originally a collaborative project by several Rackspace employees, and was originally built specifically to support Red Hat Enterprise Linux systems. It has always been an open source project so it wasn't restricted to internal use at the Rack (so this isn't just an internal tool - its used by people all over the world), however, it was, shall we say mainly used by Rackspace employees to provide support to our customers running Red Hat Enterprise Linux Systems and the Linux, Apache, MySQL, and PHP (LAMP) stack. Without revealing too much in terms of Intellectual Property or internal processes of Rackspace, we have to acknowledge it's birthplace to get a true grip on the factors that influenced it's development.
In apachebuddy.pl's hey-day, Red Hat Enterprise Linux (RHEL) was the only available operating system we sold / supported on the Linux support floor, so of course it made sense to code apachebuddy.pl accordingly, this may go some way to explain the difference in the size of the code base of apachebuddy.pl when we compare it with apache2buddy.pl (see Fig 1), since it was only written to cater to RHEL systems:
Of course that was 5+ years ago now and times move on, a need arose for further support, as support in the Linux Department expanded to cater mainly for Ubuntu and Debian systems, in addition to Red Hat Enterprise Linux, and it's open source cousin, CentOS. However the apachebuddy project was abandoned by that point and is no longer maintained, but was kept on a external facing github account and has the domain "apachebuddy.pl".
You can view the original project here.
If you're also into raw code, you can view it at the domain above or click here.
The idea behind the domain pointing at the raw code was to allow one to run apachebuddy.pl through a commandline process known as "curl and perl", where you curl the domain and pipe it into perl (see Fig 2):
# curl -sL apachebuddy.pl | perl
Note: the domain apachebuddy.pl points to https://raw.githubusercontent.com/will-parsons/apachebuddy.pl/master/apachebuddy.pl, NOT the "gusmaskowitz" one.
It's important to note that no one maintains apachebuddy.pl, it is an abandoned project that is hosted for posterity reasons. Some people still use it, but many people have now started using apache2buddy.pl instead, so lets find out why....
The .pl dns top level domain is indeed Polish. However, it is (probably not in the spirit of the IETF/IANA but hey, people are people) used (possibly inappropriately but from a tech point-of-view is a stroke of genius) to host perl scripts because they also happen to share the same file extension (.pl). As a thank you to the polish people I may one day translate this page in to polish! #challengemuch #przepraszam
One of the most iconic examples of these domains is the MySQLTuner project, looked after by Major Hayden. mysqltuner is hosted on the mysqltuner.pl domain. This may also explain the phenomenon known as the perl and curl method we discussed earilier.
I spoke to Will Parsons who hosts the project on his github account, about 5 years ago (circa 2013-14), asking him if there were any plans to update it to support Debian and Ubuntu Systems.
I was told that the code wasn't actually his, and that it was no longer being maintained anyway, and he had no plans to make any updates to apachebuddy.pl.
I asked him if he didnt mind if I took a copy of it and forked it into a new project, to see what I could do to solve the problems with it not running on Debian or Ubuntu Systems.
I say "fork" here loosely, as at that time I had no github/git training or knowledge, so I copied the code into a vim editor and originally hosted the apache2buddy project and its associated domain on a LAMP server running in the cloud. I editted the file manually eacg tiime it needed updating, and had no change control or proper backups in place.
This, as you can imagine, wasnt very easy to maintain, but, over time I grew to learn and adopt git/github repositories and learnt how to commit changes, slowly over time I migrated away from a LAMP server to hosting the code on my own github repository:
Fig 4: The home of apache2buddy.pl
Originally I had the domain apache2buddy.pl pointed to the raw code, just like Will did with the original apachebuddy.pldomain (see curl and perl method).
However I got itchy feet and a tick in my face whenever I gave any thought to the fact that the curl and perl method was open to malicious bad actors typocamping/typosquatting  on slightly mispelled domains. It kept me up at night that this also had to be done as root, on systems globally. Even though I knew I wasn't really responsible for end users doing something silly like curling and mistyping a domain name, and then piping that into perl, as root (and enabling some bad actor which maybe registered a mis-spelt version of the apache2buddy.pl domain in the hope they catch a fish with a malicious root compromise perl script or bitcoin miner or something equally unwelcome).
So to that end I pointed the domain away from the code and forced people who still wished to curl and perl, use the whole complete github path to the raw code, since it was much more difficult to land in hot water due to a mistyped URL:
Fig5: The curl and perl method we ran originally. This no longer works, you will get errors.
Fig 6: The curl and perl method we run today.
While the first example looks more aesthetically pleasing to the eye, and is easy to remember, as commands go, it was vulnerable to a typosquatting attack, consider the following BAD command (trigger warning - I have no idea if that domain is registered and if it has a malicious perl code behind it - it's just an example to illustrate what I mean by typo-squatting attacks).
Fig 7: An example of a mistyped domain in a command that could run other, potentially arbitraryor malicious perl code (as root).
Did you notice in Figure 7 that there were three d's in this last example? Suppose someone wrote a bad perl script and you mistyped the domain as above.... well, through your own fat, sausage fingers, you now potentiially aided in a third party root compromising your server. Slow clap...
Though I could never take responsibility for your own faux pas, I decided to no longer support using the domain name as part of the curl and perl technique as a saftey precaution, and so repointed it - originally to a blog, then to this page (you get some random perl errors because perl would try to parse HTML).
For about a year during the transition amd before I finally pointed the domain to a wordpress blog, effectively breaking it, I first created a simple perl script that printed a public service announcement on the terminal window, advising them that the new method was as per Fig 5 (above), and that eventually curling the domain and piping it into perl would not work, and pointed the domain to that.[ reference needed ]
That pretty much brings you up to speed on how apache2buddy.pl came to be, and how I got involved, without delving too deep into code, but don't be fooled - I'm breaking you in gently...
The original apachebuddy.pl had a couple of major shortcomings. Firstly it only ever gave you a calculation of Maxclients/MaxRequestWorkers based on total installed RAM and not taking into consideration any running services that consume RAM.
Secondly, but more pressingly at the time, the main issue was that apachebuddy.pl only worked on Red Hat Enterprise Linux systems, it didn't run on Ubuntu or Debian systems* which came into support at Rackspace around 5 years ago, a solution had to be found.
* I beleive it does now, but it is not maintained, and hasnt been committed to in over 5 years, and can be hit and miss.
Fig3: The original and unmaintained apachebuddy.pl script output, not to be confused with apache2buddy.pl. It doesnt like CentOS 7 / apache 2.4 it seems.
That demo using CentOS 7 backfired, however I posted it's output as a show of just how unreliable it has become.
Fig3a: Pretty sure thats apache.
I will do another run on a CentOS 6 machine to get you a copy of the full output. Here goes....
Fig3: The original and unmaintained apachebuddy.pl script output from CentOS 6, not to be confused with apache2buddy.pl. It doesnt like CentOS 7 / apache 2.4 it seems
Here we see a successful run on a CentOS 6.10 machine. But since it doesnt run on CentOS 7, its quickly becoming less functional. Note also how it has calculated a maxclients value based on 990MB of RAM and a largest proces size of 4.74 MB of memory. This is a fairly rudimentary calculation that doesn't consider any other running services on the system that might consume memory. So the recommendation based on total installed RAM is of limited value, what we need is a calculation based on remaining RAM after key services have been taken into consideration.
Though it seems like I might be just completely dismissive of apachebuddy.pl, I actually owe it a ton of respect, like a village elder, as apachebuddy.pl is the forefather of apache2buddy.pl, which builds upon the foundations laid down by apachebuddy.pl originally.
Apache2buddy caught the attention of TechRepublic a few years back and they wrote an article telling people "How to tune apache webserver in seconds". This was pretty awesome and I was stoked that I made a respected online publication.
You can read that article here.
These reviews were completely independent, I have no idea who wrote them or how they came to be, they were certainly not solicited by me…..but…..welcome all the same :)
At the end of the day, I can only do so much testing, and I chose, for my own sanity, to only support the big four OSes, Debian, and its most famous branch-off - Ubuntu, and of course RHEL, and the two other major distros based on that CentOS and Scientific.
I have never prevented anyone from hacking their own private version of the script, so long as they adhere to the spirit of the Apache 2.0 License.
So lets go on a journey of discovery as I walk you through a fictional scenario: we are going to fudge apache2buddy to run on an unsupported OS Version.
You have a requirement to tune apache on a CloudLinux 7 machine, but apache2buddy exits with an error stating that it only currently supports a certain list of RED Hat based OSes, but we know that CloudLinux is based on Red Hat Enterprise Linux.
We are going to download the code locally with wget, and then hack the perl script and make some changes, and try to execute it again.
Our first job is to download the perlscript locally, lets save it to root's home area, /root
# wget -O /root/apache2buddy.pl https://raw.githubusercontent.com/richardforth/apache2buddy/master/apache2buddy.pl
Fig 8: Saving the script locally with wget.
Before we proceed further, I do want to talk a bit about editors. Honestly its up to you what editor you use, if it's nano, a small kitten dies so lets avoid that, just kidding. I recommend that you use an editor that allows you to quickly and easily identify line numbers, and it must be a command-line editor (the assumption here is your production servers arent running a GUI (though seriously one time I found a rackmounted server running X windows and gnome-screensaver....take as long as you need on that one...
Anyway the point I am getting at is that in these examples I will be walking you through the edits with vi/vim, and using line numbers that made sense as of this writing, but as you know time stands still for no man and, well, line numbers may not be the same in a few years time. Its still an active project.
So if I say, on line 80 change this value to that value, and the line i am quoting loks nothing like the line 80 youre working on, have a look in that general area of the code, as things get added or removed, the code I may have been referring to may not be on line 80 any more, but it won't have drifted too far I would'nt have thought. Use your noggin before you complain.
So, if you havent already, make sure you have the following in your .vimrc file in root's home as a minimum:
Fig 9: Setting line numbers in vim using a runtime config file /root/.vimrc
Now edit the file:
# vim /root/apache2buddy.pl
Fig 10: Opening up apache2buddy.pl in the vim editor.
Next, we go to line 337 and heed the warning, you can either scroll down untill you get to line 337 or press ESC and then type:
Fig 11: Moving quickly to line 337 in the vim editor.
Observe the warning in the comments:
# Please dont make pull requests to add your distro to this list, that doesnt make it supported. # The following distros are what I use to test and deploy apache2buddy and only these distro's are supported.
Fig 12: Comments save lives, always read comments.
Add "CloudLinux" to the list of supported Oses:
my @supported_os_list = ('Ubuntu', 'ubuntu', 'Debian', 'debian', 'Red Hat Enterprise Linux', 'Red Hat Enterprise Linux Server', 'redhat', 'CentOS Linux', 'CentOS', 'centos', 'Scientific Linux', 'SUSE Linux Enterprise Server', 'SuSE', 'CloudLinux');
Fig 13: Adding CloudLinux to the list of supported OSes.
Next we need to move to line 360
Fig 14: Moving quickly to line 360 in the vim editor.
Again, add "CloudLinux" to the list of supported Oses:
my @redhat_os_list = ('Red Hat Enterprise Linux', 'redhat', 'CentOS Linux', 'Scientific Linux','CloudLinux');
Fig 15: Adding CloudLinux to the list of supported OSes.
I know what youre thinking; this was way too easy, why dont I just add support for cloudlinux and be done with it.
So.... all we have done so far is tricked the AI here into thinking that apache2buddy will run fine on CloudLinux. but of course, theres no guarrantee it will.
I have even added a --skip-os-version-check option to make it even easier to execute:
# curl -sL https://raw.githubusercontent.com/richardforth/apache2buddy/master/apache2buddy.pl | perl - --help Usage: apache2buddy.pl [OPTIONS] If no options are specified, the basic tests will be run. -h, --help Print this help message -p, --port=PORT Specify an alternate port to check (default: 80) --pid=PID Specify a PID to bypass the "Multiple PIDS listening on port 80" error. -v, --verbose Use verbose output (this is very noisy, only useful for debugging) -n, --nocolor Use default terminal color, dont try to be all fancy! -H, --noheader Do not show header title bar. -N, --noinfo Do not show informational messages. -K, --no-ok Do not show OK messages. -W, --nowarn Do not show warning messages. -L, --light-term Show colours for a light background terminal. -r, --report Implies -HNWK or --noinfo --nowarn --no-ok --noheader --skip-maxclients --skip-php-fatal --skip-updates -P, --no-check-pid DON'T Check the Parent Pid File Size (only use if desperate for more info, results may be skewed). --skip-maxclients Skip checking in maxclients was hit recently, can be slow, especialy if you have large log files. --skip-php-fatal Skip checking for PHP FATAL errors, can be slow, especialy if you have large log files. --skip-updates Skip checking for package updates, can be slow or problematic, causing the script to hang. -O, --skip-os-version-check Skips past the OS version check. Allows one to bypass EOL version showstopper but be mindful: skipping the os version check is not recommended as features may be deprecated or removed and apache2buddy is not backward compatible with end of life operating systems, this may cause errors and unpredictable behaviour. Key: [ -- ] = Information [ @@ ] = Advisory [ >> ] = Warning [ !! ] = Critical #
Fig 16: Help, outlining the option to skip OS version checks altogether, note that things might not go as well as you might hope.
The method I outlined here will get you so far, but you may have to fix other errors that crop up as you'll be running apache2buddy.pl in unknown territory. The above edits may or may not allow the script to run to completion.
If you are a really competent perl / python coder, you can do what psytester did and submit a pull request to add OS support, which psytester did to add support to SuSE Linux systems, I merged psytester's changes here, but note that I haven't acknowledged that SuSE is a Supported OS as I do not have the capability or capacity to test against SuSE systems. If it works, then great, if it does'nt, one hopes that psytester will submit further pull requests in the future to maintain support for SuSE.
Let's deal with the elephant in the room: apache2buddy.pl produces a LOT of output, and it is NOT designed to be the club with which you beat your customers over the head.
Think of it like a dashboard, or a fighter-pilot's HUD (Heads Up Display), so that you, the discerning System Administrator, can use the information to help you come to a business decision and use PARTS of the output, to help your case when influencing others, be they a customer in a support ticket, your technical director, server teams, management, or other invested parties.
I can categorically affirm that it was not built to be user friendly or customer-facing - it's a technical tool for technical people. That's not to say that technically minded end-users are forbidden from running it, but if you as a Systems Admnistrator are slapping your non-technical customers with the full output from apache2buddy.pl, or even apachebuddy.pl - like the Hammer of Thor - then you're not using it correctly.
In the next section we will learn how to get, execute, and run the script, as well as how to interpret the results and present your case in your own words, appropriately, using small snippets of output to highlight certain facts. At the end of that section, I will do a fake update/email/ticket that shows best practice in using apache2buddy findings effectively in your communications.
In this section I walk you through an execution run of apache2buddy.pl, breaking down relevant sections and providing a commentary around it, what I check, how it checks, and any remedial action necessary. I will finish up with an example update to a customer using apache2buddy as my guide and reference point.
Before we break it down, let's take a look as a full execution run output. We can then take some time to break each section down for discussion.
[root@web1 ~]# curl -sL https://raw.githubusercontent.com/richardforth/apache2buddy/master/apache2buddy.pl | perl ############################################################ apache2buddy.pl report for web1.example.com (xxx.xxx.xx.xxx) ############################################################ [ OK ] This script is being run as root. [ OK ] The utility 'pmap' exists and is available for use: /usr/bin/pmap [ OK ] The utility 'netstat' exists and is available for use: /usr/bin/netstat [ OK ] 'php' exists and is available for use: /usr/bin/php [ OK ] The utility 'apachectl' exists and is available for use: /usr/sbin/apachectl [ OK ] The 'python' binary exists and is available for use: /usr/bin/python [ OK ] The port (port 80) is a valid port. [ -- ] We are attempting to discover the operating system type and version number ... [ -- ] Distro: CentOS Linux [ -- ] Version: 7.6.1810 [ -- ] Codename: Core [ OK ] This distro is supported by apache2buddy.pl. [ OK ] This distro version is supported by apache2buddy.pl. [ -- ] Hostname: web1.example.com [ -- ] Primary IP: xxx.xxx.xx.xxx [ -- ] We are checking the service running on port 80... [ -- ] The process listening on port 80 is /usr/sbin/httpd [ -- ] The process running on port 80 is Apache/2.4.6 (CentOS). [ -- ] The full path to the Apache config file is: /etc/httpd/conf/httpd.conf [ -- ] Apache is using prefork model. [ -- ] pidfile setting is CONFIG NOT FOUND. [ -- ] Actual pidfile is /run/httpd/httpd.pid. [ -- ] Parent PID: 4769. [ OK ] Memory usage of parent PID is less than 50MB: 5396 Kilobytes. [ -- ] Apache has been running 0d 01h 03m 25s. [ !! ] *** LOW UPTIME ***. [ @@ ] The following recommendations may be misleading - apache has been restarted within the last 24 hours. [ -- ] Your server has 985 MB of PHYSICAL memory. [ >> ] ServerLimit directive not found, assuming default values. [ -- ] Your ServerLimit setting is 256. [ >> ] MaxRequestWorkers directive not found, assuming default values. [ -- ] Your MaxRequestWorkers setting is 256. [ OK ] Current Apache Process Count is 6, including the parent PID. [ -- ] Number of vhosts detected: 0. [ OK ] Current Apache vHost Count is less than maxrequestworkers. [ @@ ] vHost Count works only when we have NameVirtualHosting enabled, check config manually, they may only have the default vhost. [ >> ] MaxRequestsPerChild directive not found. [ -- ] This server is NOT running Plesk. [ -- ] This server is NOT running cPanel. [ -- ] This server is NOT running Virtualmin. [ -- ] Your PHP Memory Limit (Per-Process) is 128M. [ OK ] No additional services were detected. [ OK ] No large log files were found in /var/log/httpd. [ OK ] MaxClients has not been hit recently. [ >> ] Apache only logs maxclients/maxrequestworkers hits once in a lifetime, if no restart has happened this event may have been rotated away. [ >> ] As a backup check, please compare number of running apache processes (minus 1 for parent) against maxclients/maxrequestworkers. [ >> ] For more information see https://github.com/apache/httpd/blob/0b61edca6cdda2737aa1d84a4526c5f9d2e23a8c/server/mpm/prefork/prefork.c#L809 [ OK ] No PHP Fatal Errors were found. [ OK ] No package updates found. [ -- ] httpd is currently using 41.52 MB of memory. [ -- ] The smallest apache process is using 5.27 MB of memory [ -- ] The average apache process is using 5.27 MB of memory [ -- ] The largest apache process is using 5.27 MB of memory [ !! ] Going by the average Apache process, Apache can potentially use 1349.12 MB RAM: Without considering services: 136.97 % of total installed RAM Considering extra services: 136.97 % of remaining RAM [ !! ] Going by the largest Apache process, Apache can potentially use 1349.12 MB RAM: Without considering services: 136.97 % of total installed RAM Considering extra services: 136.97 % of remaining RAM -------------------------------------------------------------------------------- ### GENERAL FINDINGS & RECOMMENDATIONS ### -------------------------------------------------------------------------------- Apache2buddy.pl report for server: web1.example.com (xxx.xxx.xx.xxx): Settings considered for this report: [ !! ] *** LOW UPTIME ***. [ @@ ] The following recommendations may be misleading - apache has been restarted within the last 24 hours. Your server's physical RAM: 985 MB Remaining Memory after other services considered: 985 MB Apache's MaxRequestWorkers directive: 256 <--------- Current Setting Apache MPM Model: prefork Largest Apache process (by memory): 5 MB [ !! ] Your MaxRequestWorkers setting is too high. Your recommended MaxRequestWorkers setting (based on available memory) is between 167 and 186. <------- Acceptable Range (10% of MAX) Max potential memory usage: 1349 MB Percentage of TOTAL RAM allocated to Apache: 136.97 % Percentage of REMAINING RAM allocated to Apache: 136.97 % -------------------------------------------------------------------------------- A log file entry has been made in: /var/log/apache2buddy.log for future reference. Last 5 entries: 2019/01/19 10:53:17 Uptime: "0d 01h 03m 25s" Model: "Prefork" Memory: "985 MB" MaxRequestWorkers: "256" Recommended: "186" Smallest: "5.27 MB" Avg: "5.27 MB" Largest: "5.27 MB" Highest Pct Remaining RAM: "136.97%" (136.97% TOTAL RAM)
Fig 17: Output from apache2buddy.pl - not to be used as a club to beat your customers into submission.
Next we will inspect and discuss the relevant parts of the output in more detail.
Analysis to follow.
Issues can be raised directly on GitHub here.