Memory-Usage
Often, the environments on which we tend to run web servers are also those which doesn’t come with an abundance of memory. Many websites are run on shared hosting or vps which commonly offer 1 to 2 gigs of RAM (some would opt for even less). That’s not much, considering the amount of people these servers are meant to serve. Inevitably, the amount of resources a web server uses may become no less critical part than its speed. Therefore a very important aspect of a web server is the amount of memory it requires to operate.
To make it as clear as possible I used a tool called ‘ps_mem’ which is a “utility to accurately report the in core memory usage for a program”. Each server was configured to use all of the 8 cores available on my machine.
Here are the results I got:
Server Memory Usage On Rest (After Initial Launch)
Private + Shared = RAM used Program (Number of Processes)
8.5 MiB + 5.3 MiB = 13.8 MiB apache2 (9)
1.2 MiB + 1.4 MiB = 2.6 MiB lighttpd (9)
14.9 MiB + 1.9 MiB = 16.8 MiB nginx (9)
Here we can see a couple of interesting things, the first and most obvious being Lighttpd is simply the most lightweight of the three, that’s probably not surprising anyone though. If we want to make the difference a bit more conspicuous, we can say that Nginx uses ~646% more memory than Lighttpd, and Apache ~530% more than Lighttpd. But remeber, these numbers represents the servers when they are on rest right after they were initially launched.
The second thing we can note is that Apache uses much more shared memory than Nginx does. That’s quite surprising because shared memory is commonly used to enhance performance by letting processes access the same memory simultaneously which should prevent them from creating redundant copies. So the surprising thing here is that Nginx which is geared towards high performance isn’t actually making much use of shared memory.
I took another measurment while stressing the servers a bit (10,000 requests) just to see where these numbers might escalate to, the results were:
Server Memory Usage While Stressed
Private + Shared = RAM used Program (Number of Processes)
30.2 MiB + 5.6 MiB = 35.8 MiB apache2 (9)
17.0 MiB + 1.6 MiB = 18.6 MiB lighttpd (9)
15.6 MiB + 2.4 MiB = 18.0 MiB nginx (9)
Now it seems that even when the servers are being stressed they still don’t hog a huge amount of memory, an amount which could crash your entire server that is.
The above results also shows us that while Apache and Lighttpd have both increased the amount of memory they use (Lighttpd actually went up about x7 more than on rest), Nginx on the other hand didn’t even break a sweat. But these numbers aren’t telling us the whole picture because after the stressing was done I also took a peek at the number of requests processed by each server and the reports showed that Apache had an average of 1045 failed requests, Nginx 1173 failed requests, and Lighttpd – 226 failed requests. So take it with a grain of salt.
The last relevant piece of information I think worth relating to in this section is that after the initial memory tests were done I continued gauging each server’s memory usage, I was particularly interested to know whether they would return to their initial state or not. I was a bit surprised to learn that non of the servers had return to their initial state of memory usage, though I guess that makes sense as the servers might have adjusted themselves to the amount of requests they are getting. But what really took me by surprise was that with each test I ran, and I didn’t change the test itself, both Apache and Lighttpd kept growing in memory usage – after seven test Lighttpd had grown up to using ~21.3MiB and Apache ~45.1MiB, while Nginx stablized at ~17.9 after the second test.
Server Memory Usage After Running A Number Of Tests
Private + Shared = RAM used Program (Number of Processes)
39.4 MiB + 5.6 MiB = 45.1 MiB apache2 (9)
19.6 MiB + 1.6 MiB = 21.3 MiB lighttpd (9)
15.6 MiB + 2.4 MiB = 17.9 MiB nginx (9)
Unfortunately it seems Apache won’t bust its being a resource-hog myth, nor would Lighttpd its buggy, memory-leak myths (evidently, they are not myths). In fact, both Apache and Lighttpd apparently have memory leaks.
RAM usage over time across 7 stressing tests
Category Rankings
1 Nginx
2 Lighttpd
3 Apache
Performance
For the purpose of comparing the servers’ performance I designed a simple test that involves sending HTTP GET requests to each server at a time. In each time, a server is tested against either of 2 criteria: 1) requests load, or, 2) concurrency load. For at the end of the day an http server’s job is to respond to requests and what we’re looking for is the server that could deliver the maximum responses in minimum time (and of course requiring minimum resources to do so).
To avoid as much deflection as possible – one that may be caused by external factors, all tests were made locally on the same machine, thus removing latency or different specs bias out of the way. In order to avoid other software from having an effect on the servers’ results, a simple static page was used for the tests, and the tool used for measuring each response – ApacheBench – did not parse any HTML nor did it downloaded any linked sources found on the page e.g. CSS files, JS files, images, etc…
As for the servers themselves, each was compiled with the features I thought were relevant for a standard web server (you can see which modules I had installed in the lists under the Features section). The configuration was tuned in to the maximum according to each server’s optimization guide. I have also tried different modules on each server to see what works best and I shall elaborate more on the results with different modules later on.
The results I present you are the average I got after 3 tests for each step along the way, they are summed in the following graphs:
Testing the servers against rising requests load
Testing the servers against rising concurrency load
As you can see, it was a close competition between Nginx and Lighttpd, especially in the concurrency area, Nginx did came on top though.
Unfortunately, I could not get Apache to behave nicely against the concurrency tests, it was unstable from the beginning with wide gaps between and within each mark. I actually did got it to perform 5,000 concurrency load however, the average result number was so high (14.732) that it skew the entire graph (therefore I excluded the result). Beyond 5,000 concurrent requests Apache simply kept throwing errors and refused to complete the tests, no matter the configuration I used (I raised MaxRequestWorkers, MaxClients, ThreadsPerChild, etc.. but to no avail).
Regarding the affect of different configurations on the performance, it seems that different configurations won’t make any server faster than it was to begin with, however getting the configuration right can be crucial in order for the server to be able to deal with different loads. This was quite obvious in Apache’s case where without raising the bar of MaxRequestWorkers, MaxClients and ThreadsPerChild the server simply couldn’t deal with even 2,000 concurrency load. But that phenomenon isn’t exclusive to Apache, Nginx for instance required me to raise worker_rlimit_nofile and worker_connections to be able to withstand the load.
And so, it is quite obvious that configuration can play a big role in the amount of load a server can withstand but it won’t make it magically faster or anything of that sort – try to keep that in mind the next time you search for optimization guides.
Do Modules Have Any Affect On Performance?
Another confusing part of performance tests is that some modules appear to have the ability to affect performance by adding or removing these modules (think about cache modules for instance). Some of the tuning advice I read online suggested using only the modules I need and discard others to improve performance. Moreover, having the modules built-in to the servers, as in the case of Apache’s static feature, also carries some confusion whether it can improve or worsen performance?
To test the above, I have built both Nginx and Apache with a few more extra modules, I tested Apache with the ‘static’ feature and without it. As for Lighttpd – it was quite basic to begin with so there wasn’t much room to play around with its modules as well, though, I did wanted to try it with the caching module which unfortunately, refused to compile on my system. When I tried to compile memcached module the compiler then started throwing some weird errors such as:
memcache.c:45:2: warning: #warning "Working around busted-ass Linux header include problems: use FreeBSD instead" [-Wcpp]
Anyways, I can report that non of the changes I made yielded any significant difference I could attribute to a certain module or feature. It seems the role of modules is merely to increase or decrease the functionality of a server, but other than that when it comes to performance at least, modules appear to have a very insignificant role to play. This might be considered a good thing, since you then know the server you get is already geared towards maximum performance out of the box, but on the other hand, if you’re not pleased with the server’s performance then it also means there’s very little you can do about it. Tweaking to infinity won’t help.
Category Rankings
1 Nginx
2 Lighttpd
3 Apache
Conclusions
Considering all three main aspects of a web server: features, memory-usage and performance, it appears Nginx came out on top for now. While all three presented unique features that may appeal to users in certain cases, I think the drawbacks both Apache and Lighttpd currently posses (memory leaks) outweigh their advantages to the point they might hinder their further adoption (if people were actually aware of them). Although, as mentioned, Apache is the only server of the three to offer systemwide access overriding which is perhaps an irreplaceable feature for shared hosting services.
As said earlier, tweaking the servers to infinity is basically futile and won’t get you better performance.
My conclusion regarding features, is that it really doesn’t matter which server has the most features, but rather, which can deliver the features you need/want.
Unfortunately non of the servers seem to have manage breaking the myths related to it, but if I had to make a recommendation I would start with Nginx where possible and where it isn’t then Lighttpd would be my second choice of the three (despite it isn’t supporting http2 protocol yet). But to be more realistic, the places you can’t use Nginx on are probably the same as those you can’t use Lighttpd on, so in that case it’s either Apache or something else entirely.
If you like to see how Apache and Nginx compare to Node.js server see my other comparison here: Apache Vs Nginx Vs Node.js And What It Means About The Performance Of WordPress Vs Ghost.
[Solution To Nginx Riddle]
On a separate server segment that listens only on port 80:
location / {
if ($request_uri ~ /no_redirection) {
return 410;
}
if ($request_uri !~ /no_redirection) {
return 301 https://$host$request_uri;
}
}
