Skip to content

Tag: apache

Getting a feel for the Nginx Stack

Forgive me, I’m about to ramble here.

For the past several months now I’ve been dealing with trying to get my VPS configured in such a way that it was stable and used as few resources (mostly RAM) as possible. During this process I had considered switching the web server from Apache2 to one of the lighter replacements. More and more I’ve been reading about the preference for Nginx (pronounced engine-x), along with PHP-FPM, as the defacto standard for high performance PHP sites.

Time to investigate.

The Nginx Stack

I swapped Apache 2 and mod_php out on my dev machine with Nginx and php-fpm a couple of days ago. Mostly to make sure everything would go smoothly if I decided to move my VPS over and figure out what rule changes I’d have to make to get Nginx running.

To start with Nginx doesn’t use the 1-process per connection model Apache does, instead it uses async IO. This addresses one of the biggest Apache problems I’ve had to contend with, a sudden spike in traffic spawning off a 10s of new processes, is no longer an issue at all.

Nginx’s memory foot print is comparatively tiny too. I’m seeing about 10MB total for the 2 workers + the master process, instead of 4-8MB or more per process.

Couple that with a fixed number of cgi processes on the back end (either with fastcgi or php-fpm) and you can account for most if not all of resources that will be used under any load conditions.

PHP

With Apache gone, so to goes mod_php and mod_fcgid. Neither are ideal solutions to running PHP sites, but those are the breaks (devsrv was running mod_php because it’s was what Ubuntu setup back when I installed it, and mod_fcgid is what Dreamhost uses).

Nginx does things a bit differently. PHP is run as a stand alone CGI “server” that Nginx proxies requests to. I find there are a couple of really nice advantages to this, especially if you can run php-fpm.

For example, you can pool cgi processes based on actual more broadly defined considerations rather than Apache’s process class. Say you have 3 vhosts, each running Wordpress, they can be served by a single pool of php processes that can share resources like a php-apc cache.

Ultimately, this means you can still control the number of backend processes that used for PHP, but can do so while still sharing resources where it makes the most sense.

Figuring and Managing Resources

With a VPS like Dreamhost’s where there are hard limits on memory usage, and since there’s no swap space you can’t really deal with overages when they occur other than to have the watchdog kill your VPS. In short, you really want to be able to account for resource usage and deal with transient spikes in a way that doesn’t result in spiraling resource usage.

Apache has always made this a fun exercise. Yes it can be done, but like I said, it’s fun trying to tune a fuzzy system optimally when there are hard limits.

With Nginx, I can count on the server’s staying exactly, or very nearly exactly, where they were at initialization. 2 worker processes + a controller process nets me between 8 and 12MB of RAM used, and it’ll be that much regardless of whether I have 1 connection or 100 connections. The CGI upstream servers are likewise manageable. With php-fpm you can let it spawn more processes, but you can just as easily limit it to 1 or 2 if resources are scarce. Simply put, it’s much easier to account for how much resources you actually need.

Performance

The whole point of Nginx is performance, but I’m not use to seeing 2 processes handle a lot of requests and that takes some getting use to.

I ran Apache Bench (ab) against my dev server to see just how it performed against Apache setup similarly to how it is in production and the results were certainly impressive to me.

Serving static content (cached HTML files, images, css, js, etc.) 2 Nginx processes could handle more than 2000 connections per second (over 1000mbit network). This is about 2x more than Apache2 was able to handle in about 1/30th the memory foot print.

Serving dynamic content is of course considerably slower, ~3 connections/second can be handled going though the full Wordpress PHP + MySQL stack with 4 php-fpm workers and xcache running.

It’s real hard to completely quantify all the variables, especially when I’m deliberately trying not to.

Conclusions

Okay I admit this post was sparse on details, I’ll try and rectify that in the near future. For now, let me just say, if you can switch to Nginx with your php application you should see better performance and lower resource usage than the same thing running under Apache. I certainly have.

Wordpress is a Pig and Dreamhost’s VPSes aren’t configured for it.

I’ve been fighting with this for quite sometime. I moved to a VPS over a year ago in hopes of a more stable Dreamhost experience, and for a while it was. Then about 9 months ago my site started crashing in out of the blue. I’d be chugging along just fine then, “blam!”, site down. I started aggressively caching things with WP Super Cache, then W3 Total Cache. It helped a little, but ultimately things just got more and more unstable. About 2 months ago I gave up on using PhpMyAdmin when I needed to do SQL stuff, simply because it was an instacrash for my VPS. About 3 weeks ago, I had enough and decided it was time to seriously track down the problem.

To make a long story short, Wordpress, is a massive memory hog. I’m pushing on average 30MB before I even start loading plugins. That’s not a lot if you have a 8GB server dedicated to nothing but pushing wordpress but, 30MB is 5-10% of a small VPS. I’ve gone though all the Wordpress tuning guides I can find. I’ve manually cleaned up the database. Nothing really helps. Of course if the server was configured for the load it has, the problem would be considerably smaller.

Which brings us back to Dreamhost’s VPS. They say it’s designed to scale with the RAM that’s allocated to it. Sure, maybe if you’re running static HTML pages. In which case the 69 concurrent clients configured on a 400MB server would get ~6MB a piece which is just barely enough for Apache to serve static HTML. Even then it doesn’t really work out, since there’s non Apache overhead. In fact, now that I think about it, by default under full load, Apache is configured in such a way that it can easily exceed a VPS’s memory allotment just serving static content. 😮

Then comes mod_fcgi. By default it’s configured to allow 20 instances per process class (I’ll come back to this), and the Apache default is 1000.

What are process classes and why are they important. Process classes are spawned by the same executable and share a common virtual host and identity. For example, if my virtual host for cult-of-tech spawns a CGI process, that process can’t be used by another virtual host on my server. Now here’s the kicker. When wordpress gets going and everything is loaded, that 30MB+ of Wordpress and all the PHP overhead + whatever space you allot for caching (XCacahe, APC) is how big the fcgi istance will be. In my case, that means each php.cgi instance is 60-70MB. On a 400MB server, that means once you spawn 5-6 php processes you’ve used up the entirety of your VPS’s memory and, again, blam!

The kicker though, is Dreamhost’s overly aggressive memory manager on their VPSes. Instead of killing off processes, or for that matter special casing it and just restarting Apache if it’s running, the watchdog merely kills off the VPS. Well it may do more, because it can take 10 minutes for the VPS to come back up unless you manually reboot it.

Interestingly enough the answer to all of this is not to simply throw money at it. In the process of troubleshooting this I temporary pushed my memory limits up, and even at 600MB or 800MB the config would still allow enough processes to cause the server to crash. For that matter my development server, which has a ton of RAM available, can comfortably do many of the things that was causing my VPS to crash, without exceeding a 200MB memory foot print.

Simply put, there’s no reason a lightly trafficked Wordpress site should require more than 300MB, maybe 400MB, but certainly not 600MB to simply stay upright. At least not with a properly configured server behind it.

The moral of the story is:

  • Wordpress is a memory pig, and they need to seriously consider a couple of releases focusing entirely on performance and lowering the memory footprint.
  • Dreamhost’s configuration for Apache and mod_fcgi on their VPSes is overly generous for small servers and needs to be curtailed to more reasonable numbers.
  • Dreamhost’s VPS memory watch dog is aggressive, and naive, and will take down a server in a hard to quickly recover way to insure it doesn’t use more resources than the client is paying for.

And what am I doing about this?

I’ve curtailed my Apache and mod_fcgi configs to more reasonable settings.

I’ve set mod_fcgi’s MaxProcesses directive to floor( (400 - typical_process_size) / typical_process size) and my Apache MaxClients to floor(((typ-cgi-process size * 2) - 20) /5). I won’t be anymore specific than that, because what will actually work while still being performant, varies based on site, software, traffic, caching, and number of virtual hosts.