Category: Software

Linux/Ubuntu CPU Perf Scaling with Modern Intel CPUs

Published on August 26, 2018

I’m posting this largely because all the documentation I can find, and the discussions around this appear to be out of date, or at least not entirely accurate.

I run a Ubuntu server as my home NAS, storage server, general do things host, and for development work on some sites I maintain. It’s built around an Intel Xeon E3-1220 v2, 16 GB of DDR3 RAM and storage running on ZFS.

By default, Ubuntu runs a process on boot called ondemand (/etc/init.d/ondemand). The process is simple enough, it’s a 73 line shell script that basically looks at the available CPU governors (/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors).

It then looks at what governors are available and if either “interactive”, “ondemand”, or “powersave” are, it sets those governors in that order.

So for example, if you have an E3-1220 v2; then the output of /sys/.../scaling_available_governors will be powersave perforamnce.

Since powersave shows up in the available list, it will then echo that to /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor.

So far all this seems reasonable, in a sense, and for older CPUs or maybe AMD or ARM cpus (I don’t know about this one for sure, as I don’t have any AMD or ARM systems) this may be the ideal way to go about getting the clocks on the CPU to scale dynamically.

Ending an Era of OpenBSD: Or a Brief History of my Firewalls

Published on September 16, 2017

For something approaching 20 years, I’ve used OpenBSD to firewall my network from the internet and provide basic network services (DHCP, DNS, NTP, VPN, etc.). Just recently I’ve decided to retire OpenBSD and stand alone computers from the role of firewalls for something smaller, lower power, and easier to manage and upgrade.

I’ve been steadily moving towards smaller and lower power systems for as long as I’ve been doing OpenBSD based firewalls. My first machines were nothing more than mid-tower desktops that I had upgraded away from. In 2000-2003 I made my first moves towards building something more specialized, when I switched from using old towers to building a specific micro-atx pizza box style machines; though still with standard Athlon XP CPUs and parts.

In 2010 I replaced the micro-ATX Athlon XP with a mini-ITX based Intel Atom D510 machine. This halved the power consumption, from somewhere around 80-100 W^[1] to something closer to 40 W.

Around 2015 or so I started looking into running OpenBSD off a USB flash drive instead of a standard hard drive. Part of this was to remove the power consumption of the HDD from the equation. In this, final configuration, the D510 machine with 2 NICs and 2GB of RAM turned in at a somewhat respectable 30 W. Though that was hampered by an abysmally bad PSU with almost 0 power factor correction that pulled nearly 60 VA.

Lua String Compare Performance Testing (Nginx-Lua)

Published on June 14, 2017

In another article I wrote about my ongoing attempt to move my server’s WordPress’s security plugin’s firewall functionality out of PHP and into the embedded lua environment in Nginx. While I’m certainly not nearly as the scale where the C10K problem is a real issue for me, I still do my best to insure that I’m doing things as efficiently as possible.

In my last post, I was looking at the performance degradation between doing no firewalling at all (just building the page in WordPress and serving it), and using the embedded Lua environment to do basic application firewalling tests.

In that article, I saw approximately 425 microsecond latency impact form the Lua processing compared to just building the page. Of course, that was still on the order of 2 orders of magnitude faster than doing the same work in PHP.

Part of the larger part of the actual processing that is being done, is looking for various strings in the myriad of data that’s pushed along as part of the various requests. Things like, know bad user agents, key bits used in SQL injection attacks, and various things like that.

Lua and Nginx both offer some options for searching strings. On the Lua side, there’s the built in string.find() (Lua5.1 docs) and associated functions. On the Nginx-Lua side of things there’s ngx.re.find() (lua-nginx-module docs) which allows calls into Nginx’s regex engine.

I’ve done a significant amount of digging trying to find performance informational about both of these methods, and I haven’t been able to find any. So I sat down and did my own testing.

Nginx-Lua Module: Access Control Performance Testing

Published on June 2, 2017

I’ve been playing with the Lua engine in Nginx for a while. My primary intent is to offload most, if not all, of my WordPress security stuffy from running in the PHP environment to running in something that potentially won’t use as much in the way of resources. The first question I need to answer before I can reasonably consider doing this is what kind of of overhead doing extended processing in Nginx–Lua imposes in terms of performance.

To put some perspective on this, I’ve been running the WordPress security plug-in Word Fence for a while now. When I compare my production server (which has Wordfence enabled) and my development server (which doesn’t have word fence installed, but is otherwise running the same plugins and code base), I see on average a 10–20 ms increase page rendering times, and nearly 20 additional database queries per page.

The overhead from Wordfence isn’t creating a performance problem per say, however, shaving even 15 ms off a 50–60 ms page render time is an appreciable improvement. Additionally, less resources consumed by a bad actor means more resources are available for actual users.

In any even the question here is how much performance overhead does the Nginx-Lua module carry for doing some reasonable processing.

Hidden “Features”

Published on August 7, 2016

There’s a trend in modern computing that I don’t understand; hiding features and interactions. Actually, it goes beyond just hiding features to making it difficult to discover or understand what features are available or what is causing things to happen. And honestly, I’m getting kind of sick of it.

Take this gem in Windows 10.

I just upgraded to the Anniversary Edition, build 1607, but this may apply to earlier builds as well.

The biggest outward change for me with AE, is that I can no longer disable the lock screen with a group policy. Given that, I decided, that if I can’t not use it, I might as well customize it a bit.

One of the options you can set on the lock screen is the image. The choices currently are to use; Microsoft’s stream of images, a picture of your own choosing, or a slide show of your own images. I had set a picture, but I thought that a slide show would be kind of interesting. After all I have a number of my own images that I wouldn’t mind seeing there randomly.

Only there’s a big hidden catch. If you turn the slideshow on for the lock screen, instead of turning off your displays after N minutes, it does, but it also would lock the the computer and return to the lock screen. At least that’s what it was doing to me.

Edit: There are advanced configuration options for the slideshow located on a separate screen that you get to by clicking a not-very-link-like-looking text link — this flat UI thing is really starting to be more of a pain than it seems to be worth honestly. In there, there is an option to turn off using the lock screen instead of turning off the displays. Though as long as the slideshow is being used, the computer will lock when it turns off the the displays and you’ll have to re-enter your password.

Setting up OpenVPN with Certificates

Published on May 1, 2016

I did this a couple of years ago, with certificates that had a 1 year expiry date. Then my certs expired, and I’d forgotten what to do. So I figured it out again, and this time I’m writing it down.

There are two ways to setup client auth in OpenVPN, a shared secret and TLS certificates. TLS certificates are the preferred way if you can manage them, as they make it possible to revoke access to devices without having to change the shared secret for every other device.

To do this you need to setup a certificate authority and sign and issue your own certificates. Most OpenVPN guides tell you how to do this using OpenSSL and it’s associated long cryptic commands. I like my method better.

Lets Encrypt & Nginx: Www-root method and Subject Alt Names

Published on March 14, 2016

Digital Ocean has a pretty good guide for setting up Lets Encrypt with Nginx on Ubuntu 14.04. However, their guide requires you to turn down your Nginx server while initially getting you Lets Encrypt TLS certificates, this of course is problematic for server/site operators who either need or want to continue to have service continuity while getting lets Encrypt Certificates. They also don’t explain how to use subject alternative names to handle multiple sub domains on the same server.

Lets Encrypt’s software requires that they be able to connect to your server to verify that the domain you’re attempting to register a certificate for you control. In the Digital Ocean guide, this process is handled by using the built-in web server in the lens encrypt package. However, Lets Encrypt does not need to operate in this manner to create new certificates, it can use the wwwroot/filesystem approach and your existing server configuration.

The process is very similar to Digital Oceans guide, but the order of operations are slightly different.

WordPress Hardening, Moving WP-Config

Published on March 12, 2016

Lets be clear about somethings. Security is hard. Even the so called experts get it wrong, surprisingly often at that. I’m not an expert, and I’m not proposing that I’m right. Take what I say with a grain of salt.

In part of WordPress’s hardening guide they discuss moving the WordPress config file (wp-config.php) out of the document root as a mechanism that can be used to make it more difficult to attack. Specifically they say:

You can move the wp-config.php file to the directory above your WordPress install. This means for a site installed in the root of your webspace, you can store wp-config.php outside the web-root folder.

Note: Some people assert that moving wp-config.php has minimal security benefits and, if not done carefully, may actually introduce serious vulnerabilities. Others disagree.

Note that wp-config.php can be stored ONE directory level above the WordPress (where wp-includes resides) installation. Also, make sure that only you (and the web server) can read this file (it generally means a 400 or 440 permission).

The linked discussion on the topic on Stack Exchange is also worth reading while you’re at it.

With that said, I’m going to throw my 2-cents in to the discussion.

Rebuilding the Windows System Reserved Partition

Published on July 16, 2015

I’ve been googling this like crazy for the past couple of days to find out how rebuild a Windows 7 System Reserved partition. So lets start with the back story of why I needed to do this.

A few weeks ago I upgraded my Samsung 840 EVO to a new 850 EVO, and installed the 840 EVO in another computer here. In both cases I used Samsung’s Migration Tool to copy the old drives to the new drives. In both cases, at least so far as I can tell, Samsung’s tool renamed the 100MB System Reserved partition to “Data” and filled it until it had 5MB free. For whatever reason, the file filling the partition is completely invisible.

The problem with this problem, is that while the system works just fine, if you use Window’s image backup utilities (wbadmin), for a drive that’s smaller than 500MB, there must be at least 50MB free. Well in the default configuration from a Windows install, there’s about 70MB free and everything backs up just fine. However, with only 5 MB free, the volume shadow copy can’t be made, and the backup will error out. Backups failing was the symptom that clued me in that the system reserved partition was messed up again.

In any event, I’ve tried a couple of ways to recover this situation without resorting to the most oft given advice of format and reinstall—advice I find simply appalling in almost every situation that it’s given. The first time I had this problem, I just repartitioned the disk so that I had more space on the system reserved partition so it would back up—it also helped that I needed >300MB since I was going to try and convert the computer to boot off UEFI instead of the legacy BIOS.

Fixing my slow Wordpress, Nginx, & WP-Supercache Setup

Published on March 20, 2014

I probably should have caught this one a long time ago, but I didn’t. For a while now I’ve been complaining endlessly, at least in my internal monologue, about the poor performance I’ve been seeing from WP-Supercache on my VPS. Preloaded cache files simply shouldn’t take 1-1.5 seconds to serve up. They should be quick quick quick. Yet I was seeing such slow load times.

I’ve been struggling with the issue for quite sometime. I had changed from WP-Supercache to W3 Total Cache, added memcached, cached DB operations and so on and so forth trying to figure out why my pages were so slow to load.

What struck me, was that when I rolled back my W3 Total Cache implementation to no caching, responsiveness stayed about the same or got slightly better. As a non-logged in user, the inverse should have been true. Pages should have taken longer to load without a caching implementation than with one.

Then it hit me, Nginx config files are parsed in order.

Okay, let me take a step to the side here a moment. On Dreamhost VPSes if you’re running Nginx the server looks for supplementary config files in ~/nginx/$domain/ for each of the virtual hosts it’s configured to serve. Knowing that Nginx config files are read in order, I organized mine using the ##-description.conf style. So I might have 10-rewrites.conf, 50-wordpress.conf, 60-supercache.conf.

On a lark the idea struck me that maybe I should try loading the supercache rules before I got to the regular Wordpress rules, which include the directives for passing requests for .php files back to the PHP back-end.

A quick rename of 45-supercache.conf to 30-supercache.conf, thus placing the supercache rules ahead of the Wordpress rules, and my non-logged in user (i.e. reading from the static HTML cache) page response times dropped form 1-1.5s to less than 400ms.

Suffice to say, all my grumbling about slow performance was entirely due to poor configuration on my part.

I’m sure there’s probably a note about this somewhere in one of the myriad of guides for running Wordpress and WP-Supercache on Nginx, but I missed it.

I’d still love to improve the response time for pages that are being processed with PHP, but I can live with it being a touch slower for me knowing that it’s a whole lot faster for everybody else.