Skip to content

Category: Computers

Bash: Watching Aliases

If you’re trying to watch the output of an alias, you need to make watch an alias of watch as well.

For example, if you have an alias like say:

alias zfslist='zfs list -o name,volsize,used,avaiable,referenced,compressratio,mountpoint'

And want to watch the output over time as if you ran:

user@host:~/$ watch zfslist

Then you need to set up watch as an alias to watch, as follows.

alias watch='watch '

The space after watch is necessary to get bash to expand subsequent aliases after the first one.

Writing on the iPad

When it comes to writing I have a love-hate relationship with just about every product I’ve ever used on my iPad. None of them are really the one stop solution that I want.

For me, the gold standard is MS Word. Why am I not using Word for everything you may ask? Well because Microsoft hasn’t released a version of it for the iPad, and more and more I’m working on my iPad–even if the experience is mildly frustrating at the best of times.

I like word, in a large way because you style things semantically not visually, even if it doesn’t seem that way. That is, you define text as a heading 1 and then tell Word to make a heading 1s look how you want them. Ultimately this translates very nicely to eventual publication on my sites via Wordpress. H1s become H1s and the website styling gets applied, there’s no serious translation or rewriting involved.

Working with this train of thought, I first started with Apple’s Pages. In a way, Pages does a lot of what I wanted in a word processor, well kind of. The behind the scenes things were great, pages could write out a Word doc file and save it to a WebDAV server. Since I already run a Linux development server, and Apache supports WebDAV pretty much out of the box, it was a no brainer to turn WebDAV on and mutually share a folder on both WebDAV (for the iPad) an CIFS for my Windows workstations. Easy-peasy and data is flowing from iPad to workstation for final editing and publication.

Unfortunately, the experience with Pages was less than seamless for me. It worked, but there were enough minor hassles that I couldn’t quite get over that Pages, to Word, to Wordpress wasn’t quite the way I wanted to work.

Enter WriteRoom.

When I bought it, I almost immediately had buyers remorse. The thought, “what did I just waste $5 of my money on,” was pretty prominent in my head. Now that I’ve been using it for a couple of weeks, I actually like it.

WriteRoom is not without it’s flaws, and man are they gaping holes–in my opinion at least–but since WriteRoom isn’t a fancy rich text editor there’s not a whole lot of room for incompatibilities moving the data from WriteRoom’s text files to something else. Moreover, the UI is clean, and it plays well enough with a bluetooth keyboard, which is a fundamental requirement if you’re actually trying to write on an iPad.

That said, WriteRoom falls down in three major ways.

First, there’s no way to resize the width of the document, the writing area remains the same width regardless of whether the iPad is vertical or horizontal. When writing in 18 pt, which is quite comfortable to work at, the page is only 53 characters wide, even though there’s a good inch of margin on either side of the writing area with the iPad horizontal. While this may be ideal for reading, I find it to be somewhat less than ideal when I’m writing. That said, I do appreciate the ability to change the font size and line spacing, even if things feel a bit claustrophobic.

Second, there’s no way to reorganize documents after you’ve created them. Well there is, but it’s one of those workarounds that only an idiot would consider acceptable (copy the content from one doc, paste it in to a new doc, and delete to old doc). The in ability to sort things however, is rather frustrating when you have to move more than one thing around.

Finally, WriteRoom will only sync via Dropbox and iTunes–or you can email yourself the file.

I’m sure there’s probably a ton of people who don’t have a problem with that, I’m not. To me, dropbox is a liability not a feature. It’s yet another service that I have to have an account with, yet another unique password to manage, and yet another place my email address or other personal information can be lifted or lost from.

Unfortunately, from what I’ve been able to determine from the WriteRoom feature/request discussion group, that the author would rather focus on features for WriteRoom instead of writing a WebDAV library for it. As much as I don’t want to fault him for not wanting to add WebDAV support, I do; I also fault Apple for not including WebDAV client support as a basic part of their iOS APIs.

In the end, perhaps the most amusing thing about all of this, is that ultimately I’m publishing most of what I write to a Wordpress blog and there’s a native iOS Wordpress app. So why aren’t I using it for the writing?

Well in truth I do a little. However, the Wordpress app is occasionally broken when using an external keyboard. Which is odd, but apparently there’s a bug in either their code or the iOS virtual keyboard driver/screen resizing, that causes the keyboard’s advanced formatting bar (which I don’t really need or want anyway) to cover the last line of text if the on-screen keyboard hadn’t been docked at the bottom of the screen. Since I use the on-screen keyboard split almost always, the glitch almost always bites me. It’s a big enough PITA to fix, that it’s just easier to write in WriteRoom and copy it over when I’m done.

In the end, WriteRoom is a pretty good piece of software, I certainly like it better than Pages for writing. It’s certainly made my iPad a little more productive, though ultimately I keep finding that although the iPad shows little glints of genius in terms of what it could be, the design and software continue to fall just slightly short of being a real good productivity tool.

Wordpress Live Drafts Plugin – Patch

I found Stephen Sandison’s Live Drafts Wordpress plug-in the other day and it’s been a godsend for the “publish place holder and then replace with final post” type of posts I publish on some of my other sites where I’m abusing Wordpress as a CMS. If you are looking for a way to have a draft copy of an update (including previews) of an already published post; this is the plug in you want.

That said, as a nitpicker, I found one minor detail that annoyed me to no end. The safe draft button was misaligned.

Alignment of save draft button when running Live Drafts as of version 3.0.1.

How it should look.

Correct alignment of save draft button.

With that in mind, I dug into the impressive short and clear source code (at least for a Wordpress plugin) and identified the problem which was quickly rectified with the patch below.

diff --git liveDrafts.php liveDrafts.php
index 0707b0e..93fa739 100644
--- liveDrafts.php
+++ liveDrafts.php
@@ -48,7 +48,7 @@ if (!class_exists('liveDrafts')) {
                     // Add save draft button to live pages
                     jQuery(document).ready(function() {

-                        jQuery('<input type="submit" tabindex="4" value="Save Draft" id="save-post" name="save">').appendTo('#save-action');
+                        jQuery('<input type="submit" tabindex="4" value="Save Draft" id="save-post" name="save">').prependTo('#save-action');

                     });

If you know how to apply a patch a source code file, you’re good to go. If not, or you want to make the fix though the editor in your Wordpress install, you’re looking for line 51 where it says “jQuery('<input type....“. You want to change the appendTo in .appendTo('#save-action'); at the end of the line to prependTo so the end of the line reads .prependTo('#save-action');.

Unclear Instructions: Setting Ubuntu/Unity Keyboard Shortcuts

I don’t know who wrote the text for the dialog for changing Keyboard shortcuts in Ubuntu 11.10, but wow could they have been more misleading.

The instructions read, “click the row and hold down the new keys,” in reality the only place that will allow you to change the short cut is to click the text directly under the black arrow in the above image. I spent the better part of 20 minutes trying to figure out why I couldn’t change a keyboard shortcut because the directions are utterly useless.

This should probably also be filed under: Won’t file a bug report as it’s too much bloody work, and now I won’t forget how to do it.

Enough With the Updates Already

If there’s one thing I’m really starting to get tired of, it’s software updates. It doesn’t seem like a day goes by when something doesn’t need to be updated. Sure, I could probably get away without updating as much, but given the security of most applications and the fact that most software updates are at least partially security related not applying them seems like asking for trouble.

There has to be a better way.

If there’s one thing that some/most Linux distributions get right it’s updates. Yes, it’s a little annoying that ever time I log into my Ubuntu server it says there are updates. However, at the same time, the list of necessary updates covers everything I’ve installed—at least so long as I installed it from an apt repository.

Windows on the other hand is a comparative mess. Microsoft products update though Windows/Microsoft update, 3rd party programs need to roll their own update mechanism.

Things like Adobe’s Flash player, quite possibly one of the worst security offenders, are their own little special hell. Flash seemingly only seems to check for or at least try to apply updates when you first boot your computer; which means my system that sleeps nightly and is only restarted for OS updates that require a restart—yes, Windows 7 is that stable that multi-week uptimes are not only possible but should be routine—doesn’t prompt about Flash updates virtually ever. Morever, since Firefox, Chrome, and Internet Explorer all use different installers, it’s not just a simple one click update it’s a multi-stage process including downloading the installer twice.

Then there’s the other stuff. Like Wordpress. Updated recently to 3.3. Which I might add is really quite nice compared to 3.2, though since switching to Nginx most of the egregious Wordpress performance issues have at least been mitigated.

Wordpress’s built in upgrade system is nice, when it works, but even that I’m not sure goes far enough. It’s annoying as hell to upgrade Wordpress. Then the themes. Then the pluings. Then have the updater come back and say that more plugins have updates. Why can’t I click “Update everything” and confirm it and have Wordpress update everything that has an outstanding update available?

Then again, the problem isn’t Wordpress, Windows or Linux, it’s the lack of a real software engineering standards and short deadlines aimed at maximizing profits regardless of the expense it causes to the consumers.

This is painfully obvious when you start looking at games. When I got Deus Ex: Human Revolution a couple of weeks ago the first thing it did after installing 8.something gigs of game files, is spend 30 minutes downloading a couple hundred meg patch.

Skyrim is even worse, in fact it’s to the point that I’m glad I didn’t buy it when it was released because so far it’s been an epic cluster fsck of updates and patches, some that actually broke more than they fixed.

I don’t know what the solution is, or if there even is one, but I do know this. I’m freaking tired of updates, and even more tired of software that’s written shoddily in the first place, because it can be updated later.

Wordpress on Nginx on Dreamhost

By the time this is done being written I’ll have been running Wordpress on Nginx on a Dreamhost VPS. Better yet, I’ll be doing it with a smaller more well defined resource foot print, with better response times, and faster page loads than I had with Apache. The tradeoff, some more upfront configuration.

I’ve covered the broad strokes about my motivation here, here, and here. In short, Nginx offers a more consistent dependable level resource usage while still having the capability of scaling to serve may users, reducing the possibility of crashing the VPS while under moment of heavy load.

This is long, and may end up being multiple parts, so if you’re interested follow the jump and keep reading.

Getting a feel for the Nginx Stack

Forgive me, I’m about to ramble here.

For the past several months now I’ve been dealing with trying to get my VPS configured in such a way that it was stable and used as few resources (mostly RAM) as possible. During this process I had considered switching the web server from Apache2 to one of the lighter replacements. More and more I’ve been reading about the preference for Nginx (pronounced engine-x), along with PHP-FPM, as the defacto standard for high performance PHP sites.

Time to investigate.

The Nginx Stack

I swapped Apache 2 and mod_php out on my dev machine with Nginx and php-fpm a couple of days ago. Mostly to make sure everything would go smoothly if I decided to move my VPS over and figure out what rule changes I’d have to make to get Nginx running.

To start with Nginx doesn’t use the 1-process per connection model Apache does, instead it uses async IO. This addresses one of the biggest Apache problems I’ve had to contend with, a sudden spike in traffic spawning off a 10s of new processes, is no longer an issue at all.

Nginx’s memory foot print is comparatively tiny too. I’m seeing about 10MB total for the 2 workers + the master process, instead of 4-8MB or more per process.

Couple that with a fixed number of cgi processes on the back end (either with fastcgi or php-fpm) and you can account for most if not all of resources that will be used under any load conditions.

PHP

With Apache gone, so to goes mod_php and mod_fcgid. Neither are ideal solutions to running PHP sites, but those are the breaks (devsrv was running mod_php because it’s was what Ubuntu setup back when I installed it, and mod_fcgid is what Dreamhost uses).

Nginx does things a bit differently. PHP is run as a stand alone CGI “server” that Nginx proxies requests to. I find there are a couple of really nice advantages to this, especially if you can run php-fpm.

For example, you can pool cgi processes based on actual more broadly defined considerations rather than Apache’s process class. Say you have 3 vhosts, each running Wordpress, they can be served by a single pool of php processes that can share resources like a php-apc cache.

Ultimately, this means you can still control the number of backend processes that used for PHP, but can do so while still sharing resources where it makes the most sense.

Figuring and Managing Resources

With a VPS like Dreamhost’s where there are hard limits on memory usage, and since there’s no swap space you can’t really deal with overages when they occur other than to have the watchdog kill your VPS. In short, you really want to be able to account for resource usage and deal with transient spikes in a way that doesn’t result in spiraling resource usage.

Apache has always made this a fun exercise. Yes it can be done, but like I said, it’s fun trying to tune a fuzzy system optimally when there are hard limits.

With Nginx, I can count on the server’s staying exactly, or very nearly exactly, where they were at initialization. 2 worker processes + a controller process nets me between 8 and 12MB of RAM used, and it’ll be that much regardless of whether I have 1 connection or 100 connections. The CGI upstream servers are likewise manageable. With php-fpm you can let it spawn more processes, but you can just as easily limit it to 1 or 2 if resources are scarce. Simply put, it’s much easier to account for how much resources you actually need.

Performance

The whole point of Nginx is performance, but I’m not use to seeing 2 processes handle a lot of requests and that takes some getting use to.

I ran Apache Bench (ab) against my dev server to see just how it performed against Apache setup similarly to how it is in production and the results were certainly impressive to me.

Serving static content (cached HTML files, images, css, js, etc.) 2 Nginx processes could handle more than 2000 connections per second (over 1000mbit network). This is about 2x more than Apache2 was able to handle in about 1/30th the memory foot print.

Serving dynamic content is of course considerably slower, ~3 connections/second can be handled going though the full Wordpress PHP + MySQL stack with 4 php-fpm workers and xcache running.

It’s real hard to completely quantify all the variables, especially when I’m deliberately trying not to.

Conclusions

Okay I admit this post was sparse on details, I’ll try and rectify that in the near future. For now, let me just say, if you can switch to Nginx with your php application you should see better performance and lower resource usage than the same thing running under Apache. I certainly have.

Hide from Sendto Menu

So the last time I posted I mentioned that you can quickly get to the sendto folder, to say add some shortcuts to it, by typing shell:sendto into the address bar in Explorer or from the Type to Search box in the Windows start menu. Today’s handy tip comes from wanting to remove something from the sendto without making it hugely difficult to recreate it if I so desired.

So to hide something from the sendto menu without deleting it from the sendto folder, simply open up it’s properties (right click -> properties) and check the Hide box. Easy as pie.

Reminders: So many, so frustrating.

Todo lists, tasks, reminders, whatever you want to call them I’m starting to think the correct name is pain in the asses. I currently have 5 completely incompatible implementations of tasks/reminders from 3 different providers, that don’t do me a bit of good since they can’t be synchronized between them.

First, there’s Tasks, in Mozilla Thunderbird/Lightning. It’s great that the Mozilla foundation though that a task list was worthwhile to include the only problem is no service seems to exist that they can be synced with. It won’t even let me create/sync them with the newish tasks feature that Google has added to Gmail/Google Apps for Domains.

Which brings us to Google’s tasks, long awaited and requested they finally got around to implementing them. Only catch is there doesn’t appear to be a way to synchronize them to something outside of Gmail, no support in Thunderbird, no way to get them on my phone. Maybe there’s better support if you’re on an android phone, but that has it’s issues as well.

Finally we have iCloud and Reminders, added to Apple’s iOS 5. Again, there’s no way to sync them (at least as far as I can tell) to a desktop client (at least not a Windows one), or for that matter get them out of iCloud. They sync between my phone and iCloud, I can edit and creat them from my browser though iCloud.com, but is that really ideal? Not to me. They do have a number of really cool features though, like the location based stuff.

NIH (not invented here) syndrome is driving me up a wall, why there isn’t a simple universally agreed upon standard for communicating tasks/reminders between various desktop and mobile devices. I shouldn’t have to fire up iCloud to add a task that I could have added while I was in Thunderbird replying to an email.

The more things go towards being digital, and especially in the cloud, the less they seem to be able to work well together. This, I think, needs to be fixed sooner rather than later, as it stands it’s just driving me up a wall.

Finally Done with this RAID project…

Three days of screwing around and as far as I can tell I’ve successfully moved all my data from one array to another while keeping the machine and data online the whole time–other than the few minutes of reboots to remove and replace hardware. Not bad for a SOHO file & web devel server.

Expanding things…

After the move was completed, and the array was re-syncing I expanded the LVM logical volume and the file system inside of it. Expanding the LV was simple and fast using lvextend (8) and the following command…

root@host:# lvextend data /dev/md#

It should take a couple of seconds while it allocates the new extents and returns and that’s done.

Expanding the ext4 FS takes a bit longer, but as long as it’s being extended and not contracted, it can be done while the FS is online and mounted.

root@host:# resize2fs /file/system/mount/point

Adding the -P to resize2fs (8) would be handy, if it works I didn’t try it, adding progress information as the resize is done.

/dev/md127, wtf?

Rebooting to remove the old, now archival, hard drive and bringing the system back up raised an interesting head scratcher. The MD device that should have come up as MD0, came up as MD127. The /etc/mdadm.conf file looked correct, but it wasn’t putting the device where it should have been in /dev. The fix seems to be rebuilding the intiramfs…

root@host:# update-initramfs -u

With that done and a reboot the md device shows up as md0 like it’s suppose to.

Additionally md apparently now supports assigning arrays descriptive names, like "hostname:1" and the array shows up under /dev/md/ as that name in addition to the regular /dev/md* device.

A note on the Hitachi 5k3000s

I went with these drives based on this post on the Backblaze blog. Well, with the caveat that I’m still leery of 3TB drives, so I’m using 2TB drives. I don’t even think that the board or the SATA controllers I have in this box support >2TB drives (though the update later this month will).

More interestingly, the Hitachi 5k3000s are 512-byte sector drives, not 4K sector “advance format” drives. While the 4K sectors do have advantages on large drives in terms of insuring data integrity they also become somewhat fun trying to partition and align around. Partitions have to aligned to 4K boundaries (fdisk, as well as Windows’ partition tools, align to 1M (2048- 512-byte sectors) when DOS compatibility mode is disabled), on top of that you have to be careful where the MD device places the metadata information, as that can shift the FS alignment as well. And for that matter, I have no idea what kind of overhead LVM adds in terms of alignment. In short, 4K drives are, IMO, still something of a mess, and probably will be fore sometime.

One nice thing is I’m seeing about 2x the performance of these Hitachi drives than I was with WD Greens, even though I believe they were properly aligned. Benchmarks show the drives I have can do 140MB/s on the other tracks, I don’t get that yet, but I’m hopeful that a new system with a faster CPU (Xeon E3-1220) and more modern SATA controllers (not the ancient SiI3114 on this Tyan Thunder K8W) will get me closer to that.

The Brass Tacks: What I learned

  • madam’s support for RAID 10 is lacking compared to all the other levels
  • LVM, especially pvmove (8) was more useful than just having a realizable volumes.
  • Planning storage, especially growth is a pain in the rear, when you can’t just throw a ton of disks at it