Machine Check Exception

Today I noticed that my IMAP accounts were getting close to quota, so I decided it was time to back up the mailboxes, stash those tar.gz files somewhere, and clean out the old stuff.

Step 1 was find a command line tool to copy the contents of an IMAP folder, with all nested folders, to some sort of mbox/maildir/whatever local disk format. This sounds easy, but I bounced around between a bunch of perl and python tools to do this, all of which were *WAY* more high powered than what I wanted. Most everything was geared toward IMAP-to-IMAP copy or IMAP to local synchronization, with the ability to detect and propagate changes in either copy. Even after getting one of the sync tools to do a simple copy, I soon discovered that Python’s imaplib has some MemoryError bug, and cannot download multi-MB emails. That pretty much killed every Python based tool. (I couldn’t find anything based on Twisted, and my brain is too weak to write a Twisted-based IMAP tool myself.)

So then I went to Perl. The problem here is that one of my IMAP servers only supports SSL connections, and the default implementation of Mail::IMAPClient does not make it easy to establish a SSL connection (basically, they make you do the socket setup and IMAP login by hand). Thus, most all of the Perl-based tools refused to connect. (Interestingly, Debian has a patched Mail::IMAPClient with a SSL option, but by that point I had given up.)

So, having eliminated Perl and Python tools, I finally just caved in and used Thunderbird. Setup your IMAP account, enable offline browsing on all your folders, and then go to offline mode. Click “Download” when it asks, and *bang*, in your profile directory under ImapMail, you will have an mbox tree of all your mail. Tar, gzip, and go home.

After sleeping on the problem, and some further research, I think there might be an alternate way to use SATA for my drive stack. The port multiplier solution was appealing because it effectively turned SATA into a bus, with multiple drives communicating over one SATA link. However, there is no real reason for me to require a bus. At most, I can have 8 drives installed in the HD tower, and I have no intention of expanding that. SATA cables are skinny, so running separate cables for every drive, while ugly, is not impossible.

An additional option is to look into SATA multilane cables. These are cables (frequently including “Infiniband” in the description) which can carry 4 independent SATA links. This is different than the port multiplier, which puts all the data on one link. In principle, the multilane cable is just like 4 eSATA cables bundled together. It certainly is priced that way, with a single 1m multilane cable running for $60 vs. a 4-pack of 1m eSATA cables at $70. Of course, if you don’t own a super-expensive SATA controller card with a multilane connector built-in, you will also need to buy the fan-out PC bracket to go from the multilane connector to 4 separate SATA cables inside your computer. A similar bracket (but with screw holes drilled for a bridgeboard mount) will be needed in the HD tower to merge the 4 hard drives into one cable.

Initially, I had thought that going with eSATA would be cheaper than the more obscure SATA multilane gear, but the cost of buying individual eSATA cables made it almost the same. Later, I stumbled on a deal Coolgear/Cooldrive/whatever was running on a SATA multilane retro kit which bundled the PC fan-out bracket, the HD fan-out bracket, and the cable that goes between for $100. Now the cost looks like:

• Multilane cable, PC and HD tower brackets - $100
• Internal SATA cables x 8 (4 in PC, 4 in HD tower) - $17
• IDE< ->SATA converters x 4 - $52
• 4 port PCI SATA controller - $25

That’s $195, which is a little cheaper than the port multiplier solution, and most importantly, it works today. (Plus, I can sell off all the firewire gear I’ve acquired and will no longer need to help recoup some of this expense.)

The downside to a SATA solution like this is that it is not a bus. With firewire, the marginal cost of adding another drive was basically the cost of another cable and interface board. Here, if I want to bring a 5th drive online, I need to buy another multilane kit, SATA controller, etc. After that, adding the 6th drive is very cheap (2 internal SATA cables + IDE< ->SATA converter, or ~$20). But, if it works, I’ll take a slightly more expensive SATA solution over an unreliable Firewire solution any day.

While I have reached an uneasy truce with my external Firewire disk array, I am not happy with the current arrangement. The Initio 2430 FW800/IDE bridge chips seem to be weak link, though I’m not totally certain. Adding a FW400/IDE bridge based on the Oxford 911 chipset to the chain seems to have diddled the timing just enough to make the system behave. I still dare not use any sort of RAID1 or RAID5, which produced enough congestion on the bus in previous tests to cause certain death. For now, I protect against disk failure by having two files systems, each composed of separate drives, which I manually rsync from time to time. Even then, the traffic of two disk sets being accessed at once makes me nervous.

Being afraid to use my own hardware does not sit well with me, so I’m always on the lookout for ways to fix my external storage dilemma. I still wonder if switching all my bridgeboards to Oxford chips would fix things, but my experiences have really turned me off to Firewire disks in general. The reliability and error handling seems to be really poor. There has to be another way that doesn’t involve SCSI and the 5x increase in cost that implies.

SATA seems to be getting close, bringing lots of SCSI features to the masses, only 15 years later. The standard (and its various extensions) include all sorts of nice things like hotplug and NCQ. Unlike SCSI, SATA is a point-to-point link, which would seem to preclude using it for large drive sets, where you would rather have a bus like Firewire or SCSI. However, there is a port multiplier (PM) standard which allows several drives to be multiplexed over a single link. This makes a lot of sense since the link speed is around 150 MB/sec or 300 MB/sec, which outpaces even the fastest hard drives by a factor of 2 to 5, easily. Multiplexing several drives over one cable should be quite possible without much, if any, performance loss. In addition, SATA includes the possibility of connecting to external devices with an eSATA cable. Sounds like we have a real winner here.

So here are the ingredients I would need to convert my system over:

• SATA host controller that is PM aware - I already have a SATAII TX4 controller that according to the Linux SATA status page is PM capable. Cost: $0.
• eSATA connector - The controller does not include one, so I would need to purchase a small internal to external SATA bracket to put adjacent to the existing host controller. Cost: $7 + $5 ship = $12
• eSATA to 5x SATA port multiplier bridge board- My only option seems to be boards based on the SiI 3726 chipset, such as the Coolgear SATA port multiplier. Cost: $87
• IDE to SATA conversion boards - In order to continue using the 4 large IDE drives I already have, I will need to convert them to SATA. Fortunately, they have very small converters which are about the size of the back end of a hard drive, like this converter by Sabrent. Cost: $15 x 4 + $10 ship = $70.
• 3 ft. eSATA cable - Cost: $10
• 4x 8 inch SATA cables - Cost: 4 x $2 = $8.

So for ~$200, I could upgrade to an all-SATA system. That’s kind of steep, but getting full use of my hardware would make it worth it. Of course, there is a hitch: The Linux kernel, as of 2.6.15, does not support port multipliers, though “some folks at Google are poking at it, with assistance from [Jeff Garzik] and a controller vendor.” So this eSATA solution is only suitable for the future, really.

Too bad.