Machine Check Exception

Today I finally picked up my 8 bay external firewire rack at the post office. I’ve spent some time shopping around for one of these, and found very few options, really. Your basic choices are:

• How many drive bays? (2, 4, 5, or 8 )
• Firewire 400 or 800? (AKA IEEE 1394/1394b for Google posterity)
• With or without removable 3.5″ IDE drive trays?
• Black or beige?

And that’s about it. Once you decide those four things, there seems to be exactly one kind of device sold on the Internet. I don’t know who makes these things, but apparently there isn’t much competition.

Nevertheless, I lucked out and found a guy selling a used enclosure of the [8-bay, Firewire 800, with trays, black] variety on eBay for cheap. (There is also a guy always running auctions for new enclosures who is pretty cheap, but not nearly as cheap as the one I jumped on.)

It’s quite nice. The whole thing is about the size of a full tower case, but maybe 2 inches narrower. It has a 300W power supply inside that hooks to all the removeable drive trays. (Without trays, this thing can hold 8 5.25″ drives, for those situations when you want to burn lots of CDs very fast.) In addition to the power supply fan, the case has two 80mm fans, and a small fan on the back of each hard drive tray. Air flow is very good, and amazingly this isn’t as loud as I thought it would be. (Granted, I only put one hard drive in it to test with. 8 active disks will probably make sleeping at night a little more difficult.)

Each tray has a key switch to simultaneously power off the drive and unlock it for removal. Combining this with the hotplug support of firewire, and you can now make a massive RAID array with all the hotswap goodness of really expensive systems, but out of IDE drives. And with the price of 250 GB IDE drives falling (as people switch to SATA), my dream of a 1 TB array will be quite achievable.

The I/O architecture was the most surprising thing to me. The case has 4 IDE< ->Firewire bridges in it. Bay 1 is a master on controller 1, bay 2 is slave on controller 1, bay 3 is master on controller 2, and so on. Each controller exposes two firewire 800 and one firewire 400 port on the back of the case. I thought this was weird at first, but actually it is very smart. If you only have (or want to use) one free firewire port, you can just daisy chain the controllers. That means all 8 drives will share the same 800 Mbps pipe, which might be fine for when you just need a lot of disk space.

But, if your PC has multiple Firewire ports, you can choose to partially daisy chain the controllers and run two separate firewire cables, effectively doubling your bandwidth. Since there are only 4 controllers, the limit is 4 separate cables. Two drives will still have to share one 1394b link, but that provides them nominally with 50 MB/sec each, which isn’t bad for today’s IDE drives. Of course, you ain’t gonna be pumping 400 MB/sec through your desktop’s PCI bus, no matter how many ports your firewire card has. So, this massive bandwidth option is mostly theoretical, unless you plan to purchase something like this.

I, on the other hand, will be daisy-chaining the controllers into a single firewire 400 port. No screaming fast benchmarks for me just yet. Maybe that PCI-X 1394b controller is on eBay somewhere….

I’m not a big fan of Digital Rights Management, which mostly involves coming up with creative ways to limit my use of technology. (Technology fails so often as it is, we shouldn’t try so hard to come up with new ways to make it not work.)

So this morning, I go to open a PDF of a scholarship application in Acrobat Reader 7.0. The fields on the application form were designed to be filled in directly with Acrobat Reader, which makes me very happy. With typewriters as scarce as record players these days, I usually have to open PDF applications in Illustrator to edit them. This inevitably hoses some of the fonts and wastes a bunch of my time. (HELLO? Memo to Illustrator developers: Please go figure out why PDFs that Adobe Reader can display perfectly look like crap in Adobe Illustrator. It’s your damn format, after all.) The hassle is still better than quality issues with printing the form, then rescanning it, and then typing on top of that.

Back to the PDF form: I go to start typing information in the form and immediately the program tells me I won’t be permitted to save this document, so when I am done, I should make sure to print this document for my records. Great! We’ve taken 20 years of computer technology and given it the limitations of a typewriter. No going back to your document 2 days later and correcting a typo.

Fortunately, it is a short form, so I press on. Part way through, I want to cut and paste the rather long title of the award into my cover letter, so I click the text selection button, highlight the title, and go up to the Edit menu. Copy and Cut are disabled. No warning box this time. Apparently, I am not allowed to copy text from this PDF.

Further discouraged, I manually retype the title, and finish everything up. I’m not near my printer, so I go to print the application form and click on the “Save As PDF…” in the standard Mac OS print dialog. I figured this would let me save the finished PDF at the print driver level, even if I lost the ability to edit my entries later. No go. Adobe has found a way to circumvent a feature that I thought was part of the Mac OS printing subsystem.

Grr……. Please, PDF content producers, DON’T TURN ON THESE STUPID SECURITY RESTRICTIONS. They just annoy your users and thwart legitimate uses that you didn’t have the imagination to forsee. For an example of more sinister uses of PDF DRM, check out Ed Felten’s blog entry on CBS’s report on “Rathergate.”

Now is a good time to plug the Digital Consumer Bill of Rights, a great document on some very basic things we should all demand of the technology we use.

Having finally found a combination of hardware that would finally POST correctly, it was time to move on to the operating system.

Thankfully, there are quite a lot of nice Linux distributions for AMD64 these days. You can basically use any of the major ones, like Debian, Fedora, Mandrake or Gentoo, and many lesser known ones.

I run Gentoo on my other computers, so I opted to use that distribution. I went to Gentoo AMD64 project page and got familiar with the installation. It pretty much works like any other installation, as you might expect.

….once you get the system to boot, of course. I burned a copy of the Gentoo AMD64 Live CD and tried to boot it. The kernel would start to load, then panic and die with a “Machine Check Exception” (hence the inspiration for my blog name). The MCE crash was pretty confusing, and this early in the installation, quite demoralizing. I went round and round on it for a while, trying different editions of the Gentoo Live CD with no luck.

Fortunately, Opterons can excute older 32 bit code just fine. I dropped an x86 Live CD in, which booted flawlessly. Of course, I don’t want to handicap my Opteron by pretending it is an x86 chip, so this really wasn’t a permanent solution. A couple more hours of Google pinball and I discovered the nomce kernel option. Just pass that to the booting kernel, and it will skip the machine check. (Apparently MCEs are used to detect certain kinds of hardware faults. I was never able to decipher this error, and the system works fine anyway, so I blame a motherboard quirk.)

Adding nomce to the kernel options at boot time fixed my first problem, and I was able to partition, format, and install the base Gentoo system pretty quickly. Then it came time to restart and boot the hard drive directly using the GRUB installation I had performed. No go. All I saw was a blank screen with a blinking cursor. No matter what permutation of BIOS options and boot loaders (GRUB, LILO, EXTLINUX), the result was the same.

Eventually I found the problem purely by accident. When I had first installed Gentoo, I booted the Live CD off of a DVD/CD-ROM drive connected to the motherboard with one of those old 40 conductor IDE cables. I only had one 80 conductor IDE cable which I used for the hard drive. I figured worst case would be the CD-ROM drive would operate slower than if I had used a newer cable. On my way home from some errands, I stopped by a computer shop and picked up a proper 80 conductor IDE cable. When I swapped out the old cable and put in the new one, the hard drive magically was able to boot. This still makes no sense to me. The cable in question was only connected to the CD-ROM drive and not the hard disk, which was the device actually behaving strangely.

Overall, this was a pretty painful computer commissioning experience. I’ve never had this many different things go wrong for seemingly inscrutable reasons. But now it all runs, and I didn’t have to fix anything with duct tape.