Machine Check Exception

Quick test of the jsMath javascript I just enabled:
\[
\hat{P_1}(\vec{x}) = \frac{1}{n}\sum_{i=1}^n \prod_{j=1}^d
\frac{1}{h_{j}(\vec{t_i})} K\left(\frac{x_j -
t_{ij}}{h_{j}(\vec{t_i})}\right)
\]

And some inline $$\nu_e$$ and $$\alpha^2$$ Greek fun.

For more than a year now, I’ve been annoyed by a persistent, and seemingly random bug in Mail.app. Sometimes people would send me attachments which would not appear at the bottom of the email in Mail.app. Switching to the “Raw Source” view showed that MIME part containing the attachment was still there, with all the base64 encoded data. In fact, dumping the email source to disk and manually decoding the base64 worked just fine, so it was not as if the data was corrupted somehow. Webmail interfaces to the same IMAP mailbox also would show the attachment. My workaround since this bug appeared has been to watch the Size column in my Inbox view and go to webmail if I see a large email with no visible attachment.

Finally a few weeks ago, I identified the cause of the bug after it really started to annoy me when working with PGP signatures included as MIME attachments (rather than “ASCII armor”, which is incredibly ugly). The bug appears to be caused by a truncated newline symbol in the email as returned by Mailsnare, my IMAP provider. The IMAP standard specifies that the newline in emails as they are sent over the network should be (in hex) 0×0D 0×0A. This often known as the “MSDOS” newline symbol used by DOS and Windows operating systems. UNIX systems use just 0×0A, and older Mac systems use 0×0D, although with OS X it seems that 0×0A is more preferred now.

Something in the Mailsnare email processing pipeline truncates the final 0×0A in every email, so there is only a 0×0D at the end. Every previous newline is properly formed, though. When Apple Mail receives such an email, it seems that the MIME parser sometimes cannot cope with the damaged newline, which would come right after the final MIME boundary. This confuses the parser enough that it fails to display the MIME part right before the error, which is the file attachment.

I have checked this hypothesis by copying an email with the damaged newline to a local mail folder, quitting Mail.app, and manually editing the mail file on disk with a hex editor. If I repair the damaged newline and reload Mail.app, it magically sees the attachment that was invisible before.

The missing attachment problem appeared intermittently, in part, due to variations in how email clients terminate the email body. Some clients only put one newline at the end, but others put two newlines. The second-to-last newline is not damaged, and properly ends the MIME boundary, while the last newline is damaged, but no longer vital to correct parsing. However, some emails with only one damaged newline at the end still parse correctly, for reasons I do not understand.

The bug has been reported to Mailsnare tech support, but I have not been made aware of any progress on the matter.

Appendix

An easy way to test for this problem on an IMAP server (assuming your server uses SSL and you have a copy of python compiled with SSL support) is to start python and type:

>>> import getpass, imaplib
>>> M = imaplib.IMAP4_SSL('your_imap.server.net')
>>> M.login('your_username', getpass.getpass())
Password: [type your password here, nothing is echoed]
>>> code,data = M.fetch(1,’(RFC822)’)
>>> print data

Then look at the last part of the email body and see if it ends in \r\n or just \r. If it is just \r, then the IMAP server (or something else in the email processing chain) has chopped the \n character.

The tech news feeds have all gone crazy over the leak from Sun CEO that OS X 10.5 will use ZFS as the standard filesystem. As an unapologetic ZFS bigot, I think this is great news. (Especially if ditto blocks are in there! More on them later.) ZFS is just such a huge leap in both capability AND usability, the latter which Apple cares very much about.

However, the realist in me suspects this is the usual pre-keynote speculation madness. If Apple is really planning to switch to ZFS in 10.5, rather than just support it as a secondary filesystem, then I would be stunned. HFS+ and the Mac OS have grown up organically together, and replacing HFS+ (which will of course need to be supported for many more years) means that you need a new filesystem which can impersonate all of the quirky features of HFS+, most especially the case insensitivity. I knew that ZFS had support for extended attributes, which could be used as resource forks, but I thought case-sensitivity would be a deal breaker.

But this morning I read about new ZFS features which were fast-tracked to some kind of approval. In particular, they add a case-insensitive option to ZFS, and provide an interface for turning it on and off. I don’t know what stage these features are at, but it is possible they are already in the ZFS code that Apple has been playing with for a while.

So maybe the rumors are closer to true. The evidence suggests that ZFS is being tuned up to work as a reasonable replacement for HFS+. The only question is when. 10.5 or 10.6? We’ll find out in 15 hours…