February 16, 2005
[Research] IMAP Performance
I’ve been looking at IMAP performance over the last few days, and I’ve come up with some pretty heretical results.
The platform for these tests was FreeBSD 4.5/i386, running on a Dell PE 2400. The filesystem was UFS with softupdates enabled. RAM was 512MB, and clearcache was run prior to every test. I tested UW-IMAP using mbox format, Dovecot using both mbox and maildir formats, and Courier-IMAP using maildir.
The results (the two tests that have 0 times are for Dovecot on the THREAD ORDEREDSUBJECT command, which is not implemented in Dovecot):
| Server | mailbox format | test | time |
|---|---|---|---|
| dovecot | maildir | 1.capa | 0.053203 |
| dovecot | maildir | 1.fetch | 0.156899 |
| dovecot | maildir | 1.pine | 0.234601 |
| dovecot | maildir | 1.select | 0.002402 |
| dovecot | maildir | 1.sort | 0.140133 |
| dovecot | maildir | 1.squirrelmail | 2.692953 |
| dovecot | maildir | 1.thread | 0.272252 |
| dovecot | maildir | 2.sort | 0.046875 |
| dovecot | maildir | 2.squirrelmail | 0.183964 |
| dovecot | maildir | 2.thread | 0.0 |
| dovecot | maildir | 3.sort | 0.012698 |
| dovecot | maildir | 3.squirrelmail | 0.007863 |
| dovecot | mbox | 1.capa | 0.002634 |
| dovecot | mbox | 1.fetch | 3.066798 |
| dovecot | mbox | 1.pine | 0.844669 |
| dovecot | mbox | 1.select | 0.002329 |
| dovecot | mbox | 1.sort | 0.053704 |
| dovecot | mbox | 1.squirrelmail | 2.689958 |
| dovecot | mbox | 1.thread | 0.206899 |
| dovecot | mbox | 2.sort | 0.046262 |
| dovecot | mbox | 2.squirrelmail | 2.488003 |
| dovecot | mbox | 2.thread | 0.0 |
| dovecot | mbox | 3.sort | 0.014379 |
| dovecot | mbox | 3.squirrelmail | 0.00873 |
| courier | maildir | 1.capa | 0.002704 |
| courier | maildir | 1.fetch | 0.268682 |
| courier | maildir | 1.pine | 0.158809 |
| courier | maildir | 1.select | 0.032384 |
| courier | maildir | 1.sort | 1.258794 |
| courier | maildir | 1.squirrelmail | 1.587882 |
| courier | maildir | 1.thread | 1.232035 |
| courier | maildir | 2.sort | 1.219862 |
| courier | maildir | 2.squirrelmail | 0.364796 |
| courier | maildir | 2.thread | 1.131387 |
| courier | maildir | 3.sort | 0.107762 |
| courier | maildir | 3.squirrelmail | 0.04076 |
| uw-imap | mbox | 1.capa | 0.002472 |
| uw-imap | mbox | 1.fetch | 0.363805 |
| uw-imap | mbox | 1.pine | 0.171299 |
| uw-imap | mbox | 1.select | 0.142298 |
| uw-imap | mbox | 1.sort | 0.253514 |
| uw-imap | mbox | 1.squirrelmail | 1.16735 |
| uw-imap | mbox | 1.thread | 0.595428 |
| uw-imap | mbox | 2.sort | 0.266605 |
| uw-imap | mbox | 2.squirrelmail | 0.62801 |
| uw-imap | mbox | 2.thread | 0.35498 |
| uw-imap | mbox | 3.sort | 0.163758 |
| uw-imap | mbox | 3.squirrelmail | 0.116656 |
Or, as a graph:
The PINE and SquirrelMail tests are taken from a typical short session dump of the corresponding mail clients. The PINE test is a simple opening of the inbox, fetching the header information for a range of messages, fetching the header information for another range, and finally fetching a message. The first two SquirrelMail tests are variants on the form of open mailbox, expunge, sort by some key, and then fetch the header information for a message range. The third just lists subscribed mailboxes and gets a status for INBOX.
This seems to suggest that while maildirs are faster for certain operations, mbox is quite acceptable overall, in particular the UW-IMAP implementation (the Dovecot version seems to fall over). This is counter to the prevailing wisdom that maildirs scream. On the other hand, it seems that the functions used most often by mail clients typically involve a fair amount of sorting, something that maildirs are particularly bad at (except sorting by arrival).
Not included in this set of tests were numbers for creating a new mailbox, copying a message from INBOX to the new mailbox, expunging, copying the message back, expunging, and then deleting the new mailbox. Preliminary tests indicate that Dovecot’s maildir and UW-IMAP’s mbox implementations are about the same speed, Courier is slightly slower, and Dovecot’s mbox implementation completely falls over. This is perhaps a slightly overblown scenario, but without the mailbox creation and deletion it’s not far off from actual usage: there are a number of mail clients that will do a search of the INBOX for certain things, copy those messages to another mailbox (like, say, the spam box), and then expunge — possibly over and over.
I am still working on testing Cyrus, and I’d like to test Sun ONE as well, if I can get a server up and running for it.
Posted by Rowan Littell at February 16, 2005 09:31 AM