Jul  1 23:11:56 lance imapd: LOGIN, user=knut-olav@hoven.ws, ip=[::ffff:AAA.BBB.CCC.DDD], port=[48700], protocol=IMAP
Jul  1 23:11:56 lance imapd: knut-olav@hoven.ws: Permission denied

The solution was pretty simple.

The /tmp folder had bad permissions. This server was only meant for hosting email services, so bad permissions on /tmp folder was actually not an issue earlier.

I guess the wrong permissions were caused by my custom XEN node setup using multiple partitions, including a partition just for /tmp.

Debugging was quite hard

Authentication was successful, as I got a different error message when authenticating with a known bad password.

I debugged it using strace. It wasn’t easy, as courier imap forks out child processes for each connection, which I had to strace as well.

# strace /usr/sbin/couriertcpd -address=0 -maxprocs=40 -maxperip=20 -nodnslookup -noidentlookup 143 /usr/lib/courier/courier/imaplogin /usr/bin/imapd Maildir

Connect to port 143 using telnet.
Log in using this command:

i login MY_EMAIL_USERNAME MY_PASSWORD

Then find the imap process PID. Look for a process running as user vmail:

$ ps axuw|grep imapd
#...
vmail      362  0.0  1.0   4616  1344 ?        S    01:46   0:00 /usr/bin/imapd /var/spool/mail/vmail/hoven.ws/knut-olav/Maildir/
#...

In this case, the PID is 362. Then attach strace to it using strace -p 362, as sudo.

From the telnet interface, I entered a couple of commands like these:

2 select "INBOX"
5 UID fetch 1:10 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type)])

Then I found this somewhere down into the strace output:

open("/tmp/tmpfWsezjv", O_RDWR|O_CREAT|O_EXCL, 0600) = -1 EACCES (Permission denied)
write(2, "ERR: knut-olav@hoven.ws: Permiss"..., 43) = 43

Fixing the problem
chmod 1777 /tmp

As I wrote earier… a simple solution.