March 6, 2009
This is a local mail for local people, we’ll have no trouble here!
“… all programs that interact with e-mail are broken in one way or another. Please be careful.” – Lars Wirzenius
I seem to have a cunning knack of finding problems with configuring server software, particularly involving e-mail, where a) I can’t find answers in Google, b) most people I go and ask for help say they’d usually ask me such things, and c) if I go onto IRC or mailing lists I end up helping other people and not getting any help with my problem. It’s quite likely this is just because I’m something of a perfectionist, so the ridiculous crappy hacks people come up with and seem content to entrust their mail to are unacceptable to me for one reason or another. Anyway, in my ongoing quest for the perfect mail system, I’ve painted myself into a corner again.
(I’m currently running with postfix, postgrey, clamav-milter, dspam, dovecot using LDA, managesieve and the cmusieve & antispam plugins. If I can get the current incarnation working, I’ve had enough requests to write up a full HOWTO, and seen enough around with pretty questionable content, that I’ll probably do it before too long.)
I’ve got postfix’s local transport configured to hand mail to dspam over LMTP, using mailbox_transport = lmtp:unix:/dspam/dspam.sock. dspam is configured to listen here, add X-DSPAM-Result and signature headers, and then deliver the mail with dovecot’s deliver LDA (which I’ve set to 4750 root:dspam). From dspam.conf:
ServerParameters "--deliver=innocent,spam -d %u"
ServerDomainSocketPath "/var/spool/postfix/dspam/dspam.sock"
...
Preference "spamAction=tag"
Preference "signatureLocation=headers"
...
TrustedDeliveryAgent "/usr/lib/dovecot/deliver"
My dovecot configuration is pretty standard, using PAM for both passdb and userdb, and provides the auth-master socket that deliver needs. The problem I have is that postfix’s local transport is qualifying the local username with the FQDN of the machine before delivering it to dspam with LMTP (the local mail transfer protocol), even for locally-originated mail which was only addressed with a bare username! dspam doesn’t mangle it or care if the user is local or not, and then cheerfully invokes deliver -d robot101@omega.example.co.uk, which returns EX_NOUSER (addressee unknown) because my username is just robot101. From mail robot101:
Mar 6 02:25:40 omega postfix/pickup[13607]: DAA4942F41F: uid=1000 from=<robot101>
Mar 6 02:25:40 omega postfix/cleanup[13637]: DAA4942F41F: message-id=<20090306022540.DAA4942F41F@omega.example.co.uk>
Mar 6 02:25:40 omega postfix/qmgr[12552]: DAA4942F41F: from=<robot101@omega.example.co.uk>, size=339, nrcpt=1 (queue active)
Mar 6 02:25:40 omega dovecot: auth(default): passwd(robot101@omega.example.co.uk): unknown user
Mar 6 02:25:40 omega dspam[13527]: Delivery agent returned exit code 67: /usr/lib/dovecot/deliver -d robot101@omega.example.co.uk
Mar 6 02:25:40 omega postfix/lmtp[13640]: DAA4942F41F: to=<robot101@omega.example.co.uk>, orig_to=<robot101>, relay=omega.example.co.uk[/dspam/dspam.sock], delay=0.08, delays=0.05/0.01/0.01/0.03, dsn=4.3.0, status=deferred (host omega.example.co.uk[/dspam/dspam.sock] said: 421 4.3.0 <robot101@omega.example.co.uk> Delivered (in reply to end of DATA command))
So, no e-mail for me. Dearest lazyweb, which of the three components is behaving wrongly, and how can I fix it?
(And no, I’m not just going to switch to GMail. I store my data on hard drives, which are sometimes in servers, not “in the cloud”. Until about a month ago, most people I knew spoke about clouds which were made of particles of water in the sky, rather than as a data storage media. What if it rains? 😉
Update: The problem is fixed! Even though arguably the problem is dspam’s for not knowing which users are local or not, it’s fixable in dovecot 1.1.x using the auth_username_format = %n option. Thanks so much to Angel Marin for helping me out.
Update 2: There’s also a patch for dspam floating around which adds a StripRcptDomain option, which makes the LMTP server truncate the e-mail address at the @, so essentially assumes everyone to be a local user. The problem with both of these fixes is that they’re both blunt instruments which will break virtual users on the same host. I think the real fix would be something more like a LocalDomains option in dspam, to choose which domains should be considered local and truncated from the e-mail addresses for delivery purposes.
26 responses to “This is a local mail for local people, we’ll have no trouble here!”
Leave a Reply
Calendar
| M | T | W | T | F | S | S |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | 3 | 4 | 5 | 6 | 7 | 8 |
| 9 | 10 | 11 | 12 | 13 | 14 | 15 |
| 16 | 17 | 18 | 19 | 20 | 21 | 22 |
| 23 | 24 | 25 | 26 | 27 | 28 | 29 |
| 30 | 31 | |||||
Links
Archives
- April 2024
- February 2024
- March 2023
- November 2022
- May 2022
- February 2022
- June 2021
- January 2021
- August 2019
- October 2018
- July 2017
- May 2010
- October 2009
- August 2009
- July 2009
- March 2009
- January 2009
- July 2008
- June 2008
- April 2008
- May 2007
- January 2007
- December 2006
- June 2006
- April 2006
- March 2006
- November 2005
- October 2005
- September 2005
- August 2005
- July 2005
- May 2005
- April 2005
- March 2005
I think your problem might be solvable with the canonical address mapping and / or aliasing capabilities of Postfix:
http://www.postfix.org/rewrite.html#canonical
@1: Thanks for the suggestion, but I’m quite suspicious. Canonical mapping mangles the headers as well as the envelope address, and is performed at the cleanup stage before the mail goes into the postfix queue, so I’m pretty sure if I mangle the address there to not have a hostname, the local transport will get upset or confused. 🙁
Just passing by to point out that I like the word “cloud” a lot more than “web”. A web is something that glues onto you and grabs stuff tightly. A cloud just floats somewhere up there and distorts your view, unless you get too close, then it’s gone.
Oh, and the image doesn’t involve spiders.
@2: Exactly, it’s all very fluffy, whereas I want cold hard facts when it comes to data storage. 🙂
Why are you local delivering it to dovecot?
I also have a postfix + dovecot solution, with currently only spamassassin as spam check.
But i just let postfix handle everything, storing it in the maildir format.
Dovecot is basically just runs next to it and only looks at my maildir directory to see what’s in it.
This means you can keep the configuration about your domains and alias users in postfix.
@3: The Dovecot LDA takes care of storing the mail into the Maildir, as well as updating the dovecot indexes while it does so (a nice performance gain for large directories), and most importantly, processing the user’s sieve filters to choose different delivery directories or do vacation responses. That much works fine, as in if I put mailbox_command = /usr/lib/dovecot/deliver everything works perfectly. mailbox_* is implemented by the local transport, so all of the virtual, alias, etc mapping in postfix works perfectly, and it does the correct checks for valid local users before accepting mail.
The same is true with dspam, which is mostly indended to act as a wrapper for the LDA, and supports running as a daemon and having the local mail sent to it over LMTP. This allows dspam to run as a daemon and pool its MySQL connections, avoiding startup overhead, and then after doing the per-user spam processing, invokes the real LDA. You can also hook up dspam using postfix’s content_filter, or hacks using a recipient map to try and only send local mail via dspam, but it’s really ugly IMO – it means non-local recipients can end up getting dspam profiles if one mail goes to both local and remote people, as well as the mails getting queued up and delivered twice by postfix, giving log spam and misleading headers. How is your spamassassin invoked?
Why not just use amavis and plug dspam to it?
@4: Mostly because I don’t want to use amavis. I’m just trying to use dspam as it was intended, at the LDA stage of delivery, so each user on the system has only one dspam profile. I used to have it as a content_filter in postfix, and it was ugly, because each mail was double-queued by postfix, filling up the logs and headers with nonsense about mails going to/from localhost. Amavis is similarly ugly., as well as being a big blob of Perl which goes NOM NOM NOM with my e-mail.
The problem is not that postfix wants to use fully-qualified addresses – it is, after all, delivering over LMTP which requires them. The problem is that you have configured dspam to pass the wrong value to dovecot as the username parameter. Tell is to strip off the domain part.
@5: Any ideas how I configure dspam to do that? 🙂
Tell him I can’t have babies anyway! Tell him my insides are all wrong!
I officially approve the title of your blog-post 🙂
I use maildrop to invoke spamassassin. At first I thought it was a little messy, but its proven incredibly reliable – over 5 years running. I also use it *gasp* for vacation autoresponders.
I haven’t tried dspam yet, looks good. Need to try it out.
Have you checked to see if any household pets are intercepting the mail as it comes through the letterbox? It’s a common problem in many houses. Small puncture marks in your mail can be a pointer to this problem.
Hmm. That looks like my setup! Only that I use virtual mailboxes and want to plug it into LDAP. Because everything else is connected to the LDAP-server, just not the email. And I’m kinda tired of keeping two distinct user databases.
@15: It started out as a virtual setup using Postgres for Bluelinux, and I run the simplified local user version for Collabora and my personal mail.
You can have the same basic setup if you just tell dovecot to use LDAP for the userdb and passdb, and set virtual_transport to dspam, and use virtual_mailbox_maps to do LDAP queries in postfix to find out which addresses are valid, I think. That might be what you do already though. 🙂
Ironically, I think I wouldn’t have this fully-qualified vs local problem if I were using virtual users, as dovecot would be able to find the fully-qualified user in the userdb. 🙁
In your dovecot config use auth_username_format to fix the username passed to pam:
auth_username_format=%Ln
@17: Oooh, thanks! That looks like exactly what I need, but it doesn’t seem to work:
root@omega:~# grep auth_username_format /etc/dovecot/dovecot.confauth_username_format = %Lu
root@omega:~# /etc/init.d/dovecot restart
Restarting IMAP/POP3 mail server: dovecot.
root@omega:~# mail robot101
Subject: asdf
asdf
.
Cc:
root@omega:~#
I still get:
Mar 6 14:27:52 omega dovecot: auth(default): passwd(robot101@omega.example.co.uk): unknown userMar 6 14:27:52 omega dspam[17401]: Delivery agent returned exit code 67: /usr/lib/dovecot/deliver -d robot101@omega.example.co.uk</code<
Any suggestions?
Aha! Fixed it! %n is the local part. Huge success. Thank you so much! 🙂
I spoke too soon. That option seems not to affect deliver, just logging in over IMAP. 🙁
Then I guess we’re talking about dovecot 1.0.x here? upgrade to latest 1.1.x. If not, “dovecot -n” output would help 🙂
Aww… crap. There was me thinking with a nice shiny new Debian release I could actually run my mailserver using stock lenny dovecot packages. 🙁
I’ve got 1:1.0.15-2.3, dovecot -n output here. Is user_format implemented badly/incompletely in 1.0.x then?
yep, auth_username_format didn’t affect deliver lookups in 1.0.x, sorry.
So if you don’t want to upgrade, maybe dspam or postfix can strip it for you, or just do a simple wrapper script that invokes deliver striping @domain and configure dspam to invoke your script 🙂
BTW, if you choose to upgrade read [1], it’ll make upgrading painless 🙂
[1] http://wiki.dovecot.org/Upgrading/1.1
Turns out the dovecot 1.1 packages in Debian unstable install cleanly on lenny, so I’ve upgraded and it works now. Thank you so much! Lets me play with the expire plugin too. Do you have an Amazon wishlist or something? 🙂
I’m a bit late here, and I know your problem is solved, but do you need local mail to run through dspam? Postfix can have different local and virtual delivery systems. Why not use the mailbox command for local mail, delivering directly to dovecot, and specify virtual_transport = lmtp:unix:/tmp/dspam.sock to send virtual users mail through dspam?