So there you have it: an OpenBSD mail server that uses several base tools plus a few packages. As mentioned in the beginning, if you find anything helpful in this guide, or discover how great OpenBSD is, please consider supporting the project, either by purchasing a CD set or making a financial donation.
Thanks to the OpenBSD developers for making such great software freely available to us end users.
Tuesday, February 10, 2015
OpenBSD Mail Server - Part 7, Roundcube and httpd
1. Finally, the last bit, and pretty much optional. If a PHP webmail client is desired, install roundcubemail and php-fpm from packages. I decided to use sqlite3 (which is in OpenBSD base) so I did not install any database.
2. Read the docs in /usr/local/share/doc/pkg-readmes/roundcubemail-* and the install message and perform the steps listed to enable PHP etc.
3. Roundcube installs to /var/www/roundcubemail. Enable httpd by adding httpd_flags="" to /etc/rc.conf.local. Add "php_fpm" to pkg_scripts in /etc/rc.conf.local. Copy /etc/examples/httpd.conf to /etc/httpd.conf.
4. Put something like the following in /etc/httpd.conf:
5. Create the sqlite3 database and give it proper permissions:
6. Copy /etc/hosts to /var/www/etc/. Open up port 443 in pf and then restart pf and start php_fpm and httpd. May want to consider editing /etc/php-5.4.ini to set "allow_url_fopen = On" (roundcube says this is optional but recommended).
7. Edit /var/www/roundcubemail/config/defaults.inc.php to enable the installer. Edit /var/www/roundcubemail/config/config.inc.php so the following options are modified (the first fixes the path to the database, the second changes roundcube from using php mail to localhost, and the last bit enables two plugins, markasjunk and managesieve):
8. Go to https://mail.example.com/installer and go through the installer. After that is complete, remove the installer directory from /var/www/roundcubemail and edit /var/www/roundcubemail/config/defaults.inc.php to disable the installer.
9. At this point, everything should be working. Go back and review security options, logging options, etc. and tighten things down. Check out this wiki page on the httpd github repo about creating a "forbidden" directory and editing /etc/httpd.conf to point certain files and directories to the forbidden directory as a way of further locking things down.
2. Read the docs in /usr/local/share/doc/pkg-readmes/roundcubemail-* and the install message and perform the steps listed to enable PHP etc.
3. Roundcube installs to /var/www/roundcubemail. Enable httpd by adding httpd_flags="" to /etc/rc.conf.local. Add "php_fpm" to pkg_scripts in /etc/rc.conf.local. Copy /etc/examples/httpd.conf to /etc/httpd.conf.
4. Put something like the following in /etc/httpd.conf:
# cat /etc/httpd.conf
...
server "mail.example.com" {
listen on $ext_addr ssl port 443
root "/roundcubemail"
directory index index.php
location "*.php" {
fastcgi socket "/run/php-fpm.sock"
}
ssl certificate "/etc/ssl/mail.example.com.crt"
ssl key "/etc/ssl/private/mail.example.com.key"
}5. Create the sqlite3 database and give it proper permissions:
# cd /var/www/roundcubemail # mkdir db # sqlite3 -init SQL/sqlite.initial.sql db/sqlite.db (type .exit to quit sqlite3) # chown -R root:www db # chmod 0775 db # chmod 0660 db/sqlite.db
6. Copy /etc/hosts to /var/www/etc/. Open up port 443 in pf and then restart pf and start php_fpm and httpd. May want to consider editing /etc/php-5.4.ini to set "allow_url_fopen = On" (roundcube says this is optional but recommended).
7. Edit /var/www/roundcubemail/config/defaults.inc.php to enable the installer. Edit /var/www/roundcubemail/config/config.inc.php so the following options are modified (the first fixes the path to the database, the second changes roundcube from using php mail to localhost, and the last bit enables two plugins, markasjunk and managesieve):
# cat /var/www/roundcubemail/config/config.inc.php ... $config['db_dsnw'] = 'sqlite:////roundcubemail/db/sqlite.db?mode=0660'; ... $config['smtp_server'] = 'localhost'; ... // List of active plugins (in plugins/ directory) $config['plugins'] = array( 'archive', 'zipdownload', 'markasjunk', 'managesieve', );
8. Go to https://mail.example.com/installer and go through the installer. After that is complete, remove the installer directory from /var/www/roundcubemail and edit /var/www/roundcubemail/config/defaults.inc.php to disable the installer.
9. At this point, everything should be working. Go back and review security options, logging options, etc. and tighten things down. Check out this wiki page on the httpd github repo about creating a "forbidden" directory and editing /etc/httpd.conf to point certain files and directories to the forbidden directory as a way of further locking things down.
OpenBSD Mail Server - Part 6, Dovecot and Dovecot-Pigeonhole
1. Install dovecot and dovecot-pigeonhole from packages.
2. Edit login.conf per the docs in /usr/local/share/doc/pkg-readmes/dovecot-*.
3. Set up Dovecot certificates per dovecot install message.
4. Open port 993 in pf.conf for remote IMAP access.
5. Edit /etc/dovecot/conf.d/20-lmtp.conf and uncomment the “mail_plugins” line and add "sieve" at the end. Might need to edit /etc/dovecot/conf.d/10-mail.conf and set mail_location to Maildir (see comments for examples) and also /etc/dovecot/conf.d/15-lda.conf to set postmaster_address. Personally, I also like to edit 15-mailboxes.conf and add "auto = subscribe" for the Drafts, Junk, Trash, and Sent mailboxes so they are automatically created and subscribed to. Sometimes, I also like to add Maildir and a sieve script (perhaps with a rule to automatically drop emails that have been tagged as spam by SpamAssassin to the Junk mailbox) to /etc/skel so those are there when a new user is added.
6. Add dovecot to pkg_scripts and start it up.
7. Edit /etc/mail/smtpd.conf so mail is delivered to lmtp instead of ~/Maildir like the following and then restart smtpd:
8. So now here is what’s happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> -relay tagged CLAM_IN mail to spampd on port 10035 -> run it through SpamAssassin -> return to opensmtpd on lo0 port 10036 and tag it SPAM_IN -> deliver to dovecot/lmtp
Outoing mail (unchanged from last time):
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay to dkimproxy on port 10030 -> add DKIM headers -> return to opensmtpd on lo0 port 10029 and tag it DKIM_OUT -> relay out
9. Set up a sieve script in ~/.dovecot.sieve and send an email to the server in a way that triggers one of the sieve rules. There will be something like this in /var/log/maillog:
10. For future reference, here is the output of "doveconf -n":
11. There are other tweaks that can done with dovecot, such as disabling POP3, running SSL IMAP on a port other than 993, etc.
2. Edit login.conf per the docs in /usr/local/share/doc/pkg-readmes/dovecot-*.
3. Set up Dovecot certificates per dovecot install message.
4. Open port 993 in pf.conf for remote IMAP access.
5. Edit /etc/dovecot/conf.d/20-lmtp.conf and uncomment the “mail_plugins” line and add "sieve" at the end. Might need to edit /etc/dovecot/conf.d/10-mail.conf and set mail_location to Maildir (see comments for examples) and also /etc/dovecot/conf.d/15-lda.conf to set postmaster_address. Personally, I also like to edit 15-mailboxes.conf and add "auto = subscribe" for the Drafts, Junk, Trash, and Sent mailboxes so they are automatically created and subscribed to. Sometimes, I also like to add Maildir and a sieve script (perhaps with a rule to automatically drop emails that have been tagged as spam by SpamAssassin to the Junk mailbox) to /etc/skel so those are there when a new user is added.
6. Add dovecot to pkg_scripts and start it up.
7. Edit /etc/mail/smtpd.conf so mail is delivered to lmtp instead of ~/Maildir like the following and then restart smtpd:
# cat /etc/mail/smtpd.conf pki mail.example.com certificate "/etc/ssl/mail.example.com.crt" pki mail.example.com key "/etc/ssl/private/mail.example.com.key" listen on lo0 listen on lo0 port 10026 tag CLAM_IN # incoming mail listen on lo0 port 10028 tag CLAM_OUT # outgoing mail listen on lo0 port 10036 tag SPAM_IN # incoming mail listen on lo0 port 10029 tag DKIM_OUT # outgoing mail listen on egress tls pki mail.example.com auth-optional listen on egress port submission tls-require pki mail.example.com auth table aliases db:/etc/mail/aliases.db table vusers file:/etc/mail/vusers table vdomains file:/etc/mail/vdomains accept for local alias <aliases> deliver to maildir # tagged mail returned from dkimproxy_out relay out accept tagged DKIM_OUT for any relay # tagged mail returned from spampd deliver to lmtp accept tagged SPAM_IN for domain <vdomains> virtual <vusers> deliver to lmtp "/var/dovecot/lmtp" # tagged mail returned from clamsmtpd either send to spampd or dkimproxy_out accept tagged CLAM_IN for any relay via smtp://127.0.0.1:10035 # send to spampd accept tagged CLAM_OUT for any relay via smtp://127.0.0.1:10030 # send to dkimproxy_out # start here - untagged mail is sent to clamsmtpd accept from any for domain <vdomains> relay via smtp://127.0.0.1:10025 # incoming mail accept from local for any relay via smtp://127.0.0.1:10027 # outgoing mail
8. So now here is what’s happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> -relay tagged CLAM_IN mail to spampd on port 10035 -> run it through SpamAssassin -> return to opensmtpd on lo0 port 10036 and tag it SPAM_IN -> deliver to dovecot/lmtp
Outoing mail (unchanged from last time):
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay to dkimproxy on port 10030 -> add DKIM headers -> return to opensmtpd on lo0 port 10029 and tag it DKIM_OUT -> relay out
9. Set up a sieve script in ~/.dovecot.sieve and send an email to the server in a way that triggers one of the sieve rules. There will be something like this in /var/log/maillog:
Feb 3 22:35:34 server dovecot: lmtp(32707): Connect from local Feb 3 22:35:34 server dovecot: lmtp(32707, joe): zyueLYaT0VTDfwAAfpiiTw: sieve: msgid=<1423020926.3895159.222819613.25E9282E@somedomain.com>: stored mail into mailbox 'Test' Feb 3 22:35:34 server smtpd[5331]: delivery: Ok for 68ade8684466ea8c: from=<me@somedomain.com>, to=<joe@example.com>, user=joe, method=lmtp, delay=2s, stat=Delivered
10. For future reference, here is the output of "doveconf -n":
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: OpenBSD 5.6 i386
first_valid_uid = 1000
imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave
mbox_write_locks = fcntl
mmap_disable = yes
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = bsdauth
}
plugin {
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
protocols = imap pop3 lmtp sieve
service managesieve-login {
inet_listener sieve {
port = 4190
}
inet_listener sieve_deprecated {
port = 2000
}
}
ssl_cert = </etc/ssl/dovecotcert.pem
ssl_key = </etc/ssl/private/dovecot.pem
userdb {
driver = passwd
}
protocol lmtp {
mail_plugins = " sieve"
}
11. There are other tweaks that can done with dovecot, such as disabling POP3, running SSL IMAP on a port other than 993, etc.
Monday, February 9, 2015
OpenBSD Mail Server - Part 5, DKIMproxy
1. Follow the steps here to create public and private keys that will be used by DKIMproxy.
2. Create a TXT record for each domain the server will be hosting that looks something like this:
3. Install dkimproxy from ports (no packages available for OpenBSD 5.6). It has no dependencies that aren't already pulled in from prior packages so it's an easy and quick build.
4. Edit /etc/dkimproxy_out.conf so it looks something like this (note that the default ports are different so they don't conflict with the earlier clamsmtpd setup):
Since SpamAssassin already does DKIM checking for incoming mail, dkimproxy is only used for outgoing mail to add the DKIM keys etc. to outgoing headers.
5. Add "dkimproxy_out" to pkg_scripts in /etc/rc.conf.local and start it up. Again, check netstat -na -f inet to see if it's listening on port 10030.
6. Same drill as before. Edit /etc/mail/smtpd.conf so it looks something like this:
7. So now here is what’s happening:
Incoming mail (unchanged from before since incoming mail is not using dkimproxy):
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> -relay tagged CLAM_IN mail to spampd on port 10035 -> run it through SpamAssassin -> return to opensmtpd on lo0 port 10036 and tag it SPAM_IN -> deliver to maildir
Outoing mail:
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay to dkimproxy on port 10030 -> add DKIM headers -> return to opensmtpd on lo0 port 10029 and tag it DKIM_OUT -> relay out
8. Send an email and look at the headers. There should be some DKIM headers for the domain like these:
2. Create a TXT record for each domain the server will be hosting that looks something like this:
selector1._domainkey v=DKIM1; k=rsa; p=KEY_GOES_HERE TXT 1800 TTL
3. Install dkimproxy from ports (no packages available for OpenBSD 5.6). It has no dependencies that aren't already pulled in from prior packages so it's an easy and quick build.
4. Edit /etc/dkimproxy_out.conf so it looks something like this (note that the default ports are different so they don't conflict with the earlier clamsmtpd setup):
# cat /etc/dkimproxy_out.conf # specify what address/port DKIMproxy should listen on #listen 127.0.0.1:10027 listen 127.0.0.1:10030 # specify what address/port DKIMproxy forwards mail to #relay 127.0.0.1:10028 relay 127.0.0.1:10029 # specify what domains DKIMproxy can sign for (comma-separated, no spaces) #domain example.org domain example.com,example.net # specify what signatures to add signature dkim(c=relaxed) signature domainkeys(c=nofws) # specify location of the private key #keyfile /full/path/to/private.key keyfile /etc/mail/dkim/private.key # specify the selector (i.e. the name of the key record put in DNS) selector selector1 ...
Since SpamAssassin already does DKIM checking for incoming mail, dkimproxy is only used for outgoing mail to add the DKIM keys etc. to outgoing headers.
5. Add "dkimproxy_out" to pkg_scripts in /etc/rc.conf.local and start it up. Again, check netstat -na -f inet to see if it's listening on port 10030.
6. Same drill as before. Edit /etc/mail/smtpd.conf so it looks something like this:
# cat /etc/mail/smtpd.conf pki mail.example.com certificate "/etc/ssl/mail.example.com.crt" pki mail.example.com key "/etc/ssl/private/mail.example.com.key" listen on lo0 listen on lo0 port 10026 tag CLAM_IN # incoming mail listen on lo0 port 10028 tag CLAM_OUT # outgoing mail listen on lo0 port 10036 tag SPAM_IN # incoming mail listen on lo0 port 10029 tag DKIM_OUT # outgoing mail listen on egress tls pki mail.example.com auth-optional listen on egress port submission tls-require pki mail.example.com auth table aliases db:/etc/mail/aliases.db table vusers file:/etc/mail/vusers table vdomains file:/etc/mail/vdomains accept for local alias <aliases> deliver to maildir # tagged mail returned from dkimproxy_out relay out accept tagged DKIM_OUT for any relay # tagged mail returned from spampd deliver to maildir accept tagged SPAM_IN for domain <vdomains> virtual <vusers> deliver to maildir # tagged mail returned from clamsmtpd either send to spampd or dkimproxy_out accept tagged CLAM_IN for any relay via smtp://127.0.0.1:10035 # send to spampd accept tagged CLAM_OUT for any relay via smtp://127.0.0.1:10030 # send to dkimproxy_out # start here - untagged mail is sent to clamsmtpd accept from any for domain <vdomains> relay via smtp://127.0.0.1:10025 # incoming mail accept from local for any relay via smtp://127.0.0.1:10027 # outgoing mail
7. So now here is what’s happening:
Incoming mail (unchanged from before since incoming mail is not using dkimproxy):
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> -relay tagged CLAM_IN mail to spampd on port 10035 -> run it through SpamAssassin -> return to opensmtpd on lo0 port 10036 and tag it SPAM_IN -> deliver to maildir
Outoing mail:
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay to dkimproxy on port 10030 -> add DKIM headers -> return to opensmtpd on lo0 port 10029 and tag it DKIM_OUT -> relay out
8. Send an email and look at the headers. There should be some DKIM headers for the domain like these:
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=example.com; h=from:date :message-id:to:subject; s=selector1; bh=[KEY HASH] DomainKey-Signature: a=rsa-sha1; c=nofws; d=example.com; h=from:date :message-id:to:subject; q=dns; s=selector1; b=[KEY HASH]
OpenBSD Mail Server - Part 4, SpamAssassin and SpamPD
1. Install p5-Mail-SpamAssassin and spampd from packages.
2. Edit /etc/mail/spamassassin/local.cf and uncomment the "rewrite_header" line.
3. Spampd will be used as a proxy like clamsmtp. For purposes of this guide, only incoming mail will be scanned. Spampd by default runs on port 10025 but that port is already being used for clamsmtp. So, add the following to /etc/rc.conf.local:
With these flags, spampd will listen on port 10035 and after processing the mail through SpamAssassin, spampd will relay the mail back to port 10036, where OpenSMTPD will be listening.
UPDATE: spampd seems to have trouble binding to the right port (10035 in this case) upon a reboot even with those spampd_flags set in /etc/rc.conf.local. It tries to bind to 10025 which, as noted previously, is being used by clamsmtp, and therefore spampd fails to work and incoming mail has no place to go when opensmtpd tries to relay it to spampd. I have to manually log in and kick spampd to get it to bind to 10035. Still investigating a solution other than changing all the ports around ...
Add "spamassassin" and "spampd" to pkg_scripts in /etc/rc.conf.local and then start both spamassassin and spampd. A "netstat -na -f inet" should show spampd listening on port 10035.
4. Once spampd was processing mail, there were errors in /var/log/maillog along the lines of: “spampd Insecure dependency -T switch at Socket.pm” and it wasn't working. Turns out spampd needs patching for newer Perl. See this: https://github.com/mpaperno/spampd/issues/2. Here is a patch to /usr/local/sbin/spampd (also found here):
5. Restart spampd after applying that patch.
6. Now, modify /etc/mail/smtpd.conf similar to what was done for clamsmtp:
7. There were still some errors in /var/log/maillog. First, there was something like this:
Turns out, SpamAssassin had broken DNS lookups. Here is the patch to /usr/local/libdata/perl5/site_perl/Mail/SpamAssassin/DnsResolver.pm (also found here):
8. Then, there was this:
It appeared that although /var/spampd was set to _spampd:_spampd, the /var/spampd/.spamassassin was set to root:_spampd and the permissions were 700 (IIRC). Anyway, chown that directory to also be _spampd:_spampd and then it appears to work fine.
9. So now here is what's happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> relay tagged CLAM_IN mail to spampd on port 10035 -> run it through SpamAssassin -> return to spampd -> return to opensmtpd on lo0 port 10036 and tag it SPAM_IN -> deliver to maildir
Outoing mail (unchanged from before since outgoing mail is not sent to spampd):
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay out
10. Test again, both ways. Use the GTUBE test to see if it’s flagged as spam. There should be SpamAssassin headers in the incoming email. SpamAssassin can be further set up for Bayesian training and cron entries for running sa-learn on designated directories.
2. Edit /etc/mail/spamassassin/local.cf and uncomment the "rewrite_header" line.
3. Spampd will be used as a proxy like clamsmtp. For purposes of this guide, only incoming mail will be scanned. Spampd by default runs on port 10025 but that port is already being used for clamsmtp. So, add the following to /etc/rc.conf.local:
spampd_flags="--port=10035 --relayhost=127.0.0.1:10036 --tagall -aw"
With these flags, spampd will listen on port 10035 and after processing the mail through SpamAssassin, spampd will relay the mail back to port 10036, where OpenSMTPD will be listening.
UPDATE: spampd seems to have trouble binding to the right port (10035 in this case) upon a reboot even with those spampd_flags set in /etc/rc.conf.local. It tries to bind to 10025 which, as noted previously, is being used by clamsmtp, and therefore spampd fails to work and incoming mail has no place to go when opensmtpd tries to relay it to spampd. I have to manually log in and kick spampd to get it to bind to 10035. Still investigating a solution other than changing all the ports around ...
Add "spamassassin" and "spampd" to pkg_scripts in /etc/rc.conf.local and then start both spamassassin and spampd. A "netstat -na -f inet" should show spampd listening on port 10035.
4. Once spampd was processing mail, there were errors in /var/log/maillog along the lines of: “spampd Insecure dependency -T switch at Socket.pm” and it wasn't working. Turns out spampd needs patching for newer Perl. See this: https://github.com/mpaperno/spampd/issues/2. Here is a patch to /usr/local/sbin/spampd (also found here):
--- spampd.orig Thu Jan 29 23:19:45 2015
+++ spampd Thu Jan 29 23:21:31 2015
@@ -824,6 +824,22 @@ if ( $logsock !~ /^(unix|inet)$/ ) {
usage(0);
}
+# Untaint some options provided by admin command line.
+$pidfile =~ /^(.*)$/;
+$pidfile = $1;
+
+$relayhost =~ /^(.*)$/;
+$relayhost = $1;
+
+$relayport =~ /^(.*)$/;
+$relayport = $1;
+
+$host =~ /^(.*)$/;
+$host = $1;
+
+$port =~ /^(.*)$/;
+$port = $1;
+
if ( $options{tagall} ) { $tagall = 1; }
if ( $options{'log-rules-hit'} ) { $rh = 1; }
if ( $options{debug} ) { $debug = 1; $nsloglevel = 4; }
5. Restart spampd after applying that patch.
6. Now, modify /etc/mail/smtpd.conf similar to what was done for clamsmtp:
# cat /etc/mail/smtpd.conf pki mail.example.com certificate "/etc/ssl/mail.example.com.crt" pki mail.example.com key "/etc/ssl/private/mail.example.com.key" listen on lo0 listen on lo0 port 10026 tag CLAM_IN # incoming mail listen on lo0 port 10028 tag CLAM_OUT # outgoing mail listen on lo0 port 10036 tag SPAM_IN # incoming mail listen on egress tls pki mail.example.com auth-optional listen on egress port submission tls-require pki mail.example.com auth table aliases db:/etc/mail/aliases.db table vusers file:/etc/mail/vusers table vdomains file:/etc/mail/vdomains accept for local alias <aliases> deliver to maildir # tagged mail returned from spampd deliver to maildir accept tagged SPAM_IN for domain <vdomains> virtual <vusers> deliver to maildir # tagged mail returned from clamsmtpd either send to spampd or relay accept tagged CLAM_IN for any relay via smtp://127.0.0.1:10035 # send to spampd accept tagged CLAM_OUT for any relay # start here - untagged mail is sent to clamsmtpd accept from any for domain <vdomains> relay via smtp://127.0.0.1:10025 # incoming mail accept from local for any relay via smtp://127.0.0.1:10027 # outgoing mail
7. There were still some errors in /var/log/maillog. First, there was something like this:
Feb 03 16:48:44 server spampd[22524]: spf: lookup failed: available_nameservers: No DNS servers available! Feb 03 16:48:44 server spampd[22524]: rules: failed to run USER_IN_DEF_DKIM_WL test, skipping: (available_nameservers: No DNS servers available! )
Turns out, SpamAssassin had broken DNS lookups. Here is the patch to /usr/local/libdata/perl5/site_perl/Mail/SpamAssassin/DnsResolver.pm (also found here):
--- DnsResolver.pm.orig Fri Feb 7 03:36:28 2014
+++ DnsResolver.pm Thu Nov 13 16:04:01 2014
@@ -204,8 +204,10 @@
@ns_addr_port = @{$self->{conf}->{dns_servers}};
dbg("dns: servers set by config to: %s", join(', ',@ns_addr_port));
} elsif ($res) { # default as provided by Net::DNS, e.g. /etc/resolv.conf
- @ns_addr_port = map(untaint_var("[$_]:" . $res->{port}),
- @{$res->{nameservers}});
+ my @ns = $res->UNIVERSAL::can('nameservers') ? $res->nameservers
+ : @{$res->{nameservers}};
+ my $port = $res->UNIVERSAL::can('port') ? $res->port : $res->{port};
+ @ns_addr_port = map(untaint_var("[$_]:" . $port), @ns);
dbg("dns: servers obtained from Net::DNS : %s", join(', ',@ns_addr_port));
}
return @ns_addr_port;
8. Then, there was this:
Feb 03 16:48:44 server spampd[22524]: plugin: eval failed: bayes: (in learn) locker: safe_lock: cannot create tmp lockfile /var/spampd/.spamassassin/bayes.lock.mail.example.com.22524 for /var/spampd/.spamassassin/bayes.lock: Permission denied
It appeared that although /var/spampd was set to _spampd:_spampd, the /var/spampd/.spamassassin was set to root:_spampd and the permissions were 700 (IIRC). Anyway, chown that directory to also be _spampd:_spampd and then it appears to work fine.
9. So now here is what's happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> relay tagged CLAM_IN mail to spampd on port 10035 -> run it through SpamAssassin -> return to spampd -> return to opensmtpd on lo0 port 10036 and tag it SPAM_IN -> deliver to maildir
Outoing mail (unchanged from before since outgoing mail is not sent to spampd):
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay out
10. Test again, both ways. Use the GTUBE test to see if it’s flagged as spam. There should be SpamAssassin headers in the incoming email. SpamAssassin can be further set up for Bayesian training and cron entries for running sa-learn on designated directories.
Friday, February 6, 2015
OpenBSD Mail Server - Part 3, ClamAV and ClamSMTP
1. Install clamav and clamsmtp from packages.
2. Edit /etc/freshclam.conf -- comment out the “Example” line and uncomment the "DatabaseMirror" line and add the relevant country code in place of the "XY."
Run ‘freshclam’ to update the database. Add a freshclam command to root’s crontab to have periodic updates:
3. Once freshclam has updated the database, edit /etc/clamd.conf. Comment out the “Example” line, uncomment “TCPSocket” and “TCPAddr” lines and change them so clamd listens on port 3310 at 127.0.0.1.
Add “clamd” to pkg_scripts in /etc/rc.conf.local and then start clamd. Check netstat -na -f inet to see if clamd is running on 127.0.0.1:3310. Check out both /etc/freshclam.conf and /etc/clamd.conf to look at logging options or actions (in VirusEvent) to take when a virus is found. Can set it up so it drops an email into root's mailbox when a virus is found.
4. Now, set up clamsmtp, which is a proxy for clamd. Two config files will be used, one for incoming mail and one for outgoing mail. OpenSMTPD will accept mail, send it to clamsmtp on one port for incoming mail (10025) and a different port (10027) for outgoing mail. Clamsmtp will run the mail through clamd, and then return it to OpenSMTPD for incoming mail (10026) or outgoing mail (10028). Depending on which port the mail is returned to, OpenSMTPD will tag it CLAM_IN or CLAM_OUT.
So copy /etc/clamsmtpd.conf and create /etc/clamsmtpd-in.conf and /etc/clamsmtpd-out.conf. Modify the files like so:
5. Start them both:
(add something similar to /etc/rc.local so they start at boot)
6. Edit /etc/mail/smtpd.conf so it looks like this:
So here is what's happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> deliver to maildir
Outoing mail:
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay out
7. Send some emails both ways. This should be in the header:
2. Edit /etc/freshclam.conf -- comment out the “Example” line and uncomment the "DatabaseMirror" line and add the relevant country code in place of the "XY."
# cat /etc/freshclam.conf #Example ... DatabaseMirror db.us.clamav.net ...
Run ‘freshclam’ to update the database. Add a freshclam command to root’s crontab to have periodic updates:
20 * * * * /usr/local/bin/freshclam >/dev/null 2>&1
3. Once freshclam has updated the database, edit /etc/clamd.conf. Comment out the “Example” line, uncomment “TCPSocket” and “TCPAddr” lines and change them so clamd listens on port 3310 at 127.0.0.1.
# cat /etc/clamd.conf #Example ... TCPSocket 3310 ... TCPAddr 127.0.0.1 ...
Add “clamd” to pkg_scripts in /etc/rc.conf.local and then start clamd. Check netstat -na -f inet to see if clamd is running on 127.0.0.1:3310. Check out both /etc/freshclam.conf and /etc/clamd.conf to look at logging options or actions (in VirusEvent) to take when a virus is found. Can set it up so it drops an email into root's mailbox when a virus is found.
4. Now, set up clamsmtp, which is a proxy for clamd. Two config files will be used, one for incoming mail and one for outgoing mail. OpenSMTPD will accept mail, send it to clamsmtp on one port for incoming mail (10025) and a different port (10027) for outgoing mail. Clamsmtp will run the mail through clamd, and then return it to OpenSMTPD for incoming mail (10026) or outgoing mail (10028). Depending on which port the mail is returned to, OpenSMTPD will tag it CLAM_IN or CLAM_OUT.
So copy /etc/clamsmtpd.conf and create /etc/clamsmtpd-in.conf and /etc/clamsmtpd-out.conf. Modify the files like so:
# cat /etc/clamsmtpd-in.conf OutAddress: 10026 ... Listen: 0.0.0.0:10025 ... ClamAddress: 127.0.0.1:3310 ...
# cat /etc/clamsmtpd-out.conf OutAddress: 10028 ... Listen: 0.0.0.0:10027 ... ClamAddress: 127.0.0.1:3310 ...
5. Start them both:
# /usr/local/sbin/clamsmtpd -f /etc/clamsmtpd-in.conf # /usr/local/sbin/clamsmtpd -f /etc/clamsmtpd-out.conf
(add something similar to /etc/rc.local so they start at boot)
6. Edit /etc/mail/smtpd.conf so it looks like this:
# cat /etc/mail/smtpd.conf pki mail.example.com certificate "/etc/ssl/mail.example.com.crt" pki mail.example.com key "/etc/ssl/private/mail.example.com.key" listen on lo0 listen on lo0 port 10026 tag CLAM_IN # incoming mail listen on lo0 port 10028 tag CLAM_OUT # outgoing mail listen on egress tls pki mail.example.com auth-optional listen on egress port submission tls-require pki mail.example.com auth table aliases db:/etc/mail/aliases.db table vusers file:/etc/mail/vusers table vdomains file:/etc/mail/vdomains accept for local alias <aliases> deliver to maildir # tagged mail returned from clamsmtpd either deliver or relay accept tagged CLAM_IN for domain <vdomains> virtual <vusers> deliver to maildir accept tagged CLAM_OUT for any relay # start here - untagged mail is sent to clamsmtpd accept from any for domain <vdomains> relay via smtp://127.0.0.1:10025 # incoming mail accept from local for any relay via smtp://127.0.0.1:10027 # outgoing mail
So here is what's happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10025 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10026 and tag it CLAM_IN -> deliver to maildir
Outoing mail:
opensmtpd on lo0 -> relay untagged mail to clamsmtpd on port 10027 -> relay to clamd on port 3310 -> return to clamsmtpd -> return to opensmtpd on lo0 port 10028 and tag it CLAM_OUT -> relay out
7. Send some emails both ways. This should be in the header:
X-Virus-Scanned: ClamAV using ClamSMTP
Thursday, February 5, 2015
OpenBSD Mail Server - Part 2, OpenSMTPD and spamd
1. Read the man page for smtpd and smtpd.conf and review the configuration files.
2. Set up virtual users and virtual domains:
3. Create SSL certificates as described in man 5 smtpd.conf:
4. Create ~/Maildir for user ("joe" in this example).
5. Edit /etc/mail/smtpd.conf so it listens on egress with tls (for incoming mail) and egress port 587 (submission) with tls and authentication (for outgoing mail), accepts mail for virtual users and virtual domains, and delivers this mail to Maildir. Note that the smtpd.conf man page clearly says: "For each message processed by the daemon, the filter rules are evaluated in sequential order, from first to last. The first matching rule decides what action is taken." Therefore, the order of the rules in smtpd.conf is very important and will become more important as additional bits are added (e.g. for clamsmtp, spampd, and dkimproxy).
6. Edit pf.conf to allow connections on smtp port 25 and port 587, such as:
7. Reload pf and start /etc/rc.d/smtpd.
8. Test sending mail to/from the user's account. Since there is no imap client yet, might want to install mutt or something similar and point to the user's ~/Maildir to check incoming mail. The user should be able to connect to OpenSMTPD on port 587 from an outside client to send mail through OpenSMTPD to another party. Sending outbound mail from the command line should also work. Perhaps telnet into the server or run a couple of SMTP checks against the server like this one to verify things are working correctly. The session transcript should look something like this:
9. If that works, set up spamd. This is a very simple and standard setup and there are lots of resources out there on how to do this, but here is the shorthand: Add spamd_flags=”-v” to /etc/rc.conf.local. Edit /etc/mail/spamd.conf to add override/whitelist if desired (file /etc/mail/nospamd in sample pf rules). Add spamd pf rules from example /etc/pf.conf and comment out prior rule that passed smtp on egress (because now we want incoming mail to be redirected to spamd running on localhost port 8025):
Reload pf and start /etc/rc.d/spamd. Check netstat to see if spamd is listening on port 8025:
10. Send test emails again and check logs and 'spamdb' to see if email is getting greylisted. Once spamd is working, those third-party SMTP checks won't work because spamd is intercepting incoming mail. Same with telnet, if you can stand waiting for the stuttering. ;-) Anyway, now the session transcript should look something like this:
Haha. Love spamd.
11. So here is what's happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> deliver to maildir
Outoing mail:
opensmtpd on lo0 -> relay out
2. Set up virtual users and virtual domains:
# cat /etc/mail/vusers joe@example.com joe @example.com joe joe@example.net joe @example.net joe # cat /etc/mail/vdomains example.com example.net
3. Create SSL certificates as described in man 5 smtpd.conf:
# openssl genrsa -out /etc/ssl/private/mail.example.com.key 4096
# openssl req -new -x509 -key /etc/ssl/private/mail.example.com.key \
-out /etc/ssl/mail.example.com.crt -days 365
# chmod 600 /etc/ssl/mail.example.com.crt
# chmod 600 /etc/ssl/private/mail.example.com.key
4. Create ~/Maildir for user ("joe" in this example).
5. Edit /etc/mail/smtpd.conf so it listens on egress with tls (for incoming mail) and egress port 587 (submission) with tls and authentication (for outgoing mail), accepts mail for virtual users and virtual domains, and delivers this mail to Maildir. Note that the smtpd.conf man page clearly says: "For each message processed by the daemon, the filter rules are evaluated in sequential order, from first to last. The first matching rule decides what action is taken." Therefore, the order of the rules in smtpd.conf is very important and will become more important as additional bits are added (e.g. for clamsmtp, spampd, and dkimproxy).
# cat /etc/mail/smtpd.conf pki mail.example.com certificate "/etc/ssl/mail.example.com.crt" pki mail.example.com key "/etc/ssl/private/mail.example.com.key" listen on lo0 listen on egress tls pki mail.example.com auth-optional listen on egress port submission tls-require pki mail.example.com auth table aliases db:/etc/mail/aliases.db table vusers file:/etc/mail/vusers table vdomains file:/etc/mail/vdomains accept for local alias <aliases> deliver to maildir accept from any for domain <vdomains> virtual <vusers> deliver to maildir accept from local for any relay
6. Edit pf.conf to allow connections on smtp port 25 and port 587, such as:
# cat /etc/pf.conf ... pass in on egress proto tcp to any port smtp pass in on egress proto tcp to any port submission ...
7. Reload pf and start /etc/rc.d/smtpd.
8. Test sending mail to/from the user's account. Since there is no imap client yet, might want to install mutt or something similar and point to the user's ~/Maildir to check incoming mail. The user should be able to connect to OpenSMTPD on port 587 from an outside client to send mail through OpenSMTPD to another party. Sending outbound mail from the command line should also work. Perhaps telnet into the server or run a couple of SMTP checks against the server like this one to verify things are working correctly. The session transcript should look something like this:
Connecting to 123.456.789.000 220 mail.example.com ESMTP OpenSMTPD [624 ms] EHLO MXTB-PWS3.mxtoolbox.com 250-mail.example.com Hello MXTB-PWS3.mxtoolbox.com [64.20.227.133], pleased to meet you 250-8BITMIME 250-ENHANCEDSTATUSCODES 250-SIZE 36700160 250-DSN 250-STARTTLS 250 HELP [640 ms] MAIL FROM: <supertool@mxtoolbox.com> 250 2.0.0: Ok [640 ms] RCPT TO: <test@example.com> 550 Invalid recipient [640 ms] MXTB-PWS3v2 3260ms
9. If that works, set up spamd. This is a very simple and standard setup and there are lots of resources out there on how to do this, but here is the shorthand: Add spamd_flags=”-v” to /etc/rc.conf.local. Edit /etc/mail/spamd.conf to add override/whitelist if desired (file /etc/mail/nospamd in sample pf rules). Add spamd pf rules from example /etc/pf.conf and comment out prior rule that passed smtp on egress (because now we want incoming mail to be redirected to spamd running on localhost port 8025):
# cat /etc/pf.conf ... #pass in on egress proto tcp to any port smtp pass in on egress proto tcp to any port submission # rules for spamd(8) table <spamd-white> persist table <nospamd> persist file "/etc/mail/nospamd" pass in on egress proto tcp from any to any port smtp \ rdr-to 127.0.0.1 port spamd pass in on egress proto tcp from <nospamd> to any port smtp pass in log on egress proto tcp from <spamd-white> to any port smtp pass out log on egress proto tcp to any port smtp ...
Reload pf and start /etc/rc.d/spamd. Check netstat to see if spamd is listening on port 8025:
# netstat -na -f inet
10. Send test emails again and check logs and 'spamdb' to see if email is getting greylisted. Once spamd is working, those third-party SMTP checks won't work because spamd is intercepting incoming mail. Same with telnet, if you can stand waiting for the stuttering. ;-) Anyway, now the session transcript should look something like this:
Connecting to 123.456.789.000 220 mail.example.com ESMTP spamd IP-based SPAM blocker; Sat Jan 31 11:33:21 2015 [11716 ms] EHLO MXTB-PWS3.mxtoolbox.com 250 Hello, spam sender. Pleased to be wasting your time. [640 ms] MAIL FROM: <supertool@mxtoolbox.com> 250 You are about to try to deliver spam. Your time will be spent, for nothing. [640 ms] RCPT TO: <test@example.com> 250 This is hurting you more than it is hurting me. [640 ms] MXTB-PWS3v2 14602ms
Haha. Love spamd.
11. So here is what's happening:
Incoming mail:
pf -> relay to spamd -> send to opensmtpd on lo0 -> deliver to maildir
Outoing mail:
opensmtpd on lo0 -> relay out
OpenBSD Mail Server - Part 1, Initial Setup
1. Install OpenBSD 5.6. If using the auto-partitioner, make sure enough space is allocated to /usr and /usr/src to allow for extracting the sources (below). Edit /etc/rc.conf.local and add “-s” to ntpd_flags so time is set at boot if desired.
2. Add a rule to default /etc/pf.conf to allow incoming ssh connections, such as:
3. Reload pf with:
4. Update the system by fetching the sources via ftp and patching.
5. Set up $PKG_PATH to install packages.
6. Configure MX records etc. at domain registrar, perhaps with an unused domain for testing purposes.
2. Add a rule to default /etc/pf.conf to allow incoming ssh connections, such as:
# cat /etc/pf.conf ... pass in on egress proto tcp to any port ssh ...
3. Reload pf with:
# pfctl -f /etc/pf.conf
4. Update the system by fetching the sources via ftp and patching.
5. Set up $PKG_PATH to install packages.
6. Configure MX records etc. at domain registrar, perhaps with an unused domain for testing purposes.
Wednesday, February 4, 2015
OpenBSD Mail Server - Intro
The goal:
Create a reasonably secure email server on OpenBSD 5.6 using base plus a few packages. The base components will be OpenSMTPD, spamd, pf, and httpd. The primary packages will be ClamAV, ClamSMTP, SpamAssassin, SpamPD, DKIMproxy, Dovecot, Dovecot-Pigeonhole, and Roundcube. This will be a multi-post guide (7 or 8 posts) on the steps taken to accomplish this goal.
The final setup:
Incoming mail:
pf -> spamd -> opensmtpd -> clamsmtpd -> clamd -> clamsmtpd -> opensmtpd -> spampd -> SpamAssassin -> spampd -> opensmtpd -> deliver to dovecot/lmtp
Outoing mail:
opensmtpd -> clamsmtpd -> clamd -> clamsmtpd -> opensmtpd -> dkimproxy -> opensmtpd -> relay out
Other:
SSL IMAP access: Dovecot
SSL webmail access: httpd and Roundcube
Why OpenBSD?
Because I think it's a great operating system created and maintained by a lot of very smart people. Plus, I like how there are so many excellent bits of software included in the base system. If you find anything helpful in this guide, or discover how great OpenBSD is, please consider supporting the project, either by purchasing a CD set or making a financial donation. The OpenBSD team does amazing work that benefits the whole community in a wide variety of ways and they can always use the support.
Why SpamAssassin in addition to spamd?
Spamd works wonderfully well and it has a very light footprint. It trapped the bulk of my spam (more than 95%) so I almost just let it go instead of bothering with SpamAssassin. But I thought it would be interesting to try and integrate the two.
Why not <insert name of favorite operating system/software/tool>?
Never heard of it. ;-)
Why bother with setting up your own email server to begin with? Why not just keep using Gmail?
I used to run my own email server back when I hosted the Linux Reality podcast and decided it would be a fun exercise to try it again. The email server I set up using the steps in this guide might become my primary email server. Or, I might take the server down tomorrow and go back to using AOL and working on my Geocities page. Who knows?
Assumptions:
This guide assumes an understanding of how to install and configure OpenBSD and an understanding of networking and email, both in general and in regards to OpenBSD in particular. Additionally, this guide assumes an understanding of how to install packages with a properly configured $PKG_PATH, how to work from the command line and edit configuration files, how to change DNS records and MX records, and other general nuts and bolts. These kinds of basic topics will not be covered in this guide.
Disclaimer:
I am an ordinary OpenBSD user. I am not a sysadmin, developer, programmer, kung-fu master, or expert in any of these areas. This guide is mainly a writeup for myself so I can replicate these steps in the future. If someone finds it helpful, fine, but it is by no means the only way or even the best way to configure an email server. There are most likely mistakes in this guide, so take it for what it's worth and YMMV. If your email breaks because of this guide, then don't run your own email server. Feedback and corrections are welcome.
Updates:
- Updated the last line of example smtpd.conf from "for any" to "for domain <vdomains>". Thanks to Christoph on the opensmtpd-misc mailing list.
- Removed bit about enabling pf since it's enabled by default. Duh. Also changed notations of port 587 to 'submission' which is the name of that port in /etc/services. Thanks to rjc.
Resources:
- OpenBSD FAQ (required)
- OpenBSD man pages (required)
- OpenSMTPD wiki
- Helpful thread on OpenSMTPD mailing list re: tagging and proxying
- http://www.kernel-panic.it/openbsd/mail/ (the bits about ClamAV and SpamAssassing are helpful)
- https://coderwall.com/p/eejzja/simple-smtp-server-with-opensmtpd
- http://blog.ehouse.io/mail-server-basic-smtp.html
- http://blog.admiral0.it/computing/mail-server-with-opensmtpd-dovecot-and-amavisdspamassassin/
Subscribe to:
Posts (Atom)