Tutorial : Setup your mail server (courier-imap + postfix + postgresql)
By Greg on Tuesday, January 27 2009, 15:58 - Network - Permalink
Purpose
To have a mail server Which is working with sessions organised in a postgresql database structure and not based on the unix user sessions
Version of the used software for this tutorial
- Ubuntu 8.10 Intrepid server edition - Courier-imap 4.3.1 - Postfix 2.5.5 - postgresql 8.3
I started from a new installation of my distribution so all the actions described here are from scratch.
Step 1 : Prepare the system
Install postfix, courier-imap, courier-authdaemon and postgresql
The usefull debian command line :
sudo apt-get install postfix courier-imap postgresql courier-authdaemon courier-authlib-postgresql postfix-pgsql
During the installation you will be asked for some questions about postfix. Just answer with the common responses, you will be able to change this later.
After install, you can check if the right ports are open on the localhost 127.0.0.1 :
PORT STATE SERVICE 22/tcp open ssh 25/tcp open smtp 143/tcp open imap 5432/tcp open postgresql
Create a postfix user
Normally the postfix user is automatically created by the apt-get install, otherwise you can do :
# adduser postfix # adduser postfix postfix
Step 2 : Postgresql
Configure postgresql for the postfix user :
Login as root in postgres is done with the "postgres" user :
# sudo -s -u postgres $ psql postgres# CREATE USER postfix WITH PASSWORD 'postfix'; postgres# CREATE DATABASE postfix; postgres# GRANT ALL PRIVILEGES ON DATABASE postfix to postfix;
Ok now you'll have a postfix user and an associated database called postfix. In this database we'll put all the informations about our mail configuration.
To prevent permissions problem with postgresql it could be usefull to allow the postfix user to have a history file, so you need to give read/write access to the folder /var/spool/postfix for postfix user :
# chown -R postfix.postfix /var/spool/postfix
Open external ports for postgresql (only if you need to access postgresql from outside)
Configure by opening the access to the database not only for your loopback interface in the file /etc/postgresql/main/8.3/postgresql.conf :
# - Connection Settings - listen_addresses = '*' # what IP address(es) to listen on; # comma-separated list of addresses; # defaults to 'localhost', '*' = all
and now in the file /etc/postgresql/main/8.3/pg_hba.conf we need to allow the used network, here we are using 10.0.2.0.24 (last line):
# "local" is for Unix domain socket connections only local all all ident sameuser # IPv4 local connections: host all all 127.0.0.1/32 md5 # IPv6 local connections: host all all ::1/128 md5 # this line add the authorisation for all the subnetwork 10.0.2.* to connect to the # local postgresql server host all all 10.0.2.0/24 md5
Now your can restart postgresql :
/etc/init.d/postgresql-8.3 restart
To check the accessibility for postgresql, lets test the ports on your external IP (for me 10.0.2.15) and you should have :
PORT STATE SERVICE 22/tcp open ssh 25/tcp open smtp 143/tcp open imap 5432/tcp open postgresql
Create the tables for the mail
This is the table which will be used to match the aliases to the existing emails :
CREATE TABLE aliases ( alias varchar(255) NOT NULL default '', address text NOT NULL, domain varchar(255) NOT NULL default '', created time with time zone NOT NULL default now(), modified time with time zone NOT NULL default now(), active int NOT NULL default '1', PRIMARY KEY (address) );
This table content the several domains that are managed by this mail server instance :
CREATE TABLE domain( domain varchar(255) NOT NULL default '', description varchar(255) NOT NULL default '', aliases int NOT NULL default '0', mailboxes int NOT NULL default '0', maxquota int NOT NULL default '0', transport varchar(255) default NULL, backupmx int NOT NULL default '0', created time with time zone NOT NULL default now(), modified time with time zone NOT NULL default now(), active int NOT NULL default '1', PRIMARY KEY (domain) ) ;
The mailbox table have all the emails of the users with theirs passwords and mail directory :
CREATE TABLE mailbox ( username varchar(255) NOT NULL default '', password varchar(255) NOT NULL default '', name varchar(255) NOT NULL default '', maildir varchar(255) NOT NULL default '', quota int NOT NULL default '0', domain varchar(255) NOT NULL default '', created time with time zone NOT NULL default now(), modified time with time zone NOT NULL default now(), active int NOT NULL default '1', PRIMARY KEY (username) ) ;
Instantiate the tables of the mail database :
This will allow you to test your system if it is working successfully :
INSERT INTO domain (domain,description) VALUES ('example.lan', 'Example domain');
INSERT INTO mailbox (username,password,name,maildir) VALUES ('greg@example.lan','$1$zO3SJ$atwB0hrEgp5KWbrJG.zwE0','Mailbox User','greg@example.lan/');
INSERT INTO aliases (alias,address) VALUES ('gregoire@example.lan', 'greg@example.lan');
INSERT INTO mailbox (username,password,name,maildir) VALUES ('test@example.lan','$1$8evSJ$CC92TOtQQzdull3QNb4AZ0','Mailbox User','test@example.lan/');
Note : the passwords used here are resectively : - greg@example.lan : secret - test@example.lan : test To generate other passwords you can use the command 'authpasswd' :
# authpasswd Password: Reenter password: $1$0h8fJ$w4sbGbaoX487cytcGpmqF1
Create the directory for your mails
# mkdir /home/postfix # mkdir /home/postfix/Maildir # chown -R postfix.postfix /home/postfix
Now let's create a folder for each user :
# sudo -s -u postfix $ cd /home/postfix/Maildir $ maildirmake greg@example.lan $ maildirmake test@example.lan
We are now ready to go ahead to the postfix configuration
Step 3 : Configure the authentication system
Configure authdaemon for the postgresql sessions
Edit the file ‘authdaemonrc‘ in the directory ‘/etc/courier/ ‘. This will configure the database access like where it is, how to connect, which user and password and which tables and fields to use.
authmodulelist="authpgsql" daemons=5 authmodulelistorig="authuserdb authpam authldap authmysql authcustom authpipe" DEBUG_LOGIN=2 # this can be usefull to get some debug log authdaemonvar=/var/run/courier/authdaemon
In the file /etc/courier/authpgsqlrc you will find all the configuration data of your system database :
PGSQL_HOST localhost PGSQL_PORT 5432 PGSQL_USERNAME postfix PGSQL_PASSWORD postfix PGSQL_DATABASE postfix PGSQL_USER_TABLE mailbox PGSQL_CRYPT_PWFIELD password # for the uid and the gid you need to use the right for your installation # use the command 'id postfix' to find it out PGSQL_UID_FIELD 111 PGSQL_GID_FIELD 119 PGSQL_LOGIN_FIELD username PGSQL_HOME_FIELD '/home/postfix/Maildir' PGSQL_MAILDIR_FIELD maildir
Test authdaemon with authtest
First you need to restart the daemons to reload the configuration files
# /etc/init.d/courier-authdaemon restart
To test if the authentication works with the database you can use authtest :
$ authtest greg@example.lan secret
Authentication succeeded.
Authenticated: greg@example.lan (uid 109, gid 117)
Home Directory: /var/spool/postfix
Maildir: greg@example.lan/
Quota: (none)
Encrypted Password: $1$zO3SJ$atwB0hrEgp5KWbrJG.zwE0
Cleartext Password: secret
Options: (none)
If it don't work you can find a lot of informations in the file /var/log/mail.log :
$ tail -n 10 /var/log/mail.log Dec 18 15:25:25 ubuntu authdaemond: modules="authpgsql", daemons=5 Dec 18 15:25:25 ubuntu authdaemond: Installing libauthpgsql Dec 18 15:25:25 ubuntu authdaemond: Installation complete: authpgsql Dec 18 15:25:40 ubuntu authdaemond: received auth request, service=login, authtype=login Dec 18 15:25:40 ubuntu authdaemond: authpgsql: trying this module Dec 18 15:25:41 ubuntu authdaemond: SQL query: SELECT username, '', password, 109, 117, '/var/spool/postfix', Maildir, '', '', '' FROM mailbox WHERE username = 'greg@example.lan' Dec 18 15:25:41 ubuntu authdaemond: authpgsql: sysusername=<null>, sysuserid=109, sysgroupid=117, homedir=/var/spool/postfix, address=greg@example.lan, fullname=<null>, maildir=greg@example.lan/, quota=<null>, options=<null> Dec 18 15:25:41 ubuntu authdaemond: authpgsql: clearpasswd=secret, passwd=<null> Dec 18 15:25:41 ubuntu authdaemond: Authenticated: sysusername=<null>, sysuserid=109, sysgroupid=117, homedir=/var/spool/postfix, address=greg@example.lan, fullname=<null>, maildir=greg@example.lan/, quota=<null>, options=<null> Dec 18 15:25:41 ubuntu authdaemond: Authenticated: clearpasswd=secret, passwd=<null>
Here, as you can see everything is working fine for me 
cool
Step 4 : Postfix
Configure postfix + postgresql
First we'll need the user id and the group id of postfix :
# id postfix
here are the lines you'll need to append to the /etc/postfix/main.cf file (don't forget to adapt the gid and the uid for your own user, otherwise you'll expect some permissions problems)
home_mailbox = mail/ smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu/GNU) # my additions for the virtual domain administration # to use the Postgresql database. virtual_gid_maps = static:117 virtual_uid_maps = static:109 virtual_transport = virtual virtual_mailbox_limit = 51200000 virtual_mailbox_base = /home/postfix/Maildir virtual_alias_maps = pgsql:/etc/postfix/pgsql_virtual_aliases_maps.cf virtual_mailbox_domains = pgsql:/etc/postfix/pgsql_virtual_domains_maps.cf virtual_mailbox_maps = pgsql:/etc/postfix/pgsql_virtual_mailbox_maps.cf
Now let's create the several maps databases for the aliases, domains and mailbox in the correct files :
/etc/postfix/pgsql_virtual_aliases_maps.cf :
# The hosts that Postfix will try to connect to hosts = localhost # The user name and password to log into the pgsql server. user = postfix password = postfix # The database name on the servers. dbname = postfix query = SELECT address FROM aliases WHERE alias='%s'
/etc/postfix/pgsql_virtual_domains_maps.cf :
user = postfix password = postfix hosts = localhost dbname = postfix query = SELECT domain FROM domain WHERE domain='%s'
/etc/postfix/pgsql_virtual_mailbox_maps.cf :
user = postfix password = postfix hosts = localhost dbname = postfix query = SELECT maildir FROM mailbox WHERE username='%s' AND active = 1
Test postfix with postmap
Let's check if the configuration we used before is correct and is working :
# postmap -q greg@example.lan pgsql:/etc/postfix/pgsql_virtual_aliases_maps.cf # postmap -q greg@example.lan pgsql:/etc/postfix/pgsql_virtual_domains_maps.cf # postmap -q greg@example.lan pgsql:/etc/postfix/pgsql_virtual_mailbox_maps.cf greg@example.lan/
Here as you can see I have no errors prompted so it's working fine :)-
Note : be sure this works before going further in this tutorial.
Configure the SMTP authentication with SASL
Create a directory in the chrooted directory of postfix to enable access to the authdaemon app :
cd /var/spool/postfix mkdir courier-authdaemon-socket
first check where is installed the socket of the authdaemon. Usually on debian systems it's installed into the /var/run/courier/authdaemon directory. Adapt the tutorial if it's different for you.
Modify your /etc/fstab file so you can mount the right directory in the right place to access to the authdaemon socket from the chrooted directory. Add this line into /etc/fstab :
/var/run/courier/authdaemon /var/spool/postfix/courier-authdaemon-socket none bind 0 0
Now let's mount the directory into the right place and test if it works :
mount /var/spool/postfix/courier-authdaemon-socket chown -R postfix.postfix /var/spool/postfix/courier-authdaemon-socket
Now we need to configure postfix to authenticate the SMTP requests on the same way as for the IMAP requests. Therefore we will ask to postfix to use authdaemon to authenticate. The authentication system for postfix is called SASL. Let's install it.
apt-get install sasl2-bin libsasl2-modules
Activate SASL in the postfix configuration file /etc/postfix/sasl/smtpd.conf (be sure the directory to the authdaemon socket is right in a chrooted perspective):
pwcheck_method: authdaemond mech_list: PLAIN LOGIN authdaemond_path: /courier-authdaemon-socket/socket log_level: 4
Append the following lines at the end the postfix configuration file /etc/postfix/main.cf :
#
# The settings for the SASL authentication using the autdaemon.
smtpd_sasl_auth_enable = yes
smtpd_sasl2_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = no
smtpd_client_restrictions = permit_mynetworks
permit_sasl_authenticated
Step 5: Courier Imap and the rest
Configure courier-imap
Configuration is done into the file /etc/courier/imapd. We need to modify this line with the right directory :
MAILDIRPATH=/home/postfix/Maildir
The default configuration is ok for the rest.
Now let's reboot all your daemons in use to be sure every configuration is considered:
# /etc/init.d/postfix restart && /etc/init.d/courier-imap restart && /etc/init.d/courier-authdaemon restart
Configure thunderbird
You can just try to add an account. The data you'll need is :
- IMAP server : your mail server IP
- SMTP server : same IP
- Username : greg@example.lan
- Password : secret
- Use TLS/SSL : no
- Port : 143
Known errors
Note : if you get the following error in /var/log/mail.log :
postfix/trivial-rewrite[19109]: warning: do not list domain example.lan in BOTH mydestination and virtual_mailbox_domain
then it means you configured the "mydestination" variable in /etc/postfix/main.cf to the same domain name as your email server. Please remove "example.lan" from it and it should work now
Postfix configure anti spam with blacklist
Postfix is free and powerful MTA. You can easily configure Postfix to block spam. You need to add following directives to /etc/postfix/main.cf file:
- disable_vrfy_command = yes : Disable the SMTP VRFY command. This stops some techniques used to harvest email addresses.
- smtpd_delay_reject = yes : It allows Postfix to log recipient address information when rejecting a client name/address or sender address, so that it is possible to find out whose mail is being rejected.
- smtpd_helo_required = yes : Require that a remote SMTP client introduces itself at the beginning of an SMTP session with the HELO or EHLO command. Many spam bot ignores HELO/EHLO command and you save yourself from spam. Following lines further restrictions on HELO command:
- smtpd_helo_restrictions = permit_mynetworks, reject_non_fqdn_hostname, Reject email if remote hostname is not in fully-qualified domain form. Usually bots sending email don't have FQDN names.
- reject_invalid_hostname, Reject all bots sending email from computers connected via DSL/ADSL computers. They don't have valid internet hostname.
You can put the following access restrictions that the Postfix SMTP server applies in the context of the RCPT TO command.
- smtpd_recipient_restrictions =
- reject_invalid_hostname, - Reject email if it not valid hostname
- reject_non_fqdn_hostname, - Reject email if it not valid FQDN
- reject_non_fqdn_sender, - Reject the request when the MAIL FROM address is not in fully-qualified domain form. For example email send from xyz or abc is rejected.
- reject_non_fqdn_recipient, - Reject the request when the RCPT TO address is not in fully-qualified domain form
- reject_unknown_sender_domain, - Reject email, if sender domain does not exists
- reject_unknown_recipient_domain, Reject email, if recipient domain does not exists
- permit_mynetworks,
- reject_rbl_client list.dsbl.org, Configure spam black lists
- reject_rbl_client sbl.spamhaus.org,
- reject_rbl_client cbl.abuseat.org,
- reject_rbl_client dul.dnsbl.sorbs.net,
- permit
Open /etc/postfix/main.cf file :
$ sudo vim /etc/postfix/main.cf
Set/modify configuration as follows
disable_vrfy_command = yes
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks,
reject_non_fqdn_hostname,
reject_invalid_hostname,
permit
smtpd_recipient_restrictions =
permit_sasl_authenticated,
reject_invalid_hostname,
reject_non_fqdn_hostname,
reject_non_fqdn_sender,
reject_non_fqdn_recipient,
reject_unknown_sender_domain,
reject_unknown_recipient_domain,
permit_mynetworks,
reject_rbl_client list.dsbl.org,
reject_rbl_client sbl.spamhaus.org,
reject_rbl_client cbl.abuseat.org,
reject_rbl_client dul.dnsbl.sorbs.net,
permit
smtpd_error_sleep_time = 1s
smtpd_soft_error_limit = 10
smtpd_hard_error_limit = 20
Also force (last lines) Postfix to limit incoming or receiving email rate to avoid spam.
Save and close the file. Restart postfix:
# /etc/init.d/postfix restart
Watch out maillog file. Now you should see lots of spam email blocked by above configuration directive:
# tail -f /var/log/mail.log
Output:
Jan 9 06:07:22 server postfix/smtpd[10308]: NOQUEUE: reject: RCPT from 183-12-81.ip.adsl.hu[81.183.12.81]: 554 Service unavailable; Client host [81.183.12.81] blocked using dul.dnsbl.sorbs.net; Dynamic IP Addresses See: http://www.sorbs.net/lookup.shtml?81.183.12.81; from= to= proto=ESMTP helo=<183-12-230.ip.adsl.hu> Jan 9 06:07:23 server postfix/smtpd[10308]: lost connection after RCPT from 183-12-81.ip.adsl.hu[81.183.12.81] Jan 9 06:07:23 server postfix/smtpd[10308]: disconnect from 183-12-81.ip.adsl.hu[81.183.12.81] Jan 9 06:10:43 server postfix/anvil[10310]: statistics: max connection rate 1/60s for (smtp:81.183.12.81) at Jan 9 06:07:17 Jan 9 06:10:43 server postfix/anvil[10310]: statistics: max connection count 1 for (smtp:81.183.12.81) at Jan 9 06:07:17 Jan 9 06:10:43 server postfix/anvil[10310]: statistics: max cache size 1 at Jan 9 06:07:17 Jan 9 06:16:58 server postfix/smtpd[10358]: warning: 81.92.197.249: address not listed for hostname unassigned.or.unconfigured.reverse.nfsi-telecom.net Jan 9 06:16:58 server postfix/smtpd[10358]: connect from unknown[81.92.197.249] Jan 9 06:17:00 server postfix/smtpd[10358]: NOQUEUE: reject: RCPT from unknown[81.92.197.249]: 550 : Recipient address rejected: User unknown in virtual alias table; from=<> to= proto=ESMTP helo= Jan 9 06:17:00 server postfix/smtpd[10358]: disconnect from unknown[81.92.197.249]
Comments
It's possible that this tutorial is not optimised and that some action not perfect are... but it's the first time for me I install a mail server so do not hesitate to add some comments if you see how I could improve this tutorial.
hi there
I noticed this line, "authmodulelistorig="authuserdb authpam authldap authmysql authcustom authpipe" .
Shouldn't it be authmodulelistorig="authuserdb authpam authldap authmysql authcustom authpipe"
Here is part of the log:
---Mar 16 22:41:06 test-server.com.nz authdaemond: PQexec failed, reconnecting: ERROR: relation "mailbox" does not exist.
Ah sorry , ignore the lines about authpgsql.
Anyway, it still fails on authtest.
Plus, are you writing this guide for Postgresql or Mysql?
If it is just for Postgresql, you might want to update the comment under the section with the heading of 'Configure postfix + postgresql' where you mentioned this:
virtual_gid_maps = static:117
virtual_uid_maps = static:109
virtual_transport = virtual
virtual_mailbox_limit = 51200000
virtual_mailbox_base = /home/postfix/Maildir
virtual_alias_maps = pgsql:/etc/postfix/pgsql_virtual_aliases_maps.cf
virtual_mailbox_domains = pgsql:/etc/postfix/pgsql_virtual_domains_maps.cf
virtual_mailbox_maps = pgsql:/etc/postfix/pgsql_virtual_mailbox_maps.cf
Hi, guys!
Kakimoto, I looked at your postings and well, I think the author will need to make the following adjustments. It worked for me.
0) after creating the tables, it is best to use the 'postfix' database.
I exited psql and entered again with the command of 'psql postfix'.
Else, if following your tute, the database table creation and insertion statements and such will not have data go into the database, postfix and hence, the table mailbox cannot be found during authtest (as observed via the logs).
1) your " grant all priveleges on database postfix" doesn't work.
postfix=# GRANT ALL PRIVILEGES ON DATABASE postfix to postfix;
GRANT
postfix=# \z
+-+-+-(3 rows)
** notice no access privileges.
postfix=# GRANT ALL PRIVILEGES ON table aliases to postfix;
GRANT
postfix=# GRANT ALL PRIVILEGES ON table domain to postfix;
GRANT
postfix=# GRANT ALL PRIVILEGES ON table mailbox to postfix;
GRANT
voila:
postfix=# \z
+-+-+--public | aliases | table | {postgres=arwdxt/postgres,postfix=arwdxt/postgres} public | domain | table | {postgres=arwdxt/postgres,postfix=arwdxt/postgres} public | mailbox | table | {postgres=arwdxt/postgres,postfix=arwdxt/postgres}(3 rows)
Run authtest:
Authentication succeeded.
Authenticated: anexiole@my-testserver.com (uid 116, gid 117) Home Directory: /home/postfix/my_maildir Maildir: anexiole@my-testserver.com/ Quota: (none)Encrypted Password: $1$ZrlvJ$x0dbQlcb9TTXi3xxs7gDlu0
Cleartext Password: skylines_kick_holdens
--- now, i shall continue with the rest of the tute.
Thanks guys for your precisions and your tests. I'll correct my tute asap considering your suggestions.
hi there
Good job. This tutorial was very helpful for me!. Thanks.
Good job. I was up and running in notime!
Small change in alias table in order to support multiple aliases.
Unique aliases and unique primary key.
Great explanation. Only one issue, though:
Whenever I restart my server, I'm forced to drop into a root console and comment out the '/etc/fstab' entry, then reboot. After reboot, I can uncomment the entry, make the '/var/run/courier/authdaemon' directory then mount the file for the chrooted postfix -- then everything is wonderful. Any idea why my system might have issues attempting to mount this file?
I'm running Ubuntu 10.04 (lucid) - server edition. Is it possible that the contents of '/var/run/' are getting wiped on restart, and therefore the file system has nothing to mount in the first place?
Thanks in advance for any help/advice.