Dovecot backups with doveadm

As everything is a file in the linux world, migrating or backing up email boxes can be as simple as moving those files around in one way or another. But if you’re using the Dovecot/Postfix combination to handle your email, like I do for my family email accounts, there’s an additional tool offered by Dovecot that may come in really handy.

This tool is called Dovecot’s administration utility – doveadm. It can be used to manage various parts of Dovecot, as well as to access users’ mailboxes for the purpose of backing up or synchronizing email boxes between two locations.

All the information below can be found on the utility’s page https://wiki2.dovecot.org/Tools/Doveadm but it took me some time to grasp some of the basic concepts that I needed for my particular use cases.

One of such uses cases would be migrating email boxes to a new location. Let’s assume that the old box can be accessed on imap.example.com using a combination of email and password. Once I’ve prepared a new empty box on a new mail server, I can pull all the content from the old location using the doveadm utility by running this command on the fresh new server:

# this is run on the mail server we are migrating to
doveadm -v \
  -o imapc_host=imap.example.com\
  -o imapc_user=name@example.com\
  -o imapc_password=mypassword
  -o imapc_features="rfc822.size fetch-headers"\
  -o imapc_ssl_verify=no\
  -o imapc_ssl=imaps\
  -o imapc_port=993 \
   backup -R -u name@example.com imapc:

Some of the options, such as imap host, user, password, those are pretty self explanatory. The important part here is the subcommand backup because it tells the utility to completely ignore the contents of the new, local email box and to replace it with the contents of the remote, old email box. It happened to me that this command failed because some content, such as a skeleton Maildir structure, was already present in the new location. In such cases, I simply deleted any content in the new location before I actually started the migration and that solved the problem.

Besides synchronizing remote content to a new, empty mailbox, there’s this use case where we already have most of the content in the new location and we only need to pull the latest changes in. This would be done like this:

doveadm -v \
  -o imapc_host=imap.example.com\
  -o imapc_user=name@example.com\
  -o imapc_password=mypassword
  -o imapc_features="rfc822.size fetch-headers"\
  -o imapc_ssl_verify=no\
  -o imapc_ssl=imaps\
  -o imapc_port=993 \
   sync -1 -R -u name@example.com imapc:

Again, the important part is sync -1. The sync subcommand in general can perform two way synchronization. Adding -1 makes it perform one-way synchronization only, in other words, it will try to bring over everything from the old location to the new one, but not the other way round.

This is especially useful when I perform the first round of full backup synchronization, as shown by the first command to migrate an email box, then I update the MX records so that any new messages would start to arrive to the new location already. This is all well, except sometimes, depending on the TTL, a handful of email messages can still be delivered into the old destination. Running doveadm sync -1 after a couple of hours will bring any such stray messages back into the fold in the new location.

Accounts with unknown password

Now, this will work well if I know the username and password of the account. If I’m administering accounts other than my own, though, I would not know other people’s password as those are hashed (for a good reason). In such a case, I would not be able to synchronize those accounts, but dovecot has a neat way of catering for these situations – configuring super users who are able to log in on behalf other users.

Once again, the source for reference can be found here: https://doc.dovecot.org/configuration_manual/authentication/master_users/

The basic concept is that in Dovecot’s configuration, you define three pieces of information: a superuser username, superuser password, and a separator of your choice. To log into any account administered by this given dovecot, you can then take the normal user’s username, combine it with the superuser’s username, using the separator to make it one string, and then you can use this combined username and the superuser’s password to access any account. Suppose my normal user would be user@example.com, my supe username would be superuser, the super password would be superpassword and the separator would be *, I would be able to log into an account this like this:

Username: user@example.com*superuser
Password: superpassword

And this is how all of it should all be set up in the Dovecot configuration:

# cat /etc/dovecot/conf.d/auth-master.conf.ext

auth_master_user_separator = *

# Example master user passdb using passwd-file. You can use any passdb though.
passdb {
  driver = passwd-file
  master = yes
  args = /etc/dovecot/passwd.masterusers
}

The file /etc/dovecot/passwd.masterusers (or any other file of your choice) would contain superusers’ usernames and passwords, because it’s a file-based password database, it might look like this:

superuser:{SHA1}nU4eI71bcnBGqeO0t9tXvY1u5oQ=

Make sure that this extension configuration is taken into account, this or any other way:

# cat /etc/dovecot/dovecot.conf | grep auth-master
!include conf.d/auth-master.conf.ext

and after a reload, you should be able to log into any account using this method.

Now, back to the backup/replication doveadm utility and why the superadmin accounts can be useful there. If I was about to back up or synchronize a mailbox for which I don’t know the password, it can be done using the above mentioned superuser instead:

doveadm -v \
  -o imapc_host=imap.example.com\
  -o imapc_user=name@example.com*superuser\
  -o imapc_password=superuserpassword
  -o imapc_features="rfc822.size fetch-headers"\
  -o imapc_ssl_verify=no\
  -o imapc_ssl=imaps\
  -o imapc_port=993 \
   backup -R -u name@example.com imapc:

The difference from the initial example in this post is the imap login credentials, i.e. name@example.com*superuser and superuserpassword, replacing the regular username/password combination. It may be obvious, but I guess it’s worth pointing out anyway, the master/superadmin authentication has to be set up on the machine from which I’m trying to pull content, imap.example.com in this case. The machine I’m running this doveadm command on can access the local account bacause it’s, well, it’s a Dovecot administration utility so that’s what it does.

I would also add that while this is very convenient, it may be good to have master authentication set up only when it’s needed, I usually only enable it when I happen to do a migration of some sorts.

P.S. it’s just a guess, but I suppose that in some future versions of Dovecot, the term ‘master’ will be replaced eventually so the configuration will need to be updated.