An investigation of moving from a web host running Squirrelmail to using Google Apps Gmail while preserving all users' emails with imapsync.

[ ----------- 100% ------------ ]

Status:   Completed

After struggling to keep users within their email quotas across the multiple domains I am hosting, I have finally decided to migrate from hosting email with my server and using Squirrelmail as the web app, to using Google Apps for my domain(s) and having users access their webmail through Gmail. The process of changing the MX entries so my old server forwards mail to Gmail’s servers is trivial, but the process of moving the users saved emails is an entirely different challenge.

After some Googling I found that the best source of information was on a Wiki page from Dreamhost. It shows a few overviews and brief examples of the arguments needed to run the various scripts, and I chose to use imapsync since it seemed to be the most versatile.

Imapsync is no longer free, but it used to be free and licensed under GPL up until v1.299, so there still are archives of v1.286 floating around on the Internet which are easily accessible. Assuming some version of Perl is already installed on the host, configuring imapsync and installing its supporting packages is fairly straightforward when following the INSTALL file. The tricky part comes when you attempt to connect to the two hosts and try to transfer some emails.

The most important argument that imapsync takes is --dry which will execute the script with any arguments you pass in, and it outputs what it would do without actually modifying any files. An example of a simple transfer of the user ‘myUser’ from old.server.com to new.server.com might look something like this:

perl imapsync --host1 old.server.com --user1 myUser --host2 new.server.com --user2 myUser --password1 oldServerPassword --password2 newServerPassword

The other features I found to be especially useful are:
     --passfile1 and --passfile2 which allow files to be specified that store the users’ passwords so that the password won’t be visible if someone runs ps while imapsync is running
     --authmech1 and --authmech2 which specify the type of authentication: PLAIN, LOGIN, CRAM-MD5 (Gmail uses LOGIN)
     --allowsizemismatch which ignores the retrieved message size (and allowed me to transfer from my old host)
     --regextrans2 which lets you enter a regular expression to apply to the new server’s folder names
     --exclude which ignores copying messages from any folders matching a regular expression

Running on a Windows environment, it was easiest to construct a single command that would execute the imap command for each user/password combination I had saved in a CSV file. The CSV was formatted like so:

      Email Address,   First Name,   Last Name,   Password

Which was also compatible with Google Apps’ “Bulk Import Users from CSV file” option. Executing in the same directory as the CSV file, my command looked like this:

for /F "skip=1 delims=, tokens=1,2,3,4*" %i in (input.csv) do C:\Perl64\bin\perl C:\imapsync-1.286\imapsync --host1 old.server.com --user1 %i --password1 %l --host2 imap.gmail.com --user2 %i --password2 %l --authmech1 PLAIN --authmech2 LOGIN --exclude "[Tt]rash|TRASH" --regextrans2 s/"INBOX[._]"// --regextrans2 s/"Junk"/"[Gmail]\/Spam"/ --regextrans2 s/"Drafts"/"[Gmail]\/Drafts"/ --regextrans2 s/"Sent"/"[Gmail]\/Sent Mail"/