Company Mail Server Upgrade

Last night was the company mail server upgrade. This was basically to get Cyrus IMAPd 2.2 running on a box before I stop being a full-time employee. We used Fedora Core 2 for the underlying distribution. cyrus-imapd-2.2.3 is included as an FC2 package, which made everything easier as we no longer have to hand-roll RPMs for the IMAP server. The new box (another Dell Dimension 4600-series machine) is slightly faster at 2.8Ghz compared to 2.4Ghz, and we’re using 120GB drives instead of the 80s in the old box.

FC2 ships with openldap 2.1.29. This was actually the biggest headache, or, rather, our somewhat ad hoc schema for tracking company-specific information didn’t conform to the now-strictly-enforced object and attribute hierarchy for 2.1.29, compared to 2.0, resulting in some untidiness in the migration. Schema objects had to be recomposed as AUXILIARY instead of STRUCTURAL, and the nightly ldap dump that was used for import had to be massaged to conform. This basically took a morning to map out, but went pretty smoothly on the night of the migration following the notes that were made.

Another nuissance: the FC2 cyrus-imapd by default apparently puts its syslog messages into the “mail” signal instead of something reasonable like local6. This was fixed by getting the cyrus-imapd SRPM and editing the SYSLOGFACILITY parameter in the .spec file, and then rolling out our own slightly modified RPM. We couldn’t quite avoid doing it, but having an office package made things much easier, as this was only a two line change (the log facility parameter itself, and the package version).

FC2 also ships with an up-to-date Squirrelmail. Nice. We’ll also be better conformed to RPM updates for Squirrelmail in the future. We also have an up-to-date ClamAV (milter), though we had to go find our own milter-spamc. Snert’s version reads /etc/mail/access and uses it to exclude ALLOW domains from spam scanning, which is a good feature.

FC2 doesn’t ship with mod_auth_pam for Apache authentication, but I got a package from duke.edu after a quick Google search. It worked fine, with the following minor issues:

  • It uses its own conf.d/auth_pam.conf to load itself, so the old load lines in conf/httpd.conf had to be cleaned up to make apachectl configtest happy
  • The included /etc/pam.d/httpd didn’t work. The old /etc/pam.d/httpd did. Here are the contents of the old file:

    #%PAM-1.0
    auth required /lib/security/pam_stack.so service=system-auth
    account required /lib/security/pam_stack.so service=system-auth

After the pam.d change, it worked fine.

Also, /etc/httpd/conf.d/ssl.conf had to be edited to point at the right SSL key and cert files, even though there were directives in various virtual hosts pointing at the right place. This resulted in a confusing FireFoxerror message, basically saying that the cert couldn’t be trusted, and then preventing you from going any further. At the time, I was worried about having to go back to Thawte to get a revised cert, but I pointed IE at the site to see what would happen, and IE gave a more useful error message. This message was enough to realize that Apache was using the default dummy certs that come with the package instead of the real ones.

Those are the main points. The bulk of the time was spent waiting for IO to finish, as the mail spool was copied from the old drive to the new ones. The old drive will become a hot spare for the RAID-1 when we’re sure we don’t need anything else from it. Sadly, I didn’t have Doom 3 on any machine in the office to pass the time. During my first systems administration all-nighters at Earthweb, we had Quake LAN parties on our workstations as we waited for dark hours to start in Denver for our upgrades. And on one surreal night, I came out of our offices at 34th and Park and saw elephants marching down the street at midnight. Good times.

Arguably, this will be my last late night sysadmin campaign for a long time to come. When I got out of the office (I was considering crashing there overnight, since the couch didn’t look that bad compared to the air mattress), I was slightly giddy, and I wished I had my camera with me to take late night photos of Lower Manhattan (which still had more pedestrians and cars on it than I saw in parts of Cleveland during the day). If I did have my camera (and if wasn’t cloudy), I would have stayed up, crossed the Brooklyn Bridge, and taken pictures of the city as it caught the dawn light.

Update: There seems to be something wrong with the way CRAM-MD5 works with saslauthd. Various clients that use CRAM-MD5 for the auth mechanism fail to authenticate. This includes pine (where I first noticed a few weeks ago, but attributed it to pine itself rather than the auth library), SnapperMail SMTP, emacs sieve and possibly Thunderbird (though TB seems to fall back to PLAIN on encrypted channels). It’s a definitely a CRAM-MD5 problem because of the various clients, which are auth’ing against IMAP or SMTP, depending on the case. This isn’t a showstopper, because the main clients still work, but it is annoying. There doesn’t appear to be an updated cyrus-sasl-md5 package, but we haven’t dug into Fedora bugzilla for any reports as of yet.

Comments are closed.