Links

Archives

A lot of Cyrus work

This week has been mostly a continuation of Cyrus IMAP work. The system itself is ready for production use, but there are a number of helper scripts and utilities that need to be developed before we can start the migration.

• Tuesday I discovered that / and /usr were both at 99% full, even before thinking about going production, and after agonizing about this, I decided that the best thing to do was to repartition and re-Jumpstart. To make it slightly easier on myself (although building Cyrus and friends really isn’t that hard, as my SSL troubles last week showed), I made Solaris packages out of most of the software: Cyrus SASL, Cyrus IMAP, Perdition (and vanessa libs), and some perl libs — Cyrus::IMAP, and Mail::IMAP. Sendmail is still an after-build, and there are a number of other perl libs that I’ve since added to support the migration, user management, and quota scripts.

• Top of the list of things yet to do, of course, is the migration utility itself. I’ve got one almost completely ready — it temporarily resets the user’s password, copies all the subscribed folders from KE to SIPALA, gets the user’s quota from the self service quota system, sets the LDAP mailHost attribute to sipala, and then resets the password back to the original. Still to complete is the part that grabs the forward and vacation settings, if present. I’d also like to extend it to general sieve copying and some other Cyrus admin work, for future use in migrating users from one Cyrus box to another, but that’s future.

• As implied, there’s a quota system tie-in. I’ve been working on an XML-RPC interface to the quota system, or at least a subset of it, to use on SIPALA rather than the direct PostgreSQL connection. I think the XML-RPC is more portable, and it means I don’t have to install PostgreSQL libraries on SIPALA. It also means we could use MySQL as the database for the quota system if we wanted to. The part of the XML-RPC server that supports the migration script is done, and some parts that support the cron update scripts are done, but there’s more work there. I also need to tool the web interface for the quota system to take you to the mail host that your mail is on, rather than assuming everyone’s on the same mail host.

• Also, as implied, there’s a WebDB tie in. WebDB is where forwarding and vacations are set, so it needs to be made aware of sieve. I’ve considered putting that info into LDAP and letting both KE and SIPALA generate the necessary stuff from there, but I think that’s actually not the best way. I think WebDB needs to be looking at the mailHost attribute itself and then either talking KE’s own peculiar language or sieve as appropriate.

Spyware User Agents Update

Apparently a couple of the user agent rules I applied to the PacketShaper were overly broad and were catching systems that did not have spyware running. My guess is that spyware had at one time been on these systems but had been removed. However, the removal probably didn’t revert IE’s user agent string back to the original, and thus tripped the shaper rules. I’ve removed those particular rules, keeping the ones that seem to be associated with standalone spyware agents.

Spyware User Agents

I’ve spent a little while the last few days building a list of PacketShaper rules to identify spyware web activity based primarily on user agent strings that the spyware programs use. Seems to be doing the right thing and blocking those user agents, though I don’t have a handy spyware-infected box to test it with.

I put the blocked class on the outbound link only, and used the class criterion for HTTP traffic to identify matching user agent strings. I had to move this class above the “Services” class in order for this HTTP traffic to be classified here rather than in the general allowed HTTP traffic.

I have identified user agent strings by running Snort on BIFROST and using the bleeding-snort malware rulesets. Mostly this has caught MarketScore, but there have been a number of other ones falling into the pot as well.

Mailman tools

After clearing up some of my inbox, I spent most of the day writing some utilities for dealing with Mailman.

One script that synchronizes Mailman aliases with LDAP, both for future LDAP routing and for current use in getting these aliases over to KE in a simple fashion. And the other script was for extracting aliases from LDAP so that non-LDAP-enabled Sendmail can get them in alias and virtual user files.

SAgate nearing readiness

I got the rewrite of the MIMEDefang filter for SAgate mostly complete today and have been testing it a bit. It’s looking good.

Naturally, a bunch of errors the first few times, but thanks to MIMEDefang’s -test mode, I got the bulk of those straightened out. There were a few I had to pin down with syslog debugs to figure out what was going on.

The main things left at this point are putting in more logging of the appropriate types (primarily in filter_end() for SpamAssassin status), making the PHP web interface understand the XML config file, and testing, testing, testing. Some command line config tools wouldn’t hurt, either.

I’d like to get a greylist implementation fitted in, but that may come in round two. I have, at least, found some good greylist reference implementations that use both BDB and MySQL as backends, so attaching that wouldn’t be too hard — just have to fit it in.

SAgate Revisitation

Revisiting SAgate and rewriting large parts of it.

I can never seem to get MIMEDefang filters exactly right, less so with ones that I’ve never put into production.

I’ve written the SAgate config file in XML, parseable with XML::Simple. SAgate will operate in MX-only mode, meaning that we won’t tell our users that they should use it as the SMTP host (or POP or IMAP host with it doing proxy, either). This means we can simplify a lot of the logic to figure out whether we should apply filters (particularly SpamAssassin) to any particular message (it had been based on a munging of recipient).

I’ve introduced the idea of multiple domains for SAgate to operate under, and built a robust routine to identify which domain is active for a particular message/recipient. I’m also building in logic to have the whole filter for a domain turned on or off (or logging-only) with exceptions for particular addresses.

All of this should end up making it more robust, slightly more flexible, and actually more simple.