Majordomo?

TERENA has been using Majordomo as it's mailing list manager (MLM) for many years. This software first appeared somewhere in 1993 and is written in Perl. According to some people:

...Reliable old standard.
Crufty perl4, but hey, it works.

It's indeed old and clunky but it does what we need: work with our MTA (Postfix), and handle multi-domain e-mail lists with separated namespaces (so f.i. mylist@domain1.com and mylist@domain2.com).

The really big downside of majordomo is that it stores subscriber and list data in flat files, sitting in your filesystem.
Over the years TERENA has set up hundreds of lists, containing thousands of subscribers. Working with this data means editing flat files, which is OK if you only require good-old grep or sed, but it quickly becomes a pain when you need to do anything beyond this.
Updating and querying lists from outside majordomo is an example of this. Lots of our PHP web-applications require an interface to majordomo's subscriber data. Currently we have hacked written a PHP interface to majordomo that uses FTP/libcurl, but clearly this is not the way to go; what we really want is some kind of SQL backend support.
There are other MLM's out there that look like they could do the job, but upon closer inspection they are less suitable for us:

  • Majordomo2 - promises to have SQL support, but from the start in 2000 this has never actually worked
  • Mailman - no SQL support, too much overhead.
  • Sympa - supports SQL, but management interface has limitations. Also too much overhead.

After speding lots of effort figuring out how to automate the creation of new majordomo lists, I came to the conclusion that majordomo in its current form is just not suitable for this. Therefore I tried to hack SQL support into the current version of majordomo (1.94.5). After spending lots of effort on this as well, I again came to the conclusion that this is not the way to go. In the end I decided to write a replacement for majordomo, from the ground up, which i will call Listman. This was a nice oppurtunity to polish up Perl skills as well ;-)

Listman

Listman is a piece of software written in Perl, that was intended to be replace Majordomo on our systems. It currently works with MySQL-5.0, Perl-5.8.7, and Postfix-2.3.3 (since no flashy/latest/greatest features are used, I expect that it will work with older versions as well).
Because it needs to replace Majordomo, the old email interface looks very similar. I have choosen to implement only a subset of the available majordomo commands in the email interface. This is done because every feature costs time to implement, and some features were never used here.
At this moment (7 Feb 2007) I have a working version running that can distribute emails, and that implements the subscribe, unsubscribe, and who commands. Confirmation cookies are sent if necessary. Some other features/highlights that already work:

  • Compatible with Majordomo. It is set-up to recognize majordomo commands to easy transition.
  • Multiple domains with sepated namespaces.
  • All subsriber data is completely in database tables. No flat files anymore.
  • All list configuration data is in database tables.
  • Works very closely with the Postfix MTA, but in theory it can work with any MTA that can do SQL lookups. Because of the tight coupling of the MTA and the database, a new list can be created just by adding it to the database. The old aliases and virtual kludges that other MLM's need are now done completely in SQL. Because of this all your (web)apps can be used to create lists on the fly.
  • Nested lists. You can put a list on another list. This is automatically exploded when f.i. checking posting access. The current version supports only one level of recursion (so no lists on lists on lists etc).
  • Configurable "allow-to-post" addresses. This is the successor of majordomo's restrict_post option. These can also contain other lists.
  • SASL authentication. You can use a successfull SASL authentication as a way of granting access to a list.
  • Header SPF checks. You can configure an SPF check on the From or Reply-To header. This fixes a problem with clever spammers bypassing the 'official' SPF restriction (which works on envelope addresses), and after that spoof list members' email addresses in the email headers.
  • Parsing of UTF-8 encoded strings in phrases. When doing a "who" you will see decoded strings an not stuff like =?UTF-8?B?w5DErcOnzrogzr3EqcWfxaHOtcWX?=
The database contains 6 tables:
With these tables a number of dedicated views can be constructed, for use by Postfix.

FAQ

to be done...

Downloads

This software is work in progress and is not yet ready for production use, but once it is, it will be made available here.

Author: Dick Visser

Spam needed!

I need some traffic for my project, so please send as much messages and/or spam as you can to the following addresses:
edna@tienhuis.us
hoebaa@terena.biz
news@terena.eu
testing@terena.info
testing2@terena.info
hosting@tienhuis.be
dod@tienhuis.us
its@terena.org