David MacQuigg – 10/13/05 Home Home/email – articles on Email Authentication
Python was chosen because it provides easy-to-read scripts that mail-system administrators can modify to make their Mail Transfer Agents (MTAs) do exactly what they want. These scripts may be embedded in a C program, and modified without having to recompile the whole program. Thus you can make program changes without disrupting a running MTA. Frequent changes will be necessary as long as email authentication methods are in the "experimental" stage.
Currently, all accepted email must be processed by an imperfect statistical spam filter. False rejects are making email communication frustrating and unreliable. The new authentication methods will allow receivers to reject forgeries and accept mail from reputable senders without any blocking or filtering. Success will be measured by how much we can reduce the ratio of filtered mail to mail accepted based on reputation of the sender. See Executive Summary.htm for a diagram showing a typical mail flow with authentication as the first step.
checkHelo.py – update to checkID.py, not yet included in the packages. This is a simplified version that uses only the HELO name for the initial query. The new strategy is to simply pass messages without a valid HELO name to the spam filter.
checkID.py – Interface between an MTA and various authentication methods (currently SPF and CSV). The MTA calls checkID with an IP address and all the envelope information. checkID calls a method, gets a result, translates the result to an action, and returns to the MTA (action, SMTP_reply, headers). Typical customization of checkID will be changing the relationship of various authentication results ( pass, fail, neutral, softfail, etc.) to the available actions ( ACCEPT, REJECT, FILTER, TEMPFAIL, DISCARD, CONTINUE ).
dnsRegRec.py – queries a DNS TXT record from the Registry of Public Email Senders
parseRegRec.py – parses the record to extract reputation and authentication information
csv.py – runs the CSV authentication method
Pre-release packages – a little more stable than the above, and no name conflicts with existing packages.
email_auth - Python package with csv.py and spf.py
.zip – Windows version .tgz – Linux version ( tar –xzf email_auth.tgz )
We need at least one good MTA integration. Sendmail is a good
choice, since it is the most popular MTA and very versatile in its configuration. We are working now on a Python milter, based on http://sourceforge.net/projects/pymilter This involves "refactoring"
checkID() into several smaller functions that fit the
milter API http://www.milter.org/milter_api/api.html
These functions may replace checkID() entirely as our standard package, since it easy
to put them back together if you need an all-in-one call.
checkID.py needs a C-wrapper that can be compiled into any MTA written in C. I'm thinking of something like Example 20-6 in Programming Python, 2nd ed. This will make it easy for anyone familiar with the MTA to call our function, not knowing anything about Python. See MTA-Integrations.htm for details.
We need pre-configured packages for receivers that don't want to upgrade their MTAs, but will consider using checkID as a separate MTA.
We need some good statistics on the percentage of spam coming from the authorized servers in each of several large domains.
We need a script to scan the headers of an email, skip over reputable forwarders, determine the authenticated Identity of the sender, and prepend a Warning: when the From: address does not match that Identity.
We need a spam filter modified to factor in the domain ratings determined by checkID.py, accumulate statistics on spam from each domain, and send an alert when there is a sudden surge in spam from any domain. A daily report on spam scores is also needed.
We need web pages that will allow senders to register their IDs and update their authentication data in the Registry and allow selected Rating Services to update the domain ratings in the Registry.
We need a script that will query DNS records from Registered Senders, and update their records in the Registry. This script should also generate an email to the sender if there is a problem with the record, like a syntax error.