David MacQuigg
Items that must be standardized to allow interoperation of different email authentication methods include the ID declaration, the authentication query, and the authentication header. Agreement on a minimal set of requirements for these items will provide a framework within which deployment and large-scale testing can begin, and will avoid the formation of a de-facto standard that may prevent later adoption of a superior method. This framework will allow widespread use of authentication, even while the different methods work out their internal problems.
Widespread adoption of email authentication has been delayed due to lack of a standard protocol accepted by all Public Mail Servers. This delay is partly due to the uncertainties about how each proposed method {see IETF drafts below} will work in large-scale deployment on the Internet. Unfortunately, large-scale testing cannot be done with the current low level of participation by Internet service providers, spam filter companies, spam-blocking services, and others who worry about investing in one technology, then having to change everything when a standard emerges.
In addition to the delay in adoption, lack of a well-planned standard raises worry that widespread use of one of the current methods will result in a "lock-in" situation where all participants in a mail transfer must use that one method. This could make it almost impossible to switch to another superior method that does not conform to the de-facto standard.
The purpose of this protocol is to distill from the competing methods just those items that are necessary for interoperation. As much as possible, we will leave to the competing methods, all details that are not needed for interoperation. This proposal is not intended as a complete authentication system, but just a framework within which other systems may operate. Getting rid of unnecessary variations will not favor any one competitor.
Following this introduction, we will jump ahead to a specific problem, the email ID declaration, and see how one possible compromise might work. This is an example intended to show how interoperability problems can be solved, not to constrain the final solution in any way. Following this example we have a more formal presentation of the proposal, starting with the fundamental requirements which any method must meet. Then we add subordinate requirements that flow from the fundamentals. Finally, we add specific requirements on implementations, step-by-step procedures, etc. This will allow an orderly discussion, starting with fundamentals that everyone can agree on, then moving to the details that everyone will hate, but hopefully most can live with.
A fundamental requirement for all email authentication methods is that the sender declare its identity to the receiver. Unfortunately, there is no agreement on how this should be done. Some believe firmly that it should be done in the initial HELO command, others insist that it should be done in the MAIL FROM command just before each email. Still others think the true identity should be extracted from one or another of the email headers that the recipient actually sees. Adding to the confusion is the fact that each of these identities may legitimately differ from the identity that is to be authenticated.
Each of the methods is now going its own way, with no thought as to how one will communicate with another as an email is forwarded across the Internet. We need a clear standard that will allow any receiver to know exactly what identity is authorizing the transfer, regardless of which authentication method is used.
One way to standardize the identity declaration is to make it sender's choice among the alternatives now promoted by each method. That will put a burden on the receiver to do the best it can with that choice, but this is better than the current confusion, where the receiver cannot even know what choice the sender is assuming. A sender can flag an identity in one of the existing email commands, or it can add a new one. To flag an identity, put the string ID=* after the declared name.
HELO my-company.com
MAIL FROM: bob@sales.my-company.com ID=*
To use an identity different than one in either command, add ID=<identity> after either of the commands.
HELO mailserver.my-company.com ID=my-company.com
It is important that the sender's ID be explicitly declared, not just assumed from existing information. Not only will this remove the current uncertainty as to which ID the sender intends to use, but false information here is evidence of lying, not just a forgivable error in passing on existing information (a long-standing tradition on the Internet). This will greatly reduce the administrative burden in deciding whether to trust a sender.
Most reputable Public Mail Servers will chose their top domain name as their ID, but it can be any name under DNS control. This could be a domain set up specifically to authorize mail servers, or it could be some other organization's ID. The latter should be allowed but discouraged, since any miscommunication over the use of someone else's ID could result in authentication failures, suspicion of forgery, and loss of reputation by the owner of the ID.
1) This
protocol must not favor any one authentication method over another.
It must allow an
arbitrary number of Forwarders using different methods to
work together in
the same authenticated transfer.
2) Each
Sending Mail Transfer Agent (MTA) in an IP-authenticated transfer
must declare, in
the SMTP session, the Identity responsible for the transfer.
3) Each
Receiving MTA in an IP-authenticated transfer, if accepting mail from
the declared
Identity, must verify that the Identity authorizes the Sending
MTA to act as its
Public Mail Server.
4) Messages that explicitly fail authorization must generate an SMTP Reject,
stating a clear reason for failure.
5) If the
message is forwarded, the results of authentication must be made
available to all
subsequent Receiving MTAs. The format
should be simple and
standardized to
facilitate later blocking and filtering.
6) A Forwarder
must not alter either the incoming headers nor the body of a
message. Prepending of headers is acceptable.
7) Forwarders should minimize Internet traffic by minimizing queries and by
rejecting forged messages at the earliest opportunity.
8) Authentication methods should avoid creating any new opportunities for
Denial-of-Service (DoS) attacks or abuse of the Internet infrastructure.
9) Spam Bounces must not go to any address that might be forged. Forwarders
must accept Spam Bounces resulting from messages they have sent within a
reasonable time,
and should attempt to forward these bounces to an Identity
they have
authenticated.
10) A means
must be provided for a legitimate and authenticated sender with a
single important message to bypass blocking and filtering. Cost to the sender
in time or money should be reasonable.
During the early days of email authentication, it may be useful to rate Public Mail Servers as to their level of compliance with this authentication standard. This will encourage all servers to provide at least minimum security, and allow mail receivers to put special trust in servers that provide the highest levels. One possible scheme could have three levels:
1)
Servers which will declare their ID, and provide a DNS record to authorize that
server.
2) Servers which will capture the IP address and any ID declared by the previous sender and prepend that information in a standard authentication header.
3) Servers which will perform an authentication check on the declared ID, using any widely-accepted method, and prepend the result of that check.
Note
that Level Two compliance will allow a complete authentication check even after
a message has passed through many forwarders.
Level Three will reduce Internet traffic, because forgeries can be
rejected before passing all the way to the final destination.
1a) Requirements
relating to IP-authenticated methods should not burden
signature-authenticated
methods.
2a) The
declaration may be done by flagging an existing domain name ( HELO or
MAIL FROM:
<name> ) or by adding a new one.
3a) Transfers within the same Administrative Domain need not be authenticated,
if the boundaries of the domain are clear from inspection of the headers.
5a) An Originating MTA must provide "authentication results" when it sends
a message on behalf of another Identity, even if that Identity is a customer.
5b) Authentication results must be communicated as a separate header line,
the first header prepended to an email arriving at a Border MTA.
6a) A Forwarder that needs to alter a message must act as a new Originator,
using an Identity for which it is authorized. Handling of Delivery Status
Notifications and Spam Bounces is then at the Forwarder's sole discretion.
8a) Authentication records should be set up so that a single DNS query can
provide complete and stable information for an entire domain, including all
needed authentication and accreditation data.
8b) Methods that allow multiple DNS queries, beyond the normal subdomain
referrals, must limit the number of extra queries, and should provide a
"one-query defense mode" which may be invoked during a DoS attack. In this
mode, mail which requires only one query should flow normally.
8c) Identities should be domain names with no more than three levels.
9a) Spam Bounces must be clearly marked, so they can be distinguished from
normal mail and Delivery Status Notifications ( DSNs ).
9b) Spam Bounces in an IP-authenticated transfer must go back to the Identity
that authorized the Sending MTA. In a signature-authenticated transfer, they
must go back to the signing Identity.
2a) This method of declaring an ID is intended to impose the minimum possible burden on operators of Public Mail Servers, especially during the early days of email authentication, when servers may chose to provide different levels of email security. Level One may be nothing but declaring a verifiable ID, and this may be nothing more than adding the character sequence ID=* after an existing name. See the section "Levels of Compliance".
4) An explicit failure is more than just a lack of authentication. It is a violation of the published policy of a domain owner. The motivation here is to make authentication failures rare and quickly repaired, without any involvement of users. An SMTP Reject gets the message to the responsible party as quickly as possible. This should be treated as a failure of the communication equipment, not something needing users to interpret error codes.
5) There is a question how this requirement should apply to an end-to-end authentication method like DomainKeys. As a minimum, a forwarder should prepend an authentication header with the sender's IP address and any declared ID, followed by the keyword UNVERIFIED.
If the forwarder can't tell what method is used, a DNS query will be necessary anyway. In that case, it may be no more work to authenticate the ID, and pass on the results. If the first query leads to more queries, and the forwarder is under heavy load, it may be appropriate to return a NEUTRAL result, depending on what method was used.
5a) This requirement arises from the fact that a recipient generally has no way of knowing the relationship between an originating ISP and its customer using a different ID. The recipient must see what looks like a simple forwarding of the message.
5b) The sequence is important. Authentication is done first, before receiving the message. Placement of the headers should reflect this. It also facilitates drawing lines to separate headers into different administrative domains.
6) Re-ordering of headers, if necessary, may be done after the message has been authenticated at the final destination. Generally, this is for Display Headers only, and should not affect the Trace Headers added after the message left the Originator.
7) This should work also with domains that use a non-IP authentication method, like DomainKeys. See note 5 above. There should be one query that returns at least a digest of whatever authentication information the domain provides, and that information should be sufficient to reject most forgeries.
8a) Records which cause additional queries should be discouraged, even if they must be allowed. Chained queries are an opportunity for abuse, especially queries to other domains.
Excessive complexity in authentication records is usually a sign of poor planning. Even the largest domains, the ones that have difficulty controlling spam from their entire IP address space, should find it advantageous to isolate their Public Mail Servers to a few easily specified IP blocks. This will protect the reputation of their domain name from any problems outside the specified blocks, even if those problems are in their IP space.
8b) A well-designed "one-query defense mode" will allow normal processing of most mail, with quick rejects of forgeries and temporary rejects of only those messages requiring more than one DNS query. The query limit might be a parameter that can be set to any value between one and the maximum allowed.
8c) Limiting the depth of subdomains in an email ID will make life simpler for email reputation services. They won't have to track millions of subdomains with reputations different than the parent. Reputable email IDs should be like broadcast licenses (a few call letters) or like brand names (short and easily recognized). Even the largest organizations won't need more than a few.
Companies that really want to delegate Public Mail Server authority to the lowest levels, can use a "flattened" name to authorize those servers. server17.ece.arizona.edu might show its email ID as "server17-ece.arizona.edu". ( More likely, it would simply use the mail server at arizona.edu. )
9-1) Some feel that Spam Bounces should not be allowed, and should be treated as more spam if received. This goes against the wishes of recipients who would like to have a "send it back" button. If we ignore this wish, recipients will continue to generate their own Spam Bounces, usually to the wrong address. A prohibition on Spam Bounces will also hinder reputable forwarders who see legitimate bounces not as an annoyance, but as valuable feedback to improve their blocking and filtering.
The strong feelings against Spam Bounces are a result of the current widespread practice of sending them to forged addresses. The new authentication methods will allow Spam Bounces to be routed correctly.
The intent of requirement 9 is to establish a default policy for unrelated parties. Forwarders may make other arrangements with related parties. (daily summaries instead of every message, etc.) Feedback on spam volumes is a vital measure of the health of the Internet.
9-2) Delivery Status Notifications ( DSNs ) should continue to use the Return-Path address, even though it might be forged. Normal mail expects that address to be used, and spam will generate only bogus DSNs that are easily blocked.
9-3) Procedures to ensure no spoofing of complaints are left to the forwarder. Mishandled complaints will lower the forwarder's reputation.
10-1) This is especially important during the adoption period, when many messages may not be properly authenticated. The forwarder's responsibility for providing a bypass could be as simple as a referral to a website. The recipient may still block email sent via the website, but that is entirely up to the recipient. The aim with this requirement is to ensure that customers who sign up with a standards-compliant forwarder won't discover their email blocked without their knowledge. If the website is well-managed, and very little spam gets through, most customers will accept the default procedure, and email will retain a good reputation for reliable delivery.
10-2) A reasonable cost is large enough to stop spam, but small enough that it is not a burden for a legitimate sender. This cost might be 30 seconds to visit a website and type in some characters. A properly-designed challenge/response system is one of the tools made possible by the proper handling of Spam Bounces.
These are desirable, but may be left out of the standard if there is too much controversy.
Benefits: Allows a domain admin to instantly detect forgery attempts against the domain, new zombies within the domain, or a legitimate server that somehow got left out of the authorization record. The extra information might also be useful in evaluating SPF records with macros, and in future scenarios, like prioritizing DNS queries during a DoS storm. Any query without an IP gets lower priority.
Costs: Will require a new DNS query type. ??? Existing TXT queries cannot carry additional data, even a few extra bytes in each query packet.
Authent:
<IP Address> <Senders-ID> SPF1 PASS
This header is
prepended by the Border MTA which first sees the message from <Senders-ID>. It appears just below the Received header from that same MTA. The order
of these headers thus follows the exact order of events, authentication then
reception.
All information
appearing in an Authent header
has been verified by the authenticating domain.
None is hearsay.
SPF1 is the authentication method. UNVERIFIED means no authentication was
attempted.
<IP Address> contains the sender's IP address. This is the minimum information which may
appear in an authentication header.
<Senders-ID> is determined by the sender's
declaration. NO-ID means there was no
valid declaration.
PASS is the result of
the authentication test, using a keyword defined by the selected method. Three
of these keywords are standard for all methods – PASS, FAIL, and NEUTRAL.
After these four
positional items, we could have any number of keyword=value pairs. These could be additional test results, or
method-unique parameters like a hash code. These might help a forwarder in
processing a bounce, but are not needed by any other participant in the
transfer.
The order of the first
four items is such that they always have the same position, even if not all are
present. The <IP Address> is
always present, even if the sender doesn't declare an ID. The <Senders ID> is present if the
sender declares an ID, even if the receiver does not authenticate the ID.
See Scenario D in Spam Scenarios.htm for an example of how this
header might be used.
Unlike Received headers, an authentication header is an affirmative statement – "I authenticated this ID, using this method, and got this result". False information here is evidence of lying or gross negligence, not just forgivable errors in processing a complex pile of headers. This will greatly reduce the administrative burden in deciding to downgrade an ID.
A simple standard format for authentication headers makes it easy for spam filters and reputation tools to skip past trusted forwarders and get straight to the ID that needs to be filtered or rated.
Two items in this header are a repeat of what is in a Received header. However, there is a benefit to having it one place, and no cost. The extra characters are not likely to require another line. The number and complexity of Received headers makes extracting these items difficult.
See IETF draft-kucherawy-sender-auth-header-01
These are set up as TXT records in the subdomain _AUTH.<domain>, so no new query type is needed, and there is no conflict with other TXT records sharing the same limited space in a DNS packet. This TXT record is then copied to some standard domain like .mail, where accreditation information may be added, and the records may be kept on fast, widely distributed servers that are secure and resistant to DoS attack.
One query to _AUTH.<domain>.mail gets all the authentication information for a domain in summary form, including a list of accreditation services and their ratings of the domain, a list of authentication methods the domain uses, and as many parameters as can be squeezed into approximately 450 bytes, a limit imposed by the 512-byte DNS response format. Links to additional records are provided if necessary.
Here is an example of a 343-byte authentication record for a
large, complex domain, with many subdomains and hundreds of servers all over
the
The record has been formatted here for easy viewing, but in DNS, it would be compressed into a sequence of strings separated by single blanks.
svc=S1:A,M2:A,H1+:B dmn=QR1,SPF1+5,DK2
QR1=ip4:?170(24.30.23;24.28.200;24.28.204;24.30.18;24.93.47;24.25.9),
+4(65.24.5.120;24.94.166.28;24.29.109.84;66.75.162.68;24.24.2.12)
DK2=dk:MHwwDQYJKoZIhvcNAQEBBQADawAwaAJhAKJ2lzDLZ8XlVambQfMXn3LRGKOD5
o6lMIgulclWjZwP56LRqdg5ZX15bhc/GsvW8xW/R5Sh1NnkJNyL/cqY1a+GzzL47t7
EXzVc+nRLWT1kwTvFNGIoAUsFUq+J6+OprwIDAQAB
The keywords svc and dmn specify information provided by the services and by the domain. Well-known services have short labels, like S1. Others may be listed with a fully-qualified domain name <FQDN>. A query to <domain>.S1.mail or <domain>.<FQDN> returns the rating for that domain by that service. Normally, these extra queries aren't necessary, because the summary above is adequate.
The example domain has A ratings from two accreditation services, and a B rating from a third. Service H1 provides one additional record at _H1.<domain>.<service>, probably some notes on the reasons for the B rating, status of cleanup efforts, etc.
This domain uses 3 authentication methods, QR1, SPF1 and DK2. These should be executed in the sequence shown. Parameters for each method are given later in the record, or (if a + follows the method name) in additional records. The SPF1 method in the record above has no parameters within the record, but the +5 after SPF1 means that additional records are available, starting with a query to _SPF1.<domain>, and requiring no more than 5 queries to complete.
See draft-qr1-00.htm for a more complete discussion of this record format, the Quick Reject method QR1, and the problem of avoiding DNS abuse.
Domains setting up their authentication records should specify at least one simple method first. Not all forwarders and receivers will implement all methods.
Domains should minimize the number of DNS queries generated by their authentication requirements. MTA's that are heavily loaded may temporarily reject messages that don't PASS with one DNS query, or they may forward the message with less than a PASS authentication result.
A query to an accreditation service should get a standardized
response. One possible standard is
detailed in draft-ietf-marid-csv-dna-02.
The ratings A, B, etc. could have the meanings defined in that
draft. One of the ratings, maybe
"C" should have a special meaning "unknown". This will allow rating services to avoid
listing large numbers of spammer-generated domain names.
MAIL FROM: postmaster@mydomain.com
RCPT TO: abuse@<senders domain>
Return-Path: <complaint>
...
or
Return-Path: <challenge>
...
The <senders domain> in this case is the authenticated domain of the immediately prior forwarder. The keyword in the Return-Path header will allow easy separation of various categories of spam-generated bounces.
Sending to abuse@<senders domain> rather than an individual address will allow the admin at <senders domain> to implement a local policy, like rejection of all spam bounces. This is important to win the confidence of admins who have until now seen only floods of improperly routed spam bounces.
The examples in this proposal may need some work to minimize conflict with existing SMTP standards and practice.
RFC 2822 section 3.6 shows the trace header blocks containing only trace fields ( Return-Path: and Received: ) interleaved with Resent-* fields. Common practice is to interleave other fields as well. (e.g. Delivered-To:, X-Sieve:, X-Authentication-Warning:, X-Spam-Status:, etc.) Paragraph 3 of section 3.6 also says " ... header fields SHOULD NOT be reordered when a message is transported or transformed. More importantly, the trace header fields and resent header fields MUST NOT be reordered, and SHOULD be kept in blocks prepended to the message." The "MUST NOT be reordered" requirement is particularly important for authentication methods using a digital signature of the headers. Even with protocols not requiring this strict ordering, it is advantageous to preserve the order of headers added by spam filters and other agents involved in the transfer.
Thus, it appears that common practice and the requirements of some authentication protocols are out of step with the ABNF syntax definition in section 3.6 of RFC 2822. This issue is discussed in the ietf-smtp mailing list {Tony Finch, ietf-smtp, 10 March 2005 6:21PM +0000}
draft-schlitt-spf-classic-00 anticipates that many domains will need to authorize servers from other domains that they are using as forwarders, and that the instability in the IP addresses of these forwarding servers will require additional DNS queries with every email at every forwarding. This inefficiency may be just a transition phase, however. As more forwarding domains adopt authentication, there will be less need for other domains to authorize their servers.
draft-schlitt-spf-classic-00, Dec 2004, Sender Policy Framework: Authorizing Use of Domains in E-MAIL
draft-lyon-senderid-core-00, Oct 2004, Sender ID: Authenticating E-Mail, Section 7.2 E-Mail Forwarders:
MAIL FROM: Forwarder SHOULD alter
Resent-From: Forwarder SHOULD add
DSNs – Forwarder SHOULD re-forward to original MAIL FROM address
Spam Bounces – Forwarder SHOULD reject
draft-katz-submitter-00, Oct 2004, SMTP Service Extension for Indicating the Responsible Submitter of an E-mail Message
MAIL FROM: alice@example.com SUBMITTER=bob@almamater.edu
Resent-From: bob@almamater.edu
draft-delany-domainkeys-base-01, Aug 2004, Domain-based Email Authentication Using Public-Keys Advertised in the DNS (DomainKeys)
draft-ietf-marid-csv-intro-02, Feb 2005, Certified Server Validation (CSV)
draft-ietf-marid-csv-csa-02, Feb 2005, Client SMTP Authorization (CSA)
draft-ietf-marid-csv-dna-02, Feb 2005, Domain Name Accreditation (DNA)
draft-kucherawy-sender-auth-header-01, Mar 2005, Message Header for Indicating Sender Authentication Status.
draft-crocker-email-arch-04, Mar 2005, Internet Mail Architecture
Summary of the current architecture and terminology.
=== End of File