Registry of Public Email Senders
David MacQuigg
Ensuring a valid identity on an email has become a vital first step in stopping spam, forgery, identity theft, and even more serious crimes. A second vital step is assessing the reputation of that identity. A lot more is needed to end the overwhelming abuse of email,[1] but allowing receivers to know the identity and reputation of senders could have a major impact.
Many pieces of the solution to the problems of authentication and reputation exist already. This article will propose a Registry of reputable senders to bring these pieces together in a smoothly working whole.
Anatomy of the Email System
Mail flows through the Internet via three kinds of Mail Transfer Agents ( MTA's ): the sender or agent who first puts the email on the public Internet, various forwarders along the way, and the receiver or agent at the final destination.[2],[3] When we say Internet with a capital I, we mean the world-wide network that shares a common set of IP addresses, not the internal networks before the sender or after the receiver. For example, the computer I am writing this article on shares a local network with other computers having addresses I can assign at will. My network connects via a router to the network of my Internet Service Provider ( ISP ), and he can assign whatever addresses he wants within his network, including the address of my router. It is only when he connects his router to the Internet that a real Internet IP address is needed.
Forwarders should not be confused with routers. There may be dozens of routers along the path from sender to receiver, but all they do is route the data packets from one station to the next. They do not change the source or destination IP addresses on the packets. When an email passes through a forwarder, however, it gets new IP addresses. This is one of the reasons it has been difficult to identify forgers. Often they work through forwarders to conceal their source address.
Forwarders are not necessary to complete a mail transfer. Any MTA on the Internet can send directly to any other MTA. They do perform useful services however, so we can't just outlaw them. Forwarders acting as agents for the sender can handle functions like spam filtering and finding the lowest cost routes. Forwarders on the receiving side can provide a permanent email address for customers who might be changing jobs or ISPs.
Authenticating Senders
Many methods have been developed to authenticate the source of an email [4],[5],[6],[7],[8]. These fall into two basic categories. With IP-based methods, you trust a forwarder to correctly identify its immediate sender, and you trust subsequent forwarders to not alter that information. With signature authentication methods, you trust the validity of a public key from the sender to verify a signature on an email. All methods require a separate "channel" to provide information that the forger can't alter. Typically, this involves storing information in the Domain Name System (DNS). This has led to speculation on what the spam gangs might do if DNS is all that is stopping them. A successful attack on DNS could take down the Internet.
No authentication method has had a significant impact on the worldwide flood of spam.[9] The problem is lack of widespread deployment. Senders have little incentive to offer authentication until they start seeing a significant amount of rejection by receivers. Receivers can't rely on authentication as long as most senders don't offer it, and a lack of authentication means nothing. Nobody can agree on which method to use, even if there were adequate incentives.
Some are calling for government regulation. Governments are reluctant to get involved, however, because of difficulties in legislating technical standards, and difficulties in law-enforcement across international boundaries. Meanwhile, the battles rage over which authentication method should be chosen as the standard. Each method has its weaknesses, and some have risks of causing new problems, like opportunities for Denial of Service attacks. A mandated standard method could be a costly mistake.
A Registry for Authentication Data and Domain Ratings
A Registry of Public Email Servers could provide a "clearing house" for authentication and domain-rating information. It could resolve some of the unnecessary incompatibilities between methods. It could provide an opportunity for each of the competing methods to deploy, while avoiding a "lock-in" for any one method. It could avoid the need for a government mandated standard.
The operation of the Registry is basically quite simple. When a forwarder or receiver wants to check the identity and reputation of an unknown sender, it queries the Registry and gets all the information needed in one packet. This could include ratings from each of several services, and whatever data is needed to run an authentication method specified by the domain owner.
Let's look at what happens when the forger in the above figure says "HELO, this is smithbarney.com". He can say whatever he wants in any command or header line, but he can't fake Smith Barney's IP address, or the information in the Registry from the real Smith Barney.[10] The forwarder queries the record in the Registry for smithbarney.com, sees that the IP address doesn't match,[11] and rejects the forgery even before transferring any data.
Now let's see what happens if the forger is more clever. "HELO, this is smith-barney.com". ( Notice the extra hyphen.) A query to the Registry shows there is a record under that name. It was registered using a stolen credit card. It even has authentication data to authorize any zombie the forger and his friends might be using as a forwarder. The only thing it doesn't have is a good reputation.[12] The record lists ratings from five services, none of which we know anything about. So the message is flagged as suspicious, and handled the same way as other email from unknown sources.[13]
Success of the Registry will depend on the motivation of senders, with little or no prodding from governments. They key difference between this Registry and earlier proposals, is that senders don't have to take the first step. They will get a default record based on their total IP allocation. They will also get a Default Identity, based on their HELO name. None of this is optimum. In fact, senders should be given plenty of opportunity to correct the records before the Registry starts operations.
Let's look at the motivations of different groups. Receivers are eager to stop incoming spam. They will adopt any and all methods they think are useful, and pay any reasonable fee to support the Registry. Senders will do only the minimum necessary to preserve their reputation and avoid rejection of their mail. The default record makes them responsible for all spam coming out of thier IP blocks, including the zombies they claim they can't control. This will give them strong motivation to edit their records and authorize just those MTAs that they use for legitimate mail.
What about forwarders? Initially, they are responsible for everything coming out of their pipe. If they authenticate their incoming mail, however, they can pass off reponsibility for content to the sending domain. The forwarder's reputation will then depend only on the reliability of its authentication. Most forwarders, the ones on the receiving side, will immediately upgrade their software with the latest and best authentication methods.
That leaves a few legitimate forwarders on the sender's side, and a thundering herd of others ranging from pure spam relays to those that simply don't care about their reputation. The legitimate forwarders will do the same as the forwarders on the receiver's side – upgrade their software and authenticate their senders.
Frequently Asked Questions
The description above is simplified in order to better understand the basic features of the proposed Registry. Further details can be found at Registry-Notes.htm. Here are some questions that have come up in discussions:
Q1) How is this Registry different than the proposals that were thoroughly discussed and rejected in the FTC's "National Do Not Email Registry" report?
A1) The key difference is that this is a Registry of senders, not receivers. There are many other differences, most of which will make things easier. See Registry-Notes.htm.
Q2) How is this Registry different than the "walled gardens" we have seen before – isolated groups of domains that trust each other, but cannot communicate with others outside their group?
A2) This Registry will include all domains that want to operate a Public Mail Server and all services that want to provide ratings of these domains. Anyone is free to join. A small fee should be sufficient to avoid spammer tricks involving massive automated registrations.
Q3) How will Rating Services be paid?
A3) Receivers will support the Registry via subscription fees, and indicate on their subscription forms which services should receive their money. Thus, the money will be distributed fairly without involving the Registry in rating the Rating Services.
Q4) How will you deal with fraudulent or incompetent Rating Services?
A4) Receivers will deal with them. The Registry is just a "clearing house" for whatever information senders and services want to provide for receivers. It may take years for a new service to earn the trust of receivers. Losing that trust will be far more costly than whatever a service might gain from becoming spam-friendly.
A low-quality Rating Service won't be able to hide its problem. Their domain ratings will be significantly different than the better services. Spam reports and other raw data will be available. Studies will be done, and articles published. Word will get around. A low-quality service will drop out when it has so few subscribers that it can't even pay its annual registration fee.
Q5) How will a new Rating Service get started? Won't the domain ratings business be dominated by just a few really big services?
A5) Senders can set up their Registry record to *recommend* Rating Services of their choice. Ratings from these services would then be included *in addition to* ratings from the most trusted services. This should be sufficient to start collecting data on the quality of the new service.
Q6) How will the Registry avoid political problems if it is controlled by one country?
A6) Most likely there will be different Registries in different parts of the world, all sharing information and operational standards under an umbrella organization like ICANN (Internet Corporation for Assigned Names and Numbers). Preventing forgery of domain names is a function not unlike preventing duplication of names and numbers.
A Look at the Future
The current deadlock between incompatible authentication methods will be resolved quickly once the Registry is running and all methods are able to use its facilities. We will finally start to see data from large-scale testing. The developers of each method will be able to spend their time more on the truly difficult problems that have arisen, and less on speculation by their competitors about what might happen. The technical problems will be resolved. Any methods that don't avoid the dire consequences predicted by their competitors will be quickly replaced.
Spam will never go away, but very little of it will get past the identity and reputation checks that most receivers will have in place. The few pieces that do get through will get an immediate and effective response – rate-limiting or even shutting down the hijacked domain, and limiting the damages to a very small fraction of todays costs.
The Registry will never be corrupted, because legitimate email users have far more economic power than email abusers. What the Registry will do is bring some of that power to bear on providing reliable identities and reputations of email senders. A modest fee from email receivers could easily generate $100 million per year, far more than what is needed.
A Registry by itself will not solve the problems of email abuse, nor will it eliminate the need for spam filters, law-enforcement, and public education, but it should provide the catalyst necessary to get the industry moving. Providing this key piece of the solution, a piece that all methods can share, is low-risk, low-cost, and should be done immediately.