by Dave Crocker, Brandenburg InternetWorking

The first Internet (ARPANET) e-mail, sent 35 years ago, was remarkably similar to a basic text e-mail of today: From, To, CC, Subject, Date, followed by lines of text, and the familiar @-sign in addresses. The right side of the address changed from a simple string into the multilevel domain name that we now use. The body can now be a set of multimedia attachments rather than just lines of text, but it can still be in its original, simpler form. The means of moving mail was the File Transfer Protocol (FTP) in the early 1970s. The current mechanism, the Simple Mail Transfer Protocol (SMTP) [1a, 1b], was not created until 10 years later, but a mere 25 years of use is not bad, either.

All of the technical specifications for e-mail have undergone many changes over the years, but a core requirement has been to protect the installed base of users and operators by incrementally adding features as options, rather than by performing wholesale replacement of any infrastructure service component. E-mail has changed the way we communicate, yet it is also now viewed as having a serious problem: As the Internet grew, it acquired the full mixture of participants, some of whom do not make nice neighbors.

Frustration with the effect of abusive users is often expressed as a belief that the solution lies in replacing some or all of the core technology of the e-mail service, or even by moving to an entirely different paradigm, such as querying Webpages using Really Simple Syndication (RSS) [2]. Although different paradigms make sense for some forms of human communications, what is forgotten in these pleas for massive change is the power of the classic mail model, whether by paper or by electrons: Spontaneous or occasional communication requires the ability to "push" the message to the recipient, without prior arrangement. This ability is, of course, also what leaves the door open for abuse—anyone may walk in, uninvited and unwanted.

The alternative proposals might work well enough for ongoing, regular communication among people who already know each other. And for most of us, that is probably 80 percent of our exchanges, or more. Unfortunately, as soon as anyone starts worrying about the remaining 20 percent, these alternative approaches require cascading hacks, producing a design that looks no better that what we have today, except that it based strictly on theory rather than decades of practice. It is easy for a paper proposal to beat a deployed system; making it work as promised is, of course, more difficult.

Mantra

I have developed a simple mantra, in response to calls for replacing today's Internet mail:

0. The basic problems we are experiencing with e-mail are really based on undesirable social behaviors, long popular outside the Internet. The Internet enables broader reach, to more victims, and in much shorter time spans, but the core misbehaviors have existed for all of recorded human history. We should not assume that there are technical solutions to social problems.
1. The beginning of changing a human service is to gain community consensus about the change that is needed, because a mechanism will not be successful unless it is perceived as needed. Only then can the engineers work on designing the change.
2. When there is community consensus about the way that e-mail needs to be changed, the folks who are currently contributing to its 35-year evolution need to try to find a way to add the desired features to the existing service. Given the record of accomplishment of e-mail, the odds seem favorable that any new requirement can also be satisfied without disrupting the installed base.
3. Alas, as those who track e-mail abuse technical discussions are well aware, we have not completed Step 1. As soon as we try to formulate community consensus about basic messaging communication policies, discussion devolves into cacophony or marginalized community fragments. It is certain that there will eventually be a change required for e-mail, which we cannot fit into the current service, but we do not yet have any evidence that e-mail abuse is going to produce that requirement.

Trust Models

One hopeful sign is that we do have a solid set of efforts to evolve e-mail to support mechanisms that are based on trust. This evolution begins with the ability to associate a validated identity to a message and then requires assessing the "safety" of that identity's owner. Until recently, only the IP address of the last-hop sending SMTP server could be used as an identifier. Using addresses as identifiers sounds reasonable at first glance, but turns out to have long-term scaling and administrative problems. As a result, there has been a broad effort to find ways to use domain names, which are more stable, and they align better with organizational boundaries. This process is well under way, with the recent IETF standardization of the Domain Keys Identified Mail (DKIM) [3] message-signing specification, as well as path-based registration schemes, such as Sender-ID [4] and SPF [5].

That took about 5 years. And now comes the hard part: developing a range of assessment mechanisms—sometimes generically called reputation services—that satisfy requirements for quality, strength, convenience, and stability. Assessment services tell recipients whether the author of the message, or the service that sent it, can be trusted. Some mechanisms need to work for small groups, others need to work for mass-market business-to-consumer mailings, and others need to work among business partners. A few startup companies have recently joined the few, surviving volunteer services, to satisfy this need. It is too early to tell whether they will suffice, or whether additional services will be needed. What is important is that these services are generally regarded as producing good results.

For the long term it seems likely that this capability will result in an Internet mail service that is logically split into two types of traffic. One has substantial trust associated with its messages, so that they can be delivered with a reasonable degree of comfort. The other is the current, open-to-all service that requires heavy filtering and the use of various heuristics, to reduce the effect of abuse mail. If the first traffic flow is sufficiently successful, filters for the second can become much more stringent. The aggregate effects of these changes will be that wanted mail is likely to be received and identified much more reliably, and unwanted mail is more likely to be rejected. [8, 9]

So the current Internet mail technical infrastructure is safe, right? Well, maybe.

Enhancements?

What gets less attention, but perhaps should worry us more, is the general lack of user-level functional enhancement for e-mail. What users can do with e-mail, today, is pretty much the same as they could do 25 years ago. The evolution of Internet mail has been primarily in support of performance, reliability, and scaling. Although important, they have not produced functional changes that are apparent to end users. Human communication is a very rich space, yet most e-mail is limited to a narrow range of styles: person-to-person informal communications, and informal, unstructured group communications. Toss in some very basic, one-way "transactional" mail, such as order confirmations from businesses to their customers, and that about covers it.

Instead, new functions for human collaboration have tended to appear in new services. Instant Messaging (IM), blogging, and wikis are the most popular examples. In each case, they rely on a centralized service, rather than the highly distributed model that e-mail uses. Users must all go to a single, centralized address to obtain a given service. Most of the IM world does not even know that there are two (!) Internet standards for distributed IM—Extensible Messaging and Presence Protocol (XMPP) [6] and SIMPLE [7]. Even for these standards, most of their production use tends to be within noninteroperable, centralized services.

Is there something about e-mail that is a barrier to functional enhancements for end users?

For these new services, the interservice relaying that is at the core of e-mail is absent. Indeed, centralized services are easier to create and operate than are distributed services, but they also carry scaling, administration, and control challenges. So the issue is not so much what is easier, but who will do the work—and when? With a centralized service, all the interesting work is done by the single provider. For a distributed model, like e-mail, the work is shared across participating organizations. The Internet was designed to avoid single points of failure (and failure), so it is ironic that these new services risk exactly these problems.

For a distributed model, like e-mail, to add end-user functions, useful adoption is required by all user software that participates, and possibly by all the intermediate, relaying services. The adoption is in three parts: agreeing on the enhancement, modifying existing software, and making it available to users. These are daunting barriers, so the appeal of centralized services is clear: a single organization decides what to change, changes it, and makes it available to end users with, at most, a natural software upgrade.

Interorganization partnerships provide the best argument for adoption of distributed services, because they do not naturally permit agreement on a central point of control. The counterforce is, again, the simplification (for the partners) that comes from agreeing to use independent third-party services. The scaling problem here is with end users having to juggle a large number of independent services. Note the emergence of IM clients that support a variety of independent IM services. Perhaps the real danger to e-mail is not its wholesale and traumatic replacement, stemming from frustration about abuses, but a gradual attrition, as portions of its traffic move to services that evolve more quickly, but leave end users with a complicated array of narrow, specialized, and noninteroperable venues.