.he 'Mail Systems and Addressing in 4.2bsd''%'
.fo 'Version 4.1'USENIX \- Jan 83'Last Mod %G%'
Mail Systems and Addressing
1919 Addison Street, Suite 105.
Berkeley, California 94704.
Routing mail through a heterogeneous internet presents many new
Among the worst of these is that of address mapping.
Historically, this has been handled on an ad hoc basis.
this approach has become unmanageable as internets grow.
Address interpretation is controlled by a production
which can parse both old and new format addresses.
a flexible technique that can
handle many common situations.
Sendmail is not intended to perform
user interface functions.
Sendmail will replace delivermail in the Berkeley 4.2 distribution.
Several major hosts are now or will soon be running sendmail.
This change will affect any users that route mail through a sendmail
The changes that will be user visible are emphasized.
\(dgA considerable part of this work
was done while under the employ
at the University of California at Berkeley.
The mail system to appear in 4.2bsd
will contain a number of changes.
Most of these changes are based on the replacement of
implements a general internetwork mail routing facility,
featuring aliasing and forwarding,
automatic routing to network gateways,
and flexible configuration.
Of key interest to the mail system user
will be the changes in the network addressing structure.
each node has an address,
and resources can be identified
with a host-resource pair;
the mail system can refer to users
using a host-username pair.
Host names and numbers have to be administered by a central authority,
but usernames can be assigned locally to each host.
multiple networks with different characteristics
the syntax and semantics of resource identification change.
Certain special cases can be handled trivially
providing network names that appear local to hosts
as with the Ethernet at Xerox PARC.
However, the general case is extremely complex.
some networks require that the route the message takes
be explicitly specified by the sender,
simplifying the database update problem
since only adjacent hosts must be entered
while others use logical addressing,
where the sender specifies the location of the recipient
but not how to get there.
Some networks use a left-associative syntax
and others use a right-associative syntax,
causing ambiguity in mixed addresses.
Internet standards seek to eliminate these problems.
Initially, these proposed expanding the address pairs
{network, host, username}
Network numbers must be universally agreed upon,
and hosts can be assigned locally
The user-level presentation was changed
comprised of a local resource identification
and a hierarchical domain specification
with a common static root.
separates the issue of physical versus logical addressing.
.q "eric@a.cc.berkeley.arpa"
organization of the address space
but not the physical networks used
(for example, this could go over different networks
or a store-and-forward network).
is intended to help bridge the gap
of networks that know nothing of each other
and the clean, tightly-coupled world
of unique network numbers.
It can accept old arbitrary address syntaxes,
resolving ambiguities using heuristics
specified by the system administrator,
as well as domain-based addressing.
It helps guide the conversion of message formats
between disparate networks.
is designed to assist a graceful transition
to consistent internetwork addressing schemes.
Section 1 defines some of the terms
when working in mail systems.
Section 2 discusses the design goals for
Section 4 discusses some of the special problems
are presented in section 5.
use names of actual people
or even an intellectual agreement
on the part of these people or organizations.
Bell Telephone Laboratories (BTL),
Digital Equipment Corporation (DEC),
Lawrence Berkeley Laboratories (LBL),
Britton-Lee Incorporated (BLI),
and the University of California at Berkeley
are not committed to any of these proposals at this time.
the personal opinions of the author.
There are four basic concepts
that must be clearly distinguished
when dealing with mail systems:
the user (or the user's agent),
the user's identification,
These are distinguished primarily by their position independence.
.sh 2 "User and Identification"
that is creating or receiving a message.
is an entity operating on behalf of the user \*-
such as a secretary who handles my mail.
or a program that automatically returns a
.q "I am at the UNICOM conference."
The identification is the tag
that goes along with the particular user.
This tag is completely independent of location.
my identification is the string
and this identification does not change
whether I am located at U.C. Berkeley,
or at a scientific institute in Austria.
Since the identification is frequently ambiguous
it is common to add other disambiguating information
that is not strictly part of the identification
.q "System Administrator"
The address specifies a location.
my address might change from
depending on my current affiliation.
an address is independent of the location of anyone else.
my address remains the same to everyone who might be sending me mail.
a person at MIT and a person at USC
and have it arrive to the same mailbox.
service would be provided to map user identifications
Currently this is handled by passing around
or by calling people on the telephone
to find out their address.
While an address specifies
As such, the route is potentially different
for every pair of people in the electronic universe.
Normally the route is hidden from the user
some networks put the burden of determining the route
Although this simplifies the software,
it also greatly impairs the usability
The UUCP network is an example of such a network.
\**This section makes no distinction between
Compatibility with the existing mail programs,
including Bell version 6 mail,
Reliability, in the sense of guaranteeing
that every message is correctly delivered
or at least brought to the attention of a human
no message should ever be completely lost.
This goal was considered essential
because of the emphasis on mail in our environment.
It has turned out to be one of the hardest goals to satisfy,
especially in the face of the many anomalous message formats
produced by various ARPANET sites.
certain sites generate improperly formated addresses,
causing error-message loops.
Some hosts use blanks in names,
mail programs that assume that an address
The semantics of some fields
are interpreted slightly differently
the obscure features of the ARPANET mail protocol
are difficult to support,
Existing software to do actual delivery
should be used whenever possible.
This goal derives as much from political and practical considerations
fairly complex environments,
connections to a single network type
(such as with multiple UUCP or Ethernets).
This goal requires consideration of the contents of an address
in order to determine which gateway to use.
Configuration information should not be compiled into the code.
A single compiled program should be able to run as is at any site
(barring such basic changes as the CPU type or the operating system).
We have found this seemingly unimportant goal
to be critical in real life.
Besides the simple problems that occur when any program gets recompiled
in a different environment,
with anything that they will be recompiling anyway.
must be able to let various groups maintain their own mailing lists,
and let individuals specify their own forwarding,
without modifying the system alias file.
Each user should be able to specify which mailer to execute
to process mail being delivered for him.
This feature allows users who are using specialized mailers
that use a different format to build their environment
without changing the system,
and facilitates specialized functions
Network traffic should be minimized
by batching addresses to a single host where possible,
without assistance from the user.
These goals motivated the architecture illustrated in figure 1.
+---------+ +---------+ +---------+
| sender1 | | sender2 | | sender3 |
+---------+ +---------+ +---------+
+----------+ + +----------+
+----------+ + +----------+
+---------+ +---------+ +---------+
| mailer1 | | mailer2 | | mailer3 |
+---------+ +---------+ +---------+
Figure 1 \*- Sendmail System Structure.
The user interacts with a mail generating and sending program.
When the mail is created,
which routes the message to the correct mailer(s).
Since some of the senders may be network servers
and some of the mailers may be network clients,
may be used as an internet mail gateway.
Arguments may be flags or addresses.
Flags set various processing options.
Following flag arguments,
address arguments may be given.
Addresses follow the syntax in RFC822
Anything in parentheses is thrown away
Anything in angle brackets (\c
This rule implements the ARPANET standard that addresses of the form
user name <machine-address>
will send to the electronic
backslashes quote characters.
Backslashes are more powerful
in that they will cause otherwise equivalent phrases
to compare differently \*- for example,
is different from either of them.
or duplicate suppression algorithms.
Parentheses, angle brackets, and double quotes
must be properly balanced and nested.
The rewriting rules control remaining parsing\**.
\**Disclaimer: Some special processing is done
after rewriting local names; see below.
Although old style addresses are still accepted
the preferred address format
is based on ARPANET-style domain-based addresses
These addresses are based on a hierarchical, logical decomposition
The addresses are hierarchical in a sense
similar to the U.S. postal addresses:
the messages may first be routed to the correct state,
with no initial consideration of the city
or other addressing details.
The addresses are logical
in that each step in the hierarchy
rather than a physical network.
would first look up the domain
in the namespace administrated by
A query could then be sent to
Eventually the mail would arrive at
which would then do final delivery
.sh 2 "Mail to Files and Programs"
Files and programs are legitimate message recipients.
Files provide archival storage of messages,
useful for project administration and history.
Programs are useful as recipients in a variety of situations,
to maintain a public repository of systems messages
Any address passing through the initial parsing algorithm
(i.e, not appearing to be a valid address for another mailer)
is scanned for two special cases.
If prefixed by a vertical bar (\c
the rest of the address is processed as a shell command.
If the user name begins with a slash mark (\c
the name is used as a file name,
.sh 2 "Aliasing, Forwarding, Inclusion"
reroutes mail three ways.
Aliasing applies system wide.
Forwarding allows each user to reroute incoming mail
destined for that account.
to read a file for a list of addresses,
in conjunction with aliasing.
Aliasing maps local addresses to address lists using a system-wide file.
This file is hashed to speed access.
Only addresses that parse as local
this guarantees a unique key
(since there are no nicknames for the local host).
if an recipient address specifies a local user
file in the recipient's home directory.
but rather to the list of addresses in that file.
this list will contain only one address,
and the feature will be used for network mail forwarding.
Forwarding also permits a user to specify a private incoming mailer.
"\^|\|/usr/local/newmail myname"
will use a different incoming mailer.
Inclusion is specified in RFC 733 [Crocker77] syntax:
An address of this form reads the file specified by
and sends to all users listed in that file.
to support direct use of this feature,
but rather to use this as a subset of aliasing.
project: :include:/usr/project/userlist
is a method of letting a project maintain a mailing list
without interaction with the system administration,
even if the alias file is protected.
It is not necessary to rebuild the index on the alias database
when a :include: list is changed.
.sh 2 "Message Collection"
Once all recipient addresses are parsed and verified,
the message is collected.
The message comes in two parts:
a message header and a message body,
separated by a blank line.
The body is an uninterpreted
The header is formated as a series of lines
Field-value can be split across lines by starting the following
lines with a space or a tab.
Some header fields have special internal meaning,
and have appropriate special processing.
Other headers are simply passed through.
Some header fields may be added automatically,
used in the UUCP environment
causes a number of serious problems.
without knowing the address of your potential correspondent.
This is typically handled
by specifying the address
it is often difficult to compute
of the topology of the network.
Although it may be easy for a human being
under many circumstances,
a program does not have equally sophisticated heuristics
certain addresses will become painfully and unnecessarily long,
as when a message is routed through many hosts in the USENET.
are impossible to parse unambiguously \*-
decvax!ucbvax!lbl-h!user@LBL-CSAM
might have many possible resolutions,
depending on whether the message was first routed
would have to be changed to use addresses
(with the hop through decvax implied).
This address would itself be a domain-based address;
an address might be of the form:
Hosts outside of Bell Telephone Laboratories
would then only need to know
how to get to a designated BTL relay,
would only be maintained inside Bell.
There are three major problems
associated with turning UUCP addresses
into something reasonable:
creating and propagating the necessary software,
and building and maintaining the database.
.sh 2 "Defining the Namespace"
Putting all UUCP hosts into a flat namespace
is not practical for a number of reasons.
with over 1600 sites already,
and (with the increasing availability of inexpensive microcomputers
several thousand more coming within a few years,
the database update problem
if the namespace is flat.
there are almost certainly name conflicts today.
as the number of sites grow
the names become ever less mnemonic.
that there be some sort of naming authority
for the set of top level names
as unpleasant a possibility
It will simply not be possible
to have one host resolving all names.
It may however be possible
in a fashion similar to that of assigning names of newsgroups
it will be essential to encourage everyone
to become subdomains of an existing domain
even though this will certainly bruise some egos.
were to be added to the UUCP network,
it would probably actually be addressed as
.sh 2 "Creating and Propagating the Software"
The software required to implement a consistent namespace
one to handle incoming mail
and one to handle outgoing mail.
must be prepared to handle either old or new style addresses.
can be passed through unchanged.
must be turned into new style addresses
It must do a database lookup on the recipient addresses
(passed on the command line)
to determine what hosts to send the message to.
If those hosts do not accept new-style addresses,
it must transform all addresses in the header of the message
into old style using the database lookup.
except for the issue of modifying the header.
It seems prudent to choose one format
Berkeley has elected to use the ARPANET protocols
this protocol is somewhat difficult to parse.
Propagation is somewhat more difficult.
There are a large number of hosts
that will want to run completely standard systems
The strategy is not to convert the entire network \*-
only enough of it it alleviate the problem.
.sh 2 "Building and Maintaining the Database"
This is by far the most difficult problem.
A prototype for this database
but it is maintained by hand
and does not pretend to be complete.
This problem will be reduced considerably
if people choose to group their hosts
This would require a global update
only when a new top level domain
A message to a host in a subdomain
could simply be routed to a known domain gateway
without notifying the rest of the world.
be notified as an efficiency measure.
There may be more than one domain gateway.
might have a dozen gateways to the outside world;
could choose the closest gateway.
would be that all gateways
maintain a consistent view of the domain
.sh 2 "Logical Structure"
domains are organized into a tree.
There need not be a host actually associated
with each level in the tree \*-
there will be no host associated with the name
an organization might group names together for administrative reasons;
CAD.research.BigCorp.UUCP
might not actually have a host representing
it may frequently be convenient to have a host
if a single host exists that
then mail from outside Berkeley
can forward mail to that host
without knowing Berkeley's
This is not unlike the operation
of the telephone network.
inside certain large domains.
at Berkeley it may be presumed
that most hosts know about other hosts
inside the Berkeley domain.
But if they process an address
Thus as new hosts are added
other hosts can be updated as convenient.
Ideally this name resolution process
would be performed by a name server
to avoid unnecessary copying
this could result in unnecessary delays.
.sh 1 "COMPARISON WITH DELIVERMAIL"
The primary differences are:
Configuration information is not compiled in.
This change simplifies many of the problems
of moving to other machines.
It also allows easy debugging of new mailers.
Address parsing is more flexible.
only supported one gateway to any network,
can be sensitive to host names
and reroute to different gateways.
features eliminate the requirement that the system alias file
(or that an update program be written,
or that the system administration make all changes).
supports message batching across networks
when a message is being sent to multiple recipients.
A mail queue is provided in
Mail that cannot be delivered immediately
but can potentially be delivered later
is stored in this queue for a later retry.
The queue also provides a buffer against system crashes;
after the message has been collected
it may be reliably redelivered
even if the system crashes during the initial delivery.
uses the networking support provided by 4.2BSD
to provide a direct interface networks such as the ARPANET
using SMTP (the Simple Mail Transfer Protocol)
over a TCP/IP connection.
Standard for the Format of ARPA Network Text Messages.
Standard for the Format of Arpa Internet Text Messages.
Network Information Center,
ARPANET Protocol Handbook.
Network Information Center,
A Dial-Up Network of UNIX Systems.
UNIX Programmer's Manual, Seventh Edition,
An Introduction to the Berkeley Network.
University of California, Berkeley California.
University of California, Berkeley.
In UNIX Programmer's Manual,
The Design of the CSNET Name Server.
The Domain Naming Convention for Internet User Applications.
Network Information Center,
A Distributed System for Internet Name Service.
Network Information Center,