| 1 | SUMMARY OF CHANGES BETWEEN SENDMAIL V5 and V8 |
| 2 | |
| 3 | The following is a summary of the changes between the last commonly |
| 4 | available version of sendmail from Berkeley (5.67) and the latest |
| 5 | version (8.6). I can't guarantee that it is complete. |
| 6 | |
| 7 | Many of these are ideas that had been tried in IDA, albeit many of |
| 8 | them with a more general syntax. |
| 9 | |
| 10 | |
| 11 | Connection Caching |
| 12 | |
| 13 | Instead of closing SMTP connections immediately, open connections are |
| 14 | cached for possible future use. There is a limit to the number of |
| 15 | simultaneous open connections and the idle time of any individual |
| 16 | connection. |
| 17 | |
| 18 | This is of best help during queue processing (since there is the |
| 19 | potential of many different messages going to one site), although |
| 20 | it can also help when processing MX records which aren't handled |
| 21 | by MX Piggybacking. |
| 22 | |
| 23 | MX Piggybacking |
| 24 | |
| 25 | If two hosts with different names in a single message happen to |
| 26 | have the same set of MX hosts, they can be sent in the same |
| 27 | transaction. Version 8 notices this and tries to batch the messages. |
| 28 | |
| 29 | For example, if two sites ``foo.com'' and ``bar.com'' are both |
| 30 | served by UUNET, they will have the same set of MX hosts and will |
| 31 | be sent in one transaction. UUNET will then split the message |
| 32 | and send it to the two individual hosts. |
| 33 | |
| 34 | RFC 1123 Changes |
| 35 | |
| 36 | A number of changes have been made to make sendmail ``conditionally |
| 37 | compliant'' (that is, it satisfies all of the MUST clauses and most |
| 38 | but not all of the SHOULD clauses in RFC 1123). |
| 39 | |
| 40 | The major areas of change are (numbers are RFC 1123 section numbers): |
| 41 | |
| 42 | 5.2.7 Response to RCPT command is fast. Previously, sendmail |
| 43 | expanded all aliases as far as it could -- this could |
| 44 | take a very long time, particularly if there were |
| 45 | name server delays. Version 8 only checks for the |
| 46 | existence of an alias and does the expansion later. |
| 47 | 5.2.8 Numeric IP addresses are logged in Received: lines. |
| 48 | This helps tracing spoofed messages. |
| 49 | 5.2.17 Self domain literal is properly handled. Previously, |
| 50 | if someone sent to user@[a.b.c.d], where a.b.c.d is |
| 51 | your IP address, the mail would probably be rejected. |
| 52 | Version 8 special cases these addresses. |
| 53 | 5.3.2 Better control over individual timeouts. RFC 821 specified |
| 54 | no timeouts. Older versions of sendmail had a single |
| 55 | timeout, typically set to two hours. Version 8 allows |
| 56 | the configuration file to set timeouts for various |
| 57 | SMTP commands individually. |
| 58 | 5.3.3 Error messages are sent as From:<>. This was urged by |
| 59 | RFC 821 and reiterated by RFC 1123, but older versions |
| 60 | of sendmail never really did it properly. Version 8 |
| 61 | does. However, some systems cannot handle this |
| 62 | perfectly legal address; if necessary, you can create |
| 63 | a special mailer that uses the `g' flag to disable this. |
| 64 | 5.3.3 Error messages are never sent to <>. Previously, |
| 65 | sendmail was happy to send responses-to-responses which |
| 66 | sometimes resulted in responses-to-responses-to-responses |
| 67 | which resulted in .... you get the idea. |
| 68 | 5.3.3 Route-addrs (the ugly ``<@hosta,@hostb:user@hostc>'' |
| 69 | syntax) are pruned. RFC 821 urged the use of this |
| 70 | bletcherous syntax. RFC 1123 has seen the light and |
| 71 | officially deprecates them, further urging that you |
| 72 | eliminate all but ``user@hostc'' should you receive |
| 73 | one of these things. Version 8 is slightly more generous |
| 74 | than the standards suggest; instead of stripping off all |
| 75 | the route addressees, it only strips hosts off up to |
| 76 | the one before the last one known to DNS, thus allowing |
| 77 | you to have pseudo-hosts such as foo.BITNET. The 'R' |
| 78 | option will turn this off. |
| 79 | |
| 80 | The areas in which sendmail is not ``unconditionally compliant'' are: |
| 81 | |
| 82 | 5.2.6 Sendmail does do header munging. |
| 83 | 5.2.10 Sendmail doesn't always use the exact SMTP message |
| 84 | text from RFC 821. This is a rather silly requirement. |
| 85 | 5.3.1.1 Sendmail doesn't guarantee only one connect for each |
| 86 | host on queue runs. Connection caching gives you most |
| 87 | of this, but it does not provide a guarantee. |
| 88 | 5.3.1.1 Sendmail doesn't always provide an adequate limit |
| 89 | on concurrency. That is, there can be several |
| 90 | independent sendmails running at once. My feeling |
| 91 | is that doing an absolute limit would be a mistake |
| 92 | (it might result in lost mail). However, if you use |
| 93 | the XLA contributed software, most of this will be |
| 94 | guaranteed (but I don't guarantee the guarantee). |
| 95 | |
| 96 | Extended SMTP Support |
| 97 | |
| 98 | Version 8 includes both sending and receiving support for Extended |
| 99 | SMTP support as defined by RFC 1425 (basic) and RFC 1427 (SIZE); |
| 100 | and limited support for RFC 1426 (BODY). |
| 101 | |
| 102 | Eight-Bit Clean |
| 103 | |
| 104 | Previous versions of sendmail used the 0200 bit for quoting. This |
| 105 | version avoids that use. However, for compatibility with RFC 822, |
| 106 | you can set option `7' to get seven bit stripping. |
| 107 | |
| 108 | Individual mailers can still produce seven bit out put using the |
| 109 | `7' mailer flag. |
| 110 | |
| 111 | User Database |
| 112 | |
| 113 | The user database is an as-yet experimental attempt to provide |
| 114 | unified large-site name support. We are installing it at Berkeley; |
| 115 | future versions may show significant modifications. |
| 116 | |
| 117 | The user database allows you to map both incoming and outgoing |
| 118 | addresses, much like IDA. However, the interface set is still |
| 119 | better with IDA. |
| 120 | |
| 121 | Improved BIND Support |
| 122 | |
| 123 | The BIND support, particularly for MX records, had a number of |
| 124 | annoying ``features'' which have been removed in this release. In |
| 125 | particular, these more tightly bind (pun intended) the name server |
| 126 | to sendmail, so that the name server resolution rules are incorporated |
| 127 | directly into sendmail. |
| 128 | |
| 129 | The major change has been that the $[ ... $] operator didn't fully |
| 130 | qualify names that were in DNS as A or MX records. Version 8 does |
| 131 | this qualification. |
| 132 | |
| 133 | Keyed Files |
| 134 | |
| 135 | Generalized keyed files is an idea taken directly from IDA sendmail |
| 136 | (albeit with a completely different implementation). They can be |
| 137 | useful on large sites. |
| 138 | |
| 139 | Version 8 includes an "NIS" map class to support NIS/YP maps. |
| 140 | |
| 141 | Multi-Word Classes |
| 142 | |
| 143 | Classes can now be multiple words. For example, |
| 144 | |
| 145 | CShofmann.CS.Berkeley.EDU |
| 146 | |
| 147 | allows you to match the entire string ``hofmann.CS.Berkeley.EDU'' |
| 148 | using the single construct ``$=S''. |
| 149 | |
| 150 | Deferred Macro Expansion |
| 151 | |
| 152 | The $&x construct has been adopted from IDA. |
| 153 | |
| 154 | IDENT Protocol Support |
| 155 | |
| 156 | The IDENT protocol as defined in RFC 1413 is supported. |
| 157 | |
| 158 | Parsing Bug Fixes |
| 159 | |
| 160 | A number of small bugs having to do with things like backslash-escaped |
| 161 | quotes inside of comments have been fixed. |
| 162 | |
| 163 | Separate Envelope/Header Processing |
| 164 | |
| 165 | Since the From: line is passed in separately from the envelope |
| 166 | sender, these have both been made visible; the $g macro is set to |
| 167 | the envelope sender during processing of mailer argument vectors |
| 168 | and the header sender during processing of headers. |
| 169 | |
| 170 | It is also possible to specify separate per-mailer envelope and |
| 171 | header processing. The SenderRWSet and RecipientRWset arguments |
| 172 | for mailers can be specified as ``envelope/header'' to give different |
| 173 | rewritings for envelope versus header addresses. |
| 174 | |
| 175 | Owner-List Propagates to Envelope |
| 176 | |
| 177 | When an alias has an associated owner-list name, that alias is used |
| 178 | to change the envelope sender address. This will cause downstream |
| 179 | errors to be returned to that owner. |
| 180 | |
| 181 | Dynamic Header Allocation |
| 182 | |
| 183 | The fixed size limit on header lines has been eliminated. |
| 184 | |
| 185 | New Command Line Flags |
| 186 | |
| 187 | The -B flag has been added to pass in body type information. |
| 188 | |
| 189 | The -p flag has been added to pass in protocol information. |
| 190 | |
| 191 | The -X flag has been added to allow logging of all protocol in and |
| 192 | out of sendmail for debugging. |
| 193 | |
| 194 | Enhanced Command Line Flags |
| 195 | |
| 196 | The -q flag can limit limit a queue run to specific recipients, |
| 197 | senders, or queue ids using -qRsubstring, -qSsubstring, or |
| 198 | -qIsubstring respectively. |
| 199 | |
| 200 | New and Old Configuration Line Types |
| 201 | |
| 202 | The `T' (Trusted users) configuration line has been deleted. It |
| 203 | will still be accepted but will be ignored. |
| 204 | |
| 205 | The `K' line has been added to declare database maps. |
| 206 | |
| 207 | The `V' line has been added to declare the configuration version |
| 208 | level. |
| 209 | |
| 210 | The `M' (mailer) line takes a D= field to specify execution |
| 211 | directory. |
| 212 | |
| 213 | New Options |
| 214 | |
| 215 | Several new options have been added, many to support new features, |
| 216 | others to allow tuning that was previously available only by |
| 217 | recompiling. Briefly: |
| 218 | |
| 219 | b Insist on a minimum number of disk blocks. |
| 220 | |
| 221 | C Delivery checkpoint interval. Checkpoint the queue (to avoid |
| 222 | duplicate deliveries) every C addresses. |
| 223 | |
| 224 | E Default error message. This message (or the contents of the |
| 225 | indicated file) are prepended to error messages. |
| 226 | |
| 227 | G Enable GECOS matching. If you can't find a local user name |
| 228 | and this option is enabled, do a sequential scan of the passwd |
| 229 | file to match against full names. Previously a compile option. |
| 230 | |
| 231 | h Maximum hop count. Previously this was compiled in. |
| 232 | |
| 233 | j Send errors in MIME-encapsulated format. |
| 234 | |
| 235 | J Forward file path. Where to search for .forward files -- defaults |
| 236 | to $HOME/.forward. |
| 237 | |
| 238 | k Connection cache size. The total number of connections that will |
| 239 | be kept open at any time. |
| 240 | |
| 241 | K Connection cache lifetime. The amount of time any connection |
| 242 | will be permitted to sit idle. |
| 243 | |
| 244 | l Enable Errors-To: header. These headers violate RFC 1123; |
| 245 | this option is included to provide back compatibility with |
| 246 | old versions of sendmail. |
| 247 | |
| 248 | O Incoming daemon options (e.g., use alternate SMTP port). |
| 249 | |
| 250 | p Privacy options. These can be used to make your SMTP server |
| 251 | less friendly. |
| 252 | |
| 253 | R Don't prune route-addrs. Normally, if version 8 sees an address |
| 254 | like "<@hostA,@hostB:user@hostC>, sendmail will try to strip off |
| 255 | as much as it can (up to user@hostC) as suggested by RFC 1123. |
| 256 | This option disables that behaviour. |
| 257 | |
| 258 | U User database spec. This is still experimental. |
| 259 | |
| 260 | V Fallback ``MX'' host. This can be thought of as an MX host |
| 261 | that applies to all addresses that has a very high preference |
| 262 | value (that is, use it only if everything else fails). |
| 263 | |
| 264 | w If set, assume that if you are the best MX host for a host, |
| 265 | you should send directly to that host. This is intended |
| 266 | for compatibility with UIUC sendmail, and may have some |
| 267 | use on firewalls. |
| 268 | |
| 269 | 7 Do not run eight bit clean. Technically, you have to assert |
| 270 | this option to be RFC 821 compatible. |
| 271 | |
| 272 | Extended Options |
| 273 | |
| 274 | The `r' (read timeout), `I' (use BIND), and `T' (queue timeout) |
| 275 | options have been extended to pass in more information. |
| 276 | |
| 277 | The `A' (alias file) option has been extended to allow multiple |
| 278 | alias files of different types. |
| 279 | |
| 280 | New Mailer Flags |
| 281 | |
| 282 | a Try to use ESMTP. It will fall back to SMTP if the initial |
| 283 | EHLO packet is rejected. |
| 284 | |
| 285 | b Ensure a blank line at the end of messages. Useful on the |
| 286 | *file* mailer. |
| 287 | |
| 288 | c Strip all comments from addresses; this should only be used as |
| 289 | a last resort when dealing with cranky mailers. |
| 290 | |
| 291 | g Never use the null sender as the envelope sender, even when |
| 292 | running SMTP. This violates RFC 1123. |
| 293 | |
| 294 | 7 Strip all output to this mailer to 7 bits. |
| 295 | |
| 296 | New Pre-Defined Macros |
| 297 | |
| 298 | $k UUCP node name from uname(2). |
| 299 | |
| 300 | $m Domain part of our full hostname. |
| 301 | |
| 302 | $_ RFC 1413-provided sender address. |
| 303 | |
| 304 | New LHS Token |
| 305 | |
| 306 | Version 8 allows `$@' on the Left Hand Side of an `R' line to match |
| 307 | zero tokens. This is intended to be used to match the null input. |
| 308 | |
| 309 | Bigger Defaults |
| 310 | |
| 311 | Version 8 allows up to 100 rulesets instead of 30. It is recommended |
| 312 | that rulesets 0-9 be reserved for sendmail's dedicated use in future |
| 313 | releases. |
| 314 | |
| 315 | The total number of MX records that can be used has been raised to |
| 316 | 20. |
| 317 | |
| 318 | The number of queued messages that can be handled at one time has |
| 319 | been raised from 600 to 1000. |
| 320 | |
| 321 | Different Default Tuning Parameters |
| 322 | |
| 323 | Version 8 has changed the default parameters for tuning queue costs |
| 324 | to make the number of recipients more important than the size of |
| 325 | the message (for small messages). This is reasonable if you are |
| 326 | connected with reasonably fast links. |
| 327 | |
| 328 | Auto-Quoting in Addresses |
| 329 | |
| 330 | Previously, the ``Full Name <email address>'' syntax would generate |
| 331 | incorrect protocol output if ``Full Name'' had special characters |
| 332 | such as dot. This version puts quotes around such names. |
| 333 | |
| 334 | Symbolic Names On Error Mailer |
| 335 | |
| 336 | Several names have been built in to the $@ portion of the $#error |
| 337 | mailer. For example: |
| 338 | |
| 339 | $#error $@NOHOST $: Host unknown |
| 340 | |
| 341 | New Built-In Mailers |
| 342 | |
| 343 | Two new mailers, *file* and *include*, are included to define options |
| 344 | when mailing to a file or a :include: file respectively. Previously |
| 345 | these were overloaded on the local mailer. |
| 346 | |
| 347 | SMTP VRFY Doesn't Expand |
| 348 | |
| 349 | Previous versions of sendmail treated VRFY and EXPN the same. In |
| 350 | this version, VRFY doesn't expand aliases or follow .forward files. |
| 351 | |
| 352 | As an optimization, if you run with your default delivery mode |
| 353 | being queue-only, the RCPT command will also not chase aliases and |
| 354 | .forward files. It will chase them when it processes the queue. |
| 355 | This speeds up RCPT processing. |
| 356 | |
| 357 | [IPC] Mailers Allow Multiple Hosts |
| 358 | |
| 359 | When an address resolves to a mailer that has ``[IPC]'' as its |
| 360 | ``Path'', the $@ part (host name) can be a colon-separated list of |
| 361 | hosts instead of a single hostname. This asks sendmail to search |
| 362 | the list for the first entry that is available exactly as though |
| 363 | it were an MX record. The intent is to route internal traffic |
| 364 | through internal networks without publishing an MX record to the |
| 365 | net. MX expansion is still done on the individual items. |
| 366 | |
| 367 | Aliases Extended |
| 368 | |
| 369 | The implementation has been merged with maps. Among other things, |
| 370 | this supports multiple alias files and NIS-based aliases. For |
| 371 | example: |
| 372 | |
| 373 | OA/etc/aliases,nis:mail.aliases |
| 374 | |
| 375 | Portability and Security Enhancements |
| 376 | |
| 377 | A number of internal changes have been made to enhance portability. |
| 378 | |
| 379 | Several fixes have been made to increase the paranoia factor. |
| 380 | |
| 381 | Miscellaneous Enhancements |
| 382 | |
| 383 | Sendmail writes a /etc/sendmail.pid file with the current process id |
| 384 | and the current invocation flags. |
| 385 | |
| 386 | Two people using the same program (e.g., submit) are considered |
| 387 | "different" so that duplicate elimination doesn't delete one of |
| 388 | them. For example, two people forwarding their email to |
| 389 | |submit will be treated as two recipients. |
| 390 | |
| 391 | The mailstats program prints mailer names and gets the location of |
| 392 | the sendmail.st file from /etc/sendmail.cf. |
| 393 | |
| 394 | Many minor bugs have been fixed, such as handling of backslashes |
| 395 | inside of quotes. |
| 396 | |
| 397 | A hook has been added to allow rewriting of local addresses after |
| 398 | aliasing. |