| 1 | FreeBSD and NFS [for a FAQ] |
| 2 | |
| 3 | Certain Ethernet adapters for ISA PC systems have limitations which |
| 4 | can lead to serious network problems, particularly with NFS. This |
| 5 | difficulty is not specific to FreeBSD, but FreeBSD systems are affected |
| 6 | by it. |
| 7 | |
| 8 | The problem nearly always occurs when (FreeBSD) PC systems are networked |
| 9 | with high-performance workstations, such as those made by Silicon Graphics, |
| 10 | Inc., and Sun Microsystems, Inc. The NFS mount will work fine, and some |
| 11 | operations may succeed, but suddenly the server will seem to become |
| 12 | unresponsive to the client, even though requests to and from other systems |
| 13 | continue to be processed. This happens to the client system, whether the |
| 14 | client is the FreeBSD system or the workstation. On many systems, there is |
| 15 | no way to shut down the client gracefully once this problem has manifested |
| 16 | itself. The only solution is often to reset the client, because the NFS |
| 17 | situation cannot be resolved. |
| 18 | |
| 19 | Though the "correct" solution is to get a higher performance and capacity |
| 20 | Ethernet adapter for the FreeBSD system, there is a simple workaround that |
| 21 | will allow satisfactory operation. If the FreeBSD system is the SERVER, |
| 22 | include the option "wsize=1024" on the mount from the client. If the |
| 23 | FreeBSD system is the CLIENT, then mount the NFS file system with the |
| 24 | option "rsize=1024". These options may be specified using the fourth |
| 25 | field of the fstab entry on the client for automatic mounts, or by using |
| 26 | the "-o" parameter of the mount command for manual mounts. |
| 27 | |
| 28 | In the following examples, "fastws" is the host (interface) name of a |
| 29 | high-performance workstation, and "freebox" is the host (interface) name of |
| 30 | a FreeBSD system with a lower-performance Ethernet adapter. Also, |
| 31 | "/sharedfs" will be the exported NFS filesystem (see "man exports"), and |
| 32 | "/project" will be the mount point on the client for the exported file |
| 33 | system. In all cases, note that additional options, such as "hard" or |
| 34 | "soft" and "bg" may be desireable in your application. |
| 35 | |
| 36 | Examples for the FreeBSD system ("freebox") as the client: |
| 37 | in /etc/fstab on freebox: |
| 38 | fastws:/sharedfs /project nfs rw,rsize=1024 0 0 |
| 39 | as a manual mount command on freebox: |
| 40 | mount -t nfs -o rsize=1024 fastws:/sharedfs /project |
| 41 | |
| 42 | Examples for the FreeBSD system as the server: |
| 43 | in /etc/fstab on fastws: |
| 44 | freebox:/sharedfs /project nfs rw,wsize=1024 0 0 |
| 45 | as a manual mount command on fastws: |
| 46 | mount -t nfs -o wsize=1024 freebox:/sharedfs /project |
| 47 | |
| 48 | Nearly any 16-bit Ethernet adapter will allow operation without the above |
| 49 | restrictions on the read or write size. |
| 50 | |
| 51 | For anyone who cares, here is what happens when the failure occurs, which |
| 52 | also explains why it is unrecoverable. NFS typically works with a "block" |
| 53 | size of 8k (though it may do fragments of smaller sizes). Since the maximum |
| 54 | Ethernet packet is around 1500 bytes, the NFS "block" gets split into |
| 55 | multiple Ethernet packets, even though it is still a single unit to the |
| 56 | upper-level code, and must be received, assembled, and ACKNOWLEDGED as a |
| 57 | unit. The high-performance workstations can pump out the packets which |
| 58 | comprise the NFS unit one right after the other, just as close together as |
| 59 | the standard allows. On the smaller, lower capacity cards, the later |
| 60 | packets overrun the earlier packets of the same unit before they can be |
| 61 | transferred to the host and the unit as a whole cannot be reconstructed or |
| 62 | acknowledged. As a result, the workstation will time out and try again, |
| 63 | but it will try again with the entire 8K unit, and the process will be |
| 64 | repeated, ad infinitum. |
| 65 | |
| 66 | By keeping the unit size below the Ethernet packet size limitation, we |
| 67 | ensure that any complete Ethernet packet received can be acknowledged |
| 68 | individually, avoiding the deadlock situation. |
| 69 | |
| 70 | Overruns may still occur when a high-performance workstations is slamming |
| 71 | data out to a PC system, but with the better cards, such overruns are |
| 72 | not guarranteed on NFS "units". When an overrun occurs, the units affected |
| 73 | will be retransmitted, and there will be a fair chance that they will be |
| 74 | received, assembled, and acknowledged. |
| 75 | -- |
| 76 | John Lind, Starfire Consulting Services |
| 77 | E-mail: john@starfire.MN.ORG USnail: PO Box 17247, Mpls MN 55417 |