This chapter is written by Ken Yap and explains how to bootstrap your computer from a program stored in non-volatile memory without accessing your hard disk. It is an ideal technique for maintaining and configure a farm of linux boxes
Network booting is an old idea. The central idea is that the computer has some bootstrap code in non-volatile memory, e.g. a ROM chip, that will allow it to contact a server and obtain system files over a network link. One goal is to avoid the use of a hard disk for booting. There are various reasons for doing this. One is to reduce the cost of maintaining software on many different machines. With network booting the files are held at a central server and can be updated at one location. Another goal is to use computers in locations where hard disks are not robust enough. For example this might be a computer on a factory floor where a hard disk might be too fragile. Finally another goal is to have a system that can be switched between different operating systems without having to reload the software. Network booting often co-exists with disk booting. For example, a system could run Windows from disk but sometimes boot Linux from the network. There are some interesting applications of this technique. For example: a friend of mine uses this to reload Windows over the network. When a Windows installation becomes corrupted, as it often does, the sysadmin can reload a fresh installation by booting Linux over the network and letting the automatic script format the disk and copy a fresh Windows installation onto it.
In order to boot over the network, the computer must get 1. an identity, 2. an operating system image and 3. usually, a working filesystem.
Consider a diskless computer (DC) that has a network boot ROM. It may be one of several identical DCs. How can we distinguish this computer from others? There is one piece of information that is unique to that computer (actually its network adapter) and that is its Ethernet address. Every Ethernet adapter in the world has a unique 48 bit Ethernet address because every Ethernet hardware manufacturer has been assigned blocks of addresses. By convention these addresses are written as hex digits with colons separating each group of two digits, for example: 00:60:08:C7:A3:D8.
The protocols used for obtaining an IP address, given an Ethernet address, are called Boot Protocol (BOOTP) and Dynamic Host Configuration Protocol (DHCP). DHCP is an evolution of BOOTP. In our discussion, unless otherwise stated, anything that applies to BOOTP also applies to DHCP. (Actually it's a small lie that BOOTP and DHCP only translate Ethernet addresses. In their foresight, the designers made provision for BOOTP and DHCP to work with any kind of hardware address. But Ethernet is what most people will be using.)
An example of a BOOTP exchange goes like this:
DC: Hello, my hardware address is 00:60:08:C7:A3:D8, please give me my IP address.
BOOTP server: (Looks up address in database.) Your name is aldebaran, your IP address is 192.168.1.100, your server is 192.168.1.1, the file you are supposed to boot from is /tftpboot/vmlinux.nb (and a few other pieces of information).
You may wonder how the DC found the address of the BOOTP server in the first place. The answer is that it didn't. The BOOTP request was broadcast on the local network and any BOOTP server that can answer the request will.
After obtaining an IP address, the DC must download an operating system image and execute it. Another Internet protocol is used here, called Trivial File Transfer Protocol (TFTP). TFTP is like a cut-down version of FTP---there is no authentication, and it runs over User Datagram Protocol (UDP) instead of Transmission Control Protocol (TCP). UDP was chosen instead of TCP for simplicity. The implementation of UDP on the DC can be small so the code is easy to fit on a ROM. Because UDP is a block oriented, as opposed to a stream oriented, protocol, the transfer goes block by block, like this:
DC: Give me block 1 of /tftpboot/vmlinux.nb.
TFTP server: Here it is.
DC: Give me block 2.
and so on, until the whole file is transferred. Handshaking is a simple acknowledge each block scheme, and packet loss is handled by retransmit on timeout. When all blocks have been received, the network boot ROM hands control to the operating system image at the entry point.
Finally, in order to run an operating system, a root filesystem must be provided. The protocol used by Linux and other Unixes is normally Network File System (NFS), although other choices are possible. In this case the code does not have to reside in the ROM but can be part of the operating system we just downloaded. However the operating system must be capable of running with a root filesystem that is a NFS, instead of a real disk. Linux has the required configuration variables to build a version that can do so.
Besides commercial boot ROMs, there are two sources for free packages for network booting. They are Etherboot and Netboot. Both can be found through the Etherboot home page. First you have to ascertain that your network card is supported by Etherboot or Netboot. Eventually you have to find a person who is willing to put the code on an EPROM (Erasable Programmable Read Only Memory) for you but in the beginning you can do network booting from a floppy.
To create a boot floppy, a special boot block is provided in the distribution. This small 512 byte program loads the disk blocks following it on the floppy into memory and starts execution. Thus to make a boot floppy, one has only to concatenate the boot block with the Etherboot binary containing the driver for one's network card like this:
cat floppyload.bin 3c509.lzrom > /dev/fd0
Before you put in the network boot floppy, you have to set up three services on Linux: BOOTP (or DHCP), TFTP and NFS. You don't have to set up all three at once, you can do them step by step, making sure each step works before going on to the next.
I assume you have installed the bootpd server from a distribution or by compiling from source. You then have to ensure that this server is waiting for bootp requests. There are two ways to do this: one is to start bootpd as a network service that is always listening when the computer is up, and the other is to start it from inetd. For the latter, /etc/inetd.conf should contain a line like this:
bootps dgram udp wait root /usr/sbin/tcpd bootpd
If you had to modify /etc/inetd.conf, then you need to restart inetd by sending the process a HUP signal.
Next, you need to give bootp a database to map Ethernet addresses to IP addresses. This database is in /etc/bootptab. It contains lines of the following form:
aldebaran.foo.com:ha=006008C7A3D8:ip=192.168.1.100:bf=/tftpboot/vmlinuz.nb
Other information can be specified but we will start simple.
Now boot the DC with the floppy and it should detect your Ethernet card and broadcast a BOOTP request. If all goes well, the server should respond to the DC with the information required. Since /tftpboot/vmlinux.nb doesn't exist yet, it will fail when it tries to load the file. Now you need to compile a special kernel, one that has the option for mounting the root filesystem from NFS turned on. You also need to enable the option to get the IP address of the kernel from the original BOOTP reply. You also need to compile the Linux driver for your network adapter into the kernel instead of loading it as a module. It is possible to download an initial ramdisk so that module loading works but this is something you can do later.
You cannot install the zImage resulting from the kernel compilation directly. It has to be turned into a tagged image. A tagged image is a normal kernel image with a special header that tells the network bootloader where the bytes go in memory and at what address to start the program. You use a program called mknbi-linux to create this tagged image. This utility can be found in the Etherboot distribution. After you have generated the image, put it in the /tftpboot directory under the name specified in /etc/bootptab. Make sure to make this file world readable because the tftp server does not have special privileges.
For TFTP, I assume that you installed tftpd from a distribution or by compiling from source. Tftpd is normally started up from inetd with a line like this in /etc/inetd.conf.
tftp dgram udp wait root /usr/sbin/tcpd in.tftpd -s /tftpboot
Again, restart inetd with a HUP signal and you can retry the boot and this time it should download the kernel image and start it. You will find that the boot will continue until the point where it tries to mount a root filesystem. At this point you must configure and export NFS partitions to proceed.
For various reasons, it's not a good idea to use the root filesystem of the server as the root filesystem of the DCs. One is simply that there are various configuration files there and the DC will get the wrong information that way. Another is security. It's dangerous to allow write access (and write access is needed for the root filesystem, for various reasons) to your server's root. However the good news is that a root filesystem for the DC is not very large, only about 30 MB and a lot of this can be shared between multiple DCs.
Ideally, to construct a root filesystem, you have to know what files your operating system distribution is expecting to see there. Critical to booting are device files, files in /sbin and /etc. You can bypass a lot of the hard work by making a copy of an existing root filesystem and modifying some files for the DC. In the Etherboot distribution, there is a tutorial and links to a couple of shell scripts that will create such a DC root filesystem from an existing server root filesystem. There are also troubleshooting tips in the Etherboot documentation as this is often the trickiest part of the setup.
The customised Linux kernel for the DC expects to see the root filesystem at /tftpboot/(IP address of the DC), for example: /tftpboot/192.168.1.100 in the case above. This can be changed when configuring the kernel, if desired.
Now create or edit /etc/exports on the server and put in a line of the following form:
/tftpboot/192.168.1.100 aldebaran.foo.com(rw,no_root_squash)
The rw access is needed for various system services. The no_root_squash attribute prevents the NFS system from mapping root's ID to another one. If this is not specified, then various daemons and loggers will be unhappy.
Start or restart the NFS services (rpc.portmap and rpc.mountd) and retry the diskless boot. If you are successful, the kernel should be able to mount a root filesystem and boot all the way to a login prompt. Most likely, you will find several things misconfigured. Most Linux distributions are oriented towards disked operation and require a little modification to suit diskless booting. The most common failing is reliance on files under /usr during the boot process, which is normally imported from a server late in the boot process. Two possible solutions are: 1. Provide the few required files under a small /usr directory on the root filesystem, which will then be overlaid when /usr is imported, and 2. Modify the paths to look for the files in the root filesystem. The files to edit are under /tftpboot/192.168.1.100 (remember, this is the root directory of the DC).
You may wish to mount other directories from the server, such as /usr (which can be exported read-only).
When you are satisfied that you can boot over the network without any problems, you may wish to put the code on an EPROM. An EPROM programmer starts at around $100 US, and is not a cost effective investment for a hobbyist who uses it sporadically. Occasionally one will appear on the used market at a bargain price, the main caveat being to ensure that the software to drive it is available. A proficient electronics hobbyist could build one using one of the several free designs published on the Internet, but for the majority of readers, the best solution is to make the acquaintance of someone who has access to one, perhaps someone in an electronics hobbyist group or working in the electronics industry.
A short note on EPROM technology: The bits of an EPROM are programmed by injecting electrons with an elevated voltage into the floating gate of a field-effect transistor where a 0 bit is desired. The electrons trapped there cause that transistor to conduct, reading as 0. To erase the EPROM, the trapped electrons are given enough energy to escape the floating gate by bombarding the chip with ultraviolet radiation through the quartz window. To prevent slow erasure over a period of years from sunlight and fluorescent lights, this quartz window is covered with an opaque label in normal use.
There is another technology, called EEPROM or Electrically Erasable PROM, sometimes called Flash PROM. Here the bits are cleared by an electrical signal. This obviates the need for an ultraviolet eraser if the EPROM is to be reused, but requires additional circuitry to support the erase phase. If one is handy with electronics, there is a contributed circuit design and driver software for an EEPROM board in the Etherboot distribution. The board is plugged into any spare ISA bus slot on a PC and boots a network card plugged into some other slot.
X-terminals are one natural use of network booting. The lack of a disk in the terminal makes it quieter and contributes to a pleasant working environment. The machine should ideally have 16MB of memory or more and the best video card you can find for it. This is an ideal use for a high-end 486 or low-end Pentium that has been obsoleted by hardware advances. Other people have used network booting for clusters of machines where the usage is light on the DC and does not warrant a disk, e.g. a cluster of classroom machines.
Your first stop should be the Etherboot home page: http://www.slug.org.au/etherboot/
There you will find links to other resources, including a mailing list you can subscribe to, where problems and solutions are discussed.