I found to make all this work I had to piece together a bunch of information from different locations. This fills in some of the blanks from the official Raspberry Pi documentation. See here, here, and here.
Image
Download the latest raspbian image from https://www.raspberrypi.org/downloads/raspbian/ and unzip it. I used the lite version as I'll install only what I need later.
To extract the files from the image we need to jump through some hoops. Inside the image are two partitions, we need data from each one.
# Make it easier to re-use these instructions by using a variable
IMG=2017-04-10-raspbian-jessie-lite.img
fdisk -l $IMG
You should see some output like:
Disk 2017-04-10-raspbian-jessie-lite.img: 1.2 GiB, 1297862656 bytes, 2534888 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x84fa8189
Device Boot Start End Sectors Size Id Type
2017-04-10-raspbian-jessie-lite.img1 8192 92159 83968 41M c W95 FAT32 (LBA)
2017-04-10-raspbian-jessie-lite.img2 92160 2534887 2442728 1.2G 83 Linux
You need to be able to mount both the boot and the root partitions. Do this by tracking the offset of each one and multiplying it by the sector size, which is given on the line saying "Sector size" (typically 512 bytes), for example with the 2017-04-01 image, boot has an offset of 8192, so I mount it like this (it is VFAT):
mount -v -o offset=$((8192 * 512)) -t vfat $IMG /mnt
# I then copy the data off:
mkdir -p /data/diskless/raspbian-lite-base-boot/
rsync -xa /mnt/ /data/diskless/raspbian-lite-base-boot/
# unmount the partition now:
umount /mnt
Then we do the same for the root partition:
mount -v -o offset=$((92160 * 512)) -t ext4 $IMG /mnt
# copy the data off:
mkdir -p /data/diskless/raspbian-lite-base-root/
rsync -xa /mnt/ /data/diskless/raspbian-lite-base-root/
# umount the partition now:
umount /mnt
DHCP
When I first set this up, I used OpenWRT on my router, and I had to patch /etc/init/dnsmasq to support setting DHCP option 43. As of the writting of this article, a similar patch has been merged, but isn't in a release yet, and, well, there may never be another release of OpenWRT. I'm now running LEDE, and the the good news is it already has the patch merged (hurrah!). If you're still on OpenWRT, then here's the patch you'll need:
https://git.lede-project.org/?p=source.git;a=commit;h=9412fc294995ae2543fabf84d2ce39a80bfb3bd6
This lets you put the following in /etc/config/dnsmasq, this says that any device that uses DHCP and has a MAC issued by the Raspberry PI Foundation, should have option 66 (boot server) and option 43 set as specified. Set the IP address on option 66 to the device that should be used for tftp on your network, if it's the same device that provides DHCP then it isn't required. I had to set the boot server, as my other network boot devices are using a different server (with an older tftpd-hpa, I explain the problem further down).
config mac 'rasperrypi'
option mac 'b8:27:eb:*:*:*'
option networkid 'rasperrypi'
list dhcp_option '66,10.1.0.253'
list dhcp_option '43,Raspberry Pi Boot'
tftp
Initially I used a version of tftpd that was too old and didn't support how the RPi tried to discover if it should use the serial number based naming scheme. The version of tftpd-hpa Debian Jessie works just fine. To find out the serial number you'll probably need to increase the logging of tftpd-hpa, do so by editing /etc/default/tftpd-hpa and adding "-v" to the TFTP_OPTIONS option. It can also be useful to watch tcpdump to see the requests and responses, for example (10.1.0.203 is the IP of the RPi I'm working with):
tcpdump -n -i eth0 host 10.1.0.203 and dst port 69
This was able to tell me the serial number of my RPi, so I made a directory in my tftpboot directory with the same serial number and copied all the boot files into there. I then found that I had to remove the init= portion from the cmdline.txt file I'm using. To ease debugging I also removed quiet. So, my current cmdline.txt contains (newlines entered for clarity, but the file has it all on one line):
idwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/nfs
nfsroot=10.1.0.253:/data/diskless/raspbian-lite-base-root,vers=3,rsize=1462,wsize=1462
ip=dhcp elevator=deadline rootwait hostname=rpi.etc.gen.nz
NFS root
You'll need to export the directories you created via NFS. My exports file has these lines:
/data/diskless/raspbian-lite-base-root 10.1.0.0/24(rw,no_root_squash,sync,no_subtree_check)
/data/diskless/raspbian-lite-base-boot 10.1.0.0/24(rw,no_root_squash,sync,no_subtree_check)
And you'll also want to make sure you're mounting those correctly during boot, so I have in /data/diskless/raspbian-lite-base-root/etc/fstab the following lines:
10.1.0.253:/data/diskless/raspbian-lite-base-root / nfs rw,vers=3 0 0
10.1.0.253:/data/diskless/raspbian-lite-base-boot /boot nfs vers=3,nolock 0 2
Network Booting
Now you can hopefully boot. Unless you into this bug, as I did. Where the RPi will sometimes fail to boot. Turns out the fix, which is mentioned on the bug report, is to put bootcode.bin (and only bootcode.bin) onto an SD card. That'll then load the fixed bootcode, and which will then boot reliably.
