Difference between revisions of "TFTP Boot and NFS Root Filesystems"

From eLinux.org
Jump to: navigation, search
m (Update to 'bootargs'; tidy bootscript names)
 
(17 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
[[Category: Parallella]]
 
[[Category: Parallella]]
<br />
 
 
= How To Set Up TFTP Boot And NFS Root Filesystems On Parallella =
 
= How To Set Up TFTP Boot And NFS Root Filesystems On Parallella =
 
This Guide describes how to set up the Parallella U-Boot to network boot using TFTP. <br />
 
This Guide describes how to set up the Parallella U-Boot to network boot using TFTP. <br />
Also, it describes how to set up mounting a root filesystem (''rootfs'') via NFS. <br />
+
Also, it describes how to set up mounting a root filesystem (''rootfs'') via NFS, so that a Parallella is run as a 'Thick Client'. <br />
See the Manual for [http://www.denx.de/wiki/DULG/Manual Das U-Boot]<br />
+
For background to the TFTP Boot in U-Boot, see the Manual for [http://www.denx.de/wiki/DULG/Manual Das U-Boot]<br />
The following is based on the original blog post by [https://plus.google.com/+shogunx/posts/K5taMhPKurp Scott Johnson] describing how to set up a Parallella cluster to use TFTP/NFS, but has been tweaked - mainly to minimise the need to edit/update the U-Boot variables via the Serial/UART cable, and provide more flexibility by using PC-based editors and storage.<br />
+
The following is based on the original blog post by [https://plus.google.com/+shogunx/posts/K5taMhPKurp Scott Johnson] describing how to set up a Parallella cluster to use TFTP/NFS, but has been tweaked - mainly to minimise the need to edit/update the U-Boot variables via the Serial/UART cable, and provide more flexibility by using PC-based editors and storage.<br /><br />
 
By editing a ''bootscript'' on the TFTP server (and re-booting), a Parallella can be re-configured to a different boot mode (headless,HDMI) and/or different root filesystem (Ubuntu, Debian, nano) - all without having to 'burn' another Parallella SD card (perhaps ever..?).<br />
 
By editing a ''bootscript'' on the TFTP server (and re-booting), a Parallella can be re-configured to a different boot mode (headless,HDMI) and/or different root filesystem (Ubuntu, Debian, nano) - all without having to 'burn' another Parallella SD card (perhaps ever..?).<br />
 +
'''Alternative root filesystems''': A Parallella may have alternative Linux root filesystems, holding different configurations or versions. The alternative root filesystems (e.g. Ubuntu, Linaro Nano, with/without SDK) are just like 'burning' and booting from multiple SD cards. <br />
 +
'''Shared root filesystems''': A root filesystem directory on the NFS server may also be shared by a number of Parallellas, say, in a cluster. The many advantages include saving space on the server, and mainly(!) not having to replicate SD cards for each cluster member.<br />
 +
Cloning and modifying the NFS directories for use by other Parallellas is also easier and quicker: just copy directories on the NFS Server (and perhaps edit a couple of files).
 +
<br />
 
Tweak the example below to suit your  needs.
 
Tweak the example below to suit your  needs.
 
== Requirements ==
 
== Requirements ==
Line 12: Line 15:
 
== Summary ==
 
== Summary ==
 
This Guide describes:
 
This Guide describes:
* An IP addressing and naming scheme to support different boot modes (headless, HDMI) and NFS root filesystems.
+
* An IP addressing and naming scheme using DHCP to support different boot modes (headless, HDMI) and NFS root filesystems.
 
* Setting up TFTP and a directory layout for TFTP ''bootscripts'' and boot mode image files.
 
* Setting up TFTP and a directory layout for TFTP ''bootscripts'' and boot mode image files.
 
* Changing the way the boot arguments are passed to the kernel, i.e. in the U-Boot 'bootargs' rather than devicetree.
 
* Changing the way the boot arguments are passed to the kernel, i.e. in the U-Boot 'bootargs' rather than devicetree.
* A directory layout for NFS root filesystems, allowing a choice of root filesystems for each Parallella.
+
* Setting up NFS and a directory layout for NFS root filesystems, allowing a choice of root filesystems for each Parallella.
* Changing the U-Boot boot command to fetch a small script file using TFTP.
+
* Changing the U-Boot boot command to fetch a small ''bootscript'' file using TFTP.
 
* An example ''bootscript'' for TFTP boot and NFS root filesystem.
 
* An example ''bootscript'' for TFTP boot and NFS root filesystem.
 
* Use of mkimage to convert the ''bootscript'' text file into a U-Boot script image file.
 
* Use of mkimage to convert the ''bootscript'' text file into a U-Boot script image file.
* A breakdown of the ''bootscript'' to show how it might be altered to suit different environments.
+
* A breakdown of the ''bootscript'' to show different options.
 
IMPORTANT NOTE:  
 
IMPORTANT NOTE:  
 
* {Sept 2014} There is a problem in Parallella U-Boot w.r.t. the Ethernet interface after any network activity in U-Boot (e.g. TFTP). (See the post on the Parallella forum post by [http://forums.parallella.org/viewtopic.php?f=48&t=1667#p10409 tajama]). The ethernet interface is left in an unuseable state.  A workaround is included in the script example below; but before attempting to apply it - you MUST check the U-Boot version. (Revisit)
 
* {Sept 2014} There is a problem in Parallella U-Boot w.r.t. the Ethernet interface after any network activity in U-Boot (e.g. TFTP). (See the post on the Parallella forum post by [http://forums.parallella.org/viewtopic.php?f=48&t=1667#p10409 tajama]). The ethernet interface is left in an unuseable state.  A workaround is included in the script example below; but before attempting to apply it - you MUST check the U-Boot version. (Revisit)
 
* {Sept 2014} the Parallella U-Boot is being updated which just might affect the script below. (Revisit).  
 
* {Sept 2014} the Parallella U-Boot is being updated which just might affect the script below. (Revisit).  
= Prepare File Service Environments =
+
<br />
== Set up IP Addresses ==
+
= Set up IP Addresses Using DHCP =
 
Allocate IP addresses for the Parallella(s) and the TFTP/NFS servers in your DHCP server based on MAC address. The Parallella MAC address is printed on the board, and is also a U-Boot variable (see later).<br />
 
Allocate IP addresses for the Parallella(s) and the TFTP/NFS servers in your DHCP server based on MAC address. The Parallella MAC address is printed on the board, and is also a U-Boot variable (see later).<br />
 
Refer to the documentation of your DHCP server, or discuss with your network admin. <br />
 
Refer to the documentation of your DHCP server, or discuss with your network admin. <br />
Line 30: Line 33:
 
* U-Boot tftpboot command requests a ''bootscript'' file, and needs the TFTP server IP address <serverip>, and its own IP address <ipaddr>.
 
* U-Boot tftpboot command requests a ''bootscript'' file, and needs the TFTP server IP address <serverip>, and its own IP address <ipaddr>.
 
* U-Boot tells Linux to use DHCP.  
 
* U-Boot tells Linux to use DHCP.  
* NFS uses IP addresses to control access to the rootfs directories.
+
* The NFS server uses IP addresses to control access to the exported rootfs directories.
 
This is an example IP address allocation and naming convention (your network may vary):
 
This is an example IP address allocation and naming convention (your network may vary):
 
<blockquote>
 
<blockquote>
 
<pre>
 
<pre>
192.168.1.201 para1 // <ipaddr> IP address of Parallella "para"<n>
+
192.168.1.21 para1 // <ipaddr> IP address of Parallella "para"<n>
192.168.1.202 para2
+
192.168.1.22 para2
 
...
 
...
192.168.1.208  para8    // ... I wish
+
192.168.1.28 para8    // ... I wish :)
 
192.168.1.50 TFTP server // IP address of TFTP server "serverip"
 
192.168.1.50 TFTP server // IP address of TFTP server "serverip"
 
192.168.1.50 NFS server // IP address of NFS server "nfs_ip"
 
192.168.1.50 NFS server // IP address of NFS server "nfs_ip"
 
</pre>
 
</pre>
 
</blockquote>
 
</blockquote>
Note: The TFTP and NFS services do not need to be on the same physical server, or even the same operating system. Here, a Debian (Wheezy) PC provides both services.<br />
+
Note: The TFTP and NFS services do not need to be on the same physical server, or even the same operating system. A Debian (Wheezy) PC provides both services in this example.
 +
 
 +
= TFTP Server =
 
== Prepare the TFTP Service ==
 
== Prepare the TFTP Service ==
 
Get the required software if not already set up. On Debian:
 
Get the required software if not already set up. On Debian:
Line 49: Line 54:
 
</pre>
 
</pre>
 
The TFTP service does not start automatically. <br />
 
The TFTP service does not start automatically. <br />
 +
(Note: There are a number of examples in various forums, etc, of how to automatically start the TFTP service - but not all are successful on all Linux distro's it seems! The following may work for you.)<br />
 
Start the tftpd-hpa service automatically by adding a command to ''/etc/rc.local'' (just before the exit 0 line).
 
Start the tftpd-hpa service automatically by adding a command to ''/etc/rc.local'' (just before the exit 0 line).
 
<pre>
 
<pre>
Line 54: Line 60:
 
</pre>
 
</pre>
 
(Edit and save)
 
(Edit and save)
 +
<blockquote>
 
<pre>
 
<pre>
 
service tftpd-hpa start
 
service tftpd-hpa start
 
</pre>
 
</pre>
To control the TFTP service use:
+
</blockquote>
 +
To control the TFTP service from the command line use:
 +
<pre>
 +
sudo service tftpd-hpa restart // to restart, or use 'stop', or 'start' as needed.
 +
</pre>
 +
To check the status of the TFTP service from the command line use:
 
<pre>
 
<pre>
sudo service tftpd-hpa restart // or 'stop', or 'start' as needed.
+
sudo service tftpd-hpa status
 +
[ ok ] in.tftpd is running.        // Service is Up
 +
[FAIL] in.tftpd is not running ... failed!    // Service is Down
 
</pre>
 
</pre>
=== Set up the TFTP Directories ===
+
 
By default, the tftpd-hpa service looks for requested files under ''/srv/tftp/''. Each Parallella has its own ''bootscript'', e.g. ''para1''. This is a U-Boot script image file, created using ''mkimage'' from a text file (see below). The ''bootscript'' points U-Boot to a boot mode directory. The ''bootscript'' also provides U-Boot with the Linux boot parameters to mount the NFS root filesystem, and finally to boot the kernel. <br />
+
== Set up the TFTP Directories ==
Each boot mode (headless, HDMI, etc) has a separate directory containing the equivalent of the BOOT partition on a SD card, containing the kernel, parallella.bit.bin and devicetree image files. <br />
+
In this scenario, the tftpd-hpa service looks for requested files under ''/srv/tftp/''. <br />
Different sets of image files can be held in different directories, e.g. ''hdmi_7010_new_kernel/'', ''test_usb_fix/''. <br />
+
The default tftpd-hpa directory may vary with distribution/release, but it is specified in the configuration file: ''/etc/default/tfptd-hpa'' <br />
Note: that is one directory per boot mode - not one per Parallella.<br />
+
Edit the line (using root priviledge): <br />
Create a boot mode directory:
+
<blockquote>
 +
<pre>
 +
TFTP_DIRECTORY="/srv/tftp"
 +
</pre>
 +
</blockquote>
 +
Restart the TFTP service if required. <br />
 +
Files in the tftpd-hpa service directory should be owned by ''nobody:nogroup''.
 +
 
 +
Each Parallella has its own ''bootscript'', e.g. ''para1'', ''para2'',  in the TFTP service directory. The ''bootscript'' is a U-Boot script image file, created using ''mkimage'' from a text file (see below). <br />
 +
The ''bootscript'' points U-Boot to a boot mode sub-directory holding the boot image files.<br />
 +
Each boot mode directory is the equivalent of the ''BOOT'' partition on a Parallella SD card, containing the kernel, parallella.bit.bin and devicetree image files.<br />
 +
Different sets of boot image files may be held in different directories, e.g. ''headless_7010'', ''hdmi_7010'', ''test_new_kernel''. By pointing at a different directory, the ''bootscript'' controls how the Parallella will be booted. <br />
 +
Note: only needs one directory per boot mode - not one per Parallella.<br />
 +
The ''bootscript'' provides U-Boot with the TFTP commands to load the image files and to boot the kernel.<br />
 +
The ''bootscript'' also provides the kernel command line boot parameters (''bootargs'') to mount the NFS root filesystem. These override the arguments in the devicetree (see ''chosen { } '').<br />
 +
 
 +
== Create a Boot Mode Directory and Prepare the Image Files ==
 
<pre>
 
<pre>
 
sudo mkdir -p /srv/tftp/<name> // replace <name> with boot mode, e.g. hdmi_7010
 
sudo mkdir -p /srv/tftp/<name> // replace <name> with boot mode, e.g. hdmi_7010
 
</pre>
 
</pre>
== Prepare the Image Files ==
+
Follow the instructions for creating the SD card (download, extract and rename the kernel uImage, parallella.bit.bin and devicetree.dtb files) but instead of 'burning' to a SD card simply copy the image files to the boot mode directory (as root).<br />
Follow the instructions for creating the SD card (that is, download, extract and rename the kernel uImage, parallella.bit.bin and devicetree.dtb files), but instead of 'burning' to a SD card simply copy the image files to the boot mode directory (as root).<br />
 
 
The TFTP service directory layout (for this example) will look something like:
 
The TFTP service directory layout (for this example) will look something like:
 
<blockquote>
 
<blockquote>
Line 84: Line 113:
 
devicetree.dtb
 
devicetree.dtb
 
uImage
 
uImage
hdmi_7010_new_kernel/ // boot mode = temp testing variation
+
test_new_kernel/ // boot mode = temp testing variation
 
paralella.bit.bin
 
paralella.bit.bin
 
devicetree.dtb
 
devicetree.dtb
Line 95: Line 124:
 
</blockquote>
 
</blockquote>
 
Note: The image files follow the same naming convention as for SD card booting.
 
Note: The image files follow the same naming convention as for SD card booting.
== Remove Bootargs from Devicetree ==
+
 
The devicetree as supplied by Adapteva passes the boot arguments to support booting from the SD card. Instead, the boot arguments are passed to the kernel by U-Boot, as defined in the ''bootscript''. <br />
+
= NFS Server =
Install the devicetree compiler if not already installed. On Debian:
+
The Official (and Unofficial) sources of Linux root filesystems for the Parallella provide instructions for extracting and 'burning' the contents to SD card. The instructions are then repeated for each physical SD card required for each Parallella in a cluster. Similar steps need to be taken to create the root filesystem contents of the NFS mounted directories - but the process is (usually) much quicker and more flexible. <br />
<pre>
+
Some minor edits to the configuration files are usually needed to ensure that the NFS variant works smoothly in a network environment. The NFS directories are available for viewing (and editing) on the NFS Server host.<br />
sudo apt-get install device-tree-compiler
+
Once fully set up and running, the root filesystem will need to be managed and maintained as normal, (e.g. ''sudo apt-get update''). This needs to be done within a session on (''one of'') the Parallella using:
</pre>
+
* the USB serial console which provides a root session, or
Extract/edit/rebuild the devicetree.dtb file to remove the parameters from the ''chosen {}'' branch.
+
* a remote SSH session after the ssh-server has been set up on the Parallella, or
{Check.. do we need the linux console line in chosen ..? Revisit}
+
* a keyboard, mouse, screen if the Parallella has been booted with HDMI support.
<pre>
+
Note: If the root filesystem is shared then this maintenance only needs to be carried on '''one''' Parallella (and not for each SD card).<br />
cd /srv/tftp/hdmi_7010
+
'''IMPORTANT WARNING:'''<br />
sudo dtc -I dtb -o dev.dts -O dts devicetree.dtb
+
When editing configuration files in the NFS root filesystem directories on the NFS Server host using root permissions:
sudo nano dev.dts
+
* Avoid the mistake of editing your host PC's configuration!
</pre>
+
*''Make Sure You Are In The Correct NFS Sub-directory!''
Look for ''chosen {'' branch, remove the arguments, and save.
+
<br />
<blockquote>
 
<pre>
 
chosen {
 
}
 
</pre>
 
</blockquote>
 
Recompile the devicetree.dtb and delete the dev.dts text file.
 
<pre>
 
sudo dtc -I dts -O dtb -o devicetree.dtb dev.dts
 
sudo rm dev.dts
 
</pre>
 
Repeat for all supplied devicetree.dtb files in each boot mode directory.
 
 
== Prepare the NFS Service ==
 
== Prepare the NFS Service ==
 
Get the required software if not already set up. On Debian:
 
Get the required software if not already set up. On Debian:
 
<pre>
 
<pre>
apt-get install nfs-kernel-server
+
sudo apt-get install nfs-kernel-server
 
</pre>
 
</pre>
The NFS service starts automatically.<br />
+
The NFS service starts automatically. To control NFS services use:
To control NFS services use:
 
 
<pre>
 
<pre>
sudo service nfs-kernel-server restart // or 'stop', or 'start' as needed.
+
sudo service nfs-kernel-server restart // to restart, or use 'stop', or 'start' as needed.
 
</pre>
 
</pre>
=== Set up the NFS Directories ===
+
To check the status of the NFS service from the command line use:
Each Parallella has at least one NFS directory containing a root filesystem. Each Parallella may have alternative root filesystems (e.g.  Ubuntu, Linaro Nano) - just like preparing and swapping alternative SD cards. <br />
 
In this example, the directories are organised under ''/srv/nfs'', which need to be created. <br />
 
 
<pre>
 
<pre>
sudo mkdir -p /srv/nfs
+
sudo service nfs-kernel-server status
sudo mkdir -p /srv/nfs/para1_ubuntu
+
nfsd running          // Service is Up
sudo mkdir -p /srv.nfs/para1_nano
+
nfsd not running    // Service is Down
 
</pre>
 
</pre>
Here is an example layout of NFS filesystem directories showing separate branches for each Parallella.
+
 
<blockquote>
+
== Set up the NFS Directories ==
 +
Create a top directory ''/srv/nfs'', and create a subdirectory under that for each NFS mounted root filesystem required. <br />
 +
Here are some examples of NFS filesystem directories, including shared root filesystems, and where a Parallella may have a choice of root filesystems.<br />
 +
In this example, ''para1'' is the only Parallella with HDMI/USB connected; the other cluster members are headless 7010's running, say, nano. For work using the SDK version, ''para1'' is booted with HDMI support and mounts the ''ubuntu_SDK'' root filesystem, otherwise, ''para1'' is booted (after changing the ''bootscript'') with the headless images and ''nano''.<br />
 
<pre>
 
<pre>
/srv/nfs/
+
sudo mkdir -p /srv/nfs
para1_ubuntu/ // rootfs for para1, based on, say, Parallella Ubuntu
+
sudo mkdir -p /srv/nfs/ubuntu // rootfs based on standard Parallella Ubuntu build, to be shared by all cluster members (including para1)
bin/
+
sudo mkdir -p /srv/nfs/nano // rootfs based on a slim linaro nano build, to be shared by all cluster members (including para1)
boot/
+
sudo mkdir -p /srv/nfs/ubuntu_SDK // rootfs based on Parallella Ubuntu build, with latest SDK (to be used by para1 only).
                etc/
+
sudo mkdir -p /srv/nfs/nano_X_server // rootfs based on a linaro nano build with SDK, X server (to be used by para1 only).
...
 
para1_deb/ // rootfs for para1, with 9600's Debian and ESDK
 
bin/
 
boot/
 
                etc/
 
...
 
para1_nano/ // rootfs for para1 based on linaro_nano
 
bin/
 
boot/
 
                etc/
 
...
 
para2_ubuntu/ // rootfs for para2, based on, say, Parallella Ubuntu
 
bin/
 
boot/
 
                etc/
 
...
 
        ...
 
para8_ubuntu/       // rootfs for para8, based on, say, Parallella Ubuntu
 
...       // etc, etc
 
 
</pre>
 
</pre>
</blockquote>
+
 
=== Update NFS Exports File ===
+
== Set up Contents of the Parallella NFS Root Filesystems ==
The NFS server requires ''/etc/exports'' to be configured correctly, to control access to each NFS filesystem directory to specific hosts.<br />
+
Instead of 'burning' a Linux root filesystem image to a partition on the SD card, place the payload root filesystem in the NFS directory, e.g. ''/srv/nfs/ubuntu/''. <br />
For each Paralella root filesystem that is created, add the export control line to ''/etc/exports'', adjusting (as you should for all examples herein) for your local IP address and directory naming scheme. The following is an example based on the layout described above.
+
Each source of root filesystem may involve slightly different preparation to enable the NFS root filesystem to work smoothly.<br />
 +
See the example in the [[Parallella NFS rootfs Setup|How-to set up Linaro Nano Root Filesystem for NFS Boot]]<br />
 +
 
 +
== Update NFS Exports File ==
 +
The NFS server requires ''/etc/exports'' to be configured correctly, to control access to each NFS filesystem directory to specific hosts. In this case the hosts are identified by their IP address.<br />
 +
For each Parallella root filesystem that is created, add the export control line to ''/etc/exports'', adjusting (as you should for all examples herein) for your local IP address and directory naming scheme. The following is an example based on the layout described above. Note:<br />
 +
* Where multiple hosts share an exported directory, the "hostIP(attributes)" are separated by <space>.
 +
* There is no <space> between "hostIP" and "(attributes)".
 +
* Continuation lines may be used for clarity. Continuation is "<space>\" only (i.e. no hidden tabs, etc).
 
<pre>
 
<pre>
 
sudo nano /etc/exports
 
sudo nano /etc/exports
 
</pre>
 
</pre>
 
(Edit and save)
 
(Edit and save)
 +
<blockquote>
 
<pre>
 
<pre>
/srv/nfs/para1_ubuntu 192.168.1.201(rw,sync,no_root_squash,no_subtree_check)
+
/srv/nfs/ubuntu 192.168.1.21(rw,sync,no_root_squash,no_subtree_check) \
/srv/nfs/para1_deb 192.168.1.201(rw,sync,no_root_squash,no_subtree_check)
+
192.168.1.22(rw,sync,no_root_squash,no_subtree_check) \
/srv/nfs/para1_nano 192.168.1.201(rw,sync,no_root_squash,no_subtree_check)
+
192.168.1.23(rw,sync,no_root_squash,no_subtree_check) \
/srv/nfs/para2_ubuntu 192.168.1.202(rw,sync,no_root_squash,no_subtree_check)
+
192.168.1.24(rw,sync,no_root_squash,no_subtree_check)
/srv/nfs/para8_ubuntu 192.168.1.208(rw,sync,no_root_squash,no_subtree_check)
+
/srv/nfs/ubuntu_SDK 192.168.1.21(rw,sync,no_root_squash,no_subtree_check)
 +
/srv/nfs/nano 192.168.1.21(rw,sync,no_root_squash,no_subtree_check) \
 +
192.168.1.22(rw,sync,no_root_squash,no_subtree_check) \
 +
192.168.1.23(rw,sync,no_root_squash,no_subtree_check) \
 +
192.168.1.24(rw,sync,no_root_squash,no_subtree_check)
 +
/srv/nfs/nano_X_server 192.168.1.22(rw,sync,no_root_squash,no_subtree_check)
 
</pre>
 
</pre>
Note: The ''bootscript'' ensures that only one of the optional filesystems will be accessed/mounted by a Parallella on a re-boot.<br />
+
</blockquote>
=== Export the NFS Directories ===
+
Note: The ''bootscript'' ensures that only one of the optional filesystems will be accessed/mounted at any one time by a Parallella on a re-boot.
 +
 
 +
== Export the NFS Directories ==
 
Ensure that these directories are made visible by the NFS service. Either:
 
Ensure that these directories are made visible by the NFS service. Either:
 
<pre>
 
<pre>
Line 192: Line 206:
 
sudo service nfs-kernel-server restart
 
sudo service nfs-kernel-server restart
 
</pre>
 
</pre>
== Set up the Pallella NFS Root Filesystems ==
 
Instead of 'burning' a filesystem image to a partition on the SD card, place the payload root filesystem in the NFS directory, e.g. ''/srv/nfs/para1/''. <br />
 
Each source of root filesystem may need different preparation to enable the NFS access to work smoothly. Some examples below.<br />
 
=== Default Parallella Ubuntu Build ===
 
{under construction - as of Sept 2014: NFS mounts OK, but doesn't complete boot, two penguins ..??}
 
=== 9600's Debian Build ===
 
{under construction - as of Sept 2014: can't extract tar file ..??}
 
=== Linaro Nano Build ===
 
See [[Parallella Linaro Nano tftpboot NFS rootfs|how-to for network booting a Linaro Nano NFS rootfs]]
 
  
= Set Up the Parallella to TFTP Boot =
+
<br /><br />
Repeat the following for each Parallella, changing the values of ''hostname'' and ''ipaddr'' as necessary.<br />
+
= Parallella TFTP Boot =
 +
The default U-Boot boot sequence looks for the boot image files on the SD card. The boot command is modified to load a ''bootscript'' using the built-in TFTP boot command.  
 +
== Set Up the Parallella to TFTP Boot ==
 
Power off the Parallella and remove the ethernet, USB peripheral and HDMI cables and SD card. <br />
 
Power off the Parallella and remove the ethernet, USB peripheral and HDMI cables and SD card. <br />
Connect a USB serial cable to the 3-pin UART header on the Parallella. (Example device http://www.adafruit.com/products/954 from Adafruit). <br />
+
Connect a USB serial cable to the 3-pin UART header on the Parallella. An [http://www.adafruit.com/products/954 example device] from Adafruit, and see [http://forums.parallella.org/viewtopic.php?f=50&t=1039#p6554 connecting a UART serial cable to the Parallella]<br />
Use a terminal program (e.g. screen, minicom) to connect to the UART Serial device.  On minicom, set hardware flow control to off, and point the logfile to, say, ~/minicomlog for diagnostics.  <br />
+
On the PC, use a terminal program (e.g. ''screen'', ''minicom'') to connect to the UART Serial device.  On ''minicom'', set hardware flow control to off, and point the logfile to, say, ''~/minicomlog' for diagnostics.  <br />
 
Power up the Parallella.  <br />
 
Power up the Parallella.  <br />
After the boot failure messages, you should get the ''zynq-uboot>'' prompt in the terminal window.  
+
After the boot failure messages, you should get the ''zynq-uboot>'' prompt in the terminal window. <br />
 
Display a list of the current state of the U-Boot environment variables using the ''printenv'' command. <br />
 
Display a list of the current state of the U-Boot environment variables using the ''printenv'' command. <br />
 
Check the value of the ''ethaddr=xx:xx:xx:xx:xx:xx'' variable; this should be the same as the label on the board, and the MAC address you are using for this Parallella in your DHCP server.<br />
 
Check the value of the ''ethaddr=xx:xx:xx:xx:xx:xx'' variable; this should be the same as the label on the board, and the MAC address you are using for this Parallella in your DHCP server.<br />
 
Enter the following commands at the ''zynq-uboot>'' prompt in order to:
 
Enter the following commands at the ''zynq-uboot>'' prompt in order to:
* set the ''hostname'' variable which will be used as the ''bootscript'' name.
+
* set the ''hostname'' variable which will be used as the ''bootscript'' name. Change for each Parallella.
* set the IP address of the Parallella and the TFTP server.
+
* set the IP address of the Parallella and the TFTP server. Change for each Parallella.
* create a variable ''tboot'' which contains the commands to fetch and execute the ''bootscript''.  
+
* set the Ip address of the TFTP server. Usually the same for all Parallellas.
* change the U-Boot bootcmd so that it will run the ''tboot'' TFTP boot sequence.  
+
* create a variable ''t-boot'' which contains the commands to fetch and then execute the ''bootscript''.  
 +
* change the U-Boot bootcmd so that it will run the ''t-boot'' TFTP boot sequence, only if no SD card is present.  
 
<pre>
 
<pre>
 
setenv hostname para1
 
setenv hostname para1
setenv ipaddr 192.168.1.201
+
setenv ipaddr 192.168.1.21
 
setenv serverip 192.168.1.50
 
setenv serverip 192.168.1.50
setenv taddr 0x100000
+
setenv t-addr 0x100000
setenv tboot 'tftpboot ${taddr} ${hostname} ; source ${taddr}'
+
setenv t-boot 'tftpboot ${t-addr} ${hostname} ; source ${t-addr}'
setenv bootcmd 'run tboot'
+
setenv bootcmd 'run t-boot'
</pre>
 
Alternatively, if preferred as a fall-back, replace the U-Boot bootcmd in the above sequence so that the boot sequence tries to boot from the SD card first. If no SD card is present, then run the TFTP boot sequence.
 
<pre>
 
setenv bootcmd 'run modeboot ; run tboot'
 
</pre>
 
Check that the variables have been set correctly and, if OK, save to flash memory.
 
<pre>
 
printenv
 
saveenv
 
 
</pre>
 
</pre>
 +
Check that the variables have been set correctly, using ''printenv'' command, and save to flash memory, using ''saveenv'' command.
 
Power off. <br />
 
Power off. <br />
Reconnect the ethernet cable. <br />
+
Reconnect the ethernet cable and any USB peripherals or HDMI you intend to use on the Parallella. <br />
Reconnect any USB peripherals or HDMI you intend to use on the Parallella. <br />
 
 
Ready to TFTP boot. <br />
 
Ready to TFTP boot. <br />
 
Notes:  
 
Notes:  
Line 242: Line 241:
 
* To remove a variable e.g. 'badvarible',  use the ''setenv'' command with no value, e.g. ''setenv badvarible''
 
* To remove a variable e.g. 'badvarible',  use the ''setenv'' command with no value, e.g. ''setenv badvarible''
 
* U-Boot autocompletes commands so ''print'', ''set'', ''save'' will also work.
 
* U-Boot autocompletes commands so ''print'', ''set'', ''save'' will also work.
= Write a Parallella Bootscript =
+
<br />
On the PC host, the ''bootscript'' image file is created and edited as a text file, and compiled into a U-Boot executable image using mkimage.<br />
+
 
 +
= Parallella Bootscript =
 +
== Writing a Parallella Bootscript ==
 +
On the TFTP server host, the ''bootscript'' image file is initially created and edited as a text file, and compiled into a U-Boot executable image using ''mkimage''.<br />
 +
<br />
 
Create a working directory to hold the ''bootscript'' text file(s).
 
Create a working directory to hold the ''bootscript'' text file(s).
 
<pre>
 
<pre>
 
mkdir ~/Documents/bootscr
 
mkdir ~/Documents/bootscr
 
</pre>
 
</pre>
Create a text file and copy/paste the text below as a template. The following is an example ''bootscript''. This loads the headless image files.<br />
+
Create a text file and copy/paste the text below as a template. The following is an example ''bootscript'' which is described below. This loads the headless image files for a 7010, and will load the shared linaro nano root filesystem.<br />
 
<pre>
 
<pre>
 
cd ~/Documents/bootscr
 
cd ~/Documents/bootscr
Line 254: Line 257:
 
</pre>
 
</pre>
 
(Edit and save)
 
(Edit and save)
<pre>
+
<blockquote><pre>
 
setenv tftp_dir headless_7010
 
setenv tftp_dir headless_7010
setenv nfs_rfs /srv/tftp/para1_ubuntu
+
setenv nfs_rfs /srv/tftp/nano
 +
 
 
setenv bootargs ip=dhcp
 
setenv bootargs ip=dhcp
 
setenv bootargs ${bootargs} console=ttyPS0,115200
 
setenv bootargs ${bootargs} console=ttyPS0,115200
 +
 
setenv nfs_ip 192.168.1.50
 
setenv nfs_ip 192.168.1.50
 
setenv bootargs ${bootargs} root=/dev/nfs rw nfsroot=${nfs_ip}:${nfs_rfs}
 
setenv bootargs ${bootargs} root=/dev/nfs rw nfsroot=${nfs_ip}:${nfs_rfs}
setenv bootargs ${bootargs} nfsrootdebug earlyprintk
+
 
setenv fpga_image parallella.bit.bin
+
setenv bitstream_image parallella.bit.bin
setenv fpga_addr 0x4000000
+
setenv bitstream_addr 0x4000000
setenv fpga_size 0x3dbafc
+
setenv bitstream_tftp 'tftpboot ${bitstream_addr} ${tftp_dir}/${bitstream_image}'
setenv fpga_tftp 'tftpboot ${fpga_addr} ${tftp_dir}/${fpga_image}'
+
setenv bitstream_load 'fpga load 0 ${bitstream_addr} ${filesize}'
setenv fpga_load 'fpga load 0 ${fpga_addr} ${fpga_size}'
+
 
setenv fdt_image devicetree.dtb
+
setenv devicetree_image devicetree.dtb
setenv fdt_addr 0x2A00000
+
setenv devicetree_addr 0x2A00000
setenv fdt_tftp 'tftpboot ${fdt_addr} ${tftp_dir}/${fdt_image}'
+
setenv devicetree_tftp 'tftpboot ${devicetree_addr} ${tftp_dir}/${devicetree_image}'
 +
 
 
setenv kernel_image uImage
 
setenv kernel_image uImage
 
setenv kernel_addr 0x3000000
 
setenv kernel_addr 0x3000000
 
setenv kernel_tftp 'tftpboot ${kernel_addr} ${tftp_dir}/${kernel_image}'
 
setenv kernel_tftp 'tftpboot ${kernel_addr} ${tftp_dir}/${kernel_image}'
 +
 
setenv phy_rst 'mw.w f8000008 df0d ; mw.w f8000140 00100801 ; mw.w f8000004 767b'
 
setenv phy_rst 'mw.w f8000008 df0d ; mw.w f8000140 00100801 ; mw.w f8000004 767b'
 
setenv rset_phy 'run phy_rst'
 
setenv rset_phy 'run phy_rst'
setenv boot_now 'bootm ${kernel_addr} - ${fdt_addr}'
+
 
run fpga_tftp fpga_load kernel_tftp fdt_tftp rset_phy boot_now
+
setenv boot_now 'bootm ${kernel_addr} - ${devicetree_addr}'
</pre>
+
 
Copy the file to create a ''bootscript'' for each Parallella, changing the filename. <br />
+
run bitstream_tftp bitstream_load kernel_tftp devicetree_tftp rset_phy boot_now
Edit the values of the ''tftp_dir'' and ''nfs_rfs'' variables in each file as required.<br />
+
</pre></blockquote>
 +
Copy the file to create a ''bootscript'' for each Parallella, changing the filename, e.g. para2.txt.
 +
Edit the values of the ''tftp_dir'' and ''nfs_rfs'' variables in each file as required. If the boot mode is the same and the NFS rootfs is shared, then there are usually no edits required. <br />
 +
To change the mode of a Parallella boot, edit the 'paraX.txt' file to change the ''tftp_dir'' and/or ''nfs_rfs'' variables, before re-making the image.<br />
 
The directory structure on the Debian PC will then look something like this:
 
The directory structure on the Debian PC will then look something like this:
<blockquote>
+
<blockquote><pre>
<pre>
 
 
/home/{username}/Documents/bootscr/ 
 
/home/{username}/Documents/bootscr/ 
para1.txt
+
      para1.txt
para2.txt
+
      para2.txt
...
+
      ...
para8.txt
+
      para8.txt
 
</pre>
 
</pre>
 
</blockquote>
 
</blockquote>
== Make the Bootscript Image ==
+
 
Install the mkimage package if not already installed. On Debian:
+
=== Make the Bootscript Image ===
 +
Install ''mkimage'', which is part of the Das U-Boot tool-set, if not already installed. On Debian:
 
<pre>
 
<pre>
 
sudo apt-get install u-boot-tools
 
sudo apt-get install u-boot-tools
 
</pre>
 
</pre>
Convert each ''bootscript'' text file into a loadable image by mkimage, and place the image in the TFTP server directory.
+
Convert each ''bootscript'' text file into a loadable image by ''mkimage'', and place each image in the TFTP server directory. For example:
 
<pre>
 
<pre>
sudo mkimage -A arm -O u-boot -T script -C none -a 0 -e 0 -n "t-Boot Script" -d ~/Documents/bootscr/para1 /srv/tftp/para1
+
cd ~/Documents/bootscr
 +
sudo mkimage -A arm -O u-boot -T script -C none -a 0 -e 0 -n "t-Boot Script" -d para1.txt /srv/tftp/para1
 +
sudo mkimage -A arm -O u-boot -T script -C none -a 0 -e 0 -n "t-Boot Script" -d para2.txt /srv/tftp/para2
 +
...
 +
sudo mkimage -A arm -O u-boot -T script -C none -a 0 -e 0 -n "t-Boot Script" -d para8.txt /srv/tftp/para8
 
</pre>
 
</pre>
  
== Reboot the Parallella ==
+
=== Reboot the Parallella ===
 
Reboot by power off, wait a few seconds, power on. <br />
 
Reboot by power off, wait a few seconds, power on. <br />
The output to the UART Serial terminal screen shows initialisation of the board and network, and then the boot sequence looking for the TFTP server.  
+
The output to the UART Serial terminal screen shows initialisation of the board and network, and then the boot sequence looking for the TFTP server.<br />
 
If the TFTP 'Loading:' line puts out a few 'T T T's, then it's retrying. Give it a few moments but if there's a line of 'T's then check that the TFTP service is actually running, and that the IP addresses and filename of the ''bootscript'' are valid. <br />
 
If the TFTP 'Loading:' line puts out a few 'T T T's, then it's retrying. Give it a few moments but if there's a line of 'T's then check that the TFTP service is actually running, and that the IP addresses and filename of the ''bootscript'' are valid. <br />
 
Once the small (~1kb) ''bootscript'' has been loaded and is executed by U-Boot, further TFTP requests will be seen as the image files are downloaded before U-Boot hands over to the kernel.
 
Once the small (~1kb) ''bootscript'' has been loaded and is executed by U-Boot, further TFTP requests will be seen as the image files are downloaded before U-Boot hands over to the kernel.
Line 308: Line 322:
 
== Breakdown of the Bootscript ==
 
== Breakdown of the Bootscript ==
 
This section takes the ''bootscript'' example and breaks it down to illustrate what may be changed to support your local environment.<br />
 
This section takes the ''bootscript'' example and breaks it down to illustrate what may be changed to support your local environment.<br />
Set the ''bootscript'' to point to the required boot mode (i.e. TFTP sub-directory) and the NFS filesystem directory chosen for this Paralella. By editing these first two lines, a Parallella can be re-booted using a different FPGA, kernel and/or a different NFS root filesystem.  
+
 
<pre>
+
* Set up the ''bootscript'' to point to the required boot mode (i.e. TFTP sub-directory) and the NFS filesystem directory chosen for this Paralella. By editing these first two lines, a Parallella can be re-booted using a different bitstream, kernel, devicetree images, and/or a different NFS root filesystem. Note that the the NFS filepath must be the same as the relevant line in NFS exports.
setenv tftp_dir headless_7010   // or, hdmi_7010, etc
+
<blockquote><pre>
setenv nfs_rootfs /srv/nfs/para1_ubuntu // or, para1_deb, etc
+
setenv tftp_dir headless_7010
</pre>
+
setenv nfs_rootfs /srv/nfs/nano
Set the initial ''bootargs'' string to tell the kernel to use DHCP when loaded.
+
</pre></blockquote>
<pre>
+
* Set the initial ''bootargs'' string to tell the kernel to use DHCP when loaded. The U-Boot ''bootargs'' override any that are set up in the devicetree image.
 +
<blockquote><pre>
 
setenv bootargs ip=dhcp
 
setenv bootargs ip=dhcp
</pre>
+
</pre></blockquote>
Add the console argument to the ''bootargs'' string. If the console is not required, delete this line.
+
* If required, add the console argument to the ''bootargs'' string. If the console is '''not''' required, delete this line.
<pre>
+
<blockquote><pre>
 
setenv bootargs ${bootargs} console=ttyPS0,115200
 
setenv bootargs ${bootargs} console=ttyPS0,115200
</pre>
+
</pre></blockquote>
Add the NFS boot parameters to the ''bootargs'' string. The NFS server IP variable is not the same as the TFTP ''serverip'' variable, although they may have the same value.  
+
* Add the NFS boot arguments to the ''bootargs'' string. Note the NFS server IP variable is not the same as the TFTP ''serverip'' variable, although they may have the same value.
<pre>
+
<blockquote><pre>
 
setenv nfs_ip 192.168.1.50
 
setenv nfs_ip 192.168.1.50
 
setenv bootargs ${bootargs} root=/dev/nfs rw nfsroot=${nfs_ip}:${nfs_rfs}
 
setenv bootargs ${bootargs} root=/dev/nfs rw nfsroot=${nfs_ip}:${nfs_rfs}
</pre>
+
</pre></blockquote>
Add additional kernel boot diagnostic flags to the ''bootargs'' string. If not required, delete this line.
+
* If required, add additional kernel boot diagnostic flags to the ''bootargs'' string. If '''not''' required, delete this line.
<pre>  
+
<blockquote><pre>
 
setenv bootargs ${bootargs} nfsrootdebug earlyprintk
 
setenv bootargs ${bootargs} nfsrootdebug earlyprintk
</pre>
+
</pre></blockquote>
Set up the variables to download the fpga image, and to load the image into the fpga.  
+
* Set up the variables for the command to tftpboot the bitstream image, and the command to load the image into the fpga. The size of the bitstream image is the size of the file set by tftpboot.
<pre>
+
<blockquote><pre>
setenv fpga_image parallella.bit.bin
+
setenv bitstream_image parallella.bit.bin
setenv fpga_addr 0x4000000
+
setenv bitstream_addr 0x4000000
setenv fpga_size 0x3dbafc
+
setenv bitstream_tftp 'tftpboot ${bitstream_addr} ${tftp_dir}/${bitstream_image}'
setenv fpga_tftp 'tftpboot ${fpga_addr} ${tftp_dir}/${fpga_image}'
+
setenv bitstream_load 'fpga load 0 ${bitstream_addr} ${filesize}'
setenv fpga_load 'fpga load 0 ${fpga_addr} ${fpga_size}'
+
</pre></blockquote>
</pre>
+
* Set up the variables for the command to tftpboot the devicetree image.
Set up the variables to download the devicetree image.
+
<blockquote><pre>
<pre>
+
setenv devicetree_image devicetree.dtb
setenv fdt_image devicetree.dtb
+
setenv devicetree_addr 0x2A00000
setenv fdt_addr 0x2A00000
+
setenv devicetree_tftp 'tftpboot ${devicetree_addr} ${tftp_dir}/${devicetree_image}'
setenv fdt_tftp 'tftpboot ${fdt_addr} ${tftp_dir}/${fdt_image}'
+
</pre></blockquote>
</pre>
+
* Set up the variables for the command to tftpboot the kernel image.
Set up the variables to dowlnload the kernel image.
+
<blockquote><pre>
<pre>
 
 
setenv kernel_image uImage
 
setenv kernel_image uImage
 
setenv kernel_addr 0x3000000
 
setenv kernel_addr 0x3000000
 
setenv kernel_tftp 'tftpboot ${kernel_addr} ${tftp_dir}/${kernel_image}'
 
setenv kernel_tftp 'tftpboot ${kernel_addr} ${tftp_dir}/${kernel_image}'
</pre>
+
</pre></blockquote>
Patch U-Boot network handling.<br />
+
* Patch U-Boot network handling. This is a workround for the problem identified by ''tajama'', and referred to above. It MAY apply to your version but YOU MUST CHECK!! This U-Boot patch applies to the following version, which is output initially by U-Boot:
'''Note:''' This is a workround for the problem identified by ''tajama'', and referred to above. <br />
+
<blockquote><pre>
This is a patch to U-Boot space and applies to the following version, which is output initially by U-Boot:
 
<blockquote>
 
<pre>
 
 
U-Boot 2012.10-00003-g792c31c (Jan 03 2014 - 12:24:08)
 
U-Boot 2012.10-00003-g792c31c (Jan 03 2014 - 12:24:08)
</pre>
+
</pre></blockquote>
</blockquote>
+
<blockquote><pre>
It MAY apply to your version but YOU MUST CHECK!! (Revisit)
 
<pre>
 
 
setenv phy_rst 'mw.w f8000008 df0d ; mw.w f8000140 00100801 ; mw.w f8000004 767b'
 
setenv phy_rst 'mw.w f8000008 df0d ; mw.w f8000140 00100801 ; mw.w f8000004 767b'
 
setenv rset_phy 'run phy_rst'
 
setenv rset_phy 'run phy_rst'
</pre>
+
</pre></blockquote>
Set up the command to boot into the kernel, passing the kernel and devicetree image addresses.
+
* Set up the variable for the command to boot into the kernel, passing the kernel and devicetree image addresses.
<pre>
+
<blockquote><pre>
setenv boot_now 'bootm ${kernel_addr} - ${fdt_addr}'
+
setenv boot_now 'bootm ${kernel_addr} - ${devicetree_addr}'
</pre>
+
</pre></blockquote>
Finally, run the sequence of tftp requests, and boot. <br />
+
* Finally, run the sequence of individual commands and boot. If any of the command sequence fails, then the 'run' command will halt. Note there are no ';' separators in the command parameters.
If any of the command sequence fails, then the 'run' command will halt (Note there are no ';' separators).
+
<blockquote><pre>
<pre>
+
run bitstream_tftp bitstream_load kernel_tftp devicetree_tftp rset_phy boot_now
run fpga_tftp fpga_load kernel_tftp fdt_tftp rset_phy boot_now
+
</pre></blockquote>
</pre>
+
<br />
 
 
 
{{Template:Parallella Navbox}}
 
{{Template:Parallella Navbox}}

Latest revision as of 04:17, 15 March 2015

How To Set Up TFTP Boot And NFS Root Filesystems On Parallella

This Guide describes how to set up the Parallella U-Boot to network boot using TFTP.
Also, it describes how to set up mounting a root filesystem (rootfs) via NFS, so that a Parallella is run as a 'Thick Client'.
For background to the TFTP Boot in U-Boot, see the Manual for Das U-Boot
The following is based on the original blog post by Scott Johnson describing how to set up a Parallella cluster to use TFTP/NFS, but has been tweaked - mainly to minimise the need to edit/update the U-Boot variables via the Serial/UART cable, and provide more flexibility by using PC-based editors and storage.

By editing a bootscript on the TFTP server (and re-booting), a Parallella can be re-configured to a different boot mode (headless,HDMI) and/or different root filesystem (Ubuntu, Debian, nano) - all without having to 'burn' another Parallella SD card (perhaps ever..?).
Alternative root filesystems: A Parallella may have alternative Linux root filesystems, holding different configurations or versions. The alternative root filesystems (e.g. Ubuntu, Linaro Nano, with/without SDK) are just like 'burning' and booting from multiple SD cards.
Shared root filesystems: A root filesystem directory on the NFS server may also be shared by a number of Parallellas, say, in a cluster. The many advantages include saving space on the server, and mainly(!) not having to replicate SD cards for each cluster member.
Cloning and modifying the NFS directories for use by other Parallellas is also easier and quicker: just copy directories on the NFS Server (and perhaps edit a couple of files).
Tweak the example below to suit your needs.

Requirements

Linux PC, Parallella board(s), USB to Serial (TTL/UART) adapter, DHCP server.

Summary

This Guide describes:

  • An IP addressing and naming scheme using DHCP to support different boot modes (headless, HDMI) and NFS root filesystems.
  • Setting up TFTP and a directory layout for TFTP bootscripts and boot mode image files.
  • Changing the way the boot arguments are passed to the kernel, i.e. in the U-Boot 'bootargs' rather than devicetree.
  • Setting up NFS and a directory layout for NFS root filesystems, allowing a choice of root filesystems for each Parallella.
  • Changing the U-Boot boot command to fetch a small bootscript file using TFTP.
  • An example bootscript for TFTP boot and NFS root filesystem.
  • Use of mkimage to convert the bootscript text file into a U-Boot script image file.
  • A breakdown of the bootscript to show different options.

IMPORTANT NOTE:

  • {Sept 2014} There is a problem in Parallella U-Boot w.r.t. the Ethernet interface after any network activity in U-Boot (e.g. TFTP). (See the post on the Parallella forum post by tajama). The ethernet interface is left in an unuseable state. A workaround is included in the script example below; but before attempting to apply it - you MUST check the U-Boot version. (Revisit)
  • {Sept 2014} the Parallella U-Boot is being updated which just might affect the script below. (Revisit).


Set up IP Addresses Using DHCP

Allocate IP addresses for the Parallella(s) and the TFTP/NFS servers in your DHCP server based on MAC address. The Parallella MAC address is printed on the board, and is also a U-Boot variable (see later).
Refer to the documentation of your DHCP server, or discuss with your network admin.
The IP addresses are used as follows:

  • U-Boot tftpboot command requests a bootscript file, and needs the TFTP server IP address <serverip>, and its own IP address <ipaddr>.
  • U-Boot tells Linux to use DHCP.
  • The NFS server uses IP addresses to control access to the exported rootfs directories.

This is an example IP address allocation and naming convention (your network may vary):

192.168.1.21	para1		// <ipaddr> IP address of Parallella "para"<n>
192.168.1.22	para2
...
192.168.1.28	para8     	// ... I wish :)
192.168.1.50	TFTP server	// IP address of TFTP server "serverip"
192.168.1.50	NFS server	// IP address of NFS server "nfs_ip"

Note: The TFTP and NFS services do not need to be on the same physical server, or even the same operating system. A Debian (Wheezy) PC provides both services in this example.

TFTP Server

Prepare the TFTP Service

Get the required software if not already set up. On Debian:

apt-get install tftpd-hpa

The TFTP service does not start automatically.
(Note: There are a number of examples in various forums, etc, of how to automatically start the TFTP service - but not all are successful on all Linux distro's it seems! The following may work for you.)
Start the tftpd-hpa service automatically by adding a command to /etc/rc.local (just before the exit 0 line).

sudo nano /etc/rc.local

(Edit and save)

service tftpd-hpa start

To control the TFTP service from the command line use:

sudo service tftpd-hpa restart	// to restart, or use 'stop', or 'start' as needed.

To check the status of the TFTP service from the command line use:

sudo service tftpd-hpa status
[ ok ] in.tftpd is running.     	     	// Service is Up
[FAIL] in.tftpd is not running ... failed!     	// Service is Down

Set up the TFTP Directories

In this scenario, the tftpd-hpa service looks for requested files under /srv/tftp/.
The default tftpd-hpa directory may vary with distribution/release, but it is specified in the configuration file: /etc/default/tfptd-hpa
Edit the line (using root priviledge):

TFTP_DIRECTORY="/srv/tftp" 

Restart the TFTP service if required.
Files in the tftpd-hpa service directory should be owned by nobody:nogroup.

Each Parallella has its own bootscript, e.g. para1, para2, in the TFTP service directory. The bootscript is a U-Boot script image file, created using mkimage from a text file (see below).
The bootscript points U-Boot to a boot mode sub-directory holding the boot image files.
Each boot mode directory is the equivalent of the BOOT partition on a Parallella SD card, containing the kernel, parallella.bit.bin and devicetree image files.
Different sets of boot image files may be held in different directories, e.g. headless_7010, hdmi_7010, test_new_kernel. By pointing at a different directory, the bootscript controls how the Parallella will be booted.
Note: only needs one directory per boot mode - not one per Parallella.
The bootscript provides U-Boot with the TFTP commands to load the image files and to boot the kernel.
The bootscript also provides the kernel command line boot parameters (bootargs) to mount the NFS root filesystem. These override the arguments in the devicetree (see chosen { } ).

Create a Boot Mode Directory and Prepare the Image Files

sudo mkdir -p /srv/tftp/<name>		// replace <name> with boot mode, e.g. hdmi_7010

Follow the instructions for creating the SD card (download, extract and rename the kernel uImage, parallella.bit.bin and devicetree.dtb files) but instead of 'burning' to a SD card simply copy the image files to the boot mode directory (as root).
The TFTP service directory layout (for this example) will look something like:

/svr/tftp/				// TFTP service directory
	hdmi_7010/			// boot mode = Zynq 7010 with HDMI support
		paralella.bit.bin
		devicetree.dtb
		uImage
	headless_7010/			// boot mode = Zynq 7010 Parallella configured as headless
		paralella.bit.bin
		devicetree.dtb
		uImage
	test_new_kernel/		// boot mode = temp testing variation
		paralella.bit.bin
		devicetree.dtb
		uImage
	para1			        // bootscript for para1
	para2			        // bootscript for para2
	...
	para8			        // bootscript for para8, etc

Note: The image files follow the same naming convention as for SD card booting.

NFS Server

The Official (and Unofficial) sources of Linux root filesystems for the Parallella provide instructions for extracting and 'burning' the contents to SD card. The instructions are then repeated for each physical SD card required for each Parallella in a cluster. Similar steps need to be taken to create the root filesystem contents of the NFS mounted directories - but the process is (usually) much quicker and more flexible.
Some minor edits to the configuration files are usually needed to ensure that the NFS variant works smoothly in a network environment. The NFS directories are available for viewing (and editing) on the NFS Server host.
Once fully set up and running, the root filesystem will need to be managed and maintained as normal, (e.g. sudo apt-get update). This needs to be done within a session on (one of) the Parallella using:

  • the USB serial console which provides a root session, or
  • a remote SSH session after the ssh-server has been set up on the Parallella, or
  • a keyboard, mouse, screen if the Parallella has been booted with HDMI support.

Note: If the root filesystem is shared then this maintenance only needs to be carried on one Parallella (and not for each SD card).
IMPORTANT WARNING:
When editing configuration files in the NFS root filesystem directories on the NFS Server host using root permissions:

  • Avoid the mistake of editing your host PC's configuration!
  • Make Sure You Are In The Correct NFS Sub-directory!


Prepare the NFS Service

Get the required software if not already set up. On Debian:

sudo apt-get install nfs-kernel-server

The NFS service starts automatically. To control NFS services use:

sudo service nfs-kernel-server restart		// to restart, or use 'stop', or 'start' as needed.

To check the status of the NFS service from the command line use:

sudo service nfs-kernel-server status
nfsd running          	// Service is Up
nfsd not running     	// Service is Down

Set up the NFS Directories

Create a top directory /srv/nfs, and create a subdirectory under that for each NFS mounted root filesystem required.
Here are some examples of NFS filesystem directories, including shared root filesystems, and where a Parallella may have a choice of root filesystems.
In this example, para1 is the only Parallella with HDMI/USB connected; the other cluster members are headless 7010's running, say, nano. For work using the SDK version, para1 is booted with HDMI support and mounts the ubuntu_SDK root filesystem, otherwise, para1 is booted (after changing the bootscript) with the headless images and nano.

sudo mkdir -p /srv/nfs
sudo mkdir -p /srv/nfs/ubuntu		// rootfs based on standard Parallella Ubuntu build, to be shared by all cluster members (including para1)
sudo mkdir -p /srv/nfs/nano		// rootfs based on a slim linaro nano build, to be shared by all cluster members (including para1)
sudo mkdir -p /srv/nfs/ubuntu_SDK	// rootfs based on Parallella Ubuntu build, with latest SDK (to be used by para1 only).
sudo mkdir -p /srv/nfs/nano_X_server	// rootfs based on a linaro nano build with SDK, X server (to be used by para1 only).

Set up Contents of the Parallella NFS Root Filesystems

Instead of 'burning' a Linux root filesystem image to a partition on the SD card, place the payload root filesystem in the NFS directory, e.g. /srv/nfs/ubuntu/.
Each source of root filesystem may involve slightly different preparation to enable the NFS root filesystem to work smoothly.
See the example in the How-to set up Linaro Nano Root Filesystem for NFS Boot

Update NFS Exports File

The NFS server requires /etc/exports to be configured correctly, to control access to each NFS filesystem directory to specific hosts. In this case the hosts are identified by their IP address.
For each Parallella root filesystem that is created, add the export control line to /etc/exports, adjusting (as you should for all examples herein) for your local IP address and directory naming scheme. The following is an example based on the layout described above. Note:

  • Where multiple hosts share an exported directory, the "hostIP(attributes)" are separated by <space>.
  • There is no <space> between "hostIP" and "(attributes)".
  • Continuation lines may be used for clarity. Continuation is "<space>\" only (i.e. no hidden tabs, etc).
sudo nano /etc/exports

(Edit and save)

/srv/nfs/ubuntu 192.168.1.21(rw,sync,no_root_squash,no_subtree_check) \
192.168.1.22(rw,sync,no_root_squash,no_subtree_check) \
192.168.1.23(rw,sync,no_root_squash,no_subtree_check) \
192.168.1.24(rw,sync,no_root_squash,no_subtree_check)
/srv/nfs/ubuntu_SDK 192.168.1.21(rw,sync,no_root_squash,no_subtree_check)
/srv/nfs/nano 192.168.1.21(rw,sync,no_root_squash,no_subtree_check) \
192.168.1.22(rw,sync,no_root_squash,no_subtree_check) \
192.168.1.23(rw,sync,no_root_squash,no_subtree_check) \
192.168.1.24(rw,sync,no_root_squash,no_subtree_check)
/srv/nfs/nano_X_server 192.168.1.22(rw,sync,no_root_squash,no_subtree_check)

Note: The bootscript ensures that only one of the optional filesystems will be accessed/mounted at any one time by a Parallella on a re-boot.

Export the NFS Directories

Ensure that these directories are made visible by the NFS service. Either:

sudo exportfs -a

or

sudo service nfs-kernel-server restart



Parallella TFTP Boot

The default U-Boot boot sequence looks for the boot image files on the SD card. The boot command is modified to load a bootscript using the built-in TFTP boot command.

Set Up the Parallella to TFTP Boot

Power off the Parallella and remove the ethernet, USB peripheral and HDMI cables and SD card.
Connect a USB serial cable to the 3-pin UART header on the Parallella. An example device from Adafruit, and see connecting a UART serial cable to the Parallella
On the PC, use a terminal program (e.g. screen, minicom) to connect to the UART Serial device. On minicom, set hardware flow control to off, and point the logfile to, say, ~/minicomlog' for diagnostics.
Power up the Parallella.
After the boot failure messages, you should get the zynq-uboot> prompt in the terminal window.
Display a list of the current state of the U-Boot environment variables using the printenv command.
Check the value of the ethaddr=xx:xx:xx:xx:xx:xx variable; this should be the same as the label on the board, and the MAC address you are using for this Parallella in your DHCP server.
Enter the following commands at the zynq-uboot> prompt in order to:

  • set the hostname variable which will be used as the bootscript name. Change for each Parallella.
  • set the IP address of the Parallella and the TFTP server. Change for each Parallella.
  • set the Ip address of the TFTP server. Usually the same for all Parallellas.
  • create a variable t-boot which contains the commands to fetch and then execute the bootscript.
  • change the U-Boot bootcmd so that it will run the t-boot TFTP boot sequence, only if no SD card is present.
setenv hostname para1
setenv ipaddr 192.168.1.21
setenv serverip 192.168.1.50
setenv t-addr 0x100000
setenv t-boot 'tftpboot ${t-addr} ${hostname} ; source ${t-addr}'
setenv bootcmd 'run t-boot'

Check that the variables have been set correctly, using printenv command, and save to flash memory, using saveenv command. Power off.
Reconnect the ethernet cable and any USB peripherals or HDMI you intend to use on the Parallella.
Ready to TFTP boot.
Notes:

  • It's worthwhile checking (again) after a power recycle that the U-Boot variables have been set correctly
  • To correct/change a variable, use the setenv command with new/correct value.
  • To remove a variable e.g. 'badvarible', use the setenv command with no value, e.g. setenv badvarible
  • U-Boot autocompletes commands so print, set, save will also work.


Parallella Bootscript

Writing a Parallella Bootscript

On the TFTP server host, the bootscript image file is initially created and edited as a text file, and compiled into a U-Boot executable image using mkimage.

Create a working directory to hold the bootscript text file(s).

mkdir ~/Documents/bootscr

Create a text file and copy/paste the text below as a template. The following is an example bootscript which is described below. This loads the headless image files for a 7010, and will load the shared linaro nano root filesystem.

cd ~/Documents/bootscr
nano para1.txt

(Edit and save)

setenv tftp_dir headless_7010
setenv nfs_rfs /srv/tftp/nano

setenv bootargs ip=dhcp
setenv bootargs ${bootargs} console=ttyPS0,115200

setenv nfs_ip 192.168.1.50
setenv bootargs ${bootargs} root=/dev/nfs rw nfsroot=${nfs_ip}:${nfs_rfs}

setenv bitstream_image parallella.bit.bin
setenv bitstream_addr 0x4000000
setenv bitstream_tftp 'tftpboot ${bitstream_addr} ${tftp_dir}/${bitstream_image}'
setenv bitstream_load 'fpga load 0 ${bitstream_addr} ${filesize}'

setenv devicetree_image devicetree.dtb
setenv devicetree_addr 0x2A00000
setenv devicetree_tftp 'tftpboot ${devicetree_addr} ${tftp_dir}/${devicetree_image}'

setenv kernel_image uImage
setenv kernel_addr 0x3000000
setenv kernel_tftp 'tftpboot ${kernel_addr} ${tftp_dir}/${kernel_image}'

setenv phy_rst 'mw.w f8000008 df0d ; mw.w f8000140 00100801 ; mw.w f8000004 767b'
setenv rset_phy 'run phy_rst'

setenv boot_now 'bootm ${kernel_addr} - ${devicetree_addr}'

run bitstream_tftp bitstream_load kernel_tftp devicetree_tftp rset_phy boot_now

Copy the file to create a bootscript for each Parallella, changing the filename, e.g. para2.txt. Edit the values of the tftp_dir and nfs_rfs variables in each file as required. If the boot mode is the same and the NFS rootfs is shared, then there are usually no edits required.
To change the mode of a Parallella boot, edit the 'paraX.txt' file to change the tftp_dir and/or nfs_rfs variables, before re-making the image.
The directory structure on the Debian PC will then look something like this:

/home/{username}/Documents/bootscr/   	
      para1.txt	
      para2.txt
      ...
      para8.txt

Make the Bootscript Image

Install mkimage, which is part of the Das U-Boot tool-set, if not already installed. On Debian:

sudo apt-get install u-boot-tools

Convert each bootscript text file into a loadable image by mkimage, and place each image in the TFTP server directory. For example:

cd ~/Documents/bootscr
sudo mkimage -A arm -O u-boot -T script -C none -a 0 -e 0 -n "t-Boot Script" -d para1.txt /srv/tftp/para1
sudo mkimage -A arm -O u-boot -T script -C none -a 0 -e 0 -n "t-Boot Script" -d para2.txt /srv/tftp/para2
...
sudo mkimage -A arm -O u-boot -T script -C none -a 0 -e 0 -n "t-Boot Script" -d para8.txt /srv/tftp/para8

Reboot the Parallella

Reboot by power off, wait a few seconds, power on.
The output to the UART Serial terminal screen shows initialisation of the board and network, and then the boot sequence looking for the TFTP server.
If the TFTP 'Loading:' line puts out a few 'T T T's, then it's retrying. Give it a few moments but if there's a line of 'T's then check that the TFTP service is actually running, and that the IP addresses and filename of the bootscript are valid.
Once the small (~1kb) bootscript has been loaded and is executed by U-Boot, further TFTP requests will be seen as the image files are downloaded before U-Boot hands over to the kernel.

Breakdown of the Bootscript

This section takes the bootscript example and breaks it down to illustrate what may be changed to support your local environment.

  • Set up the bootscript to point to the required boot mode (i.e. TFTP sub-directory) and the NFS filesystem directory chosen for this Paralella. By editing these first two lines, a Parallella can be re-booted using a different bitstream, kernel, devicetree images, and/or a different NFS root filesystem. Note that the the NFS filepath must be the same as the relevant line in NFS exports.
setenv tftp_dir headless_7010
setenv nfs_rootfs /srv/nfs/nano	
  • Set the initial bootargs string to tell the kernel to use DHCP when loaded. The U-Boot bootargs override any that are set up in the devicetree image.
setenv bootargs ip=dhcp
  • If required, add the console argument to the bootargs string. If the console is not required, delete this line.
setenv bootargs ${bootargs} console=ttyPS0,115200
  • Add the NFS boot arguments to the bootargs string. Note the NFS server IP variable is not the same as the TFTP serverip variable, although they may have the same value.
setenv nfs_ip 192.168.1.50
setenv bootargs ${bootargs} root=/dev/nfs rw nfsroot=${nfs_ip}:${nfs_rfs}
  • If required, add additional kernel boot diagnostic flags to the bootargs string. If not required, delete this line.
setenv bootargs ${bootargs} nfsrootdebug earlyprintk
  • Set up the variables for the command to tftpboot the bitstream image, and the command to load the image into the fpga. The size of the bitstream image is the size of the file set by tftpboot.
setenv bitstream_image parallella.bit.bin
setenv bitstream_addr 0x4000000
setenv bitstream_tftp 'tftpboot ${bitstream_addr} ${tftp_dir}/${bitstream_image}'
setenv bitstream_load 'fpga load 0 ${bitstream_addr} ${filesize}'
  • Set up the variables for the command to tftpboot the devicetree image.
setenv devicetree_image devicetree.dtb
setenv devicetree_addr 0x2A00000
setenv devicetree_tftp 'tftpboot ${devicetree_addr} ${tftp_dir}/${devicetree_image}'
  • Set up the variables for the command to tftpboot the kernel image.
setenv kernel_image uImage
setenv kernel_addr 0x3000000
setenv kernel_tftp 'tftpboot ${kernel_addr} ${tftp_dir}/${kernel_image}'
  • Patch U-Boot network handling. This is a workround for the problem identified by tajama, and referred to above. It MAY apply to your version but YOU MUST CHECK!! This U-Boot patch applies to the following version, which is output initially by U-Boot:
U-Boot 2012.10-00003-g792c31c (Jan 03 2014 - 12:24:08)
setenv phy_rst 'mw.w f8000008 df0d ; mw.w f8000140 00100801 ; mw.w f8000004 767b'
setenv rset_phy 'run phy_rst'
  • Set up the variable for the command to boot into the kernel, passing the kernel and devicetree image addresses.
setenv boot_now 'bootm ${kernel_addr} - ${devicetree_addr}'
  • Finally, run the sequence of individual commands and boot. If any of the command sequence fails, then the 'run' command will halt. Note there are no ';' separators in the command parameters.
run bitstream_tftp bitstream_load kernel_tftp devicetree_tftp rset_phy boot_now