[J-u-boot.git] / doc / README.iscsi

iSCSI booting with U-Boot and iPXE
==================================

Motivation
----------

U-Boot has only a reduced set of supported network protocols. The focus for
network booting has been on UDP based protocols. A TCP stack and HTTP support
are expected to be integrated in 2018 together with a wget command.

For booting a diskless computer this leaves us with BOOTP or DHCP to get the
address of a boot script. TFTP or NFS can be used to load the boot script, the
operating system kernel and the initial file system (initrd).

These protocols are insecure. The client cannot validate the authenticity
of the contacted servers. And the server cannot verify the identity of the
client.

Furthermore the services providing the operating system loader or kernel are
not the ones that the operating system typically will use. Especially in a SAN
environment this makes updating the operating system a hassle. After installing
a new kernel version the boot files have to be copied to the TFTP server
directory.

The HTTPS protocol provides certificate based validation of servers. Sensitive
data like passwords can be securely transmitted.

The iSCSI protocol is used for connecting storage attached networks. It
provides mutual authentication using the CHAP protocol. It typically runs on
a TCP transport.

Thus a better solution than DHCP/TFTP/NFS boot would be to load a boot script
via HTTPS and to download any other files needed for booting via iSCSI from the
same target where the operating system is installed.

An alternative to implementing these protocols in U-Boot is to use an existing
software that can run on top of U-Boot. iPXE[1] is the "swiss army knife" of
network booting. It supports both HTTPS and iSCSI. It has a scripting engine for
fine grained control of the boot process and can provide a command shell.

iPXE can be built as an EFI application (named snp.efi) which can be loaded and
run by U-Boot.

Boot sequence
-------------

U-Boot loads the EFI application iPXE snp.efi using the bootefi command. This
application has network access via the simple network protocol offered by
U-Boot.

iPXE executes its internal script. This script may optionally chain load a
secondary boot script via HTTPS or open a shell.

For the further boot process iPXE connects to the iSCSI server. This includes
the mutual authentication using the CHAP protocol. After the authentication iPXE
has access to the iSCSI targets.

For a selected iSCSI target iPXE sets up a handle with the block IO protocol. It
uses the ConnectController boot service of U-Boot to request U-Boot to connect a
file system driver. U-Boot reads from the iSCSI drive via the block IO protocol
offered by iPXE. It creates the partition handles and installs the simple file
protocol. Now iPXE can call the simple file protocol to load GRUB[2]. U-Boot
uses the block IO protocol offered by iPXE to fulfill the request.

Once GRUB is started it uses the same block IO protocol to load Linux. Via
the EFI stub Linux is called as an EFI application::

                  +--------+         +--------+
                  |        | Runs    |        |
                  | U-Boot |========>| iPXE   |
                  | EFI    |         | snp.efi|
    +--------+    |        | DHCP    |        |
    |        |<===|********|<========|        |
    | DHCP   |    |        | Get IP  |        |
    | Server |    |        | Address |        |
    |        |===>|********|========>|        |
    +--------+    |        | Response|        |
                  |        |         |        |
                  |        |         |        |
    +--------+    |        | HTTPS   |        |
    |        |<===|********|<========|        |
    | HTTPS  |    |        | Load    |        |
    | Server |    |        | Script  |        |
    |        |===>|********|========>|        |
    +--------+    |        |         |        |
                  |        |         |        |
                  |        |         |        |
    +--------+    |        | iSCSI   |        |
    |        |<===|********|<========|        |
    | iSCSI  |    |        | Auth    |        |
    | Server |===>|********|========>|        |
    |        |    |        |         |        |
    |        |    |        | Loads   |        |
    |        |<===|********|<========|        |       +--------+
    |        |    |        | GRUB    |        | Runs  |        |
    |        |===>|********|========>|        |======>| GRUB   |
    |        |    |        |         |        |       |        |
    |        |    |        |         |        |       |        |
    |        |    |        |         |        | Loads |        |
    |        |<===|********|<========|********|<======|        |      +--------+
    |        |    |        |         |        | Linux |        | Runs |        |
    |        |===>|********|========>|********|======>|        |=====>| Linux  |
    |        |    |        |         |        |       |        |      |        |
    +--------+    +--------+         +--------+       +--------+      |        |
                                                                      |        |
                                                                      |        |
                                                                      | ~ ~ ~ ~|

Security
--------

The iSCSI protocol is not encrypted. The traffic could be secured using IPsec
but neither U-Boot nor iPXE does support this. So we should at least separate
the iSCSI traffic from all other network traffic. This can be achieved using a
virtual local area network (VLAN).

Configuration
-------------

iPXE
^^^^

For running iPXE on arm64 the bin-arm64-efi/snp.efi build target is needed::

    git clone http://git.ipxe.org/ipxe.git
    cd ipxe/src
    make bin-arm64-efi/snp.efi -j6 EMBED=myscript.ipxe

The available commands for the boot script are documented at:

http://ipxe.org/cmd

Credentials are managed as environment variables. These are described here:

http://ipxe.org/cfg

iPXE by default will put the CPU to rest when waiting for input. U-Boot does
not wake it up due to missing interrupt support. To avoid this behavior create
file src/config/local/nap.h::

    /* nap.h */
    #undef NAP_EFIX86
    #undef NAP_EFIARM
    #define NAP_NULL

The supported commands in iPXE are controlled by an include, too. Putting the
following into src/config/local/general.h is sufficient for most use cases::

    /* general.h */
    #define NSLOOKUP_CMD            /* Name resolution command */
    #define PING_CMD                /* Ping command */
    #define NTP_CMD                 /* NTP commands */
    #define VLAN_CMD                /* VLAN commands */
    #define IMAGE_EFI               /* EFI image support */
    #define DOWNLOAD_PROTO_HTTPS    /* Secure Hypertext Transfer Protocol */
    #define DOWNLOAD_PROTO_FTP      /* File Transfer Protocol */
    #define DOWNLOAD_PROTO_NFS      /* Network File System Protocol */
    #define DOWNLOAD_PROTO_FILE     /* Local file system access */

Links
-----

* [1](https://ipxe.org) https://ipxe.org - iPXE open source boot firmware
* [2](https://www.gnu.org/software/grub/) https://www.gnu.org/software/grub/ -
  GNU GRUB (Grand Unified Bootloader)
Commit	Line	Data
76a472dc HS	1	iSCSI booting with U-Boot and iPXE
76a472dc HS	2	==================================
29a8a282	3
76a472dc HS	4	Motivation
76a472dc HS	5	----------
29a8a282 HS	6
	7	U-Boot has only a reduced set of supported network protocols. The focus for
	8	network booting has been on UDP based protocols. A TCP stack and HTTP support
	9	are expected to be integrated in 2018 together with a wget command.
	10
	11	For booting a diskless computer this leaves us with BOOTP or DHCP to get the
	12	address of a boot script. TFTP or NFS can be used to load the boot script, the
	13	operating system kernel and the initial file system (initrd).
	14
	15	These protocols are insecure. The client cannot validate the authenticity
	16	of the contacted servers. And the server cannot verify the identity of the
	17	client.
	18
	19	Furthermore the services providing the operating system loader or kernel are
	20	not the ones that the operating system typically will use. Especially in a SAN
	21	environment this makes updating the operating system a hassle. After installing
	22	a new kernel version the boot files have to be copied to the TFTP server
	23	directory.
	24
	25	The HTTPS protocol provides certificate based validation of servers. Sensitive
	26	data like passwords can be securely transmitted.
	27
	28	The iSCSI protocol is used for connecting storage attached networks. It
	29	provides mutual authentication using the CHAP protocol. It typically runs on
	30	a TCP transport.
	31
	32	Thus a better solution than DHCP/TFTP/NFS boot would be to load a boot script
	33	via HTTPS and to download any other files needed for booting via iSCSI from the
	34	same target where the operating system is installed.
	35
	36	An alternative to implementing these protocols in U-Boot is to use an existing
76a472dc	37	software that can run on top of U-Boot. iPXE[1] is the "swiss army knife" of
29a8a282 HS	38	network booting. It supports both HTTPS and iSCSI. It has a scripting engine for
	39	fine grained control of the boot process and can provide a command shell.
	40
	41	iPXE can be built as an EFI application (named snp.efi) which can be loaded and
	42	run by U-Boot.
	43
76a472dc HS	44	Boot sequence
76a472dc HS	45	-------------
29a8a282 HS	46
	47	U-Boot loads the EFI application iPXE snp.efi using the bootefi command. This
	48	application has network access via the simple network protocol offered by
	49	U-Boot.
	50
	51	iPXE executes its internal script. This script may optionally chain load a
	52	secondary boot script via HTTPS or open a shell.
	53
	54	For the further boot process iPXE connects to the iSCSI server. This includes
	55	the mutual authentication using the CHAP protocol. After the authentication iPXE
	56	has access to the iSCSI targets.
	57
	58	For a selected iSCSI target iPXE sets up a handle with the block IO protocol. It
	59	uses the ConnectController boot service of U-Boot to request U-Boot to connect a
	60	file system driver. U-Boot reads from the iSCSI drive via the block IO protocol
	61	offered by iPXE. It creates the partition handles and installs the simple file
76a472dc HS	62	protocol. Now iPXE can call the simple file protocol to load GRUB[2]. U-Boot
	63	uses the block IO protocol offered by iPXE to fulfill the request.
	64
	65	Once GRUB is started it uses the same block IO protocol to load Linux. Via
	66	the EFI stub Linux is called as an EFI application::
	67
	68	+--------+ +--------+
	69	\| \| Runs \| \|
	70	\| U-Boot \|========>\| iPXE \|
	71	\| EFI \| \| snp.efi\|
	72	+--------+ \| \| DHCP \| \|
	73	\| \|<===\|********\|<========\| \|
	74	\| DHCP \| \| \| Get IP \| \|
	75	\| Server \| \| \| Address \| \|
	76	\| \|===>\|********\|========>\| \|
	77	+--------+ \| \| Response\| \|
	78	\| \| \| \|
	79	\| \| \| \|
	80	+--------+ \| \| HTTPS \| \|
	81	\| \|<===\|********\|<========\| \|
	82	\| HTTPS \| \| \| Load \| \|
	83	\| Server \| \| \| Script \| \|
	84	\| \|===>\|********\|========>\| \|
	85	+--------+ \| \| \| \|
	86	\| \| \| \|
	87	\| \| \| \|
	88	+--------+ \| \| iSCSI \| \|
	89	\| \|<===\|********\|<========\| \|
	90	\| iSCSI \| \| \| Auth \| \|
	91	\| Server \|===>\|********\|========>\| \|
	92	\| \| \| \| \| \|
	93	\| \| \| \| Loads \| \|
	94	\| \|<===\|********\|<========\| \| +--------+
	95	\| \| \| \| GRUB \| \| Runs \| \|
	96	\| \|===>\|********\|========>\| \|======>\| GRUB \|
	97	\| \| \| \| \| \| \| \|
	98	\| \| \| \| \| \| \| \|
	99	\| \| \| \| \| \| Loads \| \|
	100	\| \|<===\|******\|<========\|******\|<======\| \| +--------+
	101	\| \| \| \| \| \| Linux \| \| Runs \| \|
	102	\| \|===>\|******\|========>\|******\|======>\| \|=====>\| Linux \|
	103	\| \| \| \| \| \| \| \| \| \|
	104	+--------+ +--------+ +--------+ +--------+ \| \|
	105	\| \|
	106	\| \|
	107	\| ~ ~ ~ ~\|
	108
	109	Security
	110	--------
29a8a282 HS	111
	112	The iSCSI protocol is not encrypted. The traffic could be secured using IPsec
	113	but neither U-Boot nor iPXE does support this. So we should at least separate
	114	the iSCSI traffic from all other network traffic. This can be achieved using a
	115	virtual local area network (VLAN).
	116
76a472dc HS	117	Configuration
76a472dc HS	118	-------------
29a8a282	119
76a472dc HS	120	iPXE
76a472dc HS	121	^^^^
29a8a282	122
76a472dc	123	For running iPXE on arm64 the bin-arm64-efi/snp.efi build target is needed::
29a8a282 HS	124
	125	git clone http://git.ipxe.org/ipxe.git
	126	cd ipxe/src
	127	make bin-arm64-efi/snp.efi -j6 EMBED=myscript.ipxe
	128
	129	The available commands for the boot script are documented at:
	130
	131	http://ipxe.org/cmd
	132
	133	Credentials are managed as environment variables. These are described here:
	134
	135	http://ipxe.org/cfg
	136
	137	iPXE by default will put the CPU to rest when waiting for input. U-Boot does
	138	not wake it up due to missing interrupt support. To avoid this behavior create
76a472dc	139	file src/config/local/nap.h::
29a8a282 HS	140
	141	/* nap.h */
	142	#undef NAP_EFIX86
	143	#undef NAP_EFIARM
	144	#define NAP_NULL
	145
	146	The supported commands in iPXE are controlled by an include, too. Putting the
76a472dc	147	following into src/config/local/general.h is sufficient for most use cases::
29a8a282 HS	148
	149	/* general.h */
	150	#define NSLOOKUP_CMD /* Name resolution command */
	151	#define PING_CMD /* Ping command */
	152	#define NTP_CMD /* NTP commands */
	153	#define VLAN_CMD /* VLAN commands */
	154	#define IMAGE_EFI /* EFI image support */
	155	#define DOWNLOAD_PROTO_HTTPS /* Secure Hypertext Transfer Protocol */
	156	#define DOWNLOAD_PROTO_FTP /* File Transfer Protocol */
	157	#define DOWNLOAD_PROTO_NFS /* Network File System Protocol */
	158	#define DOWNLOAD_PROTO_FILE /* Local file system access */
	159
76a472dc HS	160	Links
76a472dc HS	161	-----
29a8a282	162
76a472dc HS	163	* [1](https://ipxe.org) https://ipxe.org - iPXE open source boot firmware
	164	* [2](https://www.gnu.org/software/grub/) https://www.gnu.org/software/grub/ -
	165	GNU GRUB (Grand Unified Bootloader)