ROCK Pi 4: Bootloader

One of the main reasons why I opted for the ROCK Pi 4A for my home server is that I found out it runs fine with only free software. So as bootloader I want to use Das U-Boot without any binary blobs (‚blobless‘). Here I document how I installed U-Boot first on a microSD card and then the soldered SPI flash.

Boot from microSD card

Originally I wanted to use the pre-built SD card installer image for the board provided by Debian. This image includes U-Boot and is part of Debian’s main archive area, so it must comply with the DFSG and thus be free software. See the last few paragraphs of this section to find out why this installer image turned out not to be a real option for me.

Since Debian Bookworm will be stable soon and includes a fairly up-to-date U-Boot version (2023.01+dfsg-2 at the time of writing) I use this release. As explained in the README.concatenateable_images file I have to download a device-specific part (firmware.rock-pi-4-rk3399.img.gz) as well as a device-independent part (partition.img.gz) and then decompress and concatenate them.

$ wget https://deb.debian.org/debian/dists/bookworm/main/installer-arm64/current/images/netboot/SD-card-images/firmware.rock-pi-4-rk3399.img.gz
$ wget https://deb.debian.org/debian/dists/bookworm/main/installer-arm64/current/images/netboot/SD-card-images/partition.img.gz
$ zcat firmware.rock-pi-4-rk3399.img.gz partition.img.gz > sd_card_image.img

Now I use lsblk before and after I put in my microSD card to find out to which device file it is mapped (in my case that is /dev/sda), then I unmount all possibly mounted partitions and write the image. Be aware that after these steps all data on the microSD card will be lost.

$ lsblk
$ sudo umount /dev/sda?*
$ sudo dd if=sd_card_image.img of=/dev/sda
$ sync

Then I just have to insert the microSD card into my ROCK Pi 4 and connect the monitor, my keyboard and the ac adapter. The bootloader works as expected and Debian Installer starts. I could use it to install my root file system to my microSD card, probably to an eMMC (I didn’t try that, I don’t have one), to some USB mass storage device or an NVMe M.2 SSD. The bootloader however would still be on my microSD card and so I would have to keep it inserted… But I can’t.

As I wrote in my last post I’m planning to use my ROCK Pi 4 as a home server with an aluminum housing and an M.2 extension board since I want to place my NVMe M.2 SSD inside that housing. In this picture you can see the microSD card inserted before mounting the M.2 extension board on top of the ROCK Pi 4:

Now when I put the M.2 extension board on the standoff bolts with the microSD card inserted I can’t mount the aluminum part with the walls of the housing. And when I mount it without the microSD card inserted the ribbon cable covers the microSD slot…

So I definitely need another boot option. After looking at the Rockchip wiki I realized that I had two remaining options: SPI flash and eMMC. I decided to make use of the included SPI flash.

Boot from SPI flash

I couldn’t find a software project that provides an easy way to make use of the SPI flash of the ROCK Pi 4 with only free software. Armbian does provide an easy way indeed, but unfortunately the project’s builds depend on binary blobs. So I have to build U-Boot myself. The official U-Boot documentation, in this context especially the document for Rockchip supported boards, is really helpful.

The first step is to build TrustedFirmware-A (TF-A) which is required when building the U-Boot image for the ROCK Pi 4. I start by creating a directory for the build process and changing into it, then I download a tarball of TF-A v2.8, extract its contents and change into the new subdirectory:

~$ mkdir uboot-rockpi4
~$ cd uboot-rockpi4
~/uboot-rockpi4$ wget https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/snapshot/trusted-firmware-a-2.8.tar.gz
~/uboot-rockpi4$ tar -xvf trusted-firmware-a-2.8.tar.gz
~/uboot-rockpi4$ cd trusted-firmware-a-2.8

The source code includes some binary files with unknown licensing. I follow the Debian project and simply remove those files before building TF-A:

~/uboot-rockpi4/trusted-firmware-a-2.8$ find -name \*.bin -exec rm -v {} \;
~/uboot-rockpi4/trusted-firmware-a-2.8$ make CROSS_COMPILE=aarch64-linux-gnu- PLAT=rk3399
~/uboot-rockpi4/trusted-firmware-a-2.8$ cd ..

Now I download a tarball of U-Boot v2023.04, extract its contents and change into the new subdirectory:

~/uboot-rockpi4$ wget https://ftp.denx.de/pub/u-boot/u-boot-2023.04.tar.bz2
~/uboot-rockpi4$ tar -xvf u-boot-2023.04.tar.bz2
~/uboot-rockpi4$ cd u-boot-2023.04

Since U-Boot mainline doesn’t support the SPI flash of the ROCK Pi 4 yet I pieced together two patches I found at the Armbian build repository. You can download the resulting patch and apply it like this:

~/uboot-rockpi4/u-boot-2023.04$ wget https://cloud.ixypsilon.net/s/kWDo5c3Ti6T7rkP/download/rockpi4-enable-spi-flash.patch
~/uboot-rockpi4/u-boot-2023.04$ patch -p1 < rockpi4-enable-spi-flash.patch

Now I ensure that the U-Boot build process can find the TF-A binary bl31.elf, then I apply the default configuration for the board and after that I have to use the U-Boot configuration program to enable support for U-Boot scripts and switch to a more powerful command line interpreter.

For whatever reason Debian supports one boot method for the installer image mentioned above (distro boot, there is a file extlinux/extlinux.conf describing how to boot the system) and another one for the installed system (U-Boot script, there’s a file boot/boot.scr describing how to boot the system). The second boot method is not enabled by default in U-Boot 2023.04 and the boot script used by Debian needs the hush parser (the more powerful command line interpreter). That’s why I have to use the configuration program.

~/uboot-rockpi4/u-boot-2023.04$ export BL31=../trusted-firmware-a-2.8/build/rk3399/release/bl31/bl31.elf
~/uboot-rockpi4/u-boot-2023.04$ make rock-pi-4-rk3399_defconfig
~/uboot-rockpi4/u-boot-2023.04$ make menuconfig

With this last command I enter the U-Boot configuration program. First, I select Boot options, then Boot images and then I can enable Bootdev support for U-Boot scripts.

I hit Exit two times, select Command line interface and then activate Use hush shell.

I hit Exit two times to quit the configuration program, confirm that I wish to save my new configuration and finally I can start the build process:

~/uboot-rockpi4/u-boot-2023.04$ make CROSS_COMPILE=aarch64-linux-gnu-

My plan is to use the U-Boot command line to install the SPI boot image, so I write an U-Boot image that boots from a microSD card and then copy the SPI boot image to that same microSD card.

First I use lsblk before and after I insert the microSD card to make sure I use the correct device file (in my case it’s /dev/sda). After that I unmount all possibly mounted partitions. Be aware that after the next steps all data on the microSD card will be lost. I create a GPT disk label, a new partition and then build a file system on that partition. I’ll use this partition for the SPI boot image, note that it starts at sector 32768 to leave enough space in front of it for the U-Boot image that boots from the microSD card.

~/uboot-rockpi4/u-boot-2023.04$ lsblk
~/uboot-rockpi4/u-boot-2023.04$ sudo umount /dev/sda?*
~/uboot-rockpi4/u-boot-2023.04$ sudo parted /dev/sda mklabel gpt
~/uboot-rockpi4/u-boot-2023.04$ sudo parted /dev/sda mkpart noname ext4 32768s 100%
~/uboot-rockpi4/u-boot-2023.04$ sudo mkfs.ext4 /dev/sda1

Now I write the U-Boot image that boots from the microSD card:

~/uboot-rockpi4/u-boot-2023.04$ sudo dd if=u-boot-rockchip.bin of=/dev/sda seek=64
~/uboot-rockpi4/u-boot-2023.04$ sync

Finally I can generate and then copy the SPI boot image to the microSD card. For that I mount the partition I created, copy the files and then unmount the partition again.

~/uboot-rockpi4/u-boot-2023.04$ ./tools/mkimage -n rk3399 -T rkspi -d tpl/u-boot-tpl.bin:spl/u-boot-spl.bin idbloader.img
~/uboot-rockpi4/u-boot-2023.04$ sudo mkdir /media/sdcard
~/uboot-rockpi4/u-boot-2023.04$ sudo mount -t ext4 /dev/sda1 /media/sdcard
~/uboot-rockpi4/u-boot-2023.04$ sudo cp idbloader.img u-boot.itb /media/sdcard/
~/uboot-rockpi4/u-boot-2023.04$ sudo umount /media/sdcard
~/uboot-rockpi4/u-boot-2023.04$ sudo rmdir /media/sdcard

Let’s put the microSD card into the ROCK Pi 4, connect the display via HDMI, the keyboard via USB (there’s an issue with the USB 2 ports in U-Boot, so I use one of the USB 3 ports) and then the ac adapter. U-Boot starts and looks for a bootable OS but won’t find one. After that I can use the command line to flash the SPI boot image like this:

=> sf probe
=> load mmc 1:1 ${kernel_addr_r} idbloader.img
=> sf erase 0 +${filesize}
=> sf write ${kernel_addr_r} 0 ${filesize}
=> load mmc 1:1 ${kernel_addr_r} u-boot.itb
=> sf erase 0x60000 +${filesize}
=> sf write ${kernel_addr_r} 0x60000 ${filesize}

And that’s it. I can verify that U-Boot is installed on the SPI flash by removing the microSD card and then using the reset command (or by simply disconnecting and reconnecting the ac adapter).

Now all I have to do to install Debian on my NVMe SSD is to insert the microSD card with the Debian image (see first section) and connect the ac adapter. U-Boot starts from the SPI flash and then boots the Debian Installer from the microSD card. After the installation process I simply remove the card and reboot.