RJSTONE.net What This Is About
I have this Pinebook Pro and want to build u-boot firmware for it. But I’d rather cross-compile and test it out in qemu, rather than have to fiddle with booting the real hardware every time.
So I want to set up a cross-compilation environment for aarch64 (arm64) that
will let me do that (even if I can’t build for every possible hardware target
with every possible feature).
I’m going to try to do this with WSL in Windows, even though I’m not sure this is really the best option. (A dev container or VM might be more suitable.)
What I’m Using
Windows Subsystem for Linux v2 with Ubuntu 22. I think this is just the default Linux distribution for WSL these days.
> hostnamectl status
...
Chassis: container
...
Virtualization: wsl
Operating System: Ubuntu 22.04.5 LTS
Kernel: Linux 6.6.87.1-microsoft-standard-WSL2
Architecture: x86-64
> wsl.exe --version
WSL version: 2.5.7.0
Kernel version: 6.6.87.1-1
WSLg version: 1.0.66
...
Windows version: 10.0.26100.4351What I’m Doing
I’ll try to get all these steps in the right order according to how I’m doing things.
Add arm64 to dpkg
It gets annoying, but some things use the name arm64 to identify the ARM
64-bit architecture target, and other things use aarch64. For dpkg
we need:
> dpkg --add-architecture arm64
> dpkg --print-foreign-architectures
arm64Most other things will refer to ARM architecture as aarch64 (or the same
with 32 for 32 bit).
I’m not sure how much this will help, but I figure I might as well tell
dpkg that it’s allowed to install packages for the target platform just
in case it comes in handy.
Installing Necessary Packages
Here’s what I’m installing.
sudo apt update
sudo apt-get install bc bison build-essential coccinelle \
device-tree-compiler dfu-util efitools flex gdisk graphviz imagemagick \
libgnutls28-dev libncurses-dev gcc-aarch64-linux-gnu \
libpython3-dev libsdl2-dev libssl-dev lz4 lzma lzma-alone openssl \
pkg-config python3 python3-asteval python3-coverage python3-filelock \
python3-pkg-resources python3-pycryptodome python3-pyelftools \
python3-pytest python3-pytest-xdist python3-sphinxcontrib.apidoc \
python3-sphinx-rtd-theme python3-subunit python3-testtools \
python3-venv swig uuid-dev python3-setuptoolsThis is the list of
dependencies in
the u-boot docs minue the package libguestfs-tools because it depends on
mdadm which just hangs in WSL in it’s install hook script. I’m guessing it
wants to add a kernel module that doesn’t work in WSL or something. If
preparation to work around this if needed (probably not needed for what I’m
doing):
sudo apt-add-repository -s
cd <the same directory where I cloned u-boot>
ls .
u-boot/
sudo apt-get source libguestfs-tools
ls .
libguestfs-1.46.2 libguestfs_1.46.2-10ubuntu3.debian.tar.xz
libguestfs_1.46.2-10ubuntu3.dsc libguestfs_1.46.2.orig.tar.gz
u-bootBut when dealing with WSL, be prepared for some debian packages to not “just work”, especially when they are things that would expect to be able to add kernel modules.
Additionally you may need or want these:
sudo apt-get install arm-trusted-firmware-tools bsdutils bsdextrautils \
bzip2 clang gdb-multiarch gdbserver git \
qemu qemu-system qemu-user qemu-utils xz-utilsThe only one absolutely needed there is git, if it’s not already installed..
Clone u-boot Repo
git clone https://source.denx.de/u-boot/u-boot.git
git checkout v2025.07-rc5Right now, I figure that v2025.07-rc5 ought to be very recent without being
too unstable. (Usually rc5 or rc6 is the last rc before release, but the
release for v2025.07 isn’t tagged yet.)
Set Env Variables
This is the main one that needs to be set:
> export CROSS_COMPILE=aarch64-linux-gnu-(Notice the aarch64 this time.)
Or don’t set the environment variable and invoke make with it set like this
every time when needed:
> CROSS_COMPILE=aarch64-linux-gnu- make [target_name]Supplying this as a make variable also works:
> make CROSS_COMPILE=aarch64-linux-gnu- [target_name]Optional things to set if you want to try using clang:
export HOSTCC=clang
export CC=clangYou can also use a specific output directory by adding O= as in
O=/tmp/build/artifacts as an argument to the make command.
Use V=1 (or a larger number) for more verbose make output.
Also, run make help in the u-boot top level directory to get some text output
about what targets are available and what they’re for.
Cleaning
make clean- removes most things but doesn’t delete.configmake mrproper- same as clean plus removes.configand some cachesmake distclean- absolute maximum clean
Running some level of clean is important if you change any configuration.
If only the .config is changed then regular clean should be good
enough. If you change O= then mrproper will be needed.
Test Build
Try this first.
Use unset CROSS_COMPILE if set as an env var, because the
sandbox tests just use native compilation. (Optionally
add the O= or V= to the below if you want.)
> make mrproper
> make qtest
....This will generate a .config using sandbox_defconfig from the configs/
directory, then run a bunch of tests.
Don’t be surprised if some fail though because you may still be lacking some dependencies, and this tries to test every feature used on any platform. So whatever failures happen may not even be relevant for your target hardware.
Build a Boot Firmware for Real Hardware (Pinebook Pro)
Before I start I really need to read the device-specific docs. Even with the docs it can be difficult to know what they’re actually talking about, because unfortunately whoever is usually writing this stuff seems to be writing it for an audience who already knows everything.
But enough complaining, here are the main ones for this Pinebook Pro:
- https://docs.u-boot.org/en/latest/arch/arm64.html
- https://docs.u-boot.org/en/latest/board/rockchip/rockchip.html
The generic architecture doc isn’t too useful for me yet, but the second one has build instructions.
“Pinebook Pro” isn’t listed, but I know it’s using a rk3399 SoC and I know
there’s a pinebook-pro-rk3399_defconfig in configs/ so whatever build
instructions are helpful for rk3399 should be helpful to me.
TF-A (trusted-firmware-a)
There’s nothing in the above doc to tell you what this is or why you need it, but at least there are some build instructions, which is a big time saver.
From within my working base directory, which contains my u-boot directory:
git clone https://github.com/TrustedFirmware-A/trusted-firmware-a.git
cd trusted-firmware-a
make realclean
make CROSS_COMPILE=aarch64-linux-gnu- PLAT=rk3399
........build messages spew........
cd ..And thankfully that works like a charn. I now have trusted-firmware-a/build/rk3399/release/bl31/bl31.elf
which I will soon need.
What BL31.elf Actually Is
Simple answer: This is some stock ARM BSP (Board Support Package) firmware for the ARM IP (chip design) used by Rockchip on this particular model of chip (rk3399). The design won’t work without it, and it’s not worth rewriting a different version of it if the source code is available, so people just use this.
More complicated answer: This is a particularly critical part of the firmware that executes at a maximum “super supervisor” privilege level (EL3 or Exception Level 3, which is above the hypervisor level EL2) and acts as a firewall between what the documentation calls the “normal world” and the “secure world”.
Therefore it makes sense to use the smallest most solid computer program possible to control access to these sorts of functions and restrict them appropriately. Without a solid security mechanism, “rootkit” software could make itself invisible then do anything to anything by having the most unrestricted access to everything’s state.
The name “trusted” indicates that it occupies a priviledged position where it must be considered secure by the rest of the design. That doesn’t mean it’s automatically trustworthy, but if this is compromised then everything is. So it has to be trusted by design, and actually being trustworthy would be its main design objective.
To prevent tampering, ideally this especially (like the rest of the firmware) should be verified stating with a hardware root of trust. That is, it should only be loaded and executed by a SoC if it has a digital signature that should only be verifiable if the firmware image has been approved by the owner, manufacturer, etc., as indicated by a one-time-programmable CA certificate in the SoC.
But the Pinebook Pro doesn’t come with these features enabled/configured, and I have read that this particular Rockchip design (rk3399) might not be able to verify firmware signatures properly or something. (It’s one of the things I want to get to the bottom of eventually.)
Anyway, more information on the BL31 functions are here:
- ARM Trusted firmware BL31
- TF-A Firmware Image Terminology (more detail)
So, unlike BL1 (which is permanently burned into the chip) any firmware needs to include something for the BL31 functions and it might as well be the official firmware (which you can build yourself to help ensure that it isn’t “rootkitted”).
make the Config (.config)
First everything needs to be clean, then make something_defconfig where
something_defconfig is one of the device default config files in configs/.
> make distclean
....
> make -j8 pinebook-pro-rk3399_defconfig
... blabla make output blabla....
#
# configuration written to .config
#(-j8 is optional and means parallel processing, with 8 processes.)
In my case, the defconfig file is pinebook-pro-rk3399_defconfig.
That will create .config in the current directory (u-boot top level) based on
the default configuration for the Pinebook Pro.
Tamper With, uh, I mean Improve, the Config
If you ran make help you might have noticed this.
> cd u-boot/
> make help
...
Configuration targets:
config - Update current config utilising a line-oriented program
nconfig - Update current config utilising a ncurses menu based program
menuconfig - Update current config utilising a menu based program
xconfig - Update current config utilising a Qt based front-end
gconfig - Update current config utilising a GTK+ based front-end
.....blabla...These launch different editors for the .config file to change the
options from the defaults that were copied from the _defconfig file for your
device.
In my opinion, the best one to use is nconfig because it provides easy
access to all the options. menuconfig is essentially just a color version of
nconfig with less convenient keymappings, but it’s almost the same.
I don’t want to bother to install the dependencies needed to compile the GUI
equivalents, so I’m ignoring those. config is very inconvenient because it
will just ask you to confirm pretty much all settings, one after the other, and
there are zillions of options.
So unless xconfig or gconfig just instantly work in your environment,
pretty much every option besides nconfig or menuconfig is a waste of time.
Some usage notes:
nconfig
Use F2 to get some explanation of a the currently selected setting, hit F6 to save the config, and you can search for settings with F8. Esc also works as F5 “Back”.
menuconfig
Use ? for the same function as F2, and Esc does not work as “Back”.
What to Not Change
Two main bits of advice:
-
A bunch of settings are characteristics of the target device that can’t be changed unless you want to see it lock up or never do anything to begin with. These are usually pretty obvious like “Enable ARM Architecture” and “Support Rockchip RK3399” for example, but some are more obscure. Try to use some common sense.
-
Some settings might normally be perfectly fine to enable, but the feature is broken for your particular device, or there’s a dependent feature that doesn’t automatically get enabled and you’re not sure what it is. So you can end up with some seemingly valid configurations that prevent the firmware from building, or that just lock up after boothing. For this reason it’s not a good idea to change too many things before testing, otherwise it may be difficult to figure out what caused a problem.
-
If there’s a build error, check any related settings that you changed to see if it’s caused by that and a missing dependency.
So just keep in mind that not all combinations of all options are guaranteed to work even when the options seem fine. Not every feature is “there yet” on every platform.
Don’t expect any of the config to be idiot-proof.
Use U-Boot TPL or Vendor “TPL”?
Just “one more thing” for the Nth time…
There are still two environment variables I need to set for the Pinebook Pro before starting the build.
One is optional if you don’t mind “suspend to RAM” (sleep) not working properly on the Pinebook Pro. But if you do want this working:
cd ..
git clone https://github.com/rockchip-linux/rkbin
cd u-bootThis will clone the closed-source firmware driver binaries from Rockchip, one
of which is rkbin/bin/rk33/px30_ddr_333MHz_v1.16.bin.
To use this as part of the image at the end of the build, we need to set
ROCKCHIP_TPL to the correct file.
To explain this and the boot process a little better, these are the different firmware images that get executed at boot:
- BROM (BL1) - Boot ROM, can’t be changed, burned into the chip, is the first thing to start executing. This initializes the SoC (System on a Chip), (or at least things that need immediate setup like clocks, etc).
- TPL (BL2) - Tertiary Program Loader (u-boot terminology) - this initializes DRAM and returns control to the BROM. Not much else happens here. (u-boot can be configured to try adding a few extra things here when it builds this part, but not much makes sense to add.)
- SPL (BL2) - Secondary Program Loader (u-boot term) - Loaded and executed by BROM after the TPL returns. Now that RAM should all be working, this loads up the remaining firmware, that being TF-A and u-boot, then executes TF-A. (In some cases this can contain extra stuff beyond the norm.)
- TF-A (BL31) - The Trusted Firmware A for the BL31 phase (3.1) which runs at EL3 (Exception Level 3, highest privilege). Once it’s done initializing it drops the EL to EL2 or EL1 and then executes u-boot. It then stays resident, and remains dormant until something asks it to do something requiring EL3 super powers.
- u-boot (BL33) - Finally, after all that rigamarole, u-boot gets to start at EL2 or EL1. Only here do we actually get to what you think of as a normal OS boot loader. Everything else is really Firmware (what’s contained in the “BIOS” flash chip on a regular PC.) Once done with it’s stuff, it drops to EL1 and loads an OS kernel to execute at EL1. (EL2 is the hypervisor level for virtualization, so this could be used if virtualization is in play.)
So basically the vendor image px30_ddr_333MHz_v1.16.bin is a complete
replacement for what u-boot would ordinary supply as “TPL”.
So even though I’d like source code for this, I think I’ll just use the vendor’s TPL for now and see how it works for “sleep mode”. At least I was able to build the TP-A image from source.
cd u-boot
export ROCKCHIP_TPL=../rkbin/bin/rk33/px30_ddr_333MHz_v1.16.bin
export BL31=../trusted-firmware-a/build/rk3399/release/bl31/bl31.elfFinally… Run the Build
After all that trouble, I’m ready to actually build the whole image.
make CROSS_COMPILE=aarch64-linux-gnu- -j8Woohoo! Success! Worked on the first try. Probably only worked smoothly because I’m going through the process slowly enough to write all of this up.
Pick Through The Artifacts
The successful build process left us with lots of stuff, including:
> ls -l *.img *.bin *.itb
-rw-r--r-- 1 rjstone rjstone 397312 Jun 30 02:44 idbloader-spi.img
-rw-r--r-- 1 rjstone rjstone 198656 Jun 30 02:44 idbloader.img
-rw-r--r-- 1 rjstone rjstone 1230697 Jun 30 02:44 simple-bin-spi.fit.itb
-rw-r--r-- 1 rjstone rjstone 1230730 Jun 30 02:44 simple-bin.fit.itb
-rw-r--r-- 1 rjstone rjstone 1068616 Jun 30 02:44 u-boot-dtb.bin
-rw-r--r-- 1 rjstone rjstone 1069612 Jun 30 02:44 u-boot-dtb.img
-rwxr-xr-x 1 rjstone rjstone 970856 Jun 30 02:44 u-boot-nodtb.bin
-rw-r--r-- 1 rjstone rjstone 2149888 Jun 30 02:44 u-boot-rockchip-spi.bin
-rw-r--r-- 1 rjstone rjstone 9588224 Jun 30 02:44 u-boot-rockchip.bin
-rw-r--r-- 1 rjstone rjstone 1068616 Jun 30 02:44 u-boot.bin
-rw-r--r-- 1 rjstone rjstone 1069612 Jun 30 02:44 u-boot.img
-rw-r--r-- 1 rjstone rjstone 1232384 Jun 30 02:44 u-boot.itbI know I’m supposed to be able to flash idbloader.img and u-boot.bin, or
just u-boot-rockchip.bin which combines the two, but what is all this other
stuff? Tune in for a future episode this season where I will figure that out.
Flashing/Writing The Boot Loader
Warning: Don’t just assume the layout of a firmware/boot loader image you found and just write it using the offsets shown below, even if the filenames look similar. Similar images that come with different OS distributions may contain other stuff, like disk partition information, intended to be written from offset 0 because it includes a MBR and partition table. Make sure you know what a file contains and what offset it should be written at before writing it. Otherwise you end up with non-working binary at the entry points.
Block-Oriented Boot Devices and Images
If we use the single image file then we need to write it to the “disk” image or device like this:
sudo dd if=u-boot-rockchip.bin of=<file or /dev device> seek=64
syncYou may also want conv=fsync,notrunc if writing to a file.
The seek=64 will seek forward 64 * 512B blocks in the output, or 32,768 bytes
from the beginning.
If we need to write the seperate .img and .idb files, it should be done
like this:
sudo dd if=/boot/idbloader.img conv=notrunc seek=64 of=<dev or disk img>
sudo dd if=/boot/u-boot.itb conv=notrunc seek=16384 of=<dev or disk img>The single image ends up putting these two at the same offsets when the whole
thing is written starting from seek=64:
> hexdump u-boot-rockchip.bin -s 0x30690 | head -10
0030690 0074 6976 2d6e 7573 7070 796c 0000 0000
00306a0 0000 0000 0000 0000 0000 0000 0000 0000
*
0030800 ffff ffff ffff ffff ffff ffff ffff ffff
*
07f8000 0dd0 edfe 0000 000c 0000 3800 0000 c80a
07f8010 0000 2800 0000 1100 0000 0200 0000 0000
07f8020 0000 c800 0000 900a 0000 0000 0000 0000So idbloader.img (the first part) ends at about 0x3069A (in this particular
build), then is filled with all 1’s (0xFF) until 0x7F800, then the u-boot.itb
part starts. (All 1’s is the state of erased flash ram, and using that removes
the need to write to a block after erasing it.)
So that just saves some trouble, but it’s the same thing in offset terms.
Writing to SPI Flash and “Not Disk” MTD Flash
Don’t Do It
At least on the Pinebook Pro with it’s hard-coded boot order.
The first reason why anyone who’s a “noob” to this kind of thing (including me) shouldn’t do this on a Pinebook Pro is that SPI has the highest boot priority, and if it’s programmed with something that looks like it can boot, but instead hangs, then the PBP is hard bricked and can’t be unbricked without disabling the SPI flash chip somehow. This normally requires a soldering iron, since there’s no switch for it like there is with the eMMC.
SDcards Are Convenient
When testing out bootloaders on actual hardware, for the Pinebook Pro the easiest option is to erase the SPI (if there’s anything there, it’s blank from the factory), erase the firmware areas of the internal eMMC, and just always boot from SDCard while you’re tinkering.
That’s because the SDcard can be easily removed and reflashed, but it’s less convenient to disble the eMMC (internal switch) and even more inconvenient to disable the SPI flash (need soldering iron).
How It’s Done, Approximately
I don’t really want to provide directions for doing this mostly because I haven’t done it myself, but unlike eMMC and sdcard devices, things like SPI flash chips are usually programmed with specialized utilities like Linux MTD Utilities.
The reason for this is that “MTD” flash devices are “raw” and “dumb” flash chips that don’t do any automatic mapping of blocks for wear balancing, or automatic detection and re/unmapping of bad blocks. Software has to do that when programming them.
SDcards, USB flash drives, etc, all contain controllers that do this kind of stuff to create a translation layer in hardware that looks like a reliable block device. Under the hood, physical blocks are being remapped all over the place internally to avoid burning out specific physical blocks when the same logical blocks keep getting overwritten.
So trying to just dd to one of these devices would be a bad idea. If nothing
else, bad blocks wouldn’t be detected or dealt with.
Pinebook Pro SPI Info
Here are some links:
SPI Images
You will also notice that there are separate images built for SPI flash, like
u-boot-rockchip-spi.bin, which you might also notice is much smaller than
u-boot-rockchip.bin. That’s because it doesn’t have the same huge 0xFF hole in
the middle as the offsets are not the same for booting from SPI flash.
There’s a bit of a hole in that one, but not in the same location or as large:
0060680 6f74 2d72 616d 2d78 696d 7263 766f 6c6f
0060690 0074 6976 2d6e 7573 7070 796c 0000 0000
00606a0 0000 0000 0000 0000 0000 0000 0000 0000
*
0061000 ffff ffff ffff ffff ffff ffff ffff ffff
*
00e0000 0dd0 edfe 0000 000c 0000 3800 0000 b00a
00e0010 0000 2800 0000 1100 0000 0200 0000 0000
00e0020 0000 bf00 0000 780a 0000 0000 0000 0000Finishing Up
That’s it for this episode.
In the next episode, I’ll go over booting u-boot in qemu (though it won’t be exactly the same pinebook pro rockchip image).