Any Linux distro on NVIDIA Jetson Orin with JetPack 6

NVIDIA has just released the Jetpack 6 Developer Preview for the NVIDIA Jetson Orin hardware. The thing that is most exciting about this release is they finally support the ability to use upstream kernels and other Linux distributions. This means you can start to use both RHEL (9.3 and later) and Fedora on the Jetson Orin hardware! This has been a LONG time coming, something I’ve been involved with for 5 years!

So while this is a developer preview, AKA public Beta, it’s still very usable and for people that are interested in using other Linux distributions now is the time to get stuck in. Like all things it’s not perfect and there’s still work to be done, but many hands do make light work!

You start by downloading the BSP from there you can follow the following instructions and you should end up with a device you can easily install Fedora 39 or RHEL 9.3 or other distros with the appropriate bits enabled.

To flash the firmware you need to follow the Orin AGX guide for recovery and cabling, for Orin NX/nano you need to use the HW pins near the mSD card, to put the device in recovery mode and cabling then do the following for Orin AGX:

$ tar zvf Jetson_Linux_R36.2.0_aarch64.tbz2
$ cd Linux_for_Tegra/
$ lsusb|grep -i nv
Bus 003 Device 044: ID 0955:7045 NVIDIA Corp. [unknown]
Bus 003 Device 045: ID 0955:7023 NVIDIA Corp. [unknown]
$ sudo ./flash.sh p3737-0000-p3701-0000-qspi external
removed a lot of output
*** The target generic has been flashed successfully. ***
Make the target filesystem available to the device and reset the board to boot from external external.
$

The command for other Orin devices such as NX and Nano will be similar, you’ll just have to swap the p3737 variable, eg for Orin Nano use: sudo ./flash.sh p3737-0000-p3701-0000-qspi external.

Once the flash completes the device will reboot and you will be able to use the usual mechanisms to install your OS, whether the RHEL or Fedora installers or a Fedora Arm image. I’ve tested running OSes off both the microSD and a NVME card, plus installing off USB, the DisplayPort output should work in EFI console mode. The firmware is based upon the widely known TianoCore/EDK2 so the firmware interface should be straight forward. For those that may need a serial console if it’s not automatically detected you can use console=ttyAMA0,115200, This runs off the microUSB port on /dev/ttyACM2 on the host device.

For hardware vendors that have hardware based on the NVIDIA Orin hardware they will be able to adopt and make this available to their customers that may wish to run distributions other than L4T. If they are unsure feel free to reach out to me in the usual locations.

Using fwupdmgr to update NVME firmware

The fabulous fwupdmgr provides the ability to easily update firmware that is published to Linux Vendor Firmware Service (LVFS) but it can also be used to apply updates that aren’t necessarily in LVFS. One type of firmware that it supports updating is NVME firmware, that’s basically any NMVE, because the standard specifies a standardised mechanism for updating the firmware on all NVME devices.

I had a need to update a NVME firmware in an aarch64 device to see if it fixed an issue I was seeing. The Crucial P2 supported options were of course x86 only. The ISO download actually contained a little LinuxOS in an initrd on the .iso. The advice from Richard the fwupd technical lead was to “Look for a ~4mb high entropy blob” so mounting it up, I mounted the iso, extracted the initrd, and then used fwupdmmgr to apply the new firmware.

Find the NVME and check the firmware version:

$ cat /sys/class/nvme/nvme0/firmware_rev 
P2CR010 

So once I’d downloaded the update file I did the following to extract and update the firmware. Note I did this all as root, you can do most of it as non root.

# unzip iso_p2cr012.zip
# mount -o loop iso_p2cr012.iso /mnt/
# mkdir ~/tmp
# cp /mnt/boot/corepure64.gz tmp/
# cd tmp
# gunzip corepure64.gz
# cpio -iv < corepure64
# fwupdtool install-blob opt/firmware/P2CR012/1.bin
Loading…                 [-                                      ]
Loading…                 [-                                      ]
Choose a device:
0.	Cancel
1.	71b677ca0f1bc2c5b804fa1d59e52064ce589293 (CT250P2SSD8)
2.	2270d251f7c1dc37a29a2aa720a566aa0fa0ecde (spi1.0)
1
Waiting…                 [************************************** ] Less than one minute remaining…
An update requires a reboot to complete. Restart now? [y|N]: y

And away it goes, a reboot later and did it work?

$ cat /sys/class/nvme/nvme0/firmware_rev 
P2CR012

YES!!

HW video offload on Fedora Arm

There’s been the starting pieces of hardware video offload with the stateless engine support for some time and it now supports at least H264/HEVC/VP8/VP9/mepg2 decode offload depending on the hardware capabilities. The problem has been support for software/userspace has taken longer then the initial drivers but now that’s catching up now with gstreamer support landing in 2020 and with apps like clapper now using it. I’ve been meaning to play with this and work out how to make it work in Fedora as it’s useful for devices based on the AllWinner/Rockchip/NXP i.MX8 devices like the Pine64 laptops/phones plus a bunch of other devices, even NVIDIA Jetson devices should work before long.

You’ll need to ensure you have gstreamer1-plugins-bad-free installed and the video application I was testing with is clapper:

$ sudo dnf install -y gstreamer1-plugins-bad-free clapper

Seeing what hardware offload is supported:

$ gst-inspect-1.0 v4l2codecs
Plugin Details:
  Name                     v4l2codecs
  Description              V4L2 CODEC Accelerators plugin
  Filename                 /usr/lib64/gstreamer-1.0/libgstv4l2codecs.so
  Version                  1.20.0
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2022-02-03
  Binary package           Fedora GStreamer-plugins-bad package
  Origin URL               http://download.fedoraproject.org

  v4l2slh264dec: V4L2 Stateless H.264 Video Decoder
  v4l2slmpeg2dec: V4L2 Stateless Mpeg2 Video Decoder
  v4l2slvp8alphadecodebin: VP8 Alpha Decoder
  v4l2slvp8dec: V4L2 Stateless VP8 Video Decoder
  v4l2slvp9alphadecodebin: VP9 Alpha Decoder
  v4l2slvp9dec: V4L2 Stateless VP9 Video Decoder

  6 features:
  +-- 6 elements

Finally in Clapper you need to enable playbin3 option, I also enabled Pipewire audio support:
Clapper Preferences

We will also be enabling decode support in Chromium and Chromium freeworld before long, there’s a little more work to do here, but as usual once it lands it’ll all just start to work in Chromium too!

Short history of ARMv7/armhfp/arm32 in Fedora

Back in mid November I proposed a change for Fedora 37 to retire ARMv7 as an architecture, FESCo accepted the proposal. Per the Fedora 36 schedule we branched Fedora 36 this week. Last night I enacted the last of the process to disable it in rawhide so to quote “It’s dead Jim”. The last release of Fedora to support ARMv7 AKA armhfp AKA arm32 will be Fedora 36 which will go end of life around June 2023.

I thought I’d cover a few of the things we achieved with Fedora ARM and some of the impact it’s had on the wider Linux on ARM ecosystem which people may not have realised.

First a little bit of ARM history in the Fedora ecosystem. The beginnings of ARM support actually precedes Fedora all the way back to 1998 with a fork of Red Hat Linux 4.2 and more officially with Red Hat Linux 5.1 on the Corel Netwinder (I always wanted one of those but they weren’t available in Aus).

In Fedora itself the earliest details I remember was that Marvell bootstrapped ARMv5 in Fedora 7 and continued to build and support it through to Fedora 12. This “software architecture” was known as softfp. It was optimised for the ARMv5 architecture which didn’t have a hard requirement on a floating point unit so emulated it when it was needed hence “software floating point”. In Fedora 13 Seneca College took over the ARMv5 infrastructure and building from Marvell. I officially got involved in the Fedora 14 build process and soon after was also contracted by OLPC to drive Fedora on OLPC for their ARM based XO laptops as well as work on their i686 devices to have a single OS for all of them.

In mid 2011, the Fedora 15 timeframe, a small Red Hat team started to do a ARMv7 hard floating point, AKA hardfp or armhfp, bootstrap as ARM’s new v7 mandated a floating point unit. The bootstrap included the core toolchain (binutils/gcc/glibc/elfutils and friends) and ultimately the entire distribution, I drove this effort from a community, build and packaging perspective. This required 100s of patches to upstream projects that made many assumptions about ARM only being softfp, but it also allowed us at the time to fix many general architecture assumptions in these projects. The hard floating point bootstrapping was useful for the wider community too, it was used by Nokia as the base of it’s hardfp efforts for Maemo, plus other distros used it as as it’s much easier/quicker if you already have a full distro running the architecture you wish to boostrap. What wasn’t generally known at the time was also the first new architecture that has been bootstrapped in the Fedora/RHEL ecosystem since x86_64 a long time before and it allowed Red Hat to refresh it’s memory on how to do this in preparation of the then unannounced aarch64 architecture and the POWER Little Endian intentions, basically it provided a cover story. We also worked to get other languages such as Fortran, golang, rust and others building and working on armhfp and those other architectures. The final piece of this was ARMv7 being promoted to a primary Fedora architecture in Fedora 20. This then later went on to my proposal to redefine secondary architectures in Fedora.

In the wider community of Linux Fedora ARM was the first distribution to adopt the kernel “multi platform” work enabling us to go from building 5 different kernels to support a handful of arm devices to a single kernel supporting 100s of devices in a very short period of time. I worked with closely Arnd Bergmann from Linaro on issues with the early pieces of the multiplatform work. In upstream U-Boot we posted the first distro_boot patches to support booting Linux in the same way across all the devices we actively supported so we didn’t need specially wrapped kernels and know exact offsets for every SoC or device. The distro_boot support evolved, working with SUSE, into UEFI support in U-Boot further standardising the ARM boot process by abstracting the pieces that were different and letting the firmware deal with them. This work ultimately evolved into EBBR and the ARM System Ready IR spec. In Fedora 34 we moved to soley supporting UEFI on both ARM architectures. A lot of Linux distros still have specific kernels for each device and use non standard boot methods for devices and hence have an image for each device/use-case they wish to use. This was something Fedora identified very early on as something that would not scale!

Fedora also leads a lot of things in the gcc toolchain stack across all our supported architectures, we’ve actively enabled a lot of security features and other things like LTO early on. As the Fedora gcc maintainers, employed by Red Hat, are also key upstream GCC maintainers we’re almost always the first distribution to rebase onto a new release before it’s a stable release, for example Fedora 36 had just had a mass rebuild against a gcc-12 pre release snapshot. This builds all of the 50k or more source packages with the pre-release of the new toolchain making for a much better release for the wider GCC community because this picks up a number of bugs/regressions in both the general support but also in the architectures Fedora supports which means the ARMv7 hardfp support in GCC has benefited from 100s of bugs we’ve detected in gcc/binutils/glibc etc before they land in a stable gcc release. With the retirement of ARMv7 in Fedora this is going to be something the wider ARMv7 community is going to have to pick up post the GCC-12 release.

Over the subsequent 11 years of ARMv7 support in Fedora, and much longer if you include the early ARMv5 the distribution has also enabled a number of other innovative features like support for containers, support for devices like the Raspberry Pi 2 and 3 in Fedora 25. as mentioned various toolchains, and fun things like robots. Of course we also lose some things too. Devices like the BeagleBone don’t yet have a 64 bit sibling, but there’s less and less 32 bit devices coming out and the use of armhfp is waning quickly and the maintenance cost is rising as the industry moves more generally to 64 bit even in embedded use cases and the fact is with devices like the $15 Raspberry Pi Zero 2W it makes less and less sense even if I do still actively run BeagleBones, a Panda-ES and 3 different i.MX6 devices.

So I engaged with the wider Arm ecosystem and it made sense to finally sunset our ARMv7 32 bit support. We’re of course leaving it in good shape with things like gcc-12, the latest rust and golang toolchains and 5.17 kernels, much newer by the time F-36 goes EOL in June 2023, it will be in good shape if people wish to use it as the basis of some form of continuing ARMv7 supported Linux distribution.

Sail off into the sunset friend, it’s been a fun 12 years of hacking on those projects!

Fedora on NVIDIA Jetson Xavier

The last two years or so I’ve been working with NVIDIA on general distro support including UEFI and ACPI for their Jetson Xavier platforms. Their Xavier platform, except a few quirks, are mostly SystemReady-ES compliant, so having a SBBR compliant firmware goes quite some way to having a widely available, relatively affordable, platform that “just works” for the arm ecosystem. I was very excited to finally have NVIDIA finally release the first version in March this year. This firmware is a standard UEFI firmware based on the open source TianoCore/EDK2 reference firmware, it allows booting in either ACPI or Device-Tree mode and supports all the basic things needed. The ACPI mode is not as fully featured as the Device-Tree mode as yet. In ACPI you get compute (cpu/memory/virt etc), PCIe, USB, network, which is just fine if you’re just looking for standard server or for testing a SystemReady system but there’s no display or accelerator support as yet. The Device-Tree mode is more feature full but both work pretty well with upstream kernels and NVIDIA are improving and upstreaming more things regularly.

For flashing with the latest Fedora releases you’ll want the Linux for Tegra (L4T) R32.6.1 release and the latest UEFI firmware (1.1.2 ATM). The R32.6.1 release fixes issues with python3.9 and later so you’ll need that for Fedora. The following will extract everything into a directory called Linux_for_Tegra. Note the release for Xavier is different to the L4T for the TX1/TX2 series of devices such as the nano.

$ tar xvf Jetson_Linux_R32.6.1_aarch64.tbz2
$ tar xvf nvidia-l4t-jetson-uefi-R32.6.1-20211119125725.tbz2
$ cd Linux_for_Tegra

To flash either the Xavier AGX or NX you need to put them into recovery mode and connect a USB cable, USB-C for AGX or micro-USB for NX. Once you’re in recovery mode you can flash them.

For the Xavier AGX:

$ lsusb | grep -i NV
Bus 001 Device 086: ID 0955:7019 NVIDIA Corp. APX
$ sudo ./flash.sh jetson-xavier-uefi-min external

For the Xavier NX:

$ lsusb | grep -i NV
Bus 001 Device 089: ID 0955:7e19 NVIDIA Corp. APX
$ sudo ./flash.sh jetson-xavier-nx-uefi-acpi internal

There will be a bunch of output and it will eventually return to the prompt and reset the device. You can now install Fedora on the device. You can use any of the pre-canned aarch64 image or traditional installer available from the fedora website. When running in ACPI mode you don’t get display output so you’ll need to use a serial console, in both ACPI and Device-Tree mode there’s not currently support for accelerated GPU graphics/AI/ML support. If you want to be able to easily switch between ACPI/Device-Tree modes you’ll want to install the dracut-config-generic package to have a generic initrd to make it easy to reboot between both modes.

SystemReady ES support for MacchiatoBin

I’ve had a MacchiatoBin Double Shot board for some time. It runs various services for my local network and generally just works. I run a TianoCore EDK2 firmware on it using ACPI. It’s purely a network device so I don’t bother with any form of graphics and in the very few occasions I need to access it locally I do so via the built in USB serial TTL.

Recently Solid Run announced the MacciatoBin is now SystemReady ES certified. Excellent news! I’ve worked with Arm for some time on both the SytemReady ES (Embedded Server) and SystemReady IR (IoT Ready) standards and recently the certification program has been finalised so it’s nice to start to see the fruits from all the hard work myself, and may others, have done over a number of years appear.

The EDK2 firmware I was running was coming up to two years old and there’s been a number of enhancements to the various components of the firmwares that make up a complete update so I decided to download the latest firmware and update it. Eventually I am sure Solid Run will have these published to LVFS to make the process even easier but I know that to get to this stage has been a LOT of effort so it’s still a great step forward.

The first step of updating a EDK2 firmware is to download it and put it on the EFI partition:

peter@macbin:~ $ wget https://github.com/Semihalf/edk2-platforms/wiki/releases/flash-image-a8k-mcbin.bin_r20210630
peter@macbin:~ $ sudo mv flash-image-a8k-mcbin.bin_r20210630 /boot/efi
peter@macbin:~ $ sudo reboot

On reboot you’re given a prompt to interrupt the boot process. From the menu select the option for the shell:

Shell> fs0:
FS0:\> ls
Directory of: FS0:\
04/06/2021  19:08          4,096  EFI
07/25/2021  17:01           2,855,040  flash-image-a8k-mcbin.bin_r20210630
          1 File(s)   2,855,040 bytes
          1 Dir(s)

FS0:\> fupdate flash-image-a8k-mcbin.bin_r20210630
Detected w25q32bv SPI NOR flash with page size 256 B, erase size 4 KB, total 4 MB
Updating, 99%
fupdate: Update 2855040 bytes at offset 0x0 succeeded!
FS0:\> reset

It then reboots and we’re done, you see a very similar output to previously with some updated versions of various firmware and before long you’re back through grub and running Fedora again. Painless!

I’m really happy to see this is such a straightforward process, and I’m looking forward to seeing more features, enhancements and fixes to the firmware including capsule updates and the associated LVFS/fwupdmgr support, and improvements around firmware security (fwupdmgr –force security). Top marks to the Solid Run team!

Fedora on the Pinebook Pro

First thing to note here is that this is not limited to the Pinebook Pro, I’m just using it as the example for 64 bit Rockchip devices with SPI flash on Fedora. This post is focused on devices with SPI but I’ll do a separate follow-up post for other devices including details for writing to eMMC over USB.

The story of Fedora on the Pinebook Pro, and other Rockchip devices, has been a sordid story of a lack of time, bugs, rabbit holes, more bugs and various other things. Not at all sordid at all really, mostly just a lack of time on my behalf, and nobody else stepping up to assist in a way to benefit all Fedora users, mostly they do one time hacks to sort themselves. Overall the support in Fedora for Rockchip devices has been quite solid for a number of releases. The problem has been with the early boot firmware, notable because without SPI flash it wants to splat itself across the first 8Mb of the disk, and if there was SPI flash it generally wasn’t overly stable/straight forward.

Anyway we’re now in a place where devices with SPI flash should mostly work just fine, those devices without it will work with a little manual intervention, and while the support isn’t complete, and will need more polish, they’re all details we can polish with little interruption to users by standard package updates. By default users will have accelerated graphics and from my testing on GNOME 40 it’s by all accounts a pretty decent experience!

Setting up the firmware

First step is to get the firmware written to SPI flash. This is a two step process, the first is to write out a micro SD card from another device, the second is to boot that mSD card on the Pinebook Pro, or another device like the Rockpro64, and write the firmware to the SPI flash.

There’s some nuances to this process, and the way the early boot firmware works, if another version of U-Boot takes precedence that is likely OK as it should still be able to work, the fall back is to use the internal switch to turn off the eMMC temporarily. I also have no idea if the Pine64 shipped U-Boot has any display output, the Fedora build does, if not you’ll need to use the option to disable eMMC or use a serial console cable. Anyway on to the steps:

Set up the mSD card
Use a mSD card that has no data you wish to keep, this process will wipe it out. You want at least U-Boot build 2021.04-3.fc34, you can adjust the umount to be more specific, and you need to substitute XXX for the media, otherwise it’s a relatively quick and straightforward process:

sudo dnf install --enablerepo=updates-testing -y arm-image-installer uboot-images-armv8
sudo umount /run/media//*
sudo spi-flashing-disk --target=pinebook-pro-rk3399 --media=/dev/XXX

Write the firmware to flash
Now remove the mSD card from your host and put it into the Pinebook Pro. Press the power button, from experience you likely need to press and momentarily hold and in a second or two the display will light up with text output. Interrupt the boot by pressing space. Next up we write out the flash:

Hit any key to stop autoboot:  0 
=> ls mmc 1:1
   167509   idbloader.img
   335872   idbloader.spi
   975872   u-boot.itb
  9331712   u-boot-rockchip.bin

4 file(s), 0 dir(s)

=> sf probe
SF: Detected gd25q128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB

=> load mmc 1:1 ${fdt_addr_r} idbloader.spi
335872 bytes read in 39 ms (8.2 MiB/s)

=> sf update ${fdt_addr_r} 0 ${filesize}
device 0 offset 0x0, size 0x52000
61440 bytes written, 274432 bytes skipped in 0.803s, speed 427777 B/s

=> load mmc 1:1 ${fdt_addr_r} u-boot.itb
975872 bytes read in 107 ms (8.7 MiB/s)

=> sf update ${fdt_addr_r} 60000 ${filesize}
device 0 offset 0x60000, size 0xee400
914432 bytes written, 61440 bytes skipped in 9.415s, speed 106127 B/s

Once the last command above has completed eject the mSD card and type reset at the => prompt and the device should reboot and you should see output similar to before but running from the SPI flash!

If you had to turn off the eMMC you can now turn it back on.

Installing Fedora

The nice thing with the firmware on SPI flash it should now work mostly like any other laptop and you can use either the pre canned desktop images (Workstation, KDE, XFCE, Sugar), the Workstation LiveCD iso or the standard everything network installer.

To run the arm Workstation image off a micro SD card or USB stick you can do the following:

arm-image-installer --media=/dev/XXX --resizefs --target=none --image=Fedora-Workstation-34-1.2.aarch64.raw.xz

Note ATM you’ll need to use the USB port on the right hand side, I need to investigate the USB/USB-C port on the left as it appears not to currently work in firmware, but works fine once Fedora is running.

Next steps and improvements

The two biggest issues remaining for the Pinebook Pro is enabling PCIe support (supported from June 2021) and the lack of the brcmfmac firmware, both WiFi and bluetooth, being upstream. For the later issue if there’s anyone from Synaptics that can assist in resolving that problem please reach out to me! A interim WiFi firmware to use is here.

Some things at the Fedora level I’ve not really tested and will do so more, and likely polish with OS updates, in the coming weeks include sound, USB-C port (charging and display output). On the firmware level there’s still some more improvements to be done, tweaks to improve the USB support, turning on the power LED as early as possible to give an indicator, improvements to the EFI framebuffer to ensure consistent early boot output, support for UEFI BGRT to enable smooth boot etc.

For support please email the Fedora Arm mailing list or reach out on IRC via #fedora-arm on Libera.Chat.

Installing Fedora on the NVIDIA Jetson nano

Updated – Aug 2021
You now used the latest R32.6.1 release and it now works with the latest Python releases. Some minor edits below.

Overview
Nvidia launched the Jetson Nano Developer Kit in March 2019, since there there’s been a few minor refreshes including a just announced cheaper 2Gb model. I received the original 4Gb rev A device shortly after they were launched.

Over the last year or so as part of my role at Red Hat I started working with some of the NVidia Tegra team to improve support for the Jetson devices. This work has been wide ranging and though it’s taken awhile, with Fedora 33 we’re starting to see the fruits of that collaboration. The first is improved support for the Jetson Nano. The official L4T (Linux 4 Tegra) Jetson Nano images look a lot like an Android phone with numerous partitions across the mSD card. This makes it harder to support a generic Linux distribution like Fedora as there are assumptions by distributions of control they can have over the storage, so while it was certainly possible to get Fedora to run on these devices it generally wasn’t for the faint of heart. As of the recent L4T releases, you definitely want R32.4.4, it’s now a supported option to flash all the firmware to the onboard SPI flash enabling the use of the entire mSD card for the OS of your choice, which as we all know will be Fedora 😉 but the instructions here should be adaptable to work for any distribution.

Before we begin
We do it in two stages, first is to flash the new firmware to the SPI over the micro USB port, second we’ll prepare the Fedora OS for the mSD card. For the first stage you’ll need the latest L4T Release R32.6.1 and the Fedora U-Boot builds installed locally.

Before we get started you’ll need the following:

  • A USB-A to micro USB cable for flashing
  • A HDMI monitor and a USB keyboard
  • A jumper, a jumper wire or something to close the connection on the FRC pins for recovery mode
  • A 3.3v USB Serial TTY (optional)
  • An appropriate 5v barrel PSU (optional)

If you wish to use a serial TTY there’s a good guide here for connecting it to the RevA nano, the RevB has two camera connectors so they’ve moved the serial console headers to near the mSD card slot. The command to see serial output is:

screen /dev/ttyUSB0 115200

Flashing the Jetson Nano
So let’s get started with flashing the firmware. This step with the firmware on the SPI doesn’t have to be done often. First we’ll extract the L4T release and get all the bits installed that we need to flash the firmware:

sudo dnf install -y usbutils uboot-images-armv8 arm-image-installer
tar xvf ~/Downloads/Jetson-210_Linux_R32.6.1_aarch64.tbz2
cd Linux_for_Tegra
cp /usr/share/uboot/p3450-0000/u-boot.bin bootloader/t210ref/p3450-0000/

Next, based on instructions from the NVidia Jetson Nano Quick Start Guide, we need to put the Jetson Nano into Force Recovery Mode (FRC) to prepare for flashing the firmware:

  1. Ensure that your Jetson Nano Developer Kit is powered off. There’s no need for a mSD card ATM, we’re just writing to the SPI flash.
  2. Connect the Micro-USB OTG cable to the Micro USB port on the Nano. Don’t plug it into the host computer just yet.
  3. Enable Force Recovery mode by placing a jumper across the FRC pins of the Button Header on the carrier board.
    a. For carrier board revision A02, these are pins 3 and 4 of Button Header (J40) which is located near the camera header.
    b. For carrier board revision B01, these are pins 9 and 10 of Button Header (J50), which is located on the edge of the carrier board under the Jetson module.
  4. Only if you wish to use a separate PSU place a jumper across J48 to enable use of a DC power adapter.
  5. Connect a DC power adapter to J25. The developer kit powers on automatically and enters Force Recovery mode. Note it may be possible to do this with USB power but I’ve not tested it.
  6. Remove the jumper from the FRC pins of the Button Header.
  7. See if you can see the Jetson Nano is in recovery mode by running:
    lsusb | grep -i nvidia

Now we can actually flash the firmware (make sure you’re still in the Linux_for_Tegra directory):

sudo ./flash.sh p3448-0000-max-spi external

You will see a lot of output as the command runs, and if you have a serial TTY you’ll see some output there but eventually you’ll be returned to the command prompt and the system will reset. If you have a HDMI monitor attached you’ll see the NVidia logo pop up, if you have a serial console you’ll see a bunch of output and eventually the output of U-Boot and the associated U-Boot prompt.

Jetson TX1 and TX2
You can basically follow the same instructions above for the older TX1/TX2 devices except for two things. For the TX1 you can use the same L4T release, for the TX2 you need to download a different L4T release.

For the U-Boot copy there’s a different U-Boot for each device which needs to be copied to a different location. For the firmware copy I treat the eMMC as if it was the SPI flash, and run the OS off a SD card, it’s not the most efficient but it keeps things more straight forward:

TX1:

cp /usr/share/uboot/p2371-2180/u-boot* bootloader/t210ref/p2371-2180/
sudo ./flash.sh jetson-tx1 mmcblk0p1

TX2:

cp /usr/share/uboot/p2771-0000-500/* bootloader/t186ref/p2771-0000/500/
sudo ./flash.sh jetson-tx2 mmcblk0p1

Getting Fedora running
Now we have the firmware flashed we can prepare Fedora for the mSD card. Download the Fedora Workstation for aarch64 raw image. You can of course also use XFCE, Minimal or Server images. Put the mSD card in reader and after unmounting any filesystem run the following command (look at the help for other options around users/ssh-keys):

sudo arm-image-installer --media=/dev/XXX --resizefs --target=none --image=~/Downloads/Fedora-Workstation-33-1.3.aarch64.raw.xz

Note you need to replace XXX with the right device, and you don’t need a target option as we’re not writing the firmware to the mSD card.

Once that completes you should be able to pop the mSD card into your Jetson Nano and reset the device and see it boot. You will see all the output if you have a serial console attached. If you’re using HDMI it may take a little while once the NVidia logo disappears for the GNOME first user setup to appear.

Also note that while a lot of things work on this device, like the nouveau driver for display, it’s not perfect yet and we’re actively working to fix and improve the support for the Jetson Nano, most of these will come via the standard Fedora update mechanism. If you have queries please engage in the usual ways via the mailing list or #fedora-arm on Libera.Chat.

Documenting my various arm and IoT devices: quick overview

It’s been around ten years since I got my first arm single board computer, a Beagle-xM, which started me down the route of playing with Fedora on ARM and ultimately to my role in device edge/IoT at Red Hat. Shortly after that time I also moved into my current flat, almost ten years later I finally made the decision to move to a new place.

In the process of unpacking the contents of boxes from the flat into my new home office I thought I would document all my devices. This is mostly for my own reference, but I have little doubt others are interested from previous conversations. I’ve broken the list down into a few broad categories, mostly so the blog post isn’t unwieldy, there’s certainly cross overs between the categories, like some generations of the Raspberry Pi can run in either 32 or 64 bit mode, some Arm SBCs also have an integrated micro controller etc. For simplicity I’m putting those cross over devices in a single list, that of which they’re most capable, I’m also not putting devices on the list that aren’t easily able to run an open source OS such as Linux or Zephyr RTOS as I have numerous micro controllers/phones etc I can’t be bothered with and hence they’re not seen as useful for this list.

The lists, I will update with links as I post them, are going to be as follows:

  • Part one: Arm 64 bit devices (aarch64)
  • Part two: Arm 32 bit devices (ARMv7)
  • Part three: Micro controllers
  • Part four: x86 and other devices

Three ways to speed up dnf on arm devices

I have a large bunch of Arm Single Board Computers I use for testing a lot. Most of the testing ends up being pretty basic stuff like firmware, kernels, and the various bits of hardware peripherals that people use like storage, network, display and sound output, plus things like sensors and HAT support.

The problem is that these devices often aren’t the fastest in the world for various reasons so I want to be able to apply updates to the basic system as quickly as possible to find out the results. Over time I’ve worked out that these three things speed up dnf quite a bit for the sort of testing I wish to do are as follows:

  1. Disable modularity:
    sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/fe*mod*
  2. Don’t install weak dependencies:
    echo "install_weak_deps=False" >> /etc/dnf/dnf.conf
  3. Disable dnf makecache. It never seems to be up to date when you need it anyway:
    systemctl disable dnf-makecache; systemctl mask dnf-makecache

You may need to re-do some of these each major update as they seem to want to force you to have them every time.