Installing Fedora on the NVIDIA Jetson nano

Nvidia launched the Jetson Nano Developer Kit in March 2019, since there there’s been a few minor refreshes including a just announced cheaper 2Gb model. I received the original 4Gb rev A device shortly after they were launched.

Over the last year or so as part of my role at Red Hat I started working with some of the NVidia Tegra team to improve support for the Jetson devices. This work has been wide ranging and though it’s taken awhile, with Fedora 33 we’re starting to see the fruits of that collaboration. The first is improved support for the Jetson Nano. The official L4T (Linux 4 Tegra) Jetson Nano images look a lot like an Android phone with numerous partitions across the mSD card. This makes it harder to support a generic Linux distribution like Fedora as there are assumptions by distributions of control they can have over the storage, so while it was certainly possible to get Fedora to run on these devices it generally wasn’t for the faint of heart. As of the recent L4T releases, you definitely want R32.4.4, it’s now a supported option to flash all the firmware to the onboard SPI flash enabling the use of the entire mSD card for the OS of your choice, which as we all know will be Fedora 😉 but the instructions here should be adaptable to work for any distribution.

We do it in two stages, first is to flash the new firmware to the SPI over the micr USB port, second we’ll prepare the Fedora OS for the mSD card. For the first stage you’ll need the latest L4T Release R32.4.4 and the Fedora U-Boot builds installed locally.

Before we get started you’ll need the following:

  • A USB-A to micro USB cable for flashing
  • A HDMI monitor and a USB keyboard
  • A jumper, a jumper wire or something to close the connection on the FRC pins for recovery mode
  • A 3.3v USB Serial TTY (optional)
  • An appropriate 5v barrel PSU (optional)

If you wish to use a serial TTY there’s a good guide here for connecting it to the RevA nano, the RevB has two camera connectors so they’ve moved the serial console headers to near the mSD card slot. The command to see serial output is:

screen /dev/ttyUSB0 115200

So let’s get started with flashing the firmware. This step with the firmware on the SPI doesn’t have to be done often. First we’ll extract the L4T release and get all the bits installed that we need to flash the firmware:

sudo dnf install -y usbutils uboot-images-armv8 arm-image-installer
tar xvf ~/Downloads/Tegra210_Linux_R32.4.4_aarch64.tbz2
cd Linux_for_Tegra
cp /usr/share/uboot/p3450-0000/u-boot.bin bootloader/t210ref/p3450-porg/

Next, based on instructions from the NVidia Jetson Nano Quick Start Guide, we need to put the Jetson Nano into Force Recovery Mode (FRC) to prepare for flashing the firmware:

  1. Ensure that your Jetson Nano Developer Kit is powered off. There’s no need for a mSD card ATM, we’re just writing to the SPI flash.
  2. Connect the Micro-USB OTG cable to the Micro USB port on the Nano. Don’t plug it into the host computer just yet.
  3. Enable Force Recovery mode by placing a jumper across the FRC pins of the Button Header on the carrier board.
    a. For carrier board revision A02, these are pins 3 and 4 of Button Header (J40) which is located near the camera header.
    b. For carrier board revision B01, these are pins 9 and 10 of Button Header (J50), which is located on the edge of the carrier board under the Jetson module.
  4. Only if you wish to use a separate PSU place a jumper across J48 to enable use of a DC power adapter.
  5. Connect a DC power adapter to J25. The developer kit powers on automatically and enters Force Recovery mode. Note it may be possible to do this with USB power but I’ve not tested it.
  6. Remove the jumper from the FRC pins of the Button Header.
  7. See if you can see the Jetson Nano is in recovery mode by running:
    lsusb | grep -i nvidia

Now we can actually flash the firmware (make sure you’re still in the Linux_for_Tegra directory):

sudo ./flash.sh p3448-0000-max-spi external

You will see a lot of output as the command runs, and if you have a serial TTY you’ll see some output there but eventually you’ll be returned to the command prompt and the system will reset. If you have a HDMI monitor attached you’ll see the NVidia logo pop up, if you have a serial console you’ll see a bunch of output and eventually the output of U-Boot and the associated U-Boot prompt.

Now we have the firmware flashed we can prepare Fedora for the mSD card. Download the Fedora Workstation for aarch64 raw image. You can of course also use XFCE, Minimal or Server images. Put the mSD card in reader and after unmounting any filesystem run the following command (look at the help for other options around users/ssh-keys):

sudo arm-image-installer --media=/dev/XXX --resizefs --target=none --image=~/Downloads/Fedora-Workstation-33-1.3.aarch64.raw.xz

Note you need to replace XXX with the right device, and you don’t need a target option as we’re not writing the firmware to the mSD card.

Once that completes you should be able to pop the mSD card into your Jetson Nano and reset the device and see it boot. You will see all the output if you have a serial console attached. If you’re using HDMI it may take a little while once the NVidia logo disappears for the GNOME first user setup to appear.

Also note that while a lot of things work on this device, like the nouveau driver for display, it’s not perfect yet and we’re actively working to fix and improve the support for the Jetson Nano, most of these will come via the standard Fedora update mechanism. If you have queries please engage in the usual ways via the mailing list or #fedora-arm on Freenode.

Three ways to speed up dnf on arm devices

I have a large bunch of Arm Single Board Computers I use for testing a lot. Most of the testing ends up being pretty basic stuff like firmware, kernels, and the various bits of hardware peripherals that people use like storage, network, display and sound output, plus things like sensors and HAT support.

The problem is that these devices often aren’t the fastest in the world for various reasons so I want to be able to apply updates to the basic system as quickly as possible to find out the results. Over time I’ve worked out that these three things speed up dnf quite a bit for the sort of testing I wish to do are as follows:

  1. Disable modularity:
    sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/fe*mod*
  2. Don’t install weak dependencies:
    echo "install_weak_deps=False" >> /etc/dnf/dnf.conf
  3. Disable dnf makecache. It never seems to be up to date when you need it anyway:
    systemctl disable dnf-makecache; systemctl mask dnf-makecache

You may need to re-do some of these each major update as they seem to want to force you to have them every time.

Fedora 21 and ARM device support

As we slowly meander our way towards the pointy end of the Fedora 21 release, with Alpha speeding up in the rear view mirror, the Fedora ARM team are starting to discuss the best way to deal with the blossoming amount of ARMv7 devices that can and do run out of the box on Fedora.

With our 3.16 kernel containing device tree blobs for 200+ devices, the Fedora 3.17 rawhide kernel already containing 230+, it’s truly impossible to actively test and support all of those devices. So much like previous releases we’ll be focusing on testing a group of “primary devices” with the remainder being considered as secondary. This doesn’t mean they won’t work, it just means they’re not necessarily a testing focus of the regular contributors or they might not be readily available to purchase.

So what makes a device primary? Well there’s a number of considerations we’ve put into the list. Firstly the device has to be widely available and well supported upstream. Some will notice that some of the devices are no longer widely available (yes Panda, Trimslice and Calxeda I’m looking at you!) and I did consider their removal from the list but given a lot of contributors have them I think it’s worthwhile keeping them around for the moment. The primary devices list won’t be release blocking, we don’t block x86 releases for specific single devices, so I don’t believe we should for ARMv7 either.

Astute readers will notice the proposed primary list of around two dozen devices is much larger than the core devices we supported in Fedora 20! YAY! is all I have to say about that 🙂

The list is not final, at the moment it’s a suggested list and one open for discussion to some degree and what we’ll be heading from Alpha to Beta with. I fully expect it to be tweaked as we go along, there might be cool new shiny Chromebooks 😉 that arrive on the scene and end up working nicely and are hence worth actively supporting (no EXYNOS Chromebooks I’m not looking at you!) and some devices on the list below that end up not making the grade. One thing is for sure the grade includes that they support Cute Embedded Nonsense Hacks ie DeviceTree… there’s no board support here.

Primary:

  • Wandboard (all models/revisions)
  • Utilite (all models)
  • Cubox-i (all models)
  • Hummingboard (all models)
  • RIoTBoard
  • BeagleBone Black
  • Tegra K1 Jetson
  • CubieBoard (all models)
  • Banana Pi
  • Trimslice
  • PandaBoard (all models)
  • Calxeda Highbank/Midway
  • VExpress (qemu)

Secondary:

  • BeagleBone White
  • Beagle xM
  • Novena
  • UDOO
  • AC100
  • Qualcomm (IFC6410, DragonBoard)
  • Various Marvell devices (Mirabox, AX3, CuBox)
  • Various Exynos devices
  • Other AllWinner devices (as per available u-boot/DT support)
  • Gumstix Overo series of devices
  • OMAP5 EVM
  • STE SnowBall

ARM64:

  • AMD Seattle
  • APM Mustang
  • VExpress (qemu)

So what can a user expect from the primary devices above? Will all the functionality of a device work? Well it depends on the specific device and the associated SoC. For example the AllWinner SoC GPU support is far from upstream so unfortunately there will be no graphical UX for those devices, the Tegra K1 support for the GPU isn’t quite there yet but we’re hoping by GA it will be. Some will be better than others in terms of certain features but for example the AllWinner devices would make good storage devices with their SoC attached SATA and Network, no ugly usb storage/network here, so they are useful to support as a primary device and can easily have feature enablement in the F-21 cycle with a “yum upgrade” to a newer kernel.

We’ll delve deeper into the specifics of each device and the final list closer to beta.

3.16 Fedora ARM kernel status

So 3.16 is has quite a few new features in terms of newly supported devices, also some what surprisingly this blog post will be out before 3.16! In terms of new device support all the SoCs listed here are exciting for a number of reasons for Fedora ARM. Aarch64 (ARM64) makes it’s first debut with support of real hardware although we’ve actually had kernel support enable for it for some time in Fedora even if only usable on the glacial Foundation emulator.

The 3.16 release is also very likely to be the kernel that ships with Fedora 21 GA and with the Alpha due in about a month we’re starting to polish and test all the platforms and devices we want to support for GA.

Anyway without any further a do let’s get into the gritty details:

  • NVIDIA Jetson TK1 support: While we’ve had the basics of this for a while all of the bits are there now.
  • EXYNOS support: This SoC is probably the most asked about platform and finally after a long wait the multiplatform support has landed upstream. We currently ship around 20 dtb files for exynos4 and 5 (Chromebook support anyone?). Testing is sought and feedback and greatly appreciated.
  • Qualcomm MSM 8×60, 8960 and 8974 support: While the multiplatform support for these devices landed upstream a few releases a go they’re now to the point they should be relatively usable so it’s time to get wider testing. This should be the beginning of supporting the venerable ifc6410 and dragonboard devices.
  • APM X-GENE support: One of our first aarch64 supported pieces of hardware. Similar to the QCom SoC the initial support has been upstream for a while but with 3.16 it becomes usable with the vast majority of basic support upstream so minimal patches are needed. More on aarch64 soon.
  • AMD Seattle support: The other of our aarch64 supported pieces of hardware if you’re lucky enough to get your mits on a device.

The other feature we’re starting to see mature is GPU and Graphics support. I’m not exactly sure yet as to what the final state of this functionality will be for Fedora 21 GA but we potentially will have suppport for:

  • nouveau/mesa support on the NVIDIA Tegra K1
  • freedreno/mesa support on the Qualcomm boards
  • etnaviv/mesa support on i.MX6 devices
  • improved modesetting support for a number of other devices. Some of this has already landed and is usable in rawhide now.

What covered above is just a high level overview of what’s new in the upcoming release. There’s been numerous other improvements in existing supported SoCs and devices all over the place that would take too long to cover off here but in short with all the shiny that’s landed in 3.16 what Fedora ARM will look like as part of the Fedora 21 GA release is quickly starting to take shape.

3.14 Fedora ARM kernel status

With 3.14.1 out Josh and Justin are preparing to land the 3.14 kernel into Fedora 20. So what does it mean in terms of ARM on Fedora. Well it’s an evolution. There’s of course the usual raft of new devices and some new SoCs, and best of all lots of improvements in support for existing devices. Even the return of some old favourites! Generally from the stash of devices Paul, I and others have that get regularly tested things are looking pretty good with 3.14.

New SoCs and device support:

  • Tegra 4/K1 support has been enabled. For 3.14 this doesn’t mean much as there’s not a wide level of devices out there that we ship device tree blobs for but it’s a good preparation for 3.15 as we should have pretty reasonable support for the nice new Tegra K1 dev board!
  • TI devices: The 3.14 finally brings working support for the OMAP5 EVM board, this will improve further in 3.15 but it boots and generally works. It also adds support for the original BeagleBone and the USB is now finally all back for the Beagle-XM devices so they go back into the fully working list! Also the DT bits for the Overo devices has started to land so if is interested in those I’d love feedback from people with those devices.
  • Freescale i.MX: To this lovely growing list we add initial support for the various Cubox-i devices and the hummingboard. Still no HDMI support but here’s hoping for 3.15
  • Xilinx Zynq 7000: The SD controller has finally landed for this which means they should be bootable to login but from there I’m not sure the status of the rest of the support. We ship DT for the zynq-zc702, zynq-zc706 and zynq-zed devices so if you’ve got one of those and have time to test feedback would be welcome.

Interesting bugs fixed:
Nothing particularly exciting comes to mind here. The fix for the Beagle-XM usb hub power up is nice, as was the final config option for the BeagleBone White. There’s obviously a deluge of other ARM driver fixes and improvements (PTP high precision time support for modular CPSW ethernet on the BeagleBone’s anyone?).

Outstanding bugs and issues:
So this is really the list of items I have outstanding for 3.14 so that I can spin some new images. Feedback on any of these and anything I might not be aware of would be very useful.

  • OMAP DRM display. In 3.11 a new display framework landed for the OMAP DRM driver and in 3.12 the old one was dropped. This broke X on devices like the Pandaboard for 3.12+ kernels. I know roughly the problematic area but I just need to get the cycles to debug this. Any help is welcome.
  • Serial over USB. While this isn’t a kernel bug but rather just needs me to hack together some scripts it’s a blocker for easy OOTB support for devices like the BeagleBone Black
  • Tegra DRM display. After a re-write it’s back and modular again in 3.14. Just need to ensure it’s working and ready to go
  • Testing… testing… testing 🙂

That’s mostly all I have on my list. If there’s anything I’m not aware of please do let me know and I’ll endevour to help out where possible. In particular I’m very interested in boot issues for devices that would would be supportable with new ARM images based on 3.14. From the 3.11 kernel that F-20 GA shipped with there’s been a lot of change and improvements, and while non boot enhancements are easy to do with a “yum update” issues with boot aren’t quite so easy to deal with!