Supporting Fedora 24 on the Raspberry Pi 2

So I get asked the question dozens of times a day so I thought I’d outline the answer to the question “When is Fedora going to support the Raspberry Pi 2?” and “The kernel support is upstream in the 4.5 kernel, why isn’t it enabled in Fedora 24?”

Ultimately support in the kernel is great, it’s obviously a core blocker, and the first steps to supporting a new piece of hardware in Fedora. The thing is that when people say kernel support is easy they are partially right but it’s only a very small part of what’s needed to support a complex device such has an ARMv7 Single Board Computer for the average user, especially one as popular as the Raspberry Pi! To make the device work with Fedora we could just enable the kernel bits but it doesn’t make for a good user experience OOTB (Out Of The Box).

With a lot of ARMv7 devices these days a new device comes out and it just works with Fedora. It’s awesome when I read a report, or someone tells me “I tried device X with Fedora and it just worked”. That’s because of a lot of work Fedora, and others, have done to ensure upstream boot loaders and boot process just works with new devices. It’s taken a long time to get us to this point. The Raspberry Pi is sadly not like all the other ARMv7 or aarch64 devices. It doesn’t have a standard boot process, doesn’t use u-boot or uEFI, needs vFAT partitions, firmware, text config files and other things that none of our other supported devices need. All of these differences need to be taken into account.

So in terms of the support being upstream in 4.5 the answer to that is it’s “mostly” upstream, there’s still a bunch of patches we’d need to pull in to ensure a nice OOTB experience. This isn’t a blocker in my opinion, it’s something that’s relatively straight foward with most of the bits already headed upstream into 4.6 so it’s a short term issue.

For the rest of the bits what do I consider a nice out of the box experience? This:

  • A single image to support the Raspberry Pi and all out devices (more on that below)
  • Graphics and USB support from boot to login
  • Most basic peripherals working, at a minimum USB (inc keyboard/mouse), HDMI, wired ethernet, a decent selection of wireless USB dongles, storage (MMC and USB) and preferably sound (analog, digital HDMI, mic)
  • A means of easily creating a bootable SD card from at least the Fedora command line, and probably Windows or MacOSX
  • Good documentation, FAQ etc

Single Image:
We currently produce a lot of different images for ARMv7 like Workstation, Server, Minimal and various desktops. If we had to double the amount of images we make to add vFAT that would double the work needed by QA, rel-eng and also lead to confusion by end users as to which image is needed. I have no intention of doubling anyone’s work, or adding confusion for end users, there needs to be a proper engineering solution to this problem!

Serial Console
A number of people have said to me “just enable it and tell them to use the serial console” but having been working on ARMv7 for over 6 years now I know from experience that this leads to vast amounts of confusion by end users as to why “it’s not working” and it leads to a lot of time “providing support” to end users. With a device such as the Raspberry Pi this will become an order of magnitude worse which won’t provide users a good Fedora experience, and likely drive the people who are trying to support the device a even more nuts than normal!

Basic peripheral support
I feel that network, storage, display, input and sound, although I’m still on the fence about sound, are the minimum viable supported peripherals needed for a good OOTB experience for users.

I know that initially we won’t support HATs very well, you can already copy around device tree overlays in the /sys filesystem for basic support, but upstream still hasn’t finalised what a good experience will look like in this regard. I think we can live with this. This won’t stop the use of I2C or SPI devices connected to the 40 pin header, most of these should work just fine.

Image to card creation
Closely related to the single image and basic peripheral support I think the ability to easily create an image to use is important. I’m not sure we’re going to be able to easily solve the Windows/OSX problem, although to be honest I’ve not looked at what’s out there and we might be able to extend LiveUSBCreator here.

Raspberry Pi 3
So when will we support this? Well the kernel support hasn’t been released yet, at least that I could find at the time of writing this. That being said I don’t think it’ll be a particularly evasive or large patch set, the hardware around the Cortex-A53 is the same, so it’ll be just some glue and a pinctrl driver to make all the bits work together, likely not too dissimilar to other recent SoCs that have gained ARMv8 support. This won’t land in 4.6 as the major changes are already queued to land in that, so likely 4.7 will be the earliest upstream kernel. The wireless on the other hand could be more interesting.

So basically the work needed for support of the Raspberry Pi 3, with a proper 64 bit OS 😉 , shouldn’t be too hard once the kernel bits are upstream.

The other issue we had was the legal ability to redistribute the “GPU boot loader firmware” thankfully that problem was resolved about a year ago, although it did take us over 2 years to do so!

The other firmware issue which will cause problems with Raspberry Pi 3 support is that the Broadcom wireless is notoriously terrible, as any Linux running Mac user will contest to, and their wireless firmware isn’t re-distributable in the standard process of being included in the upstream linux-firmware. With luck this is something that the Raspberry Pi Foundation could assist Broadcom in improving!

So we’re well on the way to adding support. Assistance in the issues above would be very welcome, whether for supporting a single image or adding support for writing images for the Pi, to help us get the bits done sooner rather than later. Feel free to reach out to me on IRC (pbrobinson on Libera.Chat) or some other means.

My ARM grab bag device list

They say the first step of coming to terms with addiction is admitting you have a problem… I have a problem with collecting ARM devices… there I said it! How big is this problem you ask? How about I list them out and let you decide!

I’ll break the list down into categories as I believe it’s big enough to do so :-/

The aarch64 set of devices currently stands at:

  • 2x Applied Mustang (different x-gene SoC revs)
  • AMD Seattle
  • 96boards HiKey (hi6220)

The ARMv7 boards list is currently:

  • Compulabs Trimslice (tegra-2)
  • Toshiba AC100 (tegra-2)
  • nVidia Jetson TK1 (tegra-124)
  • Acer Chromebook (tegra-124)
  • BeagleBoard xM (omap3)
  • Nokia n900 (omap3)
  • Nokia n950 prototype (omap3)
  • BeagleBone (am33xx)
  • BeagleBone Black (am33xx) x3
  • BeagleBone Green (am33xx)
  • PandaBoard ES Prototype (omap4)
  • PandaBoard ES B2 (omap4)
  • CubieBoard (allwinner-a10)
  • CubieTruck (allwinner-a20)
  • Banana Pi (allwinner-a20)
  • C.H.I.P. Alpha x2 (allwinner-r8)
  • Snowball (u8500)
  • Compulabs Utilite (imx6q)
  • WandBoard Quad revb (imx6q)
  • novena board (imx6q)
  • RIoTboard (imx6s)
  • UDOO Neo (imx6sx)
  • Origen (exynos-4)
  • OLPC XO 1.75 – a number of variants (mmp2) xNumerous
  • OLPC XO-4 including XO-Touch (mmp3) xNumerous
  • Linksys WRT1900AC (armada-xp)
  • Mirabox (armada-370)
  • ifc6410 (qcom)
  • Parallella Board (zynq7000)
  • Raspberry Pi 2 x3

The Cortex-M series for IoT sensors is currently:

  • TI SensorTag 2015
  • ARM mBed IoT starter kit
  • BeeWi SmartClim

Other random related bits:

  • BeagleBone Breadboard Prototyping Cape x2
  • BeagleBone CryptoCape
  • Original 256Mb Raspberry Pi model B
  • Grove starter kit for BeagleBone Green
  • Explorer HAT
  • PiGlo HAT
  • TI CC2531 802.15.4 USB dongle x3
  • numerous random sensors

So the list above is the devices that I use for hacking on. I count 41 without listing out the dozen or so ARM based XOs I have (various prototypes and models). I also don’t have in that list phones, tablets and two drones as I don’t really hack on those as it’s not like with the list above I don’t already have enough toys! So do I have a problem?

Playing with Bluetooth Smart devices on Fedora

So I have a few Bluetooth Smart (AKA BT Low Energy or BT-LE) devices including a BeeWi SmartClim Smart Temperature & Humidity Sensor, a TI SensorTag2 and a Runtasic Orbit activity tracker. I thought I’d see if I could connect to them with something other than their proprietary apps that run on my phone and get anything useful out of them.

To do this I installed a Fedora 23 minimal image on a micro SD card for BeagleBone Black, inserted a CSR USB Bluetooth 4.0 dongle and powered it up. Once I’d completed the first boot wizard and logged in I installed the bluetooth command line utility packages with a “sudo dnf install -y bluez bluez-libs”. You don’t need to use a BeagleBone, you can use any ARM device or even a standard laptop which supports Bluetooth 4+ with BT-LE support.

Now that we have a device running we’re ready to start to play.

First lets see if our system sees our bluetooth dongle:

$ sudo hcitool dev
	hci0	AA:BB:CC:DD:EE:33

Then lets scan for any Bluetooth Smart devices that are in range:

$ sudo hcitool lescan 
LE Scan ...
A1:B2:C3:D4:E5:F6 (unknown)
A1:B2:C3:D4:E5:F6 CC2650 SensorTag
A9:B8:C7:D6:E5:F4 (unknown)
A9:B8:C7:D6:E5:F4 BeeWi SmartClim

Now create a connection to the device we discovered:

$ sudo hcitool lecc A1:B2:C3:D4:E5:F6
Connection handle 3586

In the bluez package gatttool is a tool we can use to interact with Bluetooth Smart devices. We can use gatttool in interactive mode to send commands to out previously scanned device address:

sudo gatttool -b A1:B2:C3:D4:E5:F6 -I

Now we connect to the deivce (note in older version of gattool the prefix would change from “[ ]” to “[CON]” now it changes the colour of the BT MAC):

[A1:B2:C3:D4:E5:F6][LE]> connect
Attempting to connect to A1:B2:C3:D4:E5:F6
Connection successful

We can request the device characteristics:

[A1:B2:C3:D4:E5:F6][LE]> characteristics
handle: 0x0002, char properties: 0x02, char value handle: 0x0003, uuid: 00002a00-0000-1000-8000-00805f9b34fb
handle: 0x0004, char properties: 0x02, char value handle: 0x0005, uuid: 00002a01-0000-1000-8000-00805f9b34fb
handle: 0x0006, char properties: 0x02, char value handle: 0x0007, uuid: 00002a04-0000-1000-8000-00805f9b34fb
handle: 0x0009, char properties: 0x20, char value handle: 0x000a, uuid: 00002a05-0000-1000-8000-00805f9b34fb
handle: 0x000d, char properties: 0x02, char value handle: 0x000e, uuid: 00002a23-0000-1000-8000-00805f9b34fb
handle: 0x000f, char properties: 0x02, char value handle: 0x0010, uuid: 00002a24-0000-1000-8000-00805f9b34fb
handle: 0x0011, char properties: 0x02, char value handle: 0x0012, uuid: 00002a25-0000-1000-8000-00805f9b34fb
handle: 0x0013, char properties: 0x02, char value handle: 0x0014, uuid: 00002a26-0000-1000-8000-00805f9b34fb
handle: 0x0015, char properties: 0x02, char value handle: 0x0016, uuid: 00002a27-0000-1000-8000-00805f9b34fb
handle: 0x0017, char properties: 0x02, char value handle: 0x0018, uuid: 00002a28-0000-1000-8000-00805f9b34fb
handle: 0x0019, char properties: 0x02, char value handle: 0x001a, uuid: 00002a29-0000-1000-8000-00805f9b34fb
handle: 0x001b, char properties: 0x02, char value handle: 0x001c, uuid: 00002a2a-0000-1000-8000-00805f9b34fb
handle: 0x001d, char properties: 0x02, char value handle: 0x001e, uuid: 00002a50-0000-1000-8000-00805f9b34fb
handle: 0x0020, char properties: 0x12, char value handle: 0x0021, uuid: f000aa01-0451-4000-b000-000000000000
handle: 0x0023, char properties: 0x0a, char value handle: 0x0024, uuid: f000aa02-0451-4000-b000-000000000000
handle: 0x0025, char properties: 0x0a, char value handle: 0x0026, uuid: f000aa03-0451-4000-b000-000000000000
handle: 0x0028, char properties: 0x12, char value handle: 0x0029, uuid: f000aa21-0451-4000-b000-000000000000
handle: 0x002b, char properties: 0x0a, char value handle: 0x002c, uuid: f000aa22-0451-4000-b000-000000000000
handle: 0x002d, char properties: 0x0a, char value handle: 0x002e, uuid: f000aa23-0451-4000-b000-000000000000
handle: 0x0030, char properties: 0x12, char value handle: 0x0031, uuid: f000aa41-0451-4000-b000-000000000000
handle: 0x0033, char properties: 0x0a, char value handle: 0x0034, uuid: f000aa42-0451-4000-b000-000000000000
handle: 0x0035, char properties: 0x0a, char value handle: 0x0036, uuid: f000aa44-0451-4000-b000-000000000000
handle: 0x0038, char properties: 0x12, char value handle: 0x0039, uuid: f000aa81-0451-4000-b000-000000000000
handle: 0x003b, char properties: 0x0a, char value handle: 0x003c, uuid: f000aa82-0451-4000-b000-000000000000
handle: 0x003d, char properties: 0x0a, char value handle: 0x003e, uuid: f000aa83-0451-4000-b000-000000000000
handle: 0x0040, char properties: 0x12, char value handle: 0x0041, uuid: f000aa71-0451-4000-b000-000000000000
handle: 0x0043, char properties: 0x0a, char value handle: 0x0044, uuid: f000aa72-0451-4000-b000-000000000000
handle: 0x0045, char properties: 0x0a, char value handle: 0x0046, uuid: f000aa73-0451-4000-b000-000000000000
handle: 0x0048, char properties: 0x10, char value handle: 0x0049, uuid: 0000ffe1-0000-1000-8000-00805f9b34fb
handle: 0x004d, char properties: 0x0a, char value handle: 0x004e, uuid: f000aa65-0451-4000-b000-000000000000
handle: 0x004f, char properties: 0x0a, char value handle: 0x0050, uuid: f000aa66-0451-4000-b000-000000000000
handle: 0x0052, char properties: 0x1a, char value handle: 0x0053, uuid: f000ac01-0451-4000-b000-000000000000
handle: 0x0055, char properties: 0x0a, char value handle: 0x0056, uuid: f000ac02-0451-4000-b000-000000000000
handle: 0x0057, char properties: 0x0a, char value handle: 0x0058, uuid: f000ac03-0451-4000-b000-000000000000
handle: 0x005a, char properties: 0x12, char value handle: 0x005b, uuid: f000ccc1-0451-4000-b000-000000000000
handle: 0x005d, char properties: 0x08, char value handle: 0x005e, uuid: f000ccc2-0451-4000-b000-000000000000
handle: 0x005f, char properties: 0x08, char value handle: 0x0060, uuid: f000ccc3-0451-4000-b000-000000000000
handle: 0x0062, char properties: 0x1c, char value handle: 0x0063, uuid: f000ffc1-0451-4000-b000-000000000000
handle: 0x0066, char properties: 0x1c, char value handle: 0x0067, uuid: f000ffc2-0451-4000-b000-000000000000

Request the name of the device by querying the appropriate characteristic by hhandle:

[A1:B2:C3:D4:E5:F6][LE]> char-read-hnd 0x3
Characteristic value/descriptor: 53 65 6e 73 6f 72 54 61 67 20 32 2e 30 

The response is returned as a hexadecimal string. To make it readable we need to convert it to ASCII with some simple python:

Python 2.7.10 (default, Sep  8 2015, 17:20:17) 
[GCC 5.1.1 20150618 (Red Hat 5.1.1-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> "53656e736f7254616720322e30".decode("hex")
'SensorTag 2.0'

So that’s the basics of playing with Bluetooth Smart on the command line. I want to work out all of the characteristics and write a simple python daemon to poll the device and then write the output to different locations depending on the config like to a text file or to a MQTT message bus.

Getting IoT kick started on Fedora

So a number of people have been discussing the Internet of Things on Fedora for some time. We now have a Fedora IoT mailing list where these discussions can be more centralised and directed.

So where and how do we get started here? I’m going to kick start some ideas here and repost it as a mail to the list so we can use it as a basis to start the discussion.

As I outlined in my Using Fedora as a base for the IoT revolution talk at Flock there’s a lot of use cases and components that make up a complete IoT stack. I think initially we should focus on two initial goals rather than biting off too much:

  • A IoT internet gateway device
  • A IoT sensors endpoint device

The general idea here is that both of the above would be a very minimal shared build, likely using atomic images to enable easy update/rollback with some specific components for each use case. Initially I suggest we focus on a single, or maybe a couple, of specific devices to limit the scope to something more achievable and to add features as we go.

IoT internet gateway device specs and features

  • Wired and/or wireless ethernet to provide internet connectivity
  • Bluetooth Smart (AKA LE)
  • Thread Stack support (6LoWPAN and friends)
  • 802.15.4 support
  • MQTT Broker support (not standard for a IoT GW but enables easier localised testing)
  • MQTT Client
  • Atomic support: updates, rollback etc
  • Works with both our endpoint below and other IoT OSes such as Contiki

IoT internet sensors endpoint specs and features

  • Wired or wireless ethernet IP support
  • Bluetooth Smart (AKA LE)
  • Equivalent to Thread Stack support (6LoWPAN and friends)
  • MQTT Broker support (not standard for a IoT GW but enables easier testing
  • MQTT Client
  • CoAP client
  • Atomic support: updates, rollback etc
  • Support for various inputs and outputs and sensors

I have no doubt missed a lot of details in the above use cases, it’s somewhere to start. I think we also need to look at tools like Node-RED and tools for managing the devices. IoT is a big topic, the idea is we need to get the conversation start somewhere. I’ll look forward to seeing you all on the list to do that.

Flock Rochester

I’m not going to do a day by day outline of what I did at flock, if I did it would basically be “blah blah blah I talked a lot to a lot of people about a lot of tech topics” and anyone that’s ever met me would have guessed that! It was, as in the past, a great conference. A big shout out to the organisers for an excellent event with two excellent evening events! So I’m going to give a brief summary to my talks and link to slides and video recordings.

My first talk was an overview of the state of aarch64 and POWER as secondary architectures. The slides aren’t particularly interesting as they’re just words for discussion points. The video has all the interesting bits. A related talk was Dennis’s Standardising ARMv7 booting with a memorial quote by Jon Masters 😉

My second talk was about using Fedora as a base for IoT. Slides are here but the talk was quite a bit different to the slides and is more interesting so I suggest watching the video.

I also actively participated in Dennis’s Fedora Release Engineering going forward because well obviously I’m part of it 😉 and it was interesting for where we’re going, and even where we’ve come from in the last year or so 🙂

Finally I loved the Keynote Be an inspiration, not an impostor by Major Hayden. He’s published a follow up blog post with a FAQ too.

The least memorable bit was the terrible Amtrak ride back to New York City. On the plus side it makes the worst of the British National Rail service seem amazingly on time! NEVER AGAIN!