Getting started with OpenCL using mesa/rusticl

Mesa, the open source low level graphics stack, has featured support for Open Compute Language (OpenCL) for some time via a front end called Clover. The problem was that the GPUs that it supported were limited, it didn’t have Image support, and wasn’t really under active development. Around a year ago Karol Herbst filed a merge request adding rusticl to Mesa 22.3 release, and soon after that I enabled it optionally in Fedora.. What is rusticl? It’s a OpenCL API implementation written in rust for Mesa, it will eventually replace clover. One advantage it has is image support from the outset and also works on much wider range of OpenCL capable GPUs, including some ARM GPUs, and the support is actively improving all the time.

In the year since it landed in a stable mesa release rusticl keeps evolving, faster, more HW support, more features, less crashes. I’ve tinkered with it as I’ve had spare time on the weekends and evenings as well as trying to work out details of how you’d use it to run higher level stacks.

As of writing it works with following gallium drivers: iris (Intel), nouveau (Nvidia), radeonsi/r600 (AMD ATI), and panfrost (Arm MALI). There’s other drivers that are various stages of development but are not yet upstream.

So let’s get the basics up and running on Fedora if you have supported hardware. First install the core software stack:

$ sudo dnf install -y mesa-libOpenCL mesa-dri-drivers spirv-llvm-translator spirv-tools-libs clinfo clpeak

Next we run clinfo and clpeak with required parameters, from the driver names above (in this case an Intel laptop), to enable rusticl. The two commands output a lot of information so I’m not going to post them here but the output shows OpenCL running and some details about what’s supported:

$ RUSTICL_ENABLE=iris clinfo --list
WARNING: OpenCL support via iris+clover is incomplete.
For a complete and conformant OpenCL implementation, use
https://github.com/intel/compute-runtime instead
Platform #0: rusticl
 `-- Device #0: Mesa Intel(R) Xe Graphics (TGL GT2)
Platform #1: Portable Computing Language
 `-- Device #0: cpu-11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
Platform #2: Clover
$ RUSTICL_ENABLE=iris clinfo
output a lot of details
$ RUSTICL_ENABLE=iris clpeak
output a lot of details

This gets up the basic pieces up and running for OpenCL, there’s of course more to do and not all use cases are complete. Eventually we won’t need to have the environment variables to enable rusticl. The details of drivers and other features enabled by environment variables are documented here. I plan to do some more posts as follow ups to build on this basis.

Fedora on the UP Squared

With the IoT Working Group and Edition moving forward I’ve been looking for an x86_64 device suitable for testing IoT related use cases. I was originally planning on using the MinnowBoard or Joule but given Intel has killed those product lines off it was back to the drawing board. I eventually settled on the UP², in particular I chose the UP² Pentium-4GB-32GB-PACK as it had everything I wanted in one box.

Hardware

The on paper hardware specs show a recent generation Intel Apollo Lake core, reasonable memory and storage options, an onboard FPGA, USB-3, dual ethernet and various other bits. The kit comes with options for active or passive cooling and the later the heatsink is massive, for the moment I’m running it on the passive cooling. The case is OK, I wouldn’t rave about it though. The power connector on the other hand is terrible, the PSU cable doesn’t seat well into the board and I’ve bumped it already and had it lose power, the power button is also tiny, so small in fact I mistook it for a reset button.

Fedora support

As you would expect the support in Fedora 27 is decent, accelerated 4K graphics, wired RealTek ethernet NICs, Intel m.2 PCI-e WiFI/Bluetooth, although the later is only 4.2 for IoT I would have appreciated Bluetooth 5, all work out of the box as expected. The firmware is uEFI and in theory supports secure-boot but I couldn’t work out how to turn it on in the firmware menus as it was greyed out, it also has a TPM2 module I’ve not had time to investigate. They eagle eyed would also note that I mention Fedora 27 even though Fedora 28 has been out a few days. Well for some reason F-28 doesn’t boot, I tried the network installer and the Workstation live image, they both get to the grub menu, then I get no output and nothing for a moment then it resets and we start again. I need to investigate this further but Fedora 27 Workstation livecd booted and installed fine so that’s what it’s got for the moment. Ultimately this is going to a host to test Fedora IoT so while I tested the general support this is fine.

IoT support

There’s a number of reasons I chose this particular device as an IoT test device:

  • Reasonably priced with a reasonable feature set.
  • Intel hasn’t killed it off yet like they’ve done with the Joule platform and Minnowboard 3 so I could actually buy it 🙂
  • Multiple network interfaces, reasonable WiFi and Bluetooth support.
  • Industrial IO sensors via an onboard Intel Sensor Hub. lsiio is reporting 9 sensors of various types. I’ve not checked this further yet.
  • USB-3, a Raspberry Pi HAT compatible connector and other options to add IoT related functionality or interfaces.

FPGA support

I’ve not looked at the FPGA support at all. The upstream kernel now has a FPGA Manager Framework and there’s a bunch of Altera FPGA support there but I’m not sure how it maps to this device. I also have to investigate open source toolchains for FPGA bitstreams as a lot of them just aren’t, I’ll likely do the HW enablement side of things and leave the toolchain bits to people that understand them. I also have the 96boards Ultra96 board so FPGA investigation was already on my Fedora 29 To Do list, and a lot of other people seem quite interested in them of late, no idea why 😉

Fedora IoT Edition is go!

Tap tap tap… is this on?

So the Fedora Council has approved my proposal of IoT as a Council Objective. I did a presentation on my IoT proposal to the council a few weeks ago and we had an interesting and wide ranging discussion on IoT and what it means to Fedora. I was actually expecting IoT to be a Spin with a SIG to cover it but the Council decided it would be best to go the whole way and make it an Official Edition with a Working Group to back it! Amazing! One of the side effects of IoT being an accepted Objective is that the Objective Lead has a seat on the Council.

So I would say the real work starts now, but the reality is that there’s been no small amount of work I’ve been doing to get to this point, but there is also now a lot of work to do to get us to a release. We’re going to aim this initially for Fedora 29, with the intention to have a lightweight spin style process to get things up to speed as quick as possible between now and then.

So what will be happening over the coming weeks (and months)? We’ll be getting the working group in place, getting an initial monthly release process in place so that people can start to have something to kick the tires with and provide feedback and drive discussion. With those two big pieces in place we can start to grow the Fedora IoT community and work out the bits that work and bits that don’t work. Iterate early and iterate often as is often said!

So of course the big question is how do you get involved? We’ll be tracking all of the Working Group efforts in a number of places:

  • Fedora IoT Pagure Group: We’ll be using this for issue tracking, release milestones, and for git repositories to contain things like container recipes.
  • Fedora IoT mailing list: If you don’t have a FAS account you can subscribe by emailing (blank is fine) iot-subscribe AT lists.fedoraproject.org and the list server will reply with subscription options.
  • IRC on #fedora-iot
  • Fedora IoT Tracking bug: This will be primarily for tracking dependencies and component RFEs and issues.

The above list will change and evolve as we go, I expect the pagure group, mailing list and IRC to be the primary places of communication. There will of course be updates also on this blog, no doubt Fedora Magazine, FedoraIoT on twitter and elsewhere.

What will there be to do? Well lots, and that is still obviously in flux at the moment. The things that come to mind that we’ll definitely need to address will include, but certainly won’t be limited to, awesome docs, the actual OSTree Atomic host image which will be the key foundation, CI/CD pipelines to automate testing as much as possible, release processes including landing of features once they’re ready, containers and layers to add functionality, a selection of supported reference devices (see also CI/CD in this context too), various IoT frameworks, hardware enablement such as wireless standards and distinct from the supported reference HW, security (a single word can’t even begin to describe this iceberg!) and developer experience to name but a few but there’s so much more! Is everyone excited? Of course you are!