The state of open source accelerated graphics on ARM devices

I’ve been meaning to write about the state of accelerated open source graphics options for a while now to give an update on a blog post I wrote over 5 years ago in January 2012, before the Raspberry Pi even existed! Reading back through that post it was pretty dark times for any form of GUI on ARM devices but with the massive changes in ARM devices and the massive change in SBCs (Single Board Computers) heralded by things like the Raspberry Pi have things improved at all? The answer is generally yes!

The bad

Looking back at that post the MALI situation is still just as dire with ARM still steadfastly refusing to budge. The the LIMA reverse engineering effort started with promise, but went up in smoke with a fairly public community break down, I don’t envision that situation improving any time soon although just recently there appears to be some forward movement happening finally after a long silence. This only covers the MALI-400 series and any newer GPU is a completely different architecture/IP. Even with sessions recently at Linaro Connect titled What’s happening with ARM Mali drivers I don’t see fast change here.

The Imagination Technologies PowerVR is still just as dire as situation as it was five years ago. The company’s incompetent management recently managed to avoid being bought by Apple which in turn, because they’ve screwed the open source community while milking the Apple cash cow, essentially means they’re screwed. I suspect they’ll either open source to try and remain a relevant contender or die in a tire fire. Only time will tell there, in the mean time any ARM SoC that has this IP on board is useless for anything graphical so I’d tend to avoid it, thankfully there seems to be less of them these days.

The good

Despite the two bad examples above there’s actually been a lot of good change in the last five years. We now have a number of options for fully accelerated 2D/3D graphics on ARM SoCs and I run GNOME Shell on Wayland, yes the full open source shiny, on a number of different devices regularly.

NVIDIA true to the rumours did open up all the graphics on the Tegra series of hardware. The new Tegra K/X series have GPUs similar to their x86 offerings with Kepler/Maxwell/Pascal GPU cores but NVIDIA supports these devices by contributing to the nouveau open driver rather than the closed x86 driver. The performance on 32 bit TK1 devices has been decent for a number of releases of Fedora and improves all the time, we’ll be supporting the X series (X1/X2) with their Maxwell/Pascal GPUs in Fedora 27.

In the old post I brushed past Vivante with a mere mention of Marvell and Freescale (now NXP). The Vivante GPUs ship in NXP i.MX6 and i.MX4, some Marvell chips and some TI chips. There was a reverse engineering effort called etnaviv that must have started not long after I wrote that post and after a number of years of development support landed upstream in the kernel late 2015, and in mesa in the 7.1 release allowing us to support fully accelerated Wayland in Fedora 26! Did anyone notice? I didn’t really yell about it as much as I should have! It supports fully accelerated 3D in mesa/wayland, is pretty stable and is improving all the time, well done to all the contributors to that effort!

Another I brushed past in the old post was the Qualcomm Snapdragon SoC. They ship with a Adreno GPU. This was previously closed source, with the SoC primarily used by phone/tablet manufacturers I suspect they didn’t care… until Rob Clark (and no doubt there were other contributors) decided to reverse engineer the driver with the open freedreno driver. This is now the default driver with even Qualcomm contributing to it. We’ll support this in Fedora 27, initially with the 96boards Dragonboard 410c using the freedreno driver, but I doubt it’ll be the last Qualcomm based device we support. The Snapdragon 835 SoC, the device in all the high end Android phones this year and the ARM Windows 10 laptops, is really nice with decent performance, I’d love to be able to support a device with that SoC!

Raspberry Pi, as I mentioned in the introduction, wasn’t even out in when I wrote the original post. When it fist launched there wasn’t an open driver but 5 years later there is, sponsored by Broadcom no less. We introduced initial support for the Raspberry Pi with the open vc4 driver by Eric Anholt in Fedora 25 and it’s improving regularly. It supports fully accelerated 3D in mesa/wayland, and 2D via glamor in mesa.

So in conclusion we have improved by A LOT! We now have numerous different GPUs with open drivers to choose from in all price ranges that support fully accelerated 2D/3D desktops from four different vendors on both ARMv7 and aarch64. The media acceleration offload is also looking quite good, but that’s one for another post. The biggest holdout is MALI, and that would need two open drivers or ARM to come to the table, LIMA might work out for the 400 series, but that won’t work on the newer midguard series. With support in a number of drivers for the shiny new Wayland there’s an increasing number of devices people can use to enjoy the latest desktops fully accelerated!

Raspberry Pi improvements in Fedora 26

So since I landed support for the Raspberry Pi 2 and 3 just in time for Fedora 25 Beta it’s been a bit of a fun ride. The support for Raspberry Pi is mostly done in my spare time along side all the other responsibilities I have and it’s been interesting to see people’s feedback. Going into Fedora 25 I knew it wasn’t going to be perfect but the experience was going to be reasonable for newbies to get going without generally needing serial consoles and it met Fedora’s (and mine) exacting standards on free drivers. I think we achieved that quite well but I also learned a lot in the Fedora 25 cycle and what’s coming in Fedora 26 is quite a substantial jump forward.

Hardware for a good experience

So what have I learned about the first six months or so of Raspberry Pi in Fedora? Well there’s a couple of things that the user can do to ensure a decent starting experience themselves. The biggest FAQs I’ve dealt with on the various support forums are generally fixed by these three things:

  • A proper spec power supply. For the RPi2 this means at least 2 AMPs and for the RPi3 at least 2.5 AMPs. If you want to plug in USB WiFi dongle and a USB HDD you’ll likely want to add a little more! In most cases an old phone charger will not suffice.
  • A good quality Class 10 micro SD card. I generally use Samsung EVO or SanDisk Ultra cards.
  • A Raspberry Pi 2 or 3. Yes, it’s surprising how many people hope to run it on something else. SORRY (actually, I’m not!)!

What’s in Fedora 26 Final

So enough of what to do! Everyone wants to know what improvements arrived in the Fedora 26 Final with the 4.11.x kernels:

  • Pi3 WiFi: It’s been working in F-26 since Alpha and is surprisingly stable. There’s a file you need to grab to enable it. See details in the wiki here.
  • Performance: In the process of dealing with wifi I worked out one of the reasons we were seeing poor performance on the SD card. We’ve had some minor improvements in F-25 but this fix over doubles the performance for me on the SD card.
  • HDMI video: There’s been issues around certain monitors crashing the video (vc4) driver and people getting black screens during boot. While this isn’t perfect yet (ain’t hardware great!!) it’s greatly improved across numerous devices.
  • Composite video: We’ve had support for the composite video since 4.10 but I need people to help test this.
  • Sound: HDMI audio is supported, I’ve done minor testing with the one HDMI audio capable device I have. Analogue audio out isn’t upstream yet.
  • HAT support: We now have all the support needed to do overlays in the kernel/bootloader and dtc stack. I just need to test it some more, document it and work out how we can best distribute pre built overlays to ease consumption. There’s still no consensus on an Overlay Manager from upstream to auto load overlays based on EEPROM on the HATs. In a lot of cases you want to load the overlays from u-boot anyway for things like display. Look out for docs and blog posts on this soon!

What arrived with the 4.12 kernel rebase

  • Thermal support: so if the RPi runs too hot it’ll slow it down
  • More performance improvements and tweaks.

What’s coming in the 4.13 kernel rebase

  • Bluetooth support: upstream finally tracked down the issues here. It’s been a much requested features and I should have the bits in place soon!
  • More performance, stability and graphics improvements and tweaks.

What about Fedora 25?

Some of the above pieces will be coming to Fedora 25 with the 4.12 rebase. The focus of my spare time is Fedora 27 mostly now, with the above coming to F-26. Some components are a lot harder to back port without issues or a complex series of package updates to ensure smooth upgrade. The WiFi and performance improvements were the hardest as part of that change moves around the use of hardware blocks and drivers. I managed to stop both the RPi2 and RPi3 booting numerous times in testing before I properly realised the implications of the change. Getting these changes for users back into a stable release without issues is hard and time consuming to do across all the various use cases. I tried this with some fixes in 4.9 and ended up making the RPi3 very unstable. This cost me a lot of time to debug and fix and I don’t really want a repeat of that!!

Graphics device

One of the surprising side effects was the discovery of a device that is five years old is that Fedora suffered from early adopters issues. We were one of the first distributions to adopt a fully upstream open kernel and graphics stack and with that came a number of issues around monitor detection, especially older/cheaper models that aren’t 1920/1080 “Full HD” or via HDMI to VGA adapters. We’re still working through these with upstream and have improved the situation quite a bit in Fedora 26 overall but it takes time and reproducible use cases which with random hardware isn’t easy or quick! 🙁

Next up?

I’ll leave Fedora 27 features and functionality for another, this post has been sitting in my drafts folder since June so it’s time to get it out and like my development move on to Fedora 27!

3.16 Fedora ARM kernel status

So 3.16 is has quite a few new features in terms of newly supported devices, also some what surprisingly this blog post will be out before 3.16! In terms of new device support all the SoCs listed here are exciting for a number of reasons for Fedora ARM. Aarch64 (ARM64) makes it’s first debut with support of real hardware although we’ve actually had kernel support enable for it for some time in Fedora even if only usable on the glacial Foundation emulator.

The 3.16 release is also very likely to be the kernel that ships with Fedora 21 GA and with the Alpha due in about a month we’re starting to polish and test all the platforms and devices we want to support for GA.

Anyway without any further a do let’s get into the gritty details:

  • NVIDIA Jetson TK1 support: While we’ve had the basics of this for a while all of the bits are there now.
  • EXYNOS support: This SoC is probably the most asked about platform and finally after a long wait the multiplatform support has landed upstream. We currently ship around 20 dtb files for exynos4 and 5 (Chromebook support anyone?). Testing is sought and feedback and greatly appreciated.
  • Qualcomm MSM 8×60, 8960 and 8974 support: While the multiplatform support for these devices landed upstream a few releases a go they’re now to the point they should be relatively usable so it’s time to get wider testing. This should be the beginning of supporting the venerable ifc6410 and dragonboard devices.
  • APM X-GENE support: One of our first aarch64 supported pieces of hardware. Similar to the QCom SoC the initial support has been upstream for a while but with 3.16 it becomes usable with the vast majority of basic support upstream so minimal patches are needed. More on aarch64 soon.
  • AMD Seattle support: The other of our aarch64 supported pieces of hardware if you’re lucky enough to get your mits on a device.

The other feature we’re starting to see mature is GPU and Graphics support. I’m not exactly sure yet as to what the final state of this functionality will be for Fedora 21 GA but we potentially will have suppport for:

  • nouveau/mesa support on the NVIDIA Tegra K1
  • freedreno/mesa support on the Qualcomm boards
  • etnaviv/mesa support on i.MX6 devices
  • improved modesetting support for a number of other devices. Some of this has already landed and is usable in rawhide now.

What covered above is just a high level overview of what’s new in the upcoming release. There’s been numerous other improvements in existing supported SoCs and devices all over the place that would take too long to cover off here but in short with all the shiny that’s landed in 3.16 what Fedora ARM will look like as part of the Fedora 21 GA release is quickly starting to take shape.

The state of open ARM GPU support?

Graphics support on Linux has always been something of an “interesting” topic on Linux. On the x86 side of things over the last 15 years or so there’s been somewhat of a consolidation of GPU manufacturers found in most video cards. Now days you basically have a choice of Intel, ATI or NVIDIA chipsets. They are mostly pretty well supported in Linux whether it be by the vendors themselves or through community projects. I can remember back in the 90s when there was a lot more vendors. Anyone remember when around 1998 a Matrox g200 was the best card to get for Linux, or even earlier around 95/96 (when I first started using Linux) where a S3 ViRGE was always a good bet to get a working X? No, not a lot of people do! Maybe that’s not such a bad thing, I for one was not sad to see the end of modelines and xorg.conf files.

So having got involved in the ARM project one of the less nice things at this point in time is the complete lack of decent X drivers for 3D, and in most cases for 2D the best you can do is a FBdev driver. A few off us discussed the sorry state of drivers for ARM based devices at FUDCon Blacksburg.

So what is out there? Do we have the mass variety of GPUs like the older x86 days, or is it more like big three of today with a few minor players on the side? Most definitely the later. For the devices we’d like to support within Fedora ARM we have the following devices to contend with. Firstly ARM themselves make the MALI GPU, they’re used in Samsung, ST Ericsson and other platforms. Then we have the NVIDIA ULP GeoForce used in Tegra 2 an Tegra 3 systems, apparently its nothing like it’s older brothers. Next up is the Imagination Technologies PowerVR as included in, amongst others, the TI OMAP, Samsung Hummingbird, Apple Ax. Those three seem to be the major ones we encounter at the moment. There’s also the GPUs used in the Marvell, Freescale, Qualcomm Snapdragon, Broadcom, and what ever the many other ARM platforms use, I’m not exactly sure of all those details.

At the moment the state of accelerated drivers is very similar to the time when AdamW would rant about Poulsbo drivers. Basically you grab a download of a system that ships binary only drivers and attempt to make them work with the current Fedora X and hope that it works and is mostly stable. Not ideal!

A week ago at FUDCon the state seemed dark and horrible and not about to change at any time soon. A week later it seems there’s a ray of light poking its way over the horizon that may indicate the winter might be starting to thaw on the way to a warmer spring. So why do I think there’s light at the end of the tunnel?

Firstly there’s a FOSDEM talk entitled Liberating ARM’s Mali GPU by Luc Verhaegen. Having a Nouveau style driver for the MALI GPUs will be a massive improvement for Linux on ARM as this is the core that ARM develops themselves and companies can then license it to use without having to develop their own. I think we’ll see this GPU design in a lot of devices in the coming years.

Next up there’s hints from Arnd Bergmann, a kernel developer at IBM/Linaro (according to LinkedIn), that the traditionally frosty relationship NVIDIA has with Linux X drivers might change for the Tegra platform with the comment of “see them in the same boat as Intel and AMD regarding their support for free drivers in the near future”. Which while isn’t any form of guarantee it sort of indicates they might well be at least considering it.

Samsung has also had their Exynos 4210 DRM driver merged into kernel 3.2. It’s a basic, un-accelerated DRM driver which supports KMS. It’s some forward movement, although there’s no mention yet of what the user side driver will consist of. There’s also a project for an open S3C6410 driver which is a earlier gen Samsung SoC.

Texas Instruments is also working on it’s omapdrm driver which also provides a similar basic DRM/KMS functionality. There’s also a matching X.Org driver which in conjunction with omapdrm can provide a 2D DDX driver. No 3D yet though. Not really holding my breath on that one to be honest. The FSF has had PowerVR on it’s high priority list for quite some time and as the GPU in every generation of iDevice Imagination Technologies Group has no financial incentive to support any form of X based 3D driver as they’re already selling silicon hand over fist whether it’s iOS or Android based. The working 2D driver for the aforementioned SGX derived Poulsbo plus the OMAP bits might be enough for some smart hacker to glue bits together to at least get a decent general purpose 2D SGX driver though.

Well the open drivers on ARM aren’t great, but at least there’s a few upcoming improvements to look forward to. Maybe the smaller companies might realise that an open driver will at least allow them to sell some silicon designs where they can’t get their foot in the door with the likes of Apple. Maybe people like Luc will just hack them before they do and make it a mute point.