Fedora 18 on ARM retrospective

It’s been a while since I’ve done a general update on Fedora 18 on ARM. Post release always tends to be a good have a look at what’s been achieved, where we are and what we have planned for the next release. At the beginning of last year I was trying to blog weekly on my Fedora involvement, or at least tech involvement that’s related to Linux, I need to get back into that habit, if for no other reason than to stop people bothering me ;-). As a result this is a bit of a mammoth post at around 1500 words!

Overall Fedora 18 was a reasonably standard release process for ARM. We had the usual package build issues with lots people scrambling to help us get things fixed even if some took longer than hoped, kernel issues also featured but that’s not unusual as there’s been a lot of upstream ARM churn! The extended release process wasn’t as bad for ARM as it was for some of the other teams as it allowed us to fix up some more bits and get them landed in the release. So what were the major achievements of this cycle?

  • Supported Devices: The number of devices we support has actually gone down due to the Efika devices not being updated to run on the newer kernels and then being completely dropped for the 3.7 cycle due to lack of DeviceTree support. That being said the number of devices that work with only little intervention is on the way up and this should increase dramatically over the Fedora 18 life cycle due to the unified kernel. People are also starting to play with remixes for devices and I have some ideas to support this better (see kernel section).
  • Packaging: The number of packages that didn’t build out of the total of 12,614 packages in the final F-18 GOLD release was a rather small 280 at mainline release. This has actually gone down since as we’ve had a couple of packages that were blocking a lot more land in the post F-18 updates flurry and more are on the cusp of landing too. I’m working through the remaining packages to document the remaining hold outs and classify them as 1) non ARM arch 2) simple fixes required 3) complex fixes required 4) language bootstrap required 5) unknown or 6) dependent on another package from 1-5.

    We’re also working on improving and optimising packages for ARM. Over this release cycle I’ve pushed a number of fixes to improve performance and operation of packages. I expect this to continue for a number of cycles to come as well as enabling more HW device support like UCM support for sound devices in F-19.

  • Kernels: I’ve spent an inordinate amount of time this release doing kernel work. I would never consider myself a kernel hacker but clearly I’m able to fool people as I was asked by a kernel developer if my employment by Red Hat was to become the ARM kernel maintainer, while chuffed at the complement I’m really not a kernel hacker!

    So what have we achieved in the kernel this cycle? Well we’ve landed the first iteration of the 3.7 “unified” ARM kernel which allows us to boot a single kernel on multiple devices. We have two devices currently supported in 3.7 with the qemu based Versatile Express emulation and Calxeda Highbank being the first candidates that have been tested to work, we have the Marvell mvebu support enabled in 3.7 too but there’s not enough driver support for it to work on real devices. The 3.8 kernel expands this out by adding IMX (yet to be enabled), SunXI (also known as AllWinner A1x) and some other devices. The A1x support is similar to the mvebu in 3.7 in that it’s just core SoC support so not really useful as yet. The unified kernel has allowed us to collapse a number of our kernels into one which has reduced our build times dramatically (one of the bullet points needed for primary arch) but it’s not all completely rosy. The unified kernel is still problematic and I don’t believe this is helped by its current lack of use. This is especially disappointing when Linaro don’t even build unified kernels for testing. It appears Fedora is leading here and I get to feel the pain 🙁

    Moving forward I’m starting to hack about on a new kernel idea to enable easier support and “Remixes” for non mainline ARM devices. Basically I want to do a remote clone of the Fedora kernel tree and setup a branch for each device we wish to support. From there I plan on pulling in the patches/trees needed to support the device. It’s not going to be pretty as some still use the 3.4 kernel etc but the planned outcome is an automated yum repo with a series of kernels enabling people to build their own install images for a much larger range of devices, a long with a central repo so that when/if the support goes mainline it’s easier for us to merge the changes in. Once this is working I hope for people to be able to submit new platforms via git pull requests. I’ll blog about this more as I get more bits working. I need to upshift my git skills and I suspect this will be an interesting challenge!

  • FUDCon and FOSDEM: Unfortunately due to personal commitments I was unable to attend FUDCon NA this year. I was some what amazed at the number of people that contacted me to ask why I wasn’t there and if everything was OK or just to express their disappointment at not catching up with me. Believe me it was unavoidable and I wasn’t just trying to avoid you all! From everything I’ve heard it was a great event with lots achieved both around the ARM and non ARM stuff but others have reported on this so I won’t bore you!

    On the plus side I did make it to FOSDEM this year! And what an excellent event it was! It’s an event I’ve been wanting to attend for 5 years or so. I was even recorded in a podcast by the excellent EMEA Cloud Guy talking about ARM, OLPC, all sorts of cloud computing tech as well as other things that I do in my day job. So if you really did miss my voice at FUDCon or are just having problems sleeping you can have a 30 minute odd fix there 😉

    There was a lot of things covered on the ARM side from ARM64 (or aarch64) through to the varying state of GPU support. I plan to write up more about this in other posts but it’s certainly going to be an exciting year for ARM development! You can mark my words 😀

  • Awareness: I’m constantly surprised about the people that get in contact with me about Fedora ARM support. It’s amazing and exciting the places people are looking to use and actually are starting to use it, it’s nice to know that the thousands of hours I and others are putting into Fedora on ARM aren’t going to waste. The awareness of the project both inside the Fedora community and in the wider Fedora ecosystem is expanding rapidly and that can only be a good thing! For example OLPC announced the XO-4 Touch at CES and it’s running Fedora 18 on ARM to boot.
  • Build platform: We’ve finally got enterprise hardware on it’s way. I was kind of hoping we would have made the switch a month a go but any moment now we’ll be moving the koji instance used for ARM builds over to a unified set of enterprise builders based on the Calxeda EnergyCore platform. This is in preparation for a push to promote ARMv7 to primary architecture. I must say, primary or otherwise, I’m really looking forward to having a unified set of builders running on enterprise storage and networking. While the existing platform has served us well it’s really starting to bulge around the edges while building for the three Fedora releases on two architectures. Look for the announcement Real Soon Now or just listen out to me celebrating!
  • Dropping armv5tel: We’ve decided to drop ARMv5tel support as of Fedora 19. Both Fedora 17 and 18 will continue to be supported for the entire mainline life cycle. The fact is that ARM themselves, with the introduction of the Cortex-A5, have stated that all cost points and use cases are now covered by the ARMv7 Cortex-A* series of designs and as the memory supported by the v5 series of chips is only DDR1 the devices were getting expensive to make. While there are a number of plug devices out there still the fact is there’s probably less devices there as there were PPC devices when it was decided to drop PPC so I don’t see it as a major issue. The Raspberry Pi is going to be supported by Seneca in a separate ARMv6 secondary arch effort so isn’t going to be left out.
  • Anniversary and statistics: Stats are always fun and I noticed the other day that it was in fact my two year anniversary of working on Fedora ARM builds, although like a typical male in a relationship I did forget it! In my defence I have also forgotten my own birthday in the past too 😐

    I pushed my first build on January 9th, 2011 and since then I’ve done a mere 82,876 builds via a lot of tasks (koji times out on that query!). I’ve also submitted well over 300 updates for F-17 and at a little over 125 not quite as many for F-18 putting me on the top of the updates list (sorry!) for both releases. They weren’t all for ARM though but I’m very glad F-18 is, currently, a lot less than F-17 which means that we’ve made progress on fixing ARM issues 🙂

Look for some more specific, less wordy, ARM related posts soon.

Fedora ARM F-17 and rawhide status update

or as the saying goes… the mustard indicates progress! Well there certainly has been an a-bun-dance of movement in the Fedora ARM world of late and it’s been way too long since I’ve given an update on the status of the Fedora ARM SIG.

We’re progressing well and there’s a lot of stuff going on. Of course you’re all going to be up be up at 06:00 GMT tomorrow (Wed) to order your Raspberry Pi running Fedora 14… aren’t you! I’m sure those of you that got a voucher for one at FUDCon Blacksburg will be ready and waiting.

In terms of raw package building of rawhide and F-17 branches we’ve been progressing well since we kicked off the mass rebuild shortly after the aforementioned FUDCon. In terms of numbers of a total 11348 packages in F-17 we’ve built 8861 of them giving us around 2487 left to go. Of course that’s not an exact number as there’s a number of x86 only packages and between now and the final beefy F-17 release there will be lots more releases of the packages that will need to be built. We’re already building rawhide to ensure we stay up to speed on that front too! One thing that is slowing us down is having to spend a lot of time fixing broken upstream packages and of course fixing ARM specific issues.

In terms of devices that will be supported for the Fedora 17 release the number will be quite small. The list at the moment will likely be PandaBoard (including ES), all modern BeagleBoards, Trimslice, Efika SmartTop and SmartBook, Raspberry Pi and the OLPC XO 1.75. The last two will be Fedora re-mixes. In most cases of pure Fedora the support will likely be console based or accelerated 2D UX. The device market and associated support is moving pretty quickly at the moment so that could easily change between now and release, this is very much a moving target. It’s also not to say that Fedora 17 won’t run on all devices, it’s just that Your Mileage May Vary and it depends a lot on the state of the mainline kernel support.

So what else are we working on? We’ve been doing a lot of planning in preparation for the Proposal to primary arch. We’ve also been working with various tools teams like Anaconda to get some features for building images. In this regard we also need to work with the appliance-tools and livecd-tools teams to see what can be done to allow them to support image creation. It would be cool to be able to use the appliance tools to create an qemu image that can be directly reported into libvirtd and virt-manager (hint hint 😉 ). I’m also slowly starting to investigate what’s needed to support DeviceTree so we can build less kernels and support more devices but I’m not sure it’s all quite there yet.

So what can you do to help? The most simple and best thing is to ensure your package has been built properly for mainline as part of the mass rebuild, or if we file a RHBZ bug with ARM issues to fix it or assist us in fixing it quickly. Most people have been fabulous with quick fixes and attention on bugs. Lastly we should have some minimal rootfs files available soon for F-17 😉

In summary we’ve got nearly nine thousand individual packages already built for ARM supporting both ARMv7 and ARMv5tel architectures. For our Fedora 17 release will be on a reasonable number of innovative and interesting devices. We’re aiming to release as close as is humanly possible to the Mainline Fedora release. Users should be able to run anything they would on similar x86 devices. We’re running and fast catching up 😀

Fedora 15 ARM alpha rootfs

I meant to do this over the FUDCon weekend but completely ran out of time. So I’ve cut a minimal rootfs tarfile for both softfp and hardfp Fedora 15. They are little tested but I am running something very close to them and others have had success.

The hardfp image comes with kernels for OMAP (BeagleBoard/PandaBoard) and Tegra devices (primarily Trimslice) where as the softfp image comes with OMAP and Marvell (plug computers). The root password is fedoraarm and there’s various instructions for devices all over the place.

You can get the hardfp rootfs here and the softfp rootfs here.

This will likely be the only rootfs for Fedora 15. The GUI bits are coming along slowly but from here on pretty much everything that’s working will be installable using yum.

I’m not concentrating most of my time on getting ARM rawhide on the coat tails of mainline. More on that very soon 😉

FUDCon Blacksburg Day 2 & 3

Well day two of FUDCon started with the usual bits and pieces that people have reported about including BarCamp pitches and the FPL’s state of the union address.

Again most of my day revolved around ARM bits and pieces including the Secondary Arches session. We gave away vouchers for 20 Raspberry Pi’s for people to hack on when they are released. I’m looking forward to those 20 people getting involved in the ARM project and blogging about all the cool things they do with them.

In between all the ARM sessions and lots of random hallway discussions about ARM, OLPCs and just about anything Fedora I managed to sneak off and sit in on a Gluster session to have a break from ARM related stuff. In between all of that I was hacking on rawhide for ARM but that’s another blog post entirely so stay tuned.

FUDPub was a good opportunity to have a break from the laptop and discuss various things and catch up with various people I hadn’t managed to spend much time with and get to know them a little better. I was somewhat surprised that the beer ran out so quickly.

Sunday morning was a slower kick off and was some serious ARM hacking and some further discussions. Things started to slow down as people departed to head home. Mid after noon was the second board meeting. We discussed a number of things from SOPA/PIPA support options through to Christoph’s Fedora Council proposal. In ended up being quite a long but fairly productive meeting.

Overall as always I found FUDCon very useful, this time in particular it was good to get some high bandwidth discussions over ARM related stuff with people that I’ve been speaking with daily on IRC and telephone over the last couple of months. It was also good to get it some more attention in the wider Fedora community to give people opportunity to hear about the things that we’re doing, the things we’re aiming to do and to ask questions about things like aim to promote ARM to a primary architecture.

FUDCon Blacksburg Day 1

So day one was very busy. Most of the day was spent discussing and planning various components of the ARM secondary architecture. We ran a number of sessions from koji infrastructure planning and development, building rawhide on ARM, defining requirements to promote ARM to a primary architecture. There was a LOT of information to digest and a lot of discussion had. It was all very productive.

From 16:00 we had a Fedora Board meeting to discuss board goals and Fedora goals and the success and/or failure of the current goals. We also discussed board communications with the rest of the community and projects for the board for the comming year.

Other than that even the hallway discussions were fairly high bandwidth as well and it was a very busy. I very much enjoyed sitting down for a few beers in the evening to discuss some of the many things I do within Fedora and other side projects and hobbies.

FUDCon Blacksburg Transit Day and can someone bring a spare Dell laptop charger to FUDCon

The flight was some what uneventful….. Immigration was WORSE than usual. It took 2 hours because for some reason they wouldn’t open more than four positoins at once, the queues were out the door. The there was the usual dicks that are the TSA with their terrible attitude that turns even worse if you decide to opt out of their useless scanners that will scan through everything and still require you to strip everything off.

I then discovered I’ve left my laptop charge behind in the early morning rush to get out of the house 🙁 So if anyone has a spare Dell Latitude E-Series or D-Series laptop charger they could throw in their bag I would be for ever grateful (my laptop is a E6410 but I’ve used all E and D series with it).

Repeat: Anyone coming to FUDCon that has a spare Dell charger I can borrow over FUDCon I will owe you beer (or what ever your fave beverage is!).

Preparing for FUDCon Blacksburg

Well its almost time for FUDCon Blacksburg. In fact this time next week I will have already been in Blacksburg for over a day. I’m really looking forward to the event but I know it’s going to be very busy. So what have I got planned for my time in Blacksburg? The list in fact is pretty small:

  • Fedora Board Business: We have a open board meeting planned. There will no doubt be a numbner of things to discuss and there’s already a number of things on the list.
  • Fedora ARM and Secondary Architectures in general. There’s a lot of things to discuss with ARM from koji infrastructure, building rawhide, things we need to do to progress ARM to a Primary Architecture and a lot of other things from technical to procedural and process oriented. There will also be ARM based OLPC XO 1.75s and if the rumours are true possibly even a pre production Raspberry Pi

That looks like a small list but it would be very easy to fill an entire week with those two topics on their own. If I get a spare session here and there I would also like to spend some time attending some of the Cloud SIG sessions as there’s going to be some great stuff happening there too!

My Fedora 2011 in review

Its been a some what mixed year for me personally and I certainly won’t be massively disappointed to see the end of 2011. In Fedora land the year has been completely full on!

The first major event that kicked off 2011 for me in Fedora was FUDCon Tempe which as always was brilliant. Its always awesome to catch up with a lot of Fedora Friends and to drive forward various bits of the project that are sometimes easier done face to face, whether it be in a session or at FUDPub over a beer or two! It was at Tempe that our fearless leader threw down his first challenge of the year to me… to blog about what I’m doing in the land of Fedora on a weekly basis. I didn’t quite meet this goal but have still averaged a post every fortnight.

From April a massive amount of my time was taken up with helping the Fedora ARM in particular getting Fedora 14 built for the XO 1.75 but also massively fixing packages upstream to build on ARM. With the F-14 release for the XO out I’ve been concentrating on F-15+ and later for both hard and soft floating point. The ARM project still continues to consume massive amounts of time and has basically covers a lot of the things I wanted to achieve as part of the Fedora Mobility SIG.

May saw me head off to the excellent Red Hat Summit for the second time. I was there for work and as always its very useful for the type of work I do for my $dayjob. I helped out on the Fedora booth and as always spent quite a bit of time with a number of Fedora contributors. It was there that our fearless leader convinced me that standing for The Fedora Board was also a good idea.

Fedora 15 was next up on the list, of course. A great release with gnome-3 and all sorts of other goodies. Not so good for Sugar on a Stick which had massively broken networking 🙁

October saw me attending FUDCon Milan which was a rather shorter than some FUDCons but great as always.

Fedora 16 was much more successful for Sugar on a Stick and we shipped probably the best release ever. In fact I think Fedora 16 was an awesome release.

I’m looking forward to 2012 greatly, both in the land of Fedora and in my personal life. I’m looking forward to seeing ARM progress onwards and upwards and hopefully make it to a primary architecture in the coming year. I’m not one for resolutions, I think ongoing reviews are much better so I look forward to working with all my Fedora friends throughout the coming year.

FUDCon Milan

Well another FUDCon is over, they seem to go quickly! This one seemed shorter than previous ones and that’s primarily because it was. Most FUDCon events are 2.5-3 days. This one was two. That’s not to say it wasn’t productive as it most definitely was and I was in Milan from Thursday until Tuesday along with a number of others which meant I got the opportunity to discuss a number of things face to face with a number of people that I usually speak to daily on IRC or other online means so it was certainly productive. And as always its good to meet up with friends face to face, meet new friends and eat, drink and generally be merry 🙂

So what did I achieve? Quite a bit but a lot of it isn’t directly accountable. So a bullet point list of what I can remember:

  • Discussions with dgilmore and others about ARM and various plans for F-15/16/17 and beyond
  • Secondary architectures!
  • Anaconda with wwoods and others including support for ARM, images, tablets, Live USB keys without overlay file systems to avoid corruption issues with SoaS deployments
  • Support for various OLPC XO and Sugar Labs issues we see or envision moving forward
  • Demos of ARM on the XO and other devices
  • A governance session
  • A lot of other random conversations
  • FUDPub 🙂

So what else did I get up to? Gelato with jsmith on a number of occasions, a day trip to Venice with spot and dgilmore including a VERY early start and some geocaching in both Venice and Milan and some sight seeing around Milan.

So a big THANK YOU to the organisers of FUDCon Milan with both my Fedora Board and personal hats on! 😀