Fedora Flock Presentations

For Flock I did three presentations. Two of them didn’t have slides and I’m not sure if they were recorded but the “Fedora ARM – State of the Union” was pretty well covered. My slides are here and you can watch the whole talk on YouTube here and a transcript here.

I’ll do further write ups of my various Flock activities over the coming days.

Killing zombie processes

So not really the zombie apocalypse but useful. I sometimes get zombie processes with my GNOME desktop and a lot of the GNOME apps won’t restart with a zombie brother hanging around. While sometimes the only resolution to a zombie process is to restart the computer I often find this helps.

cat /proc/1111/status | grep PPid

Where 1111 is the Zombie process number. This will give you the parent process ID (PPid). For GNOME apps this is often the gnome-shell process. Thankfully the GNOME guys allow you to restart this while all is running without losing your entire desktop session so a simple

killall -HUP gnome-shell

will restart the shell and generally kill off the Zombie so you can get back to reading your email or what ever it is you were trying to do much more quickly without the pain of a full session restart 🙂

RoundCube mail on RHEL/Fedora

I run my own mail server and for years I’ve used squirrelmail as it was a simple interface that just worked and I never really use many of the advanced web mail features anyway. The squirrelmail project hasn’t really advanced a whole lot though and while they do keep up the security fixes and make it work with the latest releases of php there’s been no real development in quite some time so with the move to a new hosted server I decided it was time for a change. I decided to go with RoundCube Mail. The instructions are identical for Fedora as well.

Initial Install
RoundCube is packaged in EPEL and while 0.8.6 isn’t the latest release the ability to “yum install” works for me.

Install the roundcubemail package, mysql and mod_nss for HTTPS (or mod_ssl if you prefer), I’m assuming here you already have a working imap/smtp server. So just a:

yum install roundcubemail mysql-server mod_nss

I plan to use a MySQL DB so to create that I did the following to create the DB and db user:

# mysql -u root -p
mysql> create database roundcubemail;
mysql> create user roundcube;
mysql> GRANT ALL PRIVILEGES ON roundcubemail.* TO roundcube@localhost IDENTIFIED BY 'changeme';
mysql> FLUSH PRIVILEGES;
mysql> quit
# mysql -u root -p roundcubemail < /usr/share/doc/roundcubemail-0.8.6/SQL/mysql.initial.sql

To configure RoundCube to access the database edit /etc/roundcubemail/db.inc.php:

$rcmail_config['db_dsnw'] = 'mysql://roundcube:changeme@localhost/roundcubemail';

To configure RoundCube for mail server settings edit /etc/roundcubemail/main.inc.php:

$rcmail_config['default_host'] = 'localhost';

The only config changes I made for the mod_nss was to change the default port from 8443 to the standard HTTPS port of 443 by editing /etc/httpd/conf.d/nss.conf

Configure RoundCube URL and various other apache config like enforcing HTTP edit /etc/httpd/conf.d/roundcubemail.conf:

Alias /webmail /usr/share/roundcubemail

    RewriteEngine on
    RewriteCond %{HTTPS} !=on
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

I'll likely tweak the apache config a little more but that ensures its running over SSL for now.

Finally restart apache to make it work 🙂 with a "service httpd restart"

Fixes to enable running SELinux in Enforcing mode
Initial on testing I wad getting an error in the apache logs about writing to logs. I figured this might be a SELinux error so I did a quick setenforce 0 to test my theory and I was right on.

[error] [client 192.168.100.1] PHP Warning:  Error writing to log file /var/log/roundcubemail/errors; Please check permissions in /usr/share/roundcubemail/program/include/main.inc on line 1965

To fix running in enforcing mode I needed to change two SELinux settings. The first was to set the log directory with http_log and the second was to allow httpd to connect to the network. Fixed easily with these two commands:

chcon -R system_u:object_r:httpd_log_t:s0 /var/log/roundcubemail
setsebool -P httpd_can_network_connect=1

Now I don't need to upset Major Hayden or make Dan weep 😉

Fixing the date display
The date column in the mail initially didn't display. Looking at the apache logs I needed to set the php date.timezone setting in /etc/php.ini. I have users in a number of timezones so I was a little concerned at first of chosing one in particular but it doesn't seem to make much difference. Just search for date.timezone in /etc/php.ini and your good to go.

Conclusion
I like my new mail setup. The migration has enabled me to clean up a number of things I've wanted to for some time and just never got around to it. All the commands are basically identical on Fedora or any other EL6 clones. Hopefully it will be useful for others, and of course feedback is welcome.

Fedora Board elections 2013

So the eagle eyed amongst you would have noticed I didn’t throw my hat into the ring for another term on the board. By the time I pass the baton on to the next candidate around July time frame I will have down well over two years on the board and while it’s not a hard requirement I believe it’s enough and it’s time for someone else to step up and contribute their effort to the future direction of Fedora. I wish all the candidates of all the elections the best of luck.

the newish Fedora ARM koji build platform

I’ve been meaning to do a post on the Fedora ARM koji build platform. I’ve been sort of busy with my day job including a load of travel as well as concentrating on Fedora ARM builds and kernel issues.

Up until the end of January for well over two years the arm.koji.fedoraproject.org platform that we all love had been hosted at The Seneca Centre for Development of Open Technology. In that time the platform was tirelessly run by Chris Tyler and his team. Over the years it has had a number of upgrades. It originally started off as a bunch of Guru plugs with a few higher end boards, over time we added in Panda Boards and then remote access to more enterprise level devices. With the increasing number of releases across two architectures it’s been clear for some time that the platform was reaching end of its usable state and building for three releases (F-17 to F-19) it was clear it was no longer up to the job.

So at the end of January the migration over to a new platform went live. The new platform builders are based on the latest revision of the Calxeda Energy Cards in the form of Boston Viridis server chassis. We have 96 of the Calxeda EnergyCore nodes spread across 4 chassis that are plugged into the usual enterprise class storage and networking. The ECX-1000 units are quad core 1.4ghz with 4Gb of RAM, local SATA storage and Gig ethernet networking which is a slight improvement over the old devices 🙂

The improvement in stability and builds has been fabulous and it’s not until you get a shiny new platform with all sorts of enterprisey features that you realise things like loopback filesystems over NFS over 100Mb ethernet over USB on pandaboards with dual core 1Ghz processors and 1Gb of RAM aren’t the best platform in the world! Nicely a number of random build issues unsurprisingly just disappeared and it’s allowed me to focus on fixing packages and other more useful things rather than a group of us coaxing builds along and generally spending too much time dealing with issues we shouldn’t have. The mass rebuild ran nicely on the new HW and it banged it’s way through it quite happily with me battling to keep it full of builds and having trouble keeping up with any issues.

Fedora 18 on ARM retrospective

It’s been a while since I’ve done a general update on Fedora 18 on ARM. Post release always tends to be a good have a look at what’s been achieved, where we are and what we have planned for the next release. At the beginning of last year I was trying to blog weekly on my Fedora involvement, or at least tech involvement that’s related to Linux, I need to get back into that habit, if for no other reason than to stop people bothering me ;-). As a result this is a bit of a mammoth post at around 1500 words!

Overall Fedora 18 was a reasonably standard release process for ARM. We had the usual package build issues with lots people scrambling to help us get things fixed even if some took longer than hoped, kernel issues also featured but that’s not unusual as there’s been a lot of upstream ARM churn! The extended release process wasn’t as bad for ARM as it was for some of the other teams as it allowed us to fix up some more bits and get them landed in the release. So what were the major achievements of this cycle?

  • Supported Devices: The number of devices we support has actually gone down due to the Efika devices not being updated to run on the newer kernels and then being completely dropped for the 3.7 cycle due to lack of DeviceTree support. That being said the number of devices that work with only little intervention is on the way up and this should increase dramatically over the Fedora 18 life cycle due to the unified kernel. People are also starting to play with remixes for devices and I have some ideas to support this better (see kernel section).
  • Packaging: The number of packages that didn’t build out of the total of 12,614 packages in the final F-18 GOLD release was a rather small 280 at mainline release. This has actually gone down since as we’ve had a couple of packages that were blocking a lot more land in the post F-18 updates flurry and more are on the cusp of landing too. I’m working through the remaining packages to document the remaining hold outs and classify them as 1) non ARM arch 2) simple fixes required 3) complex fixes required 4) language bootstrap required 5) unknown or 6) dependent on another package from 1-5.

    We’re also working on improving and optimising packages for ARM. Over this release cycle I’ve pushed a number of fixes to improve performance and operation of packages. I expect this to continue for a number of cycles to come as well as enabling more HW device support like UCM support for sound devices in F-19.

  • Kernels: I’ve spent an inordinate amount of time this release doing kernel work. I would never consider myself a kernel hacker but clearly I’m able to fool people as I was asked by a kernel developer if my employment by Red Hat was to become the ARM kernel maintainer, while chuffed at the complement I’m really not a kernel hacker!

    So what have we achieved in the kernel this cycle? Well we’ve landed the first iteration of the 3.7 “unified” ARM kernel which allows us to boot a single kernel on multiple devices. We have two devices currently supported in 3.7 with the qemu based Versatile Express emulation and Calxeda Highbank being the first candidates that have been tested to work, we have the Marvell mvebu support enabled in 3.7 too but there’s not enough driver support for it to work on real devices. The 3.8 kernel expands this out by adding IMX (yet to be enabled), SunXI (also known as AllWinner A1x) and some other devices. The A1x support is similar to the mvebu in 3.7 in that it’s just core SoC support so not really useful as yet. The unified kernel has allowed us to collapse a number of our kernels into one which has reduced our build times dramatically (one of the bullet points needed for primary arch) but it’s not all completely rosy. The unified kernel is still problematic and I don’t believe this is helped by its current lack of use. This is especially disappointing when Linaro don’t even build unified kernels for testing. It appears Fedora is leading here and I get to feel the pain 🙁

    Moving forward I’m starting to hack about on a new kernel idea to enable easier support and “Remixes” for non mainline ARM devices. Basically I want to do a remote clone of the Fedora kernel tree and setup a branch for each device we wish to support. From there I plan on pulling in the patches/trees needed to support the device. It’s not going to be pretty as some still use the 3.4 kernel etc but the planned outcome is an automated yum repo with a series of kernels enabling people to build their own install images for a much larger range of devices, a long with a central repo so that when/if the support goes mainline it’s easier for us to merge the changes in. Once this is working I hope for people to be able to submit new platforms via git pull requests. I’ll blog about this more as I get more bits working. I need to upshift my git skills and I suspect this will be an interesting challenge!

  • FUDCon and FOSDEM: Unfortunately due to personal commitments I was unable to attend FUDCon NA this year. I was some what amazed at the number of people that contacted me to ask why I wasn’t there and if everything was OK or just to express their disappointment at not catching up with me. Believe me it was unavoidable and I wasn’t just trying to avoid you all! From everything I’ve heard it was a great event with lots achieved both around the ARM and non ARM stuff but others have reported on this so I won’t bore you!

    On the plus side I did make it to FOSDEM this year! And what an excellent event it was! It’s an event I’ve been wanting to attend for 5 years or so. I was even recorded in a podcast by the excellent EMEA Cloud Guy talking about ARM, OLPC, all sorts of cloud computing tech as well as other things that I do in my day job. So if you really did miss my voice at FUDCon or are just having problems sleeping you can have a 30 minute odd fix there 😉

    There was a lot of things covered on the ARM side from ARM64 (or aarch64) through to the varying state of GPU support. I plan to write up more about this in other posts but it’s certainly going to be an exciting year for ARM development! You can mark my words 😀

  • Awareness: I’m constantly surprised about the people that get in contact with me about Fedora ARM support. It’s amazing and exciting the places people are looking to use and actually are starting to use it, it’s nice to know that the thousands of hours I and others are putting into Fedora on ARM aren’t going to waste. The awareness of the project both inside the Fedora community and in the wider Fedora ecosystem is expanding rapidly and that can only be a good thing! For example OLPC announced the XO-4 Touch at CES and it’s running Fedora 18 on ARM to boot.
  • Build platform: We’ve finally got enterprise hardware on it’s way. I was kind of hoping we would have made the switch a month a go but any moment now we’ll be moving the koji instance used for ARM builds over to a unified set of enterprise builders based on the Calxeda EnergyCore platform. This is in preparation for a push to promote ARMv7 to primary architecture. I must say, primary or otherwise, I’m really looking forward to having a unified set of builders running on enterprise storage and networking. While the existing platform has served us well it’s really starting to bulge around the edges while building for the three Fedora releases on two architectures. Look for the announcement Real Soon Now or just listen out to me celebrating!
  • Dropping armv5tel: We’ve decided to drop ARMv5tel support as of Fedora 19. Both Fedora 17 and 18 will continue to be supported for the entire mainline life cycle. The fact is that ARM themselves, with the introduction of the Cortex-A5, have stated that all cost points and use cases are now covered by the ARMv7 Cortex-A* series of designs and as the memory supported by the v5 series of chips is only DDR1 the devices were getting expensive to make. While there are a number of plug devices out there still the fact is there’s probably less devices there as there were PPC devices when it was decided to drop PPC so I don’t see it as a major issue. The Raspberry Pi is going to be supported by Seneca in a separate ARMv6 secondary arch effort so isn’t going to be left out.
  • Anniversary and statistics: Stats are always fun and I noticed the other day that it was in fact my two year anniversary of working on Fedora ARM builds, although like a typical male in a relationship I did forget it! In my defence I have also forgotten my own birthday in the past too 😐

    I pushed my first build on January 9th, 2011 and since then I’ve done a mere 82,876 builds via a lot of tasks (koji times out on that query!). I’ve also submitted well over 300 updates for F-17 and at a little over 125 not quite as many for F-18 putting me on the top of the updates list (sorry!) for both releases. They weren’t all for ARM though but I’m very glad F-18 is, currently, a lot less than F-17 which means that we’ve made progress on fixing ARM issues 🙂

Look for some more specific, less wordy, ARM related posts soon.

A change of scenery

I’ve been meaning to write this post for a number of months. Almost six in fact, I started writing it well over two months ago because I got an email over the weekend saying that it had been 90 days since I’d started at Red Hat. That’s right, I have a new job! I’d been at my previous employer for nigh on five years and I felt that I was no longer growing and that it was time for a change so when someone approached me about putting on the Red fedora I thought it was a too good an opportunity to pass up!

My official title is Senior Consultant as part of the UKI (United Kingdom and Ireland) and EMEA Infrastructure Professional Services team and I’ll be spending most of my time dealing with Enterprise customers dealing with RHEL and Cloud related things so my role doesn’t directly involve Fedora on a day to day basis. That’s not to say my role won’t involved Fedora as there’s certainly some opportunity there especially with the upstream enterprise related products such as FreeIPA, oVirt and before long OpenStack as well as many many others. It might even one day involve some ARM related work because there’s certainly interest there. Generally it won’t affect or change much in terms of my involvement as all, in most things it will just be business as usual.

One thing that has changed is the amount of travel I do has increased dramatically at least in the short term. Places I’ve been to so far include Helsinki, Munich, Paris and Cairo.

My team is recruiting more people so if you’re possibly interested in joining the Red Hat consulting team in the UK get on contact with me if you want some more information or want to have a chat about the roles.

Fedora ARM on the Google Chromebook

I’ve been asked numerous times about Fedora ARM’s plans to support the Samsung Chromebook so I thought I would do a quick blog post to outline our plans. In short “ABSOLUTELY” and “As soon as possible” are the answers 🙂

So of course we plan to support it. As a device the Samsung device is what we’ve all been waiting for in an ARM device in that it’s cheap, nice looking, has relatively decent specs and is all nicely packaged.

Unfortunately initially we’re going to need to support it as a Fedora Remix as there’s some bits that haven’t landed upstream that will require us to fork things and in particular the kernel. So in the short term we’ll do a remix and hope to have a usable version of that out soon and a more stable one not too far behind the Fedora 18 mainline release.

So what is there to be done I hear you ask? That’s a difficult answer but it breaks down roughly as follows:

  • uBoot / DevMode and how best to support it (and if we’ll be able to remove the horrible 30 second boot message)
  • Kernel support, both initially a fork and then as mainline.
  • Xorg / GPU support. We have the basics of this already in Fedora 18 but we want it to go as fast as possible 😀
  • Sound. Be aware that at the moment you’re likely to fry your speakers if you try to use it now. We don’t want that. It means we’re going to need to deal with UCM and associated support bits. It’s something I’ve only just found out about and I’m researching what we’re missing. It will be useful for all ARM platforms (and x86 too).
  • Improved HW support – basically make sure everything works as expected whether it be HDMI or WiFi.

That’s basically it (and it’s no small amount of work) but most of the rest of Fedora ARM hardfp should run just fine and dandy on it so all of that hard work is done. There’ll likely be some optimisations that can be made so it runs even faster but we should soon be able to support another stunning little device with Fedora ARM 🙂

Watch this space

GNOME 3 fall back mode

I’ve been aware of the possibility of gnome fallback mode being dropped for quite some time. I wasn’t aware a decision had been made to definitely do so.

This is somewhat disappointing to me as I know of a lot of users of it. For starters there’s the 2.5 million OLPC XOs that were in the field as of January 2012 (I’m led to believe there’s around 1.5 million more to add from this year) and there’s a lot of ARM devices that have no 3D support because it’s pretty much impossible to use ESGL with gnome-shell at the moment (yes I know there’s plans to refactor mesa and friends to cope with this but that’s not now).

This isn’t designed to be a Federico style rant and if it was it would be aimed as much at the MATE Desktop as the GNOME team. I personally believe MATE would have been better off targeting their resources at properly fixing the GNOME 3 fall back mode to do the things they miss from GNOME 2 as attempting to glue together the GNOME 2 desktop as at least all the core apps are not duplicated and they get to keep the newer code which ultimately will be a lot more maintainable moving forward through the shared development of code. I’m not sure why they went the other route.

Oh well ultimately I believe it will be yet another net loss for GNOME as possibly eventually losing 4+ million odd young users from OLPC deployments (and all the other people such as families that use the devices) when they leave school I think would be a loss for the community moving forward and then add to that the users of it on ARM devices and other old computers in the developing world I think they’re not even aware of the users they have.

N.B. As always the words and thoughts are those of my own and don’t represent anyone else’s opinions.

A long overdue status update of Fedora on ARM

Well I’ve been meaning on doing a status update on ARM as well as a number of blog posts for quite sometime but I’ve had some personal changes I’ve been settling into (more on that sometime soon).

So first and foremost we’ve announced (a slightly delayed) Fedora 18 ARM alpha. There’s a some of images there for supported devices but if there’s an image you’d like that’s not there feel free to come and help out to make it happen.

We’re looking pretty good for Fedora 18. We’ve got even more packages built than what we had for Fedora 17 and we’ve fixed and improved a lot of issues. I’ve spent a lot of time massaging the kernel into better shape with the help of the Fedora Kernel team and jwb in particular. We have people polishing desktop environments like XFCE and KDE with thanks to some of the XO 1.75s that people received as part of The Fedora Summer of Fun and I look forward to seeing more of the 50 people that received one coming and helping out. Not everything was quite so rosy though as the anaconda newui work has delayed more than just the main Fedora release. We do still have a lot of work to as while the vast majority of packages build and a lot of them even run there’s a lot of testing and optimisation to do.

But it’s not all about Fedora 18 as we’re already working on Fedora 19 (that is after all the reason we branch just prior to Alpha release)! The big thing to note to date is I’ve now landed the first support for a unified kernel. We should be able to boot around 4 different SoC platforms with the 3.7 kernel but expect this to expand rapidly in the 3.8 kernel and by the time Fedora 19 is released with what will probably be the 3.9 kernel I’m hoping we should be able to much easier support a lot more devices with just a couple of kernels.

The last couple of weeks has also been quite mammoth in general ARM news. In the shipping now news the first of the new ARM Cortex-A15 chips has made it to a great and cheap device. We hope to get it running a Fedora Remix soon. It’s the first device and the Samsung SoC based on the A15 and it’s the first of many we’ll see over the next couple of months.

In other not so shipping news it’s been mega ARMv8 / arm64 announcements week. It start with Andrew Haleyannoucing Red Hat is working on openjdk for ARM64. This was followed by Linaro announcing a ARMv8 image based on Open Embedded. Then it was AMD announcing ARMv8 chips and it’s working with Red Hat on them. And then finally today was the announcement of Linaro Enterprise Group. LEG for short (I’m already sick of the puns!). This is great news! And there should be the first starts of the Fedora bring up being available soon and a lot of work to do over the next year too!

Overall it’s been a busy six months in ARM and for me personally. I’ve known about a number of the announcements and it’s finally great to have the cat out of the bag. The soon to be released (hopefully sooner rather than later!) Fedora 18 is another pretty big step forward for ARM on Fedora. The next year is going to be a fun ride!

So one of the things that people often ask me is how do I get involved in ARM stuff. Most of the boring things like builds are mostly automated so I usually recommend to get a device and start using it, hacking on it and testing it. Even the qemu image is a good way to get started. There’s almost 12,000 source packages in Fedora so it’s impossible for us to test all use cases. There’s also a lot of bits of hardware even on the supported devices like the panda boards we don’t have time or the ability to test from GPIO to various sensors etc. So that sort of stuff is a good way to begin!