Below are 25 recent journal entries, after skipping 25</B.
past   future


  february.27.2012  12.04 pm
Red Hat Job Openings in the Fedora Universe

There are currently two job openings at Red Hat for people who wish to work on Fedora.

= Fedora Virtualization Maintainer =
Do you enjoy working with the Fedora community? Are you interested in working with the leading edge of the virtualization stack? We are looking for someone to maintain the core virtualization packages in Fedora!

Responsibilities include:
* Interract with the Fedora community on all things virt (Mailing lists, irc, FUDCon.)
* Virtualization Test Day organization
* Coordinate the Fedora feature process for virtualization features
* Maintain the fedora-virt mailing list
* Maintain the virtualization pages on the Fedora wiki
* Maintain the virtio-win drivers repository
* Maintain the virtualization preview repository for Fedora
* Primary package maintainer for the following Fedora packages:
qemu (qemu-kvm)
gpxe (ipxe)
* Work with the kernel team on kvm specific bugs
* Assist in bugs for related virt packages

Qualified candidates will have experience with:
* C, C++, Linux (kernel and userspace), KVM, qemu

(This opening is not currently live on the Red Hat website as far as I know, so please email me your resume/CV along with a short note about why you think you'd be awesome for this opening. I am _not_ the hiring manager for this position, but it is Fedora focused.)

== Web Application Developer ==

The Fedora Engineering team is looking for qualified candidates to assist us in creating and improving Free Software solutions for the Fedora Community. Fedora only uses Free Software solutions in all of its infrastructure and hosting, and you can help us in that effort. Our infrastructure is mostly driven using Python frameworks, delivering applications and contents over the web to the Fedora user and contributor communities. Travel requirements are minimal, although, participation at relevant conferences and events is expected. This position is a fantastic opportunity to develop new and interesting Free Software and Open Source solutions with a rich community of developers and users. Upstream contributions are an expected part of this position.

Existing team members in this role have created web applications like:
* Bodhi -
* Fedora Packages -
* Fedora Tagger -

These skills are the sort of skills that are desirable, however, it is not necessary that any candidate possess them all.

* Python
* TurboGears
* SQL Databases (especially Postgresql)
* Xapien
* Linux
* Pyramid
* Javascript

A bachelors degree or greater in Computer Science is preferred for this role, although, not necessary if there is extensive expertise in the desired areas for this position.

Candidates would not necessarily be required to relocate for this position.
If you're interested, please apply on the website, and send me an email telling me why you'd be an awesome fit for this position. I am the hiring manager for this position, so you would get the added bonus of working for me. ;)


  january.16.2012  01.02 pm
FUDCon Thoughts - Eucalyptus

At FUDCon, Greg DeKoenigsberg said something that resonated with me (and I paraphrase), but it was this: After 10 years of being at Red Hat, it is easy to forget that most of the rest of the world, even in the IT industry, doesn't really understand Open Source. To most folks at Red Hat, it is intuitive, something we understand/accept, myself included.

In listening to Greg talk about Eucalyptus, I forced myself to set aside any biases I may have had about that company or their history, and really try to remember that for better or worse, they were a company founded by graduate students, who really didn't understand business or open source, and they initially operated on the assumption that in order to survive as a new company, they needed to make decisions to establish the business. A lot of those decisions weren't Open Source friendly, but it is far more likely that this was out of ignorance than malice.

When I look at Eucalyptus today, I see two very smart decisions, and those decisions have names: Andy Grimm and Greg DeKoenigsberg. I've worked with both for years, and I have a huge amount of respect for both. Even above that: I know that they truly get Open Source, and that neither of them sees it as a loss leader for a proprietary offering, or a "Childrens Edition". By bringing them onboard at Eucalyptus, at least in my eyes, it shows that the decision makers at Eucalyptus:
* Realize that they're not operating as a true Open Source Company today
* Want to change that sooner rather than later (understanding that change doesn't happen overnight)
* Are empowering people with the right expertise to enact that change

I respect that greatly. Greg told me about some interesting changes coming soon for Eucalyptus, and since I'm not sure what is public and what is not, I will keep those under my hat, but I am definitely going to be watching them closely over the next year or so. Actions speak louder than words, but so far, they're moving in a better direction. I hope that it leads them away from an Open Core model.

ADDENDUM: It has been pointed out to me that there are other awesome Eucalyptus folks who understand Open Source, like Garrett Holmstrom, it was in no way my intent to omit them, or imply that all Eucalyptus employees who are not Greg or Andy do not understand Open Source, this is obviously false.


  october.26.2011  11.01 am
aggressive, repulsive, spam

I've been using the same email address for more than 10 years now, so it would be putting it mildly to say that I get a lot of spam. Lately, I've noticed a trend, a set of emails that don't trigger the spam detection routines at all, but they're clearly unwanted and unsolicited. They look like this:

To: Tom Callaway <>
From: "Arnie Boigner" <>
Subject: RE: Meeting Invitation

I hope my last message found you well. I wanted to follow up and see when we could speak further about my company's virtualization storage solutions.

Thank you,

Arnie Boigner
----- Original Message -----

From: Arnie Boigner []
Sent: Thursday, September 29, 2011
To: Tom Callaway
Subject: Meeting Invitation

Hi Tom,

I trust you're well. I'll be very brief. If you're like most of the companies I work with, you're saving money through virtualization, but you're spending more and more on data management. That's tough in a good economy, much less the one we're in. Executives tell us they struggle with:

Best Regards,

Arnie Boigner
Account Executive

Now, noteworthy is the following:

1. Arnie Boigner is a real human being. He played football at Ohio Northern University from 1989-1992 and has 3,068 yards rushing.
2. He works for Nimble Storage, a recent startup, who neither I nor Red Hat have any relationship with, nor will we likely _EVER_.
3. His email isn't coming through a Nimble Storage owned domain, it's going through a dummy domain ( This domain is owned/managed by, a company owned by Go Daddy CEO Bob Parsons that exists solely to hide true site ownership and circumvent IANA rules. As if you expected better from Go Daddy, the scumlord of domain registration.
4. The "Original Message" quoted never was sent to me. It's entirely made up.
5. I'm getting a LOT of these, all with different unique sounding dummy domain names that all refer to different companies, and similar subjects, such as "Re: Schedule Discussion", "Re: Brief Discussion", "Re: Schedule Availability", "Re: Request a Call"
, "Re: Schedule a Call", etc. See the pattern? Looks like it started at the beginning of October.
6. I'm getting them repeated in a loop. Arnie? He's "emailed" me the same spam on October 5th and again today. Note that this isn't him sending me one message, then replying to it, its the same email with bogus "Original Message" quoting at the bottom. (The earlier mail had Subject "Re: Meeting information", as opposed to "RE: Meeting information".

From this evidence, I can only conclude that there is a new "marketing" company using spammer techniques to try to get leads. The companies who have contracted with this scheme should be ashamed of themselves, and I strongly encourage you to not do business with them.

Companies I know (thanks to their toxic spam) to have partnered with this spammer:
Acorn Systems
Barry-Wehmiller International Resources
Caliber Point
CollabNet (Double shame on you.)
Nimble Storage
Results Positive

List of known bogus domain names in this racket:

All of these domains seem to be using the same DNS servers:
I don't know if those are the DNS servers that DomainsByProxy uses by default or not. is owned by Network Solutions, which appears to be a somewhat legitimate hosting company, so I'm guessing that they're not the spammers, just hosting their DNS.

Now, I'm not interested in filling out the DomainsByProxy form to report this spammer 20 something times, if for no other reason than I suspect strongly it will result in GoDaddy spamming me (they want a full header copy of each spam email). I'm mostly relying on the internet to shame these companies.

I'd love to know who this spammer is, so I can shame them equally.


  october.09.2011  11.14 pm

I got this email, and was about to delete it, but it doesn't seem like the usual spam...

Subject: Hello Sweetie!!!

My Name is Doctor River Song. from Planet Earth presently incarcerated in(Stormcage Containment Facility). I was impressed to seek my soul mate, and also i like to establish a long lasting relationship with the Doctor. I will be waiting for your reply so that i will give you my picture and more about me, but not too much, spoilers!. Thanks,waiting to hear from you soonest.

Love From,


  september.30.2011  12.16 am
FUDCon Milan: Day -1

Dennis Gilmore and I traveled together to Milan, starting from Boston (Dennis was in Boston for Red Hat meetings). We flew from Boston to Madrid on Iberia, got in the exit row, which was nice, but it was the one at the beginning of the section, so, no underseat storage. I didn't sleep, because I have a lot of trouble sleeping on planes. When we got to Madrid, it was a bit confusing, but we managed to find our gate and wait the two hours for our flight to Milan. In contrast to our half empty flight to Madrid, the flight to Milan was _PACKED_ full, mostly of teenage girls on some sort of school trip (I can only assume they were chaperoned, but I didn't see any). Iberia doesn't have much leg room at all on their smaller planes, so that flight was pretty miserable (and loud).

When we arrived in Milan (Malpensa), we had to wait a long time for Dennis's checked bag, but I'm quite used to extremely slow baggage service at European airports. Then we cleared the simplest customs process I've ever seen, which was basically just one handler with a large dog. As we approached the dog, he started to go nuts jumping and signalling all over a guy in front of us who was covered from fingertip to hairline in elaborate tattoos. We just walked around the scene, giving the dog a wide berth, and we were out.

We were hoping to be able to find a pre-paid SIM card so we could have reasonable phones with data, but we couldn't find it at the airport, so we decided to just head to the train. Having heard some horror stories about trying to take cabs in Italy, I had planned out the train route to our hotel in advance. We got tickets for the airport train into the city (after making sure we got onto the right train to Milano Centrale), which was rather nice and uneventful. Once we got to the Milano Centrale station, things got a lot more complicated, as we needed to change to the Metro subway system. We found the Metro eventually, but buying a ticket was... difficult. None of the machines would take any of our plastic, and neither of us had any Euros. To make things more complicated, there were Gypsys (at least, they were dressed as such) standing firmly in front of all of the ticket machines eagerly pressing the buttons for people on the machines, especially obvious tourists with bags. I knew that there was some sort of scam going on there (the internet lists quite a few possibilities, including simple pickpocketing), so I ended up buying our Metro tickets from a nearby store after pulling Euros from an ATM. The clerk didn't have change for 50 Euros in her register till, but she made change out of her purse. I suppose we looked pathetic. :)

Finally, on the M3 Yellow Line train, we got off at the station I'd mapped out, but as I had feared, European street signs are either in-congruent with Google Maps or just outright absent. A few wrong turns and I got my bearings and we found our hotel. We met up with Peter Robinson and Jared Smith and got a late lunch at a nearby Italian (what else?) restaurant. I had a quite nice Basil Pesto and Mozzarella Lasagna, Dennis had Beef wrapped cheese ravioli, Jared had Tuna Tartare and Peter had Steak Tartare. Since I was really dead on my feet at that point (Dennis was much better off as he slept on the long flight), I didn't have anything to drink except water, even though the bar kept pouring interesting looking shots for one of the other tables. There is plenty of time for that later. :)

After eating, I went back to the hotel and just crashed. I couldn't figure out the hotel wireless, and I was just so exhausted that I decided to watch a movie on my laptop to stay awake as long as possible, but eventually around 7 PM, I just gave in and slept. Of course, I woke up at 10 PM, so I took a shower to hose the stink of travel off of me (honestly, one of the better showers I've had in a European hotel, which isn't saying much, but it was hot and strong). I ran into John McDonough and his daughter in the elevator to the lobby, where they let me know that people were doing the traditional "Cafe American" thing for dinner and drinks, but I just wasn't up for it, so I passed along my hellos and got the wireless access info from the lobby. Went back to the room, got on the VPN through Amsterdam, and caught up on email. Now, I'm going to try to force myself to sleep some more and be ready for tomorrow.

Some minor technical notes:
* Chromium 14 "stable" is building now for all targets (updates to the chromium-stable repo coming as soon as they're done). I had to disable Native Client support (NaCl), because in order to build it, you need to have the "Native Client SDK", which is a forked and modified copy of an older GNU toolchain (binutils, glibc, gcc 4.4). I looked for the source code for it at length, but the only thing I found was a set of gitweb repos that I couldn't figure out how to clone (;a=summary for example). If you can tell me how to clone that repo, I would appreciate it, as I would like to try to package those components up and then use them to build the Chromium NaCl support.
* I had to update libjingle to 0.6.0 in order to get Chromium to build, then I had to patch Chromium to support the 0.6.0 changes. Usually, its the other way around, where I have to copy changes from Chromium's bundled libjingle fork into the libjingle upstream tree, but I guess I happened to catch it on the opposite end of the swing. Still not entirely sure how that relationship works within Google, but it sure doesn't look healthy at this angle.
* Noticed still more bundled code projects in Chromium (in case you're curious, the number just keeps going up, not down). I haven't had the time to really try to unbundle them, and a lot of them are "Google" owned projects, which means there is a good chance that the code doesn't work in a standalone capacity anyways (skia, I'm looking at you).

FUDCon Stuff:
* I'm really excited to be here, I especially want to see old friends from previous FUDCons and meet new friends. Please don't be shy, if you see a tall shaggy haired American with a nametag that says "spot", please come by and say hi. Let me know what you're excited about, what you like (or don't like, but please be nice).
* I don't have any planned topics, but if there is something you'd like me to talk about, please let me know.


  september.20.2011  12.00 pm
Want to work for Red Hat?

Normally when I say something like "Want to work for Red Hat", I have a specific position that I need to fill, but in this case, its a bit of an open book. Here's the ground rules:

* You must be willing to start work before March 1, 2012.
* You must have an impressive resume with a proven track-record of Free Software/Open Source Software experience (although, you do not necessary need to be a coder)
* You must have a skillset that Red Hat needs, specifically, think in terms of what you could be doing for Red Hat that it is not currently doing (or needs help doing)
* Bonus points if you're already a part of the Fedora Community, but this is not strictly a requirement. If you can provide references from another notable FOSS community, that will also give you bonus points.
* Bonus points if you're geographically located next to a Red Hat office. (There are some places we can't hire from, and relocation, while not impossible, makes things much more complicated, especially if there is a Visa requirement, but if you're awesome enough, anything is possible.)

Respond quickly via email ( with your resume/CV and sales pitch as to why I should consider you for a job at Red Hat. This is a limited time opportunity, and I guarantee nothing. :)

ADDENDUM: Just sending me your resume isn't going to get you anywhere. You need to sell yourself to me, tell me what you want to do at Red Hat and why I'd be a fool not to hire you.


  august.12.2011  07.49 pm
Friday in Fedora

* Generate a chromium-ffmpegsumo-13 package, easier than trying to backport changes from 14.
* Push irrlicht f15 & f16 update to stable
* Discuss cloud with Seth
* Kick Chromium repeatedly in the face
* libconfig 1.4.8 update for F16 and rawhide
* Chromium built (and for F16 even). Now, I just need to build it for all targets and test it.


  august.11.2011  04.09 pm
Thursday in Fedora

* Start working on Chromium. Again.
* Branching request for freewrl
* Branching request for perl-Switch
* Gather Fedora goodies for Linuxcon Vancouver
* Provide quote for SPDX announcement
* Switch to chromium-ffmpegsumo-14 for the stable branch (13)
* Patch chromium to support cups-1.5
* Patch chromium to support icu-4.6
* Patch chromium to fix more gcc 4.6 issues
* Patch chromium to fix glibc malloc issue (backport from chromium trunk)
* Reconsider Fedora branded keyboards, I'm not sure anyone really wants a $50 keyboard that isn't a Happy Hacker or a Model M.
* Import and build freewrl for Fedora targets (import EL versions, but I'll leave the building up to Steve)
* Import and build perl-Switch for F16 and rawhide
* Push updates for freewrl and perl-Switch
* Update ebtables in f16 and rawhide to 2.0.10-2
* Add RSS feed for freewrl Sourceforge files
* Chromium is still trying to build. Think of it like a beached whale, struggling to return to the sea.


  august.10.2011  05.35 pm
Wednesday in Fedora

* Improved my spam filtering in Thunderbird, saves me quite a bit of time
* More research into Fedora branded keyboard possibilities
* Finished xmedcon review (bz714328)
* Proposed way to prevent a python package from being installed on a RHEL 5 system where the python version is too old
* Fedora Packaging Committee Meeting, short but productive.
* Reworked freewrl package to subpackage out browser plugin, tested reported crash but could not reproduce with mesa.
* Start working on new chromium-stable packages (13.0.782.112)
* Create f16.i386 VM on treefrog
* Clarified proper packaging and licensing for Adobe PCFI
* Notice that is gone in perl-5.14.1-185.fc16
* Identified bug in perl-POE-Test-Loops (it provides perl(Switch) but should not) (bz729768)
* Created new Review Request for perl-Switch (bz729777)
* Very amused at a chromium bug report where someone submits patches but refuses to sign the Google CLA, and is described as "a denial of service attack on chromium".
* Discovered that Fedora custom keyboards would end up being $50-$60 per keyboard, depending on how many we ordered and whether we got USB wired or wireless ones. They're not Model M or Happy Hacker, just a full length modern (aka, non-clicky) US keyboard. Smallest order I could do would be 36. The art is painted onto the keyboard, not just key caps.
* Slightly less amused to discover that Google didn't bother to backport their gcc 4.6 fixes to their stable branch.
* Pass -Wno-error=uninitialized to chromium. Why be safe when you can be F-A-S-T?
* Sit in on the F16 Alpha Go/NoGo meeting. Think we're gonna slip.


  august.09.2011  05.25 pm
Tuesday in Fedora

* Only a hundred or so non-spam emails to read in INBOX
* Push tkimg-1.4-4.fc16 update to stable
* Fix up issues in freewrl review (bz726210)
* Point out mistake in scriptlet change to R package and fix it in git
* Try to figure out why system-config-date doesn't think either ntp or chrony is running on f16
* Aha! chrony didn't get installed... but why?
* comps shows the change happened properly on 07/22/11 16:39:05
* /root/install.log from the f16 alpha RC1 confirms that chrony is not in the transaction set. Maybe the RC1 was spun off old comps or before 07/22?
* Update treefrog base system to Fedora 14 via yum
* More updates on xmedcon review
* Edit R updates to have fixed packages
* Office receptionist transfers Adobe sales cold call to me, I got to explain that no, we do not use Photoshop. Tempted to ask to be transferred to Flash division so I could give them some useful feedback about x86_64.
* Discussed s-c-date and chrony issue with notting, notting pushed comps change to resolve it so that comps is present on normal install
* Explained what cla_redhat does and why no one needs it. :)
* Work on review of lzma-sdk (bz596461)
* Research ""
* Rework gambas3 package to not need write permissions on example dirs
* Work on making lzma-sdk a bit more sane and normal, with the intent of making it work with upx
I killed several hours on this, I feel so dirty, but I got it working.
* Confirmed that javascript packages are an exception to the "no-bundling rule".
* Updated libev to 4.04 in f-16 and rawhide, fixed manpage location bug (729462), pushed update for f16
* Registered #fedora-ppc, lifted the topic lock that was preventing them from updating the topic from 2008. :)


  august.08.2011  04.58 pm
Monday in Fedora

I suppose this is a sort of a test. I'm going to keep a little log of everything I do during the day related to Fedora. We'll see how far I get.

* Arrive at work, pull out my main laptop hard drive, put in the Western Digital SSD.
* Download Fedora 16 Alpha RC1 DVD image, burn to disk with k3b
* Boot DVD on my Lenovo Thinkpad T500
* Remember how slow the DVD drive is in my Thinkpad
* Installer starts, but is useless because my laptop is docked, thus, connected to a second monitor, and the installer screen is distortedly spanning both monitors. Show it to Chris Lumens, who informs me that there is a "bug filed for that and everything else". Disconnect monitor from dock, reboot.
* "systemd[1]: Failed to fully start up daemon, no such file or directory" Seems harmless though.
* Partition disk with defaults, uncheck "use LVM". Select "Graphical Desktop", this is mostly to test the install, not for long term use (although, I might migrate to this SSD at some point).
* Read while installer chews on my DVD drive in "Starting installation process"
* Read LWN while packages install.
* Daydream about pinball machines. Boy, this DVD drive is S-L-O-W.
* Packages completed: 1207 of 1207 (finally!) Installer sits there without any indicators for five minutes. I might know that it is doing post-transaction operations, but will a random user? :P I still think you should be able to play a minigame while it installs. Caldera had Tetris!
* Installation done!
* Plug external monitor back into dock.
* Reboot. Man, grub2 looks ugly! I guess it isn't themed yet.
* Wow, this SSD boots FAST!
* Firstboot has the "Synchronize date and time over the network" greyed out. Maybe ntp isn't in the default comps set? (it should be)
* Also, smolt window in firstboot seems to be broken. Big empty window where the profile should be.
* Fedora 16 Alpha background is pretty nifty. Would prefer to see more sealife, like maybe an octopus, but hey.
* Poke around GNOME 3, see no obvious bugs. (please, no trolling)
* Shutdown, reinsert old slow 7200RPM SATA drive.
* Start process of catching up on email (I don't check my Red Hat or Fedora email over the weekend). 405 new emails in INBOX (after spam filtering and mailing-list filtering). More than half of it is still spam. :/
* Push libjingle updates into stable to Fedora 14 and 15 (chromium 14 will need them)
* Send Marek Kasik's patch for poppler 0.17.0 support to Gambas upstream (
* Tell Paul Howarth that I am fine with him owning EPEL-4 branches for perl-Tree-DAG and perl-Test-Warn, as well as having commit access to devel so he can push overrides in bodhi.
* Read unread messages on fedora-devel-list, light weekend there.
* Look at failed rawhide build of libmx, adjust patch to fix compile with gdk-pixbuf, chain build with anerley.
* Rebuild easystroke for f-16 and rawhide against new boost
* Rebuild nagios-plugins-snmp-disk-proc for f-16 and rawhide against new net-snmp
* Fix libmx again, this time account for the GIR
* Push f-16 update for easystroke (
* Push f-16 update for nagios-plugins-snmp-disk-proc (
* Read fedora-legal email queue (all spam)
* libmx builds!
* Read RSS feeds while I grab a quick lunch (, groklaw, among other non-Fedora related ones)
* Put in package branching request for sar2 (bz713581). It is a helicopter flight sim.
* anerley built in rawhide!
* Kicked off Fedora-16 Alpha RC1 install on my KVM system (treefrog)
* Update R specs in git to ensure that "less" is present in the BuildRoot, and thus, selected as the default parser. Decide that the issue is not worthy of pushing an update.
* sar2 branched, do the first set of commits, builds, and updates (
* start review of xmedcon (bz714328)
* fix libtpcimgio and libtpcmisc to have PIC enabled shared libraries
* manual forced buildroot overrides for libtpcmisc builds (except rawhide) to get libtpcimgio to build against the new shared lib
* Build libtpimgio for all active targets against new libtpcmisc with shared libs
* Add the missing scriptlets for icon-cache handling to sar2, kick off new builds
* Remove the forced buildroot overrides for libtpcmisc builds, push updates for libtpcmisc and libtpcimgio for f14, f15, f16
* Delete the old (unpushed) updates for libtpcmisc and libtpcimgio to avoid a bug in bodhi where it is possible that the -3 build could get tagged, not the fixed -4 build.
* Manual override for libmx f16 to build anerley
* Ninja edit sar2 updates to switch to -2 package with proper scriptlets
* Posted my two waiting Review Requests (gambas3, freewrl) to in the hopes of getting them reviewed
* Kicked off anerley 0.3.0 build for f16
* anerley built for f16, untag libmx override, push update for both


  june.06.2011  01.49 pm
Firefox updates

Some quick updates:

The "Firefox4" repositories for Fedora 13 and Fedora 14 have been updated to 4.0.1:

I have created a "Firefox5" repository for Fedora 15 (currently at beta3). Just like the firefox4 repo, this provides xulrunner5 and firefox5 packages which install in parallel to the system xulrunner/firefox. I have no plans to backport this to Fedora 13 or 14, because that would mean that there would be a situation with three different copies of Firefox on the system at the same time. If you want to rebuild for older targets, go nuts.


  may.10.2011  12.24 pm
Freeing up a Fedora Board seat

As you can see from the Fedora Board History, I have held Elected Seat #5 since July 2008, and while I am rather proud of what we have accomplished during my tenure on the Fedora Board, I do not feel that it is healthy for a community to be run by the same people forever and ever. Accordingly, I will not be seeking re-election to the Fedora Board at this time. (I reserve the right to run in a future election, but I have no immediate plans to do so.)

Mohandas Gandhi once said "We need to be the change we wish to see in the world." I believe this is true, so I plan to spend more time working on improving Fedora at a technical level.

Lately, there has been some noise from people who are unhappy with Fedora, and I would encourage those folks to think about running for election to the Fedora Board or the Fedora Engineering Steering Committee. You can put your nomination here: Fedora Board:
Fedora Engineering Steering Committee:

Remember, Fedora is not a dictatorship (benevolent or otherwise), it is a community, and running for FESCo or the Board is an excellent way to have your voice heard.


  may.01.2011  10.36 pm
chromium-stable is here

Not too long ago, there was a blog post on Fedora Planet where someone was complaining that my chromium repo was crap, and that no one should use it because it was tracking the "unstable" branch.

Well, now, you have a choice. I've updated my scripts and made packages from the latest "stable" release, as tagged by Google (which, at the time of this writing is chromium-11.0.696.57 and v8-3.1.8). These packages have the same bundled libraries stripped out as my "unstable" packages, and are built from source on Fedora.

The main page for the chromium-stable repo is here:

You can download the file, and put it in
/etc/yum.repos.d/ .

If you have my other chromium repository enabled, and you want to switch to the "chromium-stable" repository, you'll need to do a few additional things:

1) Disable that repo in the /etc/yum.repos.d/fedora-chromium.repo file. (You can delete this file entirely, or just edit it and change enabled=1 to enabled=0.) If you don't do this, the newer packages in that repo will replace the chromium-stable repo's packages on every update.
2) Remove the chromium and v8 packages from that repo, because they are "newer" than the chromium-stable packages in versioning. You can do this by running: yum remove chromium chromium-libs v8 v8-devel
3) If you've disabled the old repo and installed the chromium-stable repo file, and removed the old chromium and v8 packages, you should be able to now run "yum install chromium" and get the "stable" packages.

Please note: If you were running the Chromium 12 packages, when you switch back to the "Stable" Chromium 11 packages, chromium will tell you that your profile is for a newer version of Chromium. In my testing, it seemed okay, but your mileage may vary. Also note that the "stable" Chromium 11 packages use the old logo, so it will look different from the latest Chromium 12 packages.

Finally, as a plea, if you see something that makes you unhappy about how one of my little spare time projects is working, instead of flaming and insulting me on your blog, why not drop me an email and ask me to fix it?


  april.12.2011  11.06 am

I'm really trying to be patient with GNOME 3. I remember when I switched from Enlightenment, but I just keep running into really annoying issues.

Issue 1: Multi-monitor support isn't working right at all. (Bugzilla 695737)
I have a LCD monitor mounted on a stand above my desk at work that is connected to a docking station. I put my Thinkpad T500 into the docking station and use it in combination with the LCD monitors in a vertical configuration. GNOME 3 sees the extra monitor, positions it properly, but won't let me open windows there or drag existing windows to it. :/

Issue 2: "Applications" noticeably grinds on my disk for several seconds before showing me a list
To be fair, I think I know why this happens:
A) I have a lot of programs installed.
B) It has to open all of their icons and cache them.

I just happen to think that the workflow model there is pretty poor, and the fact that this is slow on first load is a byproduct of that. Perhaps it should have a more reasonable default? Maybe only cache the first screen's worth of icons? Maybe pre-cache the icons on boot? Maybe put placeholders until the icons can be loaded, so I'm not sitting without any indicator of why there is a ten second delay?

Issue 3: Lack of minimize
I'm sure this makes great UI and I'm just too dumb to understand it, but my screen gets very cluttered, very quickly. Why not leave a minimize button in the title bar for each window? After all, I can still right click the title bar and minimize things, but I only discovered that by accident. (I suspect once I point that out in public, the right click minimize option will disappear too.)

Issue 4: NetworkManager
Oh, NetworkManager, I still love your autoconfiguration magic, but what have they done to your applet? I loved how I could right click you and quickly get to the configuration status, a GUI "ifconfig". Now I have to go into "Network Settings" and iterate through each possible connection. Also, I can't figure out how to turn off a specific connection, it seems that unless I go into the "Network Settings" options screen, I can't just disconnect a wireless connection without disabling wireless altogether. Oh, and when I USB tether my Nexus One, it shows up as an unlabeled "wired" connection.

Issue 5: Lack of clock customization
It took me several minutes of Google searching to discover how to adjust the date so I could see the day of the week along with the month, numeric day, and full time. I still can't figure out how to get it to show me the year, or the time in other locales (I work with international folks, and the support for other clocks at a click was very very helpful)

Issue 6: Lack of weather
Where did the weather go? I can't even figure out how to add it.

Issue 7: Lack of sane theming
Editing an XML file to change colors is not sane theming support. I cede that perhaps this is an area that will grow a nice GUI in the future, but ugh. Yes, it could be worse, but it could be a lot better. At a minimum, comments in the XML file would be very nice, as right now, my attempts at improving the theming are pretty hackish.

GNOME, I'm trying to be patient with you. We've been pals for a long time now, and I'm not giving up on you yet, but you sure are trying my patience.


  march.02.2011  04.47 pm
Fedora Says Thank You!

First of all, I do not want to take any credit for this idea, it belongs to John Rose, and has been bouncing around, trying to come to life, since July 2010. I am just happy to help it finally hatch!

John, Jared, Paul, Max, and I all want to thank Fedora users and contributors for everything that they do in our rich and diverse community. As a small token of our appreciation, we're giving away free Fedora stickers and case badges to any community member who sends us a Self-Addressed Stamped Envelope (SASE). It is our little way of saying thank you for all that you do.

Here is a short FAQ:

Q: What is a Self-Addressed Stamped Envelope?
A: A Self-Addressed Stamped Envelope (SASE) is an envelope with the sender's name and address on it, with affixed paid postage. In the UK, this is also known as a "Stamped Addressed Envelope (SAE)".

Q: Where do I send my SASE?
Red Hat
c/o Tom Callaway
314 Littleton Road
Westford, MA

Q: How long is this offer good?
A: We will return any SASEs that I receive on or before May 31, 2011.

Q: How does this work?
A: Basically, you write your mailing address on an unsealed envelope and attach paid US postage to that envelope (enough to send a letter to that address from the US). This is your SASE.

Next, you put your SASE in another envelope, which you then mail to me at the above address.

When I receive it, I will open your envelope, put in some nifty Fedora stickers and case badges, seal it up and drop it back in the mail to you.

Q: Is there a limit on how many SASEs I can send you?
A: Please, one SASE per person. If you have some sort of complicated situation, feel free to email me directly and we'll work something out (

Q: Is this offer open to contributors outside the US?
A: Of course! We realize that it may not be possible or simple for our international contributors to get their hands on US postage, so you are more than welcome to include an International Reply Coupon instead. (More info about International Reply Coupons can be found here: If you choose this option, please do not forget to include the self-addressed envelope!

Q: Can I decorate my SASE?
A: Sure. Just remember that it will be coming right back to you.

Q: Do I need to prove that I am a Fedora User or Contributor?
A: Nope. If you go to the trouble of sending us the SASE, we'll send you stickers and case badges.

Q: Can I tweet/dent/facebook/IRC/email/tell my friends about this offer?
A: Absolutely.

Q: Can I just put some money in the envelope and have you buy postage with it for me?
A: No. Any money found in the envelope will go into my 1 year old son's college fund. If postage is a problem, please send me an email (

Q: Will Fedora or Red Hat keep a record of my mailing address?
A: Nope. All we'll do is put stickers and case badges in the envelope, seal it up, and drop it back in the mail.

Q: How many stickers and case badges do I get?
A: One large Fedora Logo laptop sticker (3 inch) and four small foil "Powered by Fedora" case badges.

Q: What do the stickers look like?

Q: I have a question that isn't answered here, where can I ask it?
A: Email

A copy of all of this can also be found here:


  february.21.2011  04.43 pm
My summary of the Fedora Board Meeting (2011-02-21)

Since I was the secretary (please, no jokes), here is a brief recap of today's Fedora Board Meeting:

* An overview of last Friday's Alpha Blocker meeting was given.
* A discussion of the Fedora 15 Wallpaper situation took place. There is a "compromise" wallpaper as developed by the Fedora Design Team. If Upstream GNOME is okay with the compromise, then it will be used, if not, then the Design team will decide which wallpaper to use. If the Design team cannot (or does not) decide, the decision will be escalated to the Board to make.
* A discussion of the move from a Fedora Infrastructure maintained instance of Transifex to the hosted instance of Transifex (newer), due to lack of regular resources to maintain our local instance. This move took place on Friday, quicker than preferred, but the move was necessary to make it possible to have translations for Fedora 15.
* Mairin Duffy is taking a requested 6 week leave of absence from the board.
* There are legal concerns with the naming of the Fedora French Non Profit entity, we are working with them to resolve the issue with their use of the Fedora trademark in their name (it must be removed).
* All legal issues, including those involving trademark now go through Fedora Legal, with the exception of trademark usage approval, which is still handled by the Fedora Board.
* After consulting with Red Hat Legal, the Board unanimously agreed to lift the Legal block on sqlninja.
* After review of the data gathered from public feedback, FESCo, and FAMSco, the Fedora Board has chosen three goals, to be active until the release of Fedora 17. In no particular order, these goals are:

- Improve and simplify collaboration in the Fedora Community.
- Improve and encourage high-quality communication in the Fedora Community.
- It will be extraordinarily easy to join the Fedora community from anywhere in the world and quickly find a project to work on.

Each group within Fedora will be asked to identify tactical targets to work towards these goals. After Fedora 17 is released, a public review of the effectiveness of these goals will be done. A wider, more complete, announcement of these goals will be made by Jared Smith (FPL).

* Work on proposed governance documents as started at FUDCon Tempe continues.


Full logs of the meeting can be found here:


  february.08.2011  12.29 pm
February 1

February 1 is an interesting day for me. On February 1, 2001, I started working for a small software company called Red Hat, doing Level 2 technical support. Ten years later, on February 1, 2011, I'm still working for Red Hat, but I now manage a Fedora Engineering Team that does nothing but improve Free Software and help the Fedora Community. When I started working for Red Hat, Fedora didn't exist ( didn't come about until 2003). One of the first tasks I did was to make a set of addon kernel module RPM packages to support the DPT series of hardware raid controllers that were not included in the kernel source tree (you can still find references to that long deleted work by googling "dpt i2o tcallawa").

February 1 is also the birthday of my first (and so far, only) son, Jimmy. He turned 1 this year.

Having a child has really changed how I see the world, and has really given me a new set of perspectives on things. I know he is learning from me, not just from what I teach him, but who I am and what I do. How I interact with others, how I tackle problems, and how I deal with the world.

As I said before, I've been working with Fedora for a long time. I started working on it because it was an interesting challenge, and I kept working on it for a number of reasons, which coincidentally line up well with Fedora's Foundations.

* Freedom - I truly believe in Free Software. I'm not a zealot, nor am I an extremist, but I do believe that Free Software is the right way to develop, use, and distribute software. Everytime I can work with a software project to clarify or fix its licensing to be Free, I know that I'm helping make Fedora better, and also making the world better.

* Friends - I've met Fedora users and contributors on 4 different continents. I consider the majority of these people friends, and they have made my life better for having met them.
It is not easy for me to make friends, because I've struggled for most of my life to overcome a sense of social anxiety and depression, but in Fedora, I have met many people with similar interests and goals, which gave me confidence and made me braver.

* Features - It is a wonderful feeling to work on something, especially with others, and see it come to fruition. Fedora has always provided me with that opportunity, to take on a challenge and bring it to life.

* First - I love being on the cutting edge of Free Software, figuring out how things work, how code fits together, and what amazing things are coming. Working on Fedora is a lot like running a marathon, except that shortly after one race ends, another one starts. :) It is tiring sometimes, but I secretly enjoy the pace.

But there is a fifth reason I have stayed involved with Fedora for all these years:

* Fun - There is a great quote from Jerry Greenfield (one of the founders of Ben & Jerry's Ice Cream): "If it's not fun, why do it?" Working with, and in, Fedora is a lot of fun for me. Not everything is fun, but as a whole, working with the Free Software and the Fedora community is fun.


Lately, the things I've been doing in Fedora haven't been fun. People attack me for decisions I've made without trying to understand them, and when I try to explain, they just go off and insult me in various public and private parts of the community. People assume that Red Hat is forcing me to do things, or that Red Hat has some malicious master plan, when neither is even remotely valid. Community members are saying that Fedora has somehow been "taken from them" and are posting hateful attacks to Fedora Planet, cursing and foaming on IRC, and I'm tired of it.

When I was a new employee at Red Hat, I attended an all-hands meeting in the Durham office with the CEO, and I raised my hand to complain to Matthew Szulik about how broken some process was (I can't even remember what the process was, all these years later). He listened, nodded, then asked me how I would fix it. I had nothing, and I told him as much. Matthew looked at me without much respect and said something that has stuck with me all this time. He told me that if I was going to complain to him about something, I had better have at least one suggestion on how to improve it.

Let me offer this advice to you, the reader. If you're unhappy with any aspect of the Fedora Project, you really should be prepared to describe how you want to make it better, even further, you should be prepared to volunteer your time to improve it. If you can't manage either of those things, your complaint is not constructive in my book, merely destructive.

I'll go one step farther: These people who are proudly showing off their "give me back my Fedora" pins are behaving in a way that is demotivating and poisonous to everyone who is working hard on Fedora. If you don't like something about Fedora, don't just make hateful pins and attack other community members for their choices, propose an alternative and volunteer your time and skills to make it happen. Those people are taking the fun out of Fedora, and it clearly doesn't seem to be fun for them. So, I say to those people: If you're not willing to fix the things you think are broken, and you're not having fun, why are you still here?

No amount of past contributions and good works can undo hateful words and misdirected anger. Nothing productive or useful ever came from slander and personal attacks.

I'm not giving up on Fedora, because there are a lot of good people who put in hard work to make it awesome and fun. I want to teach my son that being positive, using your brain, and working hard are valuable life skills. But if you don't believe that, please consider finding somewhere else to participate?


  january.04.2011  11.45 am
Tuning Fedora for SSDs

A while back, I found myself reading a generic PC magazine in a waiting room, and they had an article on Tuning SSDs for Windows 7. It made me wonder what the Fedora equivalent would be. So, I asked my friend and Red Hat coworker Jeff Moyer about it. Jeff knows more about SSDs in Linux than anyone else I know, he does most (if not all) of the testing of SSDs for Red Hat's storage team. He agreed to write a little article for the Fedora Community, so here it is! Feel free to leave comments here, and I will be sure to pass them along to Jeff.


Tuning Fedora for SSDs
by Jeff Moyer

This article provides some basic background information on Solid State Disk devices (SSDs), and gives guidance on operating system tuning for

SSDs are typically associated with good random I/O performance (especially reads) for small block sizes. Unfortunately, not all SSDs are created equal. The performance spectrum for SSDs is as wide as the performance difference between a 5200rpm disk drive and a high-end RAID array. In addition, SSDs do not all implement the same feature set. Most important to this discussion are the Native Command Queuing (NCQ) and TRIM features, explained below.

A device implementing NCQ can accept multiple commands at a time. This means that the operating system can issue multiple I/O requests to the device, up to some maximum number, without waiting for previous commands to complete. The device can then perform some tricks to increase the overall I/O performance. Empirically, SSDs that lack NCQ support do not perform nearly as well as those which include this support.

TRIM (or discard, in Linux terminology) is a command that the operating system can send down to the SSD to tell the disk that a range of blocks is no longer in use. For example, if a file is removed from the file system, the file system driver can send a TRIM command to the device to mark the blocks as unused. The SSD can use
this information to free up internal erase blocks for wear leveling. In general, this is a good thing, as it allows the storage to continue remapping blocks, avoiding costly updates in place. However, the benefits of sending TRIM commands vary from device to device. On some devices, it is essential to send down TRIMs in order to get good performance from the drive, while on others, there is little or no advantage (and sometimes even a performance loss).

The downside of TRIM, in its current form, is that it is not a queued command. When a TRIM command is issued to an SSD, the operating system first has to wait for all currently outstanding commands (I/Os) to finish, then it can send the TRIM, and then it can go about sending more I/O to the device. Given the speeds of some of these devices, that can cause a big hiccup in performance.

Another technique SSD manufacturers use to provide good performance is over-provisioning. That is, SSDs are often shipped with some number of blocks that are not reported in the overall size of the device. This is key to maintaining good performance as the amount of data stored on the disk approaches the reported capacity of the disk. As you can imagine, the device firmware and the amount of over-provisioning play important roles in the overall performance of the device. Because there is limited visibility into some core pieces of SSDs that impact performance, I highly recommend reading reviews before purchasing a device.

Frequently Asked Questions

What is the erase block size for my SSD?
Good question! This information can be hard to find. It depends on the flash parts used in the SSD, which is not always advertised. You
really only need to know this for partitioning, and for that, you can use a number that should cover all SSDs. See below.

Do I need to align my partitions to the erase block size?
It is generally considered a good idea to align partitions to the erase block size of your SSD. If you do not know the erase block
size, simply align the first partition to 1MB, and the ends of your partitions to 1MB boundaries. If you don't do this, some SSDs will
perform poorly.

What file systems support discard?
btrfs, ext4, fat, gfs2, nilfs2

How do I enable discard support?
File systems supporting discard accept the 'discard' mount option. During mkfs, a discard is done on all of the free space if the device supports it (no need to pass in any options).

Should I turn on discard for my device?
This question can be answered by running the discard test kit written by Lukas Czerner (WARNING: it may brick your drive!).[1] It creates a metadata intensive workload and measures performance both with and without TRIM enabled. It also measures the performance of the disk as the number of blocks written increases without doing any TRIMming. In this manner you can see how the device degrades over time without TRIM. If your device does not degrade over time, then there is no need to enable TRIM (remember, enabling it at mount time means that you may suffer a performance penalty). Further, some device firmwares react very poorly to TRIM, in some instances rendering the device inoperable. If you want to be on the safe side, do not enable TRIM.

What about hdparm/
hdparm[2] does have a mechanism to TRIM all of the free space for a file system (see However, the hdparm author does not recommend its use without first backing up data to another device. It is worth noting that this tool allows the TRIMming of free space on some file systems which do not support discard natively. Its supported file system list includes:
online support: ext4, xfs
offline suport: ext2, ext3, and reiserfs

What tuning can I perform to make my disk go faster?
The default I/O scheduler (cfq) has optimizations built-in for SSDs. Thus, it should perform well for most users. It is worth noting, though, that not all devices advertise the proper rotational rate. You can check to see if your SSD advertises it correctly by doing the following:
# cat /sys/block/sdX/queue/rotational
replacing the 'X' with the letter of your device. If that reports a '1', then the drive is advertising itself as a rotational device, which is wrong for an SSD. You can echo a 0 into that sysfs file in order to change it, and this will enable the non-rotational media optimizations in the I/O scheduler.

In rare instances, the optimal I/O size for a device may be the erase block size. Typically, the larger the I/O size, the better performance you get from a device. However, I've seen exactly one case where sending I/O down larger than the erase block size actually resulted in decreased performance from the drive. In order to restrict the size of I/O sent down, you can echo a value to /sys/block/sdX/queue/max_sectors_kb. For example, if the performance drops off after I/Os sized 128KBs, you would do:
# echo 128 > /sys/block/sdX/queue/max_sectors_kb

Is it okay to put my swap device on an SSD?
Yes. SSDs are rated for a number of program/erase cycles that is extremely high. Plus, the sometimes random access patterns of swap can benefit greatly from being on an SSD.

Does swap support TRIM?
Yes. When enabling swap, the entire swap space is discarded.

I heard that indexing software can cause an untimely death of my SSD. Should I disable beagle, for example?
Again, NAND flash is rated for a set number of program/erase cycles. By constantly writing/overwriting/TRIMming sectors on the device, the lifetime of the device is shortened. Wear-leveling can help increase the longevity of the device, but that is controlled entirely by the drive firmware. As such, the impact of indexing software on SSDs will vary from device to device. In all cases, it will have some negative impact, but it's not clear the degree to which the software bundled with Fedora will impact the lifetime of an SSD.

Should I disable the journal on my file system?
No. Ted T'so did a nice write-up of the overhead of journaling in his blog.[3] It turns out that the journal does not generate enough I/O to be worried about in most cases.

Device-specific Recommendations*:

Intel SSDs: defaults
Sandisk G3: set /sys/block/sdX/queue/max_sectors_kb to 128
OCZ Vertex: defaults
WD SE Blue: use ext4 or btrfs with the discard mount option

* Recommendations are based on a broad range of testing performed
across several different workloads. For a specific workload, users
are encouraged to do their own testing.

Further reading:
Anandtech has a nice write-up on how SSDs work:



  november.23.2010  04.46 pm
The TeX Live Legal Audit (and how you can help)

One of the remaining big ticket items for legal audit in Fedora is TeX Live. A few years ago, we got a version of TeX Live into Fedora, and after the fact, we discovered that the licensing on much of it was confusing or non-free. Jindrich Novy has been working for a while on packaging up TeX Live in a more separated way (package per component as opposed to singe giant texlive package), and I promised him that I would help him do the Legal audit.

(This metabug is tracking the packaging effort:

TeX Live says "Therefore, to the best of our knowledge, all the software in TeX Live meets the requirements of the Free Software Foundation's definition of free software, and the Debian Free Software Guidelines. In the rare cases of conflict, we generally follow the FSF. Furthermore, the material in TeX Live should not require nonfree software to be useful."

However, there are some items which are concerning.

1) TeX Live, while they have made significant and laudable progress in licensing on their components still has a section of "OTHER-FREE":

In Fedora, nothing is "Distributable" or "Other". Everything gets properly checked and cataloged.

Going through the components in this category has been interesting. We've found at least 25 new licenses in the items in this category, many of them clearly non-free. Some of the items in this category were under common Free licenses (GPL, LPPL) and some were under licenses Fedora had seen before but are pretty rare. Some are just missing licensing altogether.

2) TeX Live has a fair number of components which are marked as being "Public Domain":

You might be saying "hey, so what?", but this is actually a complicated situation. In many parts of the world, it is not possible for a copyright holder to abandon their copyright (unless they're dead), and put a work into the Public Domain. Thus, it is important to determine where the code was written (and who wrote it).

Code written by the United States Government is in the Public Domain (unless it was written by a contractor, or certain other corner cases). Individuals in the United States, Canada, and the UK can place works in the public domain with a clear Public Domain declaration stating that they abandon their copyright on a work. (Please don't waste time arguing with me about this. I know its a gray area, but this is what we are acting on based on the advice of counsel.)

Individuals in most of Europe can almost NEVER fully abandon their copyright, thus any public domain declarations from those individuals are invalid, and we are left with little or no license on that work. Specifically, this is known to be true for France, Germany, and Poland. (Again, please don't waste time arguing with me about this. Look at it this way. If we're wrong, it cannot hurt to have an additional set of permissive license permissions. If we're right, we _need_ them.)

What this means is that Fedora needs to track down the copyright holder(s) and get them to confirm citizenship and location of where the work was created to analyze whether the PD declaration is valid, or have them give license permissions with terms that will actually accomplish close to the same end-result as PD (putting the work under any Free license is also fine by us).

3) A lot of this code is really old. Like, 25-30 years old. Nothing wrong with that, except it makes tracking down copyright holders... fun.


So, enough blather. We have a wiki page which documents the status of our efforts. If you can help us clarify a license on a work or track down the copyright holders for these works that are marked as Fedora disapproved, Improper or invalid Public Domain, or Unknown (with current email addresses), it would sure help save us some time.

Here's the page:

If you edit the wiki, please don't move things around, just add notes. You can also just send me an email (

Thanks and happy hunting.


  september.30.2010  05.19 pm
Fedora is hiring

People regularly ask me about how they can work for Red Hat, specifically, how they can work on Fedora for Red Hat. Usually, my answer is "Contribute, do good work, get noticed, and you're likely to be hired", but at the moment, two positions have opened up.

1) Fedora Program Manager:
John Poelstra has done an excellent job as the Fedora Program Manager for the last few years, but he is ready to move on to different challenges inside Red Hat, so we're looking for someone to fill his role. For more details see:
The actual job posting is here:

2) Fedora Architecture Leader:
Mike McGrath has been involved with Fedora Infrastructure for quite some time now, first as a volunteer, then as a full-time employee in the role of Architecture Lead. Mike is moving into a different role inside Red Hat Engineering, so we're faced with the difficult task of filling his position. Mike has been wonderful to work with, and I'm glad that he will still remain involved with the Fedora community, even if that is no longer going to be his primary job function.

Some details are here:

This job isn't actually posted yet (although, hopefully, it will be soon), so here's an overview:

The Role: The Fedora Project is seeking an Infrastructure Lead to be responsible for a global network of free software based software based services that supports the Fedora Project in its goals. Day to day responsibilities include volunteer recruitment, team coordination and training, systems administration and basic troubleshooting / helpdesk support of Fedora developers and systems availability/resiliency. The lead is also responsible for long term planning, budgeting, new hosting sponsorship, policy, and overall infrastructure architecture/integration. This is an on call position that requires far more then just the standard 8-5 hours as volunteers are widely distributed and systems need to be monitored and repaired in facilities around the world.

Primary Responsibilities:

  • Coordinate and oversee efforts of Fedora Infrastructure team, including facility visits

  • Ensure that all Fedora Infrastructure systems and software components are up and running properly

  • Detect and fix problems before they occur

  • Coordinate with other teams to determine needs and expectations for projects

  • Keep up on new technologies for review and early adoption

  • Ensure the current Fedora release is solid and prepared from an infrastructure perspective

  • Support Fedora Release Engineering in any tasks / issues that come up related to package updates or new releases

  • Maintain and grow community involvement in Fedora Infrastructure

Required Skills & Experience:

  • Minimum of 5 years experience in a systems administration or infrastructure role

  • Red Hat Certified Engineer (RHCE) or better is strongly preferred

  • Experience in working with and building Free Software Communities

  • Expertise in:

    • Linux (Fedora, RHEL)

    • Deployment Technologies: Kickstart, RPM, Puppet, Yum, Func, Xen/KVM

    • Security: SELinux, SSH, iptables, rkhunter, nss, clamav

    • Network: Cisco, OpenVPN, bind, VoIP

    • Web: Apache, Squid, memcached, TurboGears, PHP, MediaWiki, Django, varnish

    • Database: PostgreSQL, MySQL

    • Programming: Python, Perl, Shell

    • Backup: Bacula, rsync

    • Source Control: cvs, svn, git

    • Storage: NetApp, NFS, iscsi

    • Infrastructure: Mailman, OpenVPN, Nagios, BitTorrent, Haproxy, Yubikey/ssl authentication, CDN / GeoIP concepts

If you're interested in this job, feel free to send me an email ( with a current resume in a free format.


  august.30.2010  12.59 pm
A Copy of the email from Wim Coekaerts killing off the Sun RPC License

Since a few people have asked me for this, I asked Wim if I could repost a copy of his email and he gave me permission to do so.

(Headers are trimmed off.)

Date: Wed, 18 Aug 2010 11:25:39 -0700
From: Wim Coekaerts <>
To: Peter Brown <>,,
CC: Elena Zannoni <>
Subject: sun rpc license

Hi -

I was thinking of doing patches for this but it's probably easier to just provide you with an email with permission to relicense then it can catch the files and projects. Please let me know if any files are missing I will be happy to review that as well. just send me a note with files and which project

On behalf of Oracle America, I authorize the following list of files to be re-licensed under the license below, (3-clause BSD) and changing the old Sun Microsystems license to the one attached. :

(1) rusers (netkit)


(2) krb5



possibly also the following :

























(3) glibc


license :

Copyright (c) 2010, Oracle America, Inc.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:


Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

Neither the name of the “Oracle America, Inc.” nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.



Wim Coekaerts
Senior Vice President
Linux and Virtualization Engineering
Oracle Corporation


  august.26.2010  04.23 pm
The long, sordid tale of Sun RPC, abbreviated somewhat, to protect the guily and the irresponsible.

Once upon a time (1984), Sun created an RPC implementation for Unix, with the intent of implementing RFC 707 (High-level framework for network-based resource sharing). Now, in those days, a good way to ensure that people used code that you wrote was to upload it to usenet, and in 1985, Sun did that. (Google has one of the posts archived here: Sun RPC part 8 of 10)

Keep in mind the following: the first formal definition of free software was published by the FSF in February 1986, so there were technically no "free software" licenses when Sun RPC was written. For that era, the licensing of Sun RPC was remarkably permissive, it said:

* Sun RPC is a product of Sun Microsystems, Inc. and is provided for
* unrestricted use provided that this legend is included on all tape
* media and as a part of the software program in whole or part. Users
* may copy or modify Sun RPC without charge, but are not authorized
* to license or distribute it to anyone else except as part of a product or
* program developed by the user.
* Sun RPC is provided with no support and without any obligation on the
* part of Sun Microsystems, Inc. to assist in its use, correction,
* modification or enhancement.
* In no event will Sun Microsystems, Inc. be liable for any lost revenue
* or profits or other special, indirect and consequential damages, even if
* Sun has been advised of the possibility of such damages.
* Sun Microsystems, Inc.
* 2550 Garcia Avenue
* Mountain View, California 94043

People wanted to be able to support RPC, so this code was used all over the place. Notably, it is a core component of how NFS works on Unix (and since Linux is Unix-y, there too). Over time, this code was ported to Linux, and added, most notably, to glibc, all along, inheriting the same license Sun wrote for it in 1984.

The problem is that this license is clearly non-free, it places restrictions on distribution. Now, Debian has known about this concern for a while now, since 2002, to be precise, as you can see here: Debian Bug 181493
Fedora caught this when we first started doing serious license audits, circa 2005.

So, what do you do about this? As a distribution, your options are:

1) Pull out the non-free code. This means you don't support NFS, most notably.
2) Replace the non-free code. The big problem here is that the Sun RPC implementation was pretty much the standard RPC implementation on *nix. There were some others, but switching to them would have been really, really, painful.
3) Try to get the code relicensed, and keep it until that happens.

So, Fedora, like Debian, chose #3.

Every so often, we would check in with Sun and see if they were willing or able to relicense this code. Simon Phipps was very interested in solving the problem, but there were various concerns on Sun's side that prevented them from taking action, but in February 2009, Simon told the crowd at FOSDEM that Sun was going to "change the license to Sun's copyrights in the RPC code to a standard 3-clause BSD license". Unfortunately, Sun never actually made this change. (Simon's blog entry about this is here: )

Independently (and simultaneously), Red Hat had been trying to work with Simon and Sun to relicense a specific set of files which were under the Sun RPC license terms. We even went and tracked the lineage of these specific files and found their equivalents in OpenSolaris. When Simon made his announcement at FOSDEM, since he specifically mentioned glibc, we assumed that it was taken care of and did not explicitly call out the glibc files in our request for relicensing. In March 2009. Sun granted Red Hat permission to relicense the explicit files that we had identified to Sun as needing relicensing (to be specifc, all affected files in rpcbind 0.1.6, nfs-utils-lib 1.1.3, nfs-utils 1.1.3, libtirpc-0.1.9, and portablexdr 4.0.11).

Unfortunately, there was still remaining code in Fedora under the Sun RPC license, specifically, in glibc, krb5, and netkit-rusers. We accidentally advised glibc upstream to relicense the file, thinking that Sun had given us permission, but when we realized that we had not in fact been given that permission, we had that change reverted in upstream. By this point, Sun was too busy becoming part of Oracle to respond at all here.

So, we restarted the effort with Oracle, and on August 18, 2010, Wim Coekaerts, on behalf of Oracle America, gave permission for the remaining files that we knew about under the Sun RPC license (netkit-rusers, krb5, and glibc) to be relicensed under the 3 clause BSD license.

You can see the glibc commit here:;a=commit;h=a7ab6ec83e144dafdc7c46b8943288f450f8e320
The krb5 changes are in progress. As far as I can tell, netkit-rusers is abandoned code. Emails to and both bounce. If you know someone who could help us make this change in netkit-rusers, please let me know. In the interim, Fedora has a patch to fix the license in our copies in Fedora 14 and rawhide.

I'd like to thank Wim Coekaerts and Elena Zannoni at Oracle America, and Peter Brown at the Free Software Foundation for helping us cross the finish line here, as well as to Simon Phipps (formerly with Sun) and Richard Fontana (Red Hat Legal) for their work over the last few years to try to close this issue out.

I'm thrilled to be able to take this off my list of items which fall under category #3. Fedora has a strong commitment to Free Software, and efforts like this reflect how we are actively working towards a target of entirely Free Software.

(It is worth noting that up until August 18, 2010, any '100% Free" distribution was, well, not 100% Free, assuming they were using glibc. But I've posted on that silliness before, so I won't do it again.)


  july.30.2010  10.57 am
Chromium repos move to their new home on

I bet you're asking, where did the Fedora chromium packages go?

Don't worry, they just moved to a different location. To get them, go here:

There is even a pre-made fedora-chromium.repo file that you can drop into /etc/yum.repos.d/ and go!


  july.25.2010  11.28 am
anger and FOSS

I don't normally consider myself an angry person. I don't go around kicking puppies or punching holes in walls, but over the last day or so, I've been considering whether or not I default to anger.

Over the last week, I've been in Portland, OR for OSCON. I hadn't been to this particular show in years, primarily because it felt very very corporate on my last visit. It had a "we payed big money for this sponsorship so we're going to get to aggressively push our non-FOSS software (and culture) to you in confusingly worded sessions and keynotes" vibe. Is it snarky for me to say that? Snark is kind of an angry way of making a point, I suppose. Okay, lets try again.

It concerns me when a conference that has "Open Source" in its name in any way encourages non-FOSS software and ideals, from its sponsors or otherwise. It would be more appropriate to say that it annoys me. So, I stopped going to OSCON a few years back.

I'd heard from others that OSCON had gotten better about things, so I penciled it in for my 2010 conference attendance and got budget allocation to go. Now that it is all said and done, I think things have noticeably improved, but I felt it could still use some work. First, here were some things that I liked about OSCON:

* Good timing, on breaks, meals, snacks, events, and plenty of time to make it from one session to another (as long as no one ran too long)
* Track on new/emerging languages seemed just right for OSCON, even if it wasn't something of interest to me
* Most companies/groups on expo floor had some FOSS to talk about
* Google's generosity in giving out several hundred Nexus One phones. They're not 100% FOSS, but they're more FOSS than my old iPhone.

Things I didn't like:

* "Sponsor" sessions that were simply advertisements for non-FOSS. Does OSCON really need to do this anymore?
* Whoever gave away vuvuzuelas (spellcheck proposes "Venezuelans"). Sounds like fun, until someone walks by your booth blowing one for the 100th time. With the World Cup, I imagine you really don't need to hear anything to enjoy the game, but at a conference, being able to hear my answer to your question is rather useful.
* Defensive keynotes. Specifically, where the keynote presentation is on the defensive from the beginning. I'm looking at you, Microsoft and Eucalyptus. The fact that you need to defend your non-FOSS model and tactics outright should be telling for you, but was just embarrassing to watch.

Overall, I enjoyed OSCON, but to be honest, I came into this conference with a secret hope. I knew that Google would have a huge presence here, and I was hoping that I'd get the chance to talk about chromium (or at least, hear someone talk about it).

As many of you might know from reading this (infrequently updated) blog, I spend a fair amount of time working with the chromium source code, gently (and occasionally not as gently) massaging the source so that it builds into RPM packages for Fedora targets, while at the same time, stripping out as much of the bundled/forked third party FOSS libraries as I can. I know my work is appreciated by users and by my counterparts in other distributions who are also doing the same tasks. But I've never really felt that most of the people working on Chromium cared whether I did it or not. In fact, with the notable exception of Evan Martin, I felt that they actually were generally unhappy that I do this work.

I should clarify that statement somewhat. In the course of my work, I've needed to patch the chromium source code pretty aggressively. To date, none of my patches have been merged (or in most cases, even commented on), even though most of them have been submitted as bug reports. This is demoralizing, because it implies that they don't take the time to care about my work. However, I've tried very hard to never assume malice when other factors such as being overworked could be equally viable.

I've got pretty strong feelings about how Chromium (and v8) development is being done. I think that the design and implementation decisions that they've made have made it difficult for any real community of contributors to form around it, and in the case of v8, functionally impossible for third party FOSS projects to leverage. I've raised these feelings with the Google Chromium team in the past, and they generally don't care. (Again, Evan is a notable exception.)

I know that it is much harder to communicate online than it is in person, the nuances of inflection and tone translate poorly in bits and fonts, so I was hoping that someone from the Chromium (or v8) team would be at OSCON to talk about their roadmap, their development approach. I had hoped for this for two reasons: 1) To better understand what motivates them to act the way they do and 2) To have the opportunity to explain why their approach is so frustrating from a FOSS and a distribution packager perspective

Sadly, there were no "Chromium" or "V8" sessions on the OSCON schedule, but there were two "generic" Google sessions, so I attended both of those. The first was more of a general "Google" town hall, and the speaker (Chris DiBona) said early on that questions about Google's FOSS initiatives should be held until his later session, so I waited for that session. During the first session, it was noted that a healthy majority of attendees in the room worked for Google, which gave me hope that there were some Chromium/V8 folks present.

I attended the second session, and waited for the topic of Chromium to come up during Mr. DiBona's slides, but it was not mentioned at all. No other questioners during the Q&A session brought it up, so I raised my hand.

To be fair, I don't remember exactly what I asked, verbatim. I knew Mr. DiBona wasn't part of the Chromium team, but I was also aware that he seemed confidently (and even cheerfully) aware of Google's FOSS initiatives, so I asked a broad question that went something like this:

"Google's approach to Chromium and V8 has been really frustrating and difficult from a distribution perspective, do you know if there are any plans to improve that?"

Chris DiBona responded with a defensive ferocity and anger that caught me significantly off-guard. He accused me of being angry, then admitted he was unaware of any such concerns, then suggested it would be better handled offline, all in an extremely condescending and aggressive tone.

I left the session disappointed. I wasn't expecting him (or anyone) to say "We're so sorry! We'll change everything right now!", but I certainly wasn't expecting to be bitten and yelled at. Am I truly angry? I thought about this for the next two days. Had I said something to merit that anger, or was he simply deflecting anger back at me. I didn't feel angry, at least, not until he answered with such an aggressive tone and demeanor.

In retrospect, almost every single question asked in those two sessions was less of a question and more of a glowing fanboy praise, in the form of a question. "When will Google Foo version 2 be released?" or "Will Google Bar integrate with Google Baz?". My question summed up to "When will Google Qux stop behaving badly?". Was there a way I could have phrased that to get a better response? Would it have helped if I had been more specific (even if that would have taken 15 minutes to do properly)? Or was it simply that I was highlighting a negative that caused such backlash?

All that I knew for sure was this: In one single response, Chris DiBona made me not want to work on Chromium anymore.

After the talk, several Google employees approached me to talk about Chromium and V8, they didn't work on that effort, but they knew the people who did. I apologize for not mentioning them by name, because, well, I'm terrible with names, but we talked about my concerns, both idealogical and technical. I explained why Chromium and V8 weren't part of any Linux distribution, the work I had done to compare performance in a static vs shared configuration, why bundling and forking as an instinctual default behavior was harmful, and they explained why they suspected that the Chromium and V8 projects were handled in the way that they were. They were patient, but engaged with me, which was good, because I was admittedly pretty worked up by that point.

But I couldn't stop thinking about being angry. I know how it made me feel when Chris lashed out at me during his session, and it stacked on top of what I had already been feeling about my Chromium efforts. It made me feel unwanted, unappreciated, and unwelcome. Even though my instinct was to blame his anger for this, was I really the angry one?

Angry people are poisonous people. Sometimes, they can calm down and become productive people, but almost no one is useful and productive when they're angry. This is why snark isn't useful or productive. It's why no one wins a shouting match, ever. When it comes to FOSS, which is largely a meritocracy, when you're angry, you're not able to earn respect or credibility.

I hope that people appreciate the work that I do on Free Software and Open Source. I hope that I am respected amongst my fellow users and contributors in the community at large. I hope that I'm not angry. I'd rather be passionate than angry.

I'm not sure if I'm going to keep working on Chromium or not. I'm not sure if I was truly angry in that session at OSCON. But I do know that I'm going to make a conscious effort to identify, within myself, the difference between passion and anger, and leave any anger that I find behind. I'm not saying that it is never wrong to be angry, just that it isn't productive to be angry in FOSS, and I'd much rather be a productive contributor than an angry agitator.

I know it won't be too long before I'm giving a presentation at some FOSS or Linux event and someone asks me an angry question, but I hope I will be able to reply calmly, and channel that anger into something productive. I encourage all of you to think about being angry and how that affects your involvement with FOSS and its communities.


past   future

technorati profile