This week in anaconda

This week’s entry is all about how we create installation media.

Creating images

Before you can run the installer, you need to create some boot media that contains the installer. The boot media contains the installer which is written in Python, plus a whole lot of Python libraries, a pile of storage utilities, a bunch of base system utilities people like to have for their kickstart scripts, a lot of libraries that all these things require, and a ton of data files like translations and timezone data. Overall, creating the installation environment is a pretty slow and involved process understood only by a few. For a very long time, that process has lived in anaconda. Now, however, we are about ready to break it out into its own project. Why would we do such a thing?

• anaconda’s image building scripts contain whitelists of every file that should be included in the image. There are so many ways this can go wrong. Rebuilding a package and moving its data files from one location to another might mean they’re no longer included which could be a fatal problem. Just renaming a file could result in the same thing. This used to hit us all the time when libraries updated, but we are less susceptible to library moves anymore. Still, the whitelist is not at all foolproof.
• Adding or removing something from the image means rebuilding anaconda. For obvious reasons, this is less than ideal.
• It’s hard to tell what the image building scripts do. I’ll be honest: I’m prejudiced against large shell script projects. I find all the quoting and here documents utterly unreadable. It’s also difficult to debug.
• We don’t do a very good job of cleaning up after ourselves.

The project to redo our image building scripts is called Lorax and mgracik has been working on it for quite a while now. The main goals are to move it to Python so it’ll be more easily comprehended and debugged as well as importable in all our rel-eng tools; and use a blacklist instead of whitelist so things moving around doesn’t break the images. As was reported on anaconda-devel-list this week, we’re just aout good to start using it, too. A little surprisingly, it appears lorax runs quite a bit faster too. That’ll be a welcome change since it means speeding up tree composition, which also helps for anyone testing tree changes.

So what is left before we can start using this miracle new project? Well, lorax needs to get into Fedora first. I worked on the review ticket yesterday and today and got it into shape so now we’re just waiting on the rest of the process to magically finish up. Then, mgracik needs to do some work to make it support other architectures. While we really only do i386 and x86_64 in Fedora now, it’d be nice for lorax to support everything that anaconda (theoretically) does - ppc and s390, and some half-baked support for older processors. And then finally, we can go into the scripts directory in anaconda and remove a bunch of old crud.

While this may not be the most exciting user-level development, I’m very pleased to see some of these fragile scripts get replaced with what looks to be a more readable and less error-prone system. Hopefully this translates into a less frequently busted install image.