Thoughts on Package Management

The Change in Distro-Land

Distros have changed. In the past, they were made up of a small, tightly knit group collaborators working toward a common goal. With distributions today we now have an informal, large group of collaborators…some of which may not even be aware of the main goal of the distro. That informal collaborator may just want package foo version 2.2 included in his/her distribution so that he/she can use it on their desktop. How does that informal collaborator become empowered? How can the developers reap what that collaborator sows and harness the collective collaboration of thousands of informal contributors? The answer for many software projects is version control. But how can this system benefit package management?

What If?

What if you could combine SVN/CVS/git behavior and packages? What if when you build the package properly, it is checked into the software development tree. You’d be eliminating an entire step in the process (i.e. working more efficiently) and you’d reap all the benefits of version control (diff, merge, shadow, exports, rollbacks, tags, logs) with the actual software packages without losing the benefit of working with source or binaries. Thousands of contributions could be made in the form of ready to install packages that are CERTIFIED (see how this is possible later in this post) to work on the distribution. The contributions would come in on a version control branch designed by the distribution developers…say 1-contribs (much like a contribs rpm server would be)…but unlike most distributions, they would be certified to run on your distro before they even hit the contribs server/branch. Imagine the impact that this would have for bug testing alone.

Sound too good to be true? It’s not. It’s Conary and it is getting ready to go to version 2.0. Let’s take a look at some advantages that conary has over traditional package management and how it can empower the end user.

What Are The Advantages?
In standard distro land, packages are tracked by version/name. If you packaged wget 1.0.12, your package would include this version string in the name. What if someone else packages that same software up and names it something different than you did? What does this package with name wget-1.0.12-suse93-i586.rpm have as an advantage over wget-1.0.12-suse9.3-i586.rpm? As you might be able to tell, this is hard for a package management system to understand concepts of older or newer packages and what should be installed on a system. There is no automatic solution to this process in standard distro land. This is done manually by a developer. The developer selects which version of a package is included in the repository.

Regardless, relying on a name that is manually created and given means that dependency resolution with most distributions will break sometime. Repositories end up with version clashes, missed/overlooked patches, clashing names in between distros for the same package. A disconnect opens up between those developers working with packages…not just packages for a single desktop environment but various ones (KDE, Gnome, Openbox) because the only thing that connects them is a SVN or CVS tree. Updates to one branch do not always make it upstream. With conary, this problem is solved. It works using a distributed version tree and therefore changes always flow upstream from various branches that don’t have to be located in the same place as in traditional version control systems.

Ever done an apt-get upgrade and had only half of the packages install before a failure occurs? You may have a broken xorg after that half update was applied. In rpm land, if I updated and didn’t get through all my package installs I may end up with missing dependencies and a broken system. With Conary this NEVER happens. If an update will break the system, Conary automatically rolls back the changes so that your system is in the EXACT state it was when you began the update. This behavior is built into the package manager and happens automatically during attempted updates/upgrades. It can also be manually done to return to a state that existed before the update.

As an example, have you ever installed a group of software from a development/testing repository that you don’t want installed anymore? Don’t want to go through logs to find out what group of software you installed? Can’t remember each and every package name? With conary simply do a sudo conary rollback # where # is the number of updates you want to rollback to and conary will return your desktop to the state it was before that update.

More advantages…let’s say you’re out on rpmbone.net and you’re searching for the latest release of KAlbum. Which one do you pick out of the results

? If you see your distro represented, great! You’re set…until you need to update from that version. Will you remember what repository you got your software from so that you can get future updates? Will you add this repository to your repository sources list? This being the case you’re going to get a whole lotta updates from that new source unless you do some heavy pinning. Wouldn’t it be great if you didn’t have to remember? Wouldn’t it be great if your package manager would manage this for you as well? Conary does this lifting for you. If you install a conary package from a different conary repository, it REMEMBERS for you where you got it. No alteration of sources because there isn’t a sources file. There isn’t a list of repositories to manage. Conary remembers where all your software is from and tracks this as part of the version control.

Let’s say you roll your own rpm and make it locally on your own build environment. Upstream, the package you built is updated to a version above the local rpm/deb you’ve installed. It installs automatically over the top of your change when you update. With conary, this wouldn’t happen…it tracks local installation of conary packages as well and preserves them if you so desire.

There are countless other advantages that I won’t go into here. The advantages that I went over are meant to support the main point of this post which is that you can get more done by combining package management with version control. This turns one developer into many and allows for many collaborators to contribute certified packages into a distributed version tree repository. This is working smarter and not harder.

More Information on Conary

Certified Packages

Knowing as a developer that you’re getting a CERTIFIED version of a package onto a contribution server is a huge amount of stress relief. Certification occurs with the tool rMake, which “facilitates building packages consistently across computers with dissimilar environment”. If you click on the link to the rMake page, you’ll see plenty of the benefits of using such a tool to package software. rMake creates a chroot

and installs the package you are building in that chroot. Since a chroot is a pristine and unaltered environment, all packages can be certified to run in that pristine and unaltered environment. When a package is committed to a conary repository, you KNOW that it worked inside a chroot on a conary based system. Therefore, you have less to worry about with system conflicts. You’ll be able to build and test packages BEFORE they’re even committed to the repository.

With rMake, you don’t have to worry if that package will run on your distro…you only have to worry about the possibility of unforeseen conflicts…package foo won’t drop its use of dbus for package bar and package bar fails…and package bar was just contributed by Joe Schmoe onto your contribs server. This is what distributions SHOULD be allowed to concentrate on…conflicts between packages on an installed system and not dependency resolution or patch management. The place where developers work on conflicts should not be the packages themselves or the repository…it should be the users’ desktop.

Empowerment

Taking the worry that a package committed to the repository won’t work away, developers can empower users. Empowerment is a powerful tool in open source. Often times, a user of desktop linux becomes soured when he/she realizes that something doesn’t fit exactly what they want. For example, Joe Schmoe may want a software suite that isn’t included in the distributions repository. Should they learn how to roll an rpm or package a deb to do this? Can they? Will the developers trust the package when or even if it is submitted to the contribution server? Most likely not.

When Joe created his package, he created it on his local system. Dependencies that are unforeseen will be present in that package because Joe’s system differs from other installed systems. He didn’t package and test with a chroot to make sure that all his dependencies were resolved before submitting his package to the contribution server. If Joe were using Conary and rMake, developers wouldn’t have to worry about whether his package were up to where it needed to be…because in order for him to commit it to the contribs server it would have to have all dependencies resolved and would have to be built in a chroot before it is even submitted.

The ability to package and contribute packages is therefore available to standard users. The package is also IMMEDIATELY available for consumption by that end user…no waiting period for it to trickle down into stable…it was already stable when contributed. When it changes from 1-contrib to 1-devel, conary automagically tracks that move and updates from 1-devel in the future. It’s that smart. Developers are still in the driver seat and can control the overall flow of packages in the repository…the power still rests with them. But conary does all the lifting for them and often times the developer finds themselves letting conary manage things.

Imagine KDE and Gnome desktop distro developers working side by side and sharing fixes/patches and packages of common shared software…remember, with conary repositories, all changes can be seen by both development trees (KDE and Gnome) so the changes are automatic when put in place. Instead of having to push that fix to both branches of software, conary allows for fixes to be automatic between the two desktop development branches.

Work smarter, not harder. Conary can be an incredible tool that can empower users to contribute and merge different development trees seemlessly between various versions of software. Is there a learning curve? Of course! If you don’t know or understand the concepts of version control, you’ll have a bit of an uphill climb when learning conary. When learning to package, you’ll have to understand some of the classes and superclasses that are passed down so you can take advantage of not reinventing the wheel with each package you roll.