One of the things that team docs here has known for a while is that the rPath wiki is a fantastic tool to leverage for documentation. It’s quick. It’s easy. It allows engineers to contribute directly to the wiki. It allows community members to contribute to to the wiki.

We’ve also known for a while that this tool has a major caveat…and that is that versioned documentation is costly. For example, if we had say version 1.0 documentation of a project at wiki.rpath.com/v1/productname and version 2.0 came out, we’d have to maintain 2 separate documents with the same information in two different URI’s and 2 different name spaces. With each addition of namespace and project version, updates would be more costly and time consuming.

It’s also a bad thing that a user can search the wiki…and have the possibility of getting results from versions that they are not using…possibly information and behavior of products that no longer applies.

So how do we plan on combating this? There are a couple of phases of plans that we’ve been thinking about. The first of these is to move all product related guides (Administrative, User) offline into docbook format (pdf, html). We’ve begun testing this idea already.

This buys us the ability to keep versioned documentation separate from the wiki which empowers customers by giving them only the documentation they need and not requiring them to wade through search results to get the documentation they need.

This also is good for the community. Why? Because community docs will remain on the wiki and product documentation will be separated from it This means that documentation on using open source tools like Conary and working with rBuilder Online will be separate from rPath products making things easier to find and easier to contribute to.

The next phase of the plan after separating product specific documentation is to provide a central repository for those offline docs. This is further down the road and will only take place if product documentation has been moved offline. When this phase hits, there will be a handy website that serves as a central repository for all documentation, whether product based or community based. Currently, there are no plans to move community documentation from the wiki (Conary, rBuilder Online, rMake).

So, those are some of the updates we’ve been talking about doing. Nothing is set in concrete but we’re continuing to stay busy by keeping the information readily available to both community and customer

rPath Documentation Status Update originally appeared on Yet Another Linux Blog on June 12, 2008.

Why another package manger? Isn’t there already too many of these out there? Why use Conary when I can apt-get? Apt-get is soooo much better. Dpkg gives you sooooo much more than anything could possibly give you. Conary is still beta quality…rpm and deb are much more developed mature.

If the person(s) asking the questions above actually understood what Conary is and CAN do…they would see this is a very limited view of Conary. Not only is conary a package management system vis-a-vis a system that manages EVERY single package of software on your system…it is also a powerful version control system for software packages and packaging. It’s an enabling mechanism for packaging software quickly and easily.

I’d like to go over some of the things I think are great about Conary…clear up some of the “why is this needed” speak by showing how Conary actually gets things right and the common problems experienced by other package managers that it solves.

How Conary Organizes Packages

If you use debian or rpm repositories…you know that inside a repository directory “stable” (as an example) are all the stable packages for your distribution. The packages are versioned according to their upstream version (if the repository maintainers are sane) and maybe arch and revision number. This is done by hand. It is managed by hand. If developers/packagers cross names between repositories you are brought into dependency problems. To illustrate this concept, if you and I both packaged firefox3 and named it accordingly…and someone used both your and my repository…our versions would conflict because the packaging system wouldn’t know which one to install.

Conary takes the manual operation from this…if you use a Conary based system, your repositories ARE VERSIONED. In other words, the repositories aren’t static directories that contain a bunch of packages…they are versioned branches that contain components of software.

These components (packages) are also versioned according to upstream version…but revision is handled automatically by Conary…no manual process. This eliminates the possibility of having two packages named the same exact thing in different repositories. In other words, if Joe Schmoe is packaging Liferea for his apt.joeschmoe.com repository and names his package the same thing as say Joe Smith’s package for Liferea in his apt.joesmith.com repository we run into problems. With conary this NEVER WILL HAPPEN…EVER. This kills about 90% of dependency problems all together.

But what about arch? Arch is architecture…32bit or 64bit…PPC and more. Once again, you’re bit by the possibility of conflicting names across repositories. You’re also limited in the name because a developer has to put the architecture INSIDE THE NAME. Take a look at liferea as an example: liferea-1.2.2-2.el5.rf.x86_64.rpm. Is this easy for an end user to understand? Is it the same as liferea-1.2.2-2.el5.rf.x86-64.rpm?

Conary takes a different approach. Each package has a ‘flavor’ that it is ‘cooked’ (committed) in. There may be a 64bit flavor, 32bit flavor, Xen flavor, and so on. This flavor is visible to the user only if the user requests to see it…and it is NOT inside the name of the package. The package is still called, simply enough, liferea. Revision number, arch, upstream version, etc…are all handled automatically by Conary.

You can see how creating and maintaining software would rely less on a manual process and more on automatic source controlled one with Conary. You can also see how organized Conary is with its packages.

Sources.list Not Needed

Conary is smart enough to remember where you installed what package. There is no need to keep a sources.list. So if you install the package balsa (a Gnome mail client) from my personal repository the command would be:

sudo conary update balsa=caffeine.rpath.org@coffee:venti-1

So now what? Do you have to add caffeine into a sources.list somewhere? Nope. Conary remembers where that package came from and when an update is available later it will find it and notify you. Let’s say you did the same thing for a hypothetical repository for deb or RPM…you’d add in the repository address for where balsa is at. Then you inherit ALL packages listed at that repository…not just a single package. There once again may be problems with package names, versions, and now sources.

Conary elimiates this problem for you as well. A single package is taken from that repository and since the repository is versioned, conary knows where it came from. It knows that it doesn’t need anything else from that repository unless you tell it to install more.

Problems With RPM and Deb

The real problem I see with deb and rpm is monolithic dependency resolution. This is the term I use to describe what happens when you go to install one package and you get over 10 more packages as dependencies. Dependency resolution in RPM and Deb is left up to the developer to find when creating packages.

When packaging software with Conary, dependency resolution is automatically done FOR you. When you ‘cook’ a package, it calls out what dependencies you need to add to your ‘recipe’ (comparable to rpm spec file).

Another important characteristic of deb and rpm is that when you update a package, the old version is completely removed from your system. This means that if a program depends on another that is being removed, you’re out of luck unless it was flagged as a dependency (manually). As you can imagine, large packages like openoffice take forever to upgrade AND packages depending on one another for specific versions might find they have problems interacting. With conary, dependencies are done at the file level…so only the file(s) that requires updating is updated. This saves bandwidth for downloading and saves time for upgrading. It also allows you to get dependency resolution honed to specific files instead of just specific packages. This means that distros CAN become much smaller…that is, if you were making a liveCD and wanted to trim it down to under 200MB you could do so very easily with Conary’s fine tooth dependency resolution and packages that are componentized.

Rollback to Previous

Conary operates using something called changesets. It looks at what is on your system for software and what you want to install and creates a changeset (like a diff) between the two states. This changeset is then installed by the package manager…it reads it, fetches the software the changeset says it needs to install…and then installs it.

What if you installed a group of packages that you don’t want installed anymore? What if something you installed doesn’t work as expected? Rollback

sudo conary rollback #

where # is the number of rollbacks you would like to rollback to. Each installation action is considered one numbered change for conary. It tracks each installation/removal action and numbers it in a list. You can therefore return to a previous state on your system with ease. See Conary Rollbacks for more information.

Summary

I’ve covered quite a bit of information here…enough for a discussion I’m sure. Are there still areas about Conary you’re unsure of? Leave me a comment. Part II will be coming soon that will discuss more topics about Conary and Foresight Linux. I’d like to base Part II on answering questions from the readers.

Foresight Linux and Conary Part I originally appeared on Yet Another Linux Blog on April 23, 2008.

For those of you with developer status or above on the Foresight Linux project, there is a KDE build available for testing (along with XFCE if you really have ambitions). Login to rBuilder Online, Click on the Foresight Linux Project, go to Manage Builds and search for Foresight Linux KDE Alpha3 and click it (DVD only). I downloaded and tested this build release to see how things are going.

Foresight KDE inherits all the goodness from the Foresight Linux project. This means the install is tar based and completes in less than 7 minutes. It also means Syslinux, Compiz Fusion, and more from the Release Notes. If you are interested in helping us develop further, please visit us in freenode #foresight-kde.

Findings

I found 3 major issues during testing. First, updateall fails with glibc errors. I worked over 6 hours on getting a solution and enlisted many in the community on helping me get past it. So far, no one has been able to provide a solution. The second issue is that pango is causing odd text to display on various applications: Pic1, Pic2. No solution to this as of the time of this writing. I’m assuming that an update to pango will cure this.

Third, gtk-qt-engine wasn’t installed by default so gtk applications looked quite nasty. A quick sudo conary update gtk-qt-engine got me rolling.

Observations

Things don’t look great. While the performance of the desktop was fine (no crashes, quite speedy) the default font/text looks horrible. The installed Theme is the default KDE one…no customization. My resolution/Monitor wasn’t detected so I was at 800×600 for resolution. Menu’s are chock full of junk (default applications) and have little organization.

Looking at this, one can tell it truly is based on an alpha release but I have to say that I’m still a bit disappointed. All of these observations were made during the last alpha release as well. At the time of that alpha release, I did not have enough Conary-foo™ (OK, so it’s not trademarked…) to make anything happen. Since I now have alpha3 up and running on my main system at home and am a bit further in my conary knowledge…you can bet I’ll be working on the aesthetics of Foresight KDE. Sure, the innards are more important than the shell…but the shell is the first thing seen by the user.

Hopefully, if there is an alpha4 (or beta1 for that matter) we’ll have the aesthetics worked out by then to give the user a pleasant “Foresight” experience.

KDE4 Status

KDE4 can be installed as well. It is still pulling in from SVN snapshots though so it is not as stable as I’d like it to be. Do this at your own risk! I do not have it permanently installed on my system. To install:

sudo conary update group-kde4=/kde.rpath.org@fl:2-kde4-devel

One of the benefits of conary is show above…installing KDE4 is as simple as one command I like to compare that with the Ubuntu/Debian way of doing it. Since there is no sources.list, you don’t have to mess with it. Conary remembers the branch the software came from inherently so it goes to those same places for that software unless you tell it not too explicitly. Imagine that! A package manager that is smart and allows you to maintain less! It also allows you to have both KDE3 and KDE4 installed side by side…but remember, both KDE4 and this build release of Foresight KDE are not production releases so things WILL change. In other words, don’t put this on a system you intend to keep.

It is my understanding that the KDE4 branch for Foresight KDE will soon be reverting out of SVN checkout builds to push stability as application portage continues.

Foresight KDE Alpha3 originally appeared on Yet Another Linux Blog on January 23, 2008.

As an example, a user might not like it if you tell them their bug will not be fixed for the next release. This is normal practice in many major distributions. But if you tell a user that their bug won’t be fixed through 4 releases, you may have a problem. Unfortunately, this also is becoming a normal practice for some major distributions.

I’ve been on both sides of the fence as a user and developer. I also managed a major community incentive for PCLinuxOS called mypclinuxos.com and coordinated PCLinuxOS Magazine (which now boasts a circulation of over 50k people). The main vehicle for the massive growth it went through was empowerment. If a user needed something done, we did it right away. If we couldn’t do it right away, we found a way to do it, met later, and did it then. The users felt actively enguaged in their software and took ownership of it. I tell you about these endeavors not to boast, but to show you that I know a little bit about communities and how to make them grow.

Bureaucracy be damned.

I feel sorry for organizations that get so hung up on organization that they forget that organizing isn’t just politics and data. Organizing is people too (Keep your Soylent Green jokes to a min).

What I think is happening in Ubuntu’s case is really something that effects all linux distributions out there…a lack of a QA process to deal not with quality of software but quality of service. A lack of a process to make sure simplistic bugs like the Aumix one and what happened to address it never happen again. I know they have a person that they recruited as a liaison to do this for them but one person and a handful of processes won’t cut it. It can be done by users if it is setup correctly and if Canonical would trust and empower its community to do the lifting for them.

I guarantee if Canoncical and Ubuntu asked its users to jump 85%+ of them would ask “How High?” The point of this post is that there aren’t a lot of distros out there that empower their users. When I first started developing add-on packages with my friend Josh for MEPIS Linux, we were ostracized from the community. I experienced empowerment when I changed to PCLinuxOS as my primary distribution. Now, as I join the development team of Foresight Linux (KDE Version) I also have experienced the attitude “the tools are there…use them and allow us to help you”.

It’s refreshing to find this twice.

I’ve never been an Ubuntu user. This doesn’t mean I think it’s crap. I think it has done fantastic things and innovated many things for Linux and the desktop. I think that Ubuntu may be beginning to lose focus though on what matters. Canonical still isn’t making money hand over fist and this “desktop” thing isn’t paying the bills. More attention is being focused on selling the server side of things to fund the desktop side of things (my opinion). I sure hope Ubuntu can change prescriptions in their community glasses to bring things more clearly into focus.

Empowering the Linux Community originally appeared on Yet Another Linux Blog on January 18, 2008.