Open Source, Open Blog

Foresight Linux and Conary Part I

Apr23
2008 Written by devnet

People flat out do not understand anything about Conary. What I hear the most:

Why another package manger? Isn’t there already too many of these out there? Why use Conary when I can apt-get? Apt-get is soooo much better. Dpkg gives you sooooo much more than anything could possibly give you. Conary is still beta quality…rpm and deb are much more developed mature.

If the person(s) asking the questions above actually understood what Conary is and CAN do…they would see this is a very limited view of Conary. Not only is conary a package management system vis-a-vis a system that manages EVERY single package of software on your system…it is also a powerful version control system for software packages and packaging. It’s an enabling mechanism for packaging software quickly and easily.

I’d like to go over some of the things I think are great about Conary…clear up some of the “why is this needed” speak by showing how Conary actually gets things right and the common problems experienced by other package managers that it solves.

How Conary Organizes Packages

If you use debian or rpm repositories…you know that inside a repository directory “stable” (as an example) are all the stable packages for your distribution. The packages are versioned according to their upstream version (if the repository maintainers are sane) and maybe arch and revision number. This is done by hand. It is managed by hand. If developers/packagers cross names between repositories you are brought into dependency problems. To illustrate this concept, if you and I both packaged firefox3 and named it accordingly…and someone used both your and my repository…our versions would conflict because the packaging system wouldn’t know which one to install.

Conary takes the manual operation from this…if you use a Conary based system, your repositories ARE VERSIONED. In other words, the repositories aren’t static directories that contain a bunch of packages…they are versioned branches that contain components of software.

These components (packages) are also versioned according to upstream version…but revision is handled automatically by Conary…no manual process. This eliminates the possibility of having two packages named the same exact thing in different repositories. In other words, if Joe Schmoe is packaging Liferea for his apt.joeschmoe.com repository and names his package the same thing as say Joe Smith’s package for Liferea in his apt.joesmith.com repository we run into problems. With conary this NEVER WILL HAPPEN…EVER. This kills about 90% of dependency problems all together.

But what about arch? Arch is architecture…32bit or 64bit…PPC and more. Once again, you’re bit by the possibility of conflicting names across repositories. You’re also limited in the name because a developer has to put the architecture INSIDE THE NAME. Take a look at liferea as an example: liferea-1.2.2-2.el5.rf.x86_64.rpm. Is this easy for an end user to understand? Is it the same as liferea-1.2.2-2.el5.rf.x86-64.rpm?

Conary takes a different approach. Each package has a ‘flavor’ that it is ‘cooked’ (committed) in. There may be a 64bit flavor, 32bit flavor, Xen flavor, and so on. This flavor is visible to the user only if the user requests to see it…and it is NOT inside the name of the package. The package is still called, simply enough, liferea. Revision number, arch, upstream version, etc…are all handled automatically by Conary.

You can see how creating and maintaining software would rely less on a manual process and more on automatic source controlled one with Conary. You can also see how organized Conary is with its packages.

Sources.list Not Needed

Conary is smart enough to remember where you installed what package. There is no need to keep a sources.list. So if you install the package balsa (a Gnome mail client) from my personal repository the command would be:

sudo conary update balsa=caffeine.rpath.org@coffee:venti-1

So now what? Do you have to add caffeine into a sources.list somewhere? Nope. Conary remembers where that package came from and when an update is available later it will find it and notify you. Let’s say you did the same thing for a hypothetical repository for deb or RPM…you’d add in the repository address for where balsa is at. Then you inherit ALL packages listed at that repository…not just a single package. There once again may be problems with package names, versions, and now sources.

Conary elimiates this problem for you as well. A single package is taken from that repository and since the repository is versioned, conary knows where it came from. It knows that it doesn’t need anything else from that repository unless you tell it to install more.

Problems With RPM and Deb

The real problem I see with deb and rpm is monolithic dependency resolution. This is the term I use to describe what happens when you go to install one package and you get over 10 more packages as dependencies. Dependency resolution in RPM and Deb is left up to the developer to find when creating packages.

When packaging software with Conary, dependency resolution is automatically done FOR you. When you ‘cook’ a package, it calls out what dependencies you need to add to your ‘recipe’ (comparable to rpm spec file).

Another important characteristic of deb and rpm is that when you update a package, the old version is completely removed from your system. This means that if a program depends on another that is being removed, you’re out of luck unless it was flagged as a dependency (manually). As you can imagine, large packages like openoffice take forever to upgrade AND packages depending on one another for specific versions might find they have problems interacting. With conary, dependencies are done at the file level…so only the file(s) that requires updating is updated. This saves bandwidth for downloading and saves time for upgrading. It also allows you to get dependency resolution honed to specific files instead of just specific packages. This means that distros CAN become much smaller…that is, if you were making a liveCD and wanted to trim it down to under 200MB you could do so very easily with Conary’s fine tooth dependency resolution and packages that are componentized.

Rollback to Previous

Conary operates using something called changesets. It looks at what is on your system for software and what you want to install and creates a changeset (like a diff) between the two states. This changeset is then installed by the package manager…it reads it, fetches the software the changeset says it needs to install…and then installs it.

What if you installed a group of packages that you don’t want installed anymore? What if something you installed doesn’t work as expected? Rollback :) 

sudo conary rollback #

where # is the number of rollbacks you would like to rollback to. Each installation action is considered one numbered change for conary. It tracks each installation/removal action and numbers it in a list. You can therefore return to a previous state on your system with ease. See Conary Rollbacks for more information.

Summary

I’ve covered quite a bit of information here…enough for a discussion I’m sure. Are there still areas about Conary you’re unsure of? Leave me a comment. Part II will be coming soon that will discuss more topics about Conary and Foresight Linux. I’d like to base Part II on answering questions from the readers.

This content is published under the Attribution-Noncommercial-Share Alike 3.0 Unported license.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Posted in Linux@Work - Tagged , , ,
Similar posts
«
»
  • http://www.EeePCLinuxOS.com JMiahMan

    So me a decent front end to Conary.. Last I knew I had too do some pretty special stuff to implement Conary in my distribution. I set up a apt-get repo in 5 minutes on my own server. So that’s what bothers me about Conary. A Decent GUI is lacking and it seems to only work on Fedora or Foresight Based distributions. Also the only repos I see are hosted by rpath and in a confusing web. With rpm and apt-get I do a search online and find all my answers. With Conary a person who works at rpath can’t even help me out.

  • http://www.EeePCLinuxOS.com JMiahMan

    Oh and you also may want to take down the information about source list. It seems you’ve gotten confused. In apt-get a source list is a list of repos you can download packages from. Look in /etc/apt/source.list and you’ll see what I mean. How a package gets uninstalled is defined by the spec file in the rpm. There’s a section called %files this section tells rpm what’s being installed and then it references the same list again for uninstallation. There’s also further scripts you can run to do more then just delete those files you can unload modules etc.. as needed. Maybe you don;t understand rpm and that’s why you’re so partial to Conary. Getting a view of Conary from the rpath aspect and from you it seems way too confusing and too much of a hassle to maintain and setup. They may have got things right but that doesn’t mean it’s right for everyone else.

  • http://linux-blog.org devnet

    No, you just misunderstood what I wrote. If I have an rpm question, I just turn around and ask the person who created it who sits behind me…or I could ask my boss who literally wrote the book on rpm (Ed Bailey).

    When I say maintaining a sources.list, I’m speaking of how you know what rpm came from what repository or what .deb came from what repo…the location it came from. It’s self evident with conary…but not with rpm/deb…because with conary, the repo is versioned so you tell right away where it came from and where it looks for updates.

    With rpm/deb you may know where it came from after some slight investigation, but will it get updates from ONLY that repository? What if a version becomes available in another repository that is newer? With conary, this is handled automatically. With rpm/deb, you may not know what you’re getting.

  • http://linux-blog.org devnet

    shuttle seems to think it’s worthy enough with packagekit as a front end. Packagekit has been in Foresight longer than it’s been in any other distribution. Metadata searching will be implemented within the next few months…by then, it will be pretty slick.

    As for creating a repository for Conary, I can do this in less than 5 minutes :)

    A local repository can be done in ~10m.

    The reason the repositories are confusing to you is that you haven’t taken the time to learn how the conary version string works…what it means, what it gives you for benefits. Conary is fully open source, so you can create your own repository anywhere you want any time…rBuilder offers free repository hosting and provides the build tools to allow you to simultaneously build/release x86 x86_64bit versions in Parallels, Xen, ISO, LiveCD, VMWare, and Microsoft Virtual formats. What has your repository done for you today? Mine empowers me.

    Visit #conary anytime, we’ll show you how to search ALL repositories at once. We’ll show you how to search individual repositories. We’ll show you anything you need to.

    Remember, Conary is NOT JUST a package manager…you don’t IMPLEMENT it on other distributions to manage their packages…it MANAGES CONARY packages only in its current state.

    Packagekit is a UI for all sorts of package managers like rpm (soon to be deb on Ubuntu) and conary…it’s a front end to the backend :) So, it’s going to be a universal UI and searching for packages with it there will be the same as searching in Fedora.

    Come hang out in #conary freenode and ask the questions you need answered. The people there are VERY smart…much more so than you and I and they are helpful and will point you in the right place. If this isn’t your cup of tea, head over to http://forum.rpath.com and drop us a line there…we’ll answer any question you have.

  • JMiahMan

    Quite frankly I’ve managed thousands of Shuttles, I wasn’t impressed and I’ve seen the ForeSight Linux Version installed on it. I though it was a old and outdated version of ForeSight. It looked so vanilla it was ridiculous. Talking to the people I work with they too were not impressed and asked me is this truly what Linux has too offer. It’s much like the AsusOS that I’ve had to fight so hard against. Also like Ubuntu on dells. It’s not putting Linux’s best foot forward or truly representing what Linux can be. It falls short and then everyone sees Linux fall short and thinks that’s all it’s going to do.

    Also I shouldn’t have too learn how a repo string works. I didn’t have too learn anything but do a google search with apt and with no prior knowledge on setting up a apt repo it was done in 5 minutes. I have to read and understand a wiki to setup a conary repo. Which I couldn’t setup because I couldn’t package because I don’t want to install foresight. With apt it works debian and rpm (thanks to connectiva). Not cool or impressive. If I wanted to be stuck and not have freedom I would go to a proprietary OS.

    Also you need to talk to your rpm guys a little bit more. Yes with RPM you can have multiple repos. If one repo gets updated it sees the update. I’m running multiple repos now. I’ve updated a etcskel package bash and various other packages on my repo and they’ve updated fine.

    Also finding where the rpm comes from depends on the Frontend. In fact a lot of options come from the frontend. You’re saying rpm can’t do something or doesn’t offer it when it really has nothing to do with rpm or deb. It has to do with lack of development on a decent GUI that has the availability to show such options. There’s more then one GUI to chose from though and that leaves hope maybe someone will catch on. If fact this goes back to the age old problem in open source. People would much rather start from scratch then get there hands dirty and make something better. So we have 8 projects that end up doing similar things but not one of them is complete. So the Rpm guys get tired of working for redhat or decided so they drop rpm. Well hundreds of distributions now depend on rpm, it’s really quite unfair to say oh well we had it wrong and leave it in it’s current state to start something else. Why don’t they finish what they started instead of leaving others to finish what they started. This is one of the many issues Linux has and why it will be hard to be adopted on the Desktop. People know when things look unfinished developers also don’t want to read a wiki to find how to post there applications to a repo. You spend enough time creating and troubleshooting patching etc. Right now rpm is more finished then Conary. It’s better supported, it’s more wide spread, documentation doesn’t just lie with one distribution. This is coming Dev from someone who’s built RPM packages now managed a repo and attempted to do the same with Conary but never even got past having to install ForeSight as it crashed in VirutalBox. Maybe I’m too dumb to do it but that’s what you should assume when developing a application that the user may not be that bright. The objective of a application or any script is to make things easier and it depends on the developer to weigh functionality versus usability. Right now that’s how the other system have an advantage over conary. Conary isn’t new it’s been around at least 5 years, but it’s only being dabbled in with a few distributions in their testing beds. When a desktop oriented distribution adopts it I will be impressed and then maybe give it another look. Until then I really don’t think it’s worth my time no matter how much work it’s supposedly going to save me, it’s the curb appeal that matters and it has none.

    Hope I’m not being too harsh. That’s just how frustrated I am with the whole ordeal. I was so excited now I’m so let down.

  • http://linux-blog.org devnet

    [quote]So the Rpm guys get tired of working for redhat or decided so they drop rpm.[/quote]

    Wrong. They saw the limitations of rpm. rpm was designed to only be updated via CD…it was never meant for the web. Recognizing this and the fact that tailored operating systems were where the money would most likely be in the future, they left red hat and created specifix. It wasn’t long before they found that virtualization would be where it is at and they changed specifix to rPath.

    [quote]Right now rpm is more finished then Conary[/quote]
    Right now any package manager that has been around since the mid nineties is more finished than a package manager that’s only been around a few years. This point is moot.

    [quote]It’s better supported, it’s more wide spread, documentation doesn’t just lie with one distribution[/quote]
    Can’t believe you used the Microsoft argument on this…”Microsoft is more documented than linux, it’s been around longer, it’s more widely available…” To me, this point is also moot. How widespread something is only guarantees its popularity, not its quality.

    [quote]When a desktop oriented distribution adopts it I will be impressed and then maybe give it another look. Until then I really don’t think it’s worth my time no matter how much work it’s supposedly going to save me, it’s the curb appeal that matters and it has none.[/quote]
    Then we’re done here. It’s obvious from your comments (not even spelling Foresight correctly…ForeSight right?) that you haven’t checked into Conary as much as you say you have. If you had, much of what you had to say would be positive…there would be small ‘if onlys’ instead of huge disappointment. The reason I can say this is that I have rolled rpms in the past, I’ve created slack tarballs, I’ve done debs. I’ve used and created all of this since 1995. I’m no retard who just blindly gets behind something because he uses it. I support quality when I see it. I see it in Conary. I see the advantages it gives over traditional, legacy package management systems. It’s too bad you don’t even want to test it out. Companies like Digium, Ingres, EMC (yes, [url=http://www.emc.com/utilities/globalsiteselect.jhtml?checked=true]EMC[/url]), zimbra, and knowledgetree all use conary on a daily basis to build their software appliances. I’d say that it’s enterprise ready. Learning curve? Sounds like you’re just afraid to learn something new. Remember, it was that way the first time you switched from Windows to Linux too…good thing you didn’t give up then as easily as you have now.

    Again, since you’re my friend, I hope this isn’t harsh for you either :D

  • http://www.EeePCLinuxOS.com JMiahMan

    It’s pretty funny you can’t believe I used the Microsoft argument. I don’t believe Microsoft is really well documented at all. Documentation does play a huge role with users though. There’s more then just one reason MS has at least a 85% market share they run on just about everything. It’s also pretty funny that you’re contorting what I say. You attack me because of my poor spelling, for that I apologize. I suck at spelling in English you got me there. If you’ve ever taken philosophy you learned about deriving factual connections through statements. Somehow I don’t see JMiahMan mispells something so therefore he must not know about it the particular issue a factual statement. I miss spell my own name. That’s a sign of desperation in any disagreement.

    I have used Digium, that’s why I created my own Asterisk distribution. I’ve known quite a few people who have gotten laid off at EMC after all there headquarters is right down the street. You offer me nothing. I said Desktop distribution. When a distribution like Saybayon, or Mepis or your normal run of the mill volunteer developer can set up your easy to use Conary repos then I’ll be impressed. Show me that? Yeah of course a company paying its employees to figure something out or a company that sells support for updates may have a somewhat confusing system, it makes sense. They want you to call them. So some commercial companies have adopted Conary, you really think that impresses me? I’ve seen some commercial companies do some pretty stupid stuff. Don’t tell me what a company can do with it, with paid staff and the time too make it work. Tell me what I can do with it in my environment, with limited time and no pay. Because with rpm and apt I have something. Afraid to learn something new? No, Setting up a apt-get repo was something new. Afraid to spend hours learning something that won’t work on my distribution of choice?.. Yes, I don’t have the time or energy to spend on that. You’re missing the point. It doesn’t matter how great you think Conary is. Of course you think it’s great rPath has it’s hands all over it and you of course with rose colored glasses with support the people who pay your bills. That’s smart. Don’t give me a list though of how great it is compared to rpm or deb. With rpm/deb I can get to work great in a matter of minutes.. on the distribution of my choice. It’s been done, but your answer with Conary is to install Foresight (did I speel tha rite). Don’t tell me something show me, you failed too show me why would I invest anymore time in just words. When my objective is already done threw easier means.

  • http://linux-blog.org devnet

    Let’s say you release the next version of your software project whatever it may be.

    Do you take lessons learned and apply them to fix the problems of the previous version? Or do you just continue to make the same mistakes you made previously?

    That’s why software is versioned…so people know what has been fixed.

    Conary is rpm, the next generation. Erik saw that rpm needed something extra and created conary. He was and is in a position that no one else is being the author of the rpm format. So…if you think for a second that conary is harder than rpm, you are completely insane. There is a ONE PAGE setup instruction for packaging with conary. Conary is what rpm should be and should have been.

    So, my point is…whether you like it or not…Conary is a package manager with abilities above and beyond that of traditional package managers. I can’t force you to like it…but I can counter any advantages you say these legacy package managers have over conary.

  • http://www.EeePCLinuxOS.com JMiahMan

    You haven;t countered the fact it won;t run on my distribution of choice. You’re better doesn’t matter if I can’t use it in my environment. Who cares if it’s SVN (like) I don’t want my developers packaging anyways, even though they try, it’s not their job. I would rather them focus their energy on development. I don;t make the same mistakes with packaging because 1. I learn from my mistakes 2. I put good notes in %release in the spec file.

    So my point is Dev, Conary may be awesome and have all this wonderful stuff. That doesn;t mean it’s for everybody. The current system works great for me because:

    1. I don’t have to try and port Conary to a platform it’s not supported on. (there’s no documentation on that)

    2. I can host my own repos (not have them on rpath)

    I just works for me. So I look at Conary and say it’s overkill and for a vast majority of small projects that aren’t rPath based it’s going to be. That’s the beauty of OpenSource what you think is better technically (and is) doesn’t mean it will be better for me. So I have a choice and I can pick the best choice for me. Which I’ve done. It doesn;t need to be pushed though as the best thing since sliced bread because obviously there’s issues and it’s just not me that feels this way. I was all for it until I realized the work I would save packaging was nothing compare to the work I would have too do to:

    1. Port Conary to my platform.

    2. or completely change bases to use Conary.

    Rpm (apt-get) isn’t that broke for me so why fix it?

  • http://linux-blog.org devnet

    [quote]You haven;t countered the fact it won;t run on my distribution of choice.[/quote]
    and I don’t need to. Just like I can’t get debs to run correctly in PCLinuxOS or rpms to run in Ubuntu, why would you need to use conary packages on another system? The fact that you can’t “run” (which I assume means, use) on your distribution of choice is a fact of all package managers. However, if you want to install Conary on any distro, it’s possible…it just won’t manage packages it doesn’t create…just like deb and rpm won’t manage each other.
    [quote]I can host my own repos (not have them on rpath) [/quote]
    [url=http://wiki.rpath.com/wiki/Conary:Standalone_Repository]Need to look a Big Harder[/url]

    [quote]I just works for me. So I look at Conary and say it’s overkill and for a vast majority of small projects that aren’t rPath based it’s going to be.[/quote]
    and I don’t argue with you there…the point of this entire article was to show how people have misconceptions of what conary is and what it does…it wasn’t to convert anyone.

    [quote]Rpm (apt-get) isn’t that broke for me so why fix it?[/quote]
    To be addressed in part II…you don’t think it’s broke because you don’t know any better.

  • http://www.EeePCLinuxOS.com JMiahMan

    [quote]and I don’t need to. Just like I can’t get debs to run correctly in PCLinuxOS or rpms to run in Ubuntu, why would you need to use conary packages on another system? The fact that you can’t “run” (which I assume means, use) on your distribution of choice is a fact of all package managers. However, if you want to install Conary on any distro, it’s possible…it just won’t manage packages it doesn’t create…just like deb and rpm won’t manage each other.[/quote]
    That’s where I think you’re wrong. I can install dpkg on any distribution and install deb packages. It’s been done. I can do the same with rpm. I have a choice. I know because I’ve done it. There’s also applications to support handling debs on a rpm based distribution and visa versa. Alien would be one of those tools. If I can install and use Conary and Conary Packages (I understand it only handles conary packages) on my distribution of choice without issue then where’s the Wiki on doing it? Where’s the easy one page documentation and how do I setup a repo on my own server? If this is truly a better solution that I don’t know about show me. I’m more then happy to learn, but then what can I use for a GUI? Will PackageKit the only decent GUI I know of be stable and friendly enough at this stage to handle a normal desktop users needs?

  • http://linux-blog.org devnet

    [quote]I can install dpkg on any distribution and install deb packages.[/quote]
    You can install Conary on other distros and install Conary packages as well…the problem is, distros don’t use Conary packages.

    Let’s say that I installed dpkg on pclinuxos…think it would go without any hitches?

    Being that Conary has a [url=http://cvs.rpath.com/]checkout available[/url] and is open source…if anyone wants to experiment with it and install it on any other distro, they’re welcome to do so. Will it work out of the box? No more than rpm will on Ubuntu or deb will on pclinuxos…there always will be fiddling, heck, even directory path macros would be a source of ilk for rpm/deb.

    [quote]If I can install and use Conary and Conary Packages (I understand it only handles conary packages) on my distribution of choice without issue then where’s the Wiki on doing it? [/quote]
    There isn’t one. There has been no need to install it outside of an rPath based distribution. That being said, it isn’t a good time to give up just because it’s hard or not documented. There have been people installing it on Ubuntu, OpenSuse, and Fedora (I’ve seen wiki talk pages on it) and they’ve had success. So I know it can be done. :)
    http://wiki.rpath.com/wiki/FAQ:Conary_on_Debian_or_Ubuntu

    [quote]There’s also applications to support handling debs on a rpm based distribution and visa versa.[/quote]
    http://wiki.rpath.com/wiki/Conary:RPM_Package_Recipe_HOWTO
    rpms are covered. debs are being worked on. More on that to come (not in blog posts but in future features).

    [quote]Where’s the easy one page documentation and how do I setup a repo on my own server?[/quote]
    That was a hyperlink in my previous comment…I’ll give you the unhyperlinked version here:
    http://wiki.rpath.com/wiki/Conary:Standalone_Repository

    [quote]I’m more then happy to learn, but then what can I use for a GUI? Will PackageKit the only decent GUI I know of be stable and friendly enough at this stage to handle a normal desktop users needs?[/quote]
    The developer of Foresight is all about the desktop and making it easier for the end user. I’m sure you’ll find you have more input with him and development if you join the team/initiative or even if you just want to chat with him about the future. I can’t pretend to know what is going to happen. Try him in #foresight on freenode, nick kenvandine

  • Pingback: Eee PC Serbia » Blog Archive » Foresight Linux Mobile Edition

  • Pingback: Foresight Linux and KDE 4.2 | Yet Another Linux Blog