06 February 2010

The "dist-upgrade" misnomer & confusion

Yesterday in #ubuntu, someone asked, "I am still confused about this. Everything claims that dist-upgrade actually *upgrades* distributions...can someone please clear this up for me"

So I told them:

<maco> apt-get dist-upgrade differs from apt-get upgrade in that it will remove obsolete packages and add new dependencies, while apt-get upgrade will not. this is necessary when upgrading from one distro release to another, but it is not the *only* time it is necessary. thus, in aptitude, dist-upgrade has been renamed to full-upgrade
<maco> apt-get dist-upgrade will only change you from one release to another if you've modified /etc/apt/sources.list to point to a newer release, but this method of upgrading is not recommended

They also asked "and if i do want to upgrade the distribution (not that i do), how do i go about that?" to which I responded:

<maco> the recommended way to change distro releases is sudo do-release-upgrade

They said it was the best explanation in the shortest amount of text, so I'm posting it here, hoping it'll make it easier for people to find. By the way, man apt-get does explain all this…just in slightly more technical terms.


Victor Zamanian said...

"[...] it will remove obsolete packages and add new dependencies, while apt-get update will not."

Weeell, apt-get update most certainly won't remove anything, nor will it install anything. You probably meant "upgrade" again? :)

bytor4232 said...

Thanks for posting this. I moved from Debian to Ubuntu, and never paid much attention to "command line" upgrades, always upgraded via the update manager. I attempted to upgrade a server using dist-upgrade, and it wasn't pretty. Its always good to know the right tool for the right job!

Ben said...

I added a !dist-upgrade factoid not too long ago due to the number of people who were confused as to its use.

Mackenzie said...

Thanks Victor, fixed :)

Shane said...

Wow misnomer is a word ive never heard being used, maco the wordsmith :)

eha1990 said...

Thank you for your post. I have never heard of the "sudo do-release upgrade" command before. I until now, I've only heard about the "sudo apt-get dist-upgrade -d" command to upgrade to the newest release. I'm in the process of installing Ubuntu 9.10 on my primary workstation right now. I thought about install the alpha version of 10.04 since it is only a few months until the 10.04 LTS is released, but I've decided to simply install 9.10. Do you think it is better to upgrade a computer via "sudo do-release upgrade" or by simply doing a fresh install of the latest release.

I enjoy your blog tremendously.

Ted Tibbetts said...

So.. is there a recommended way of upgrading to a new release over ssh? do-release-upgrade gives, or at least gave, a warning against doing so. I've found that during a run of do-release-upgrade I'll be unable to ssh in to that machine ("ssh_exchange_identification: Connection closed by remote host", despite ssh having worked 10 minutes before), and my existing ssh connection will eventually time out or get dropped. So even though the do-release-upgrade process is still running under screen, and I can attach to it from a console on the server, I can't get at it via ssh. This seems to happen sometime after the download completes; the last time I tried it I was able to handle one configuration conflict before the connection dropped. Maybe this is a particularity of my configuration though, because it does sound like other people have been able to do this successfully. I guess I should file a bug.

It seems like there should be a supported way to do this, what with all the virtual servers running ubuntu. I asked about it on #ubuntu and they said basically to just use apt-get. Previously I've done this by just modifying my sources.list, which seemed to work well enough, but presumably it's better to use the release version option to apt-get/aptitude. Which is -t, rather than -d, no?

Finally, is it better to use aptitude's command-line options or apt-get? I remember reading at one point that apt-get was deprecated, but that seems to no longer be the case.

I hope you don't mind the barrage of questions; I just asked most of them on #ubuntu and didn't get very conclusive answers, so I thought you or your readers might have more info. Thanks for your blog!

Mackenzie said...

Thanks for the questions :)

I'd guess that the reason it's not recommended to upgrade to a new release over SSH in general is that things can go wrong, such as in the reboot stage of the process, which can't be fixed over SSH. Networking services being restarted in a postinst script, for example, would be problematic.

The "-d" option to apt-get dist-upgrade and do-release-upgrade is to go to the development release. You're right that -t tells which release's repositories to pull from.

It used to be that aptitude was considered better because it would remove unneeded packages once everything depending on them was remove, if used consistently. Previously, with apt-get, you'd need deborphan to tell you what cruft was on your system. Nowadays, "apt-get autoremove" will get rid of any leftover packages that were installed as dependencies and no longer needed. It also used to be the case that aptitude installed Recommends (not just Depends) by default while apt-get did not. As of Lenny, Debian (and Ubuntu followed along) switched to having apt-get install Recommends by default too (by shipping a default config file which said to do so). So nowadays, it's basically whatever's more convenient for you to type.

I did find that in installing KDE SC 4.4 from the Kubuntu backports PPA that "apt-get dist-upgrade" did not remove the no-longer-needed libkabcommon4 while "aptitude full-upgrade" noted that it intended to remove it.

Ted said...

Thanks for your answers :)

That makes sense about SSH. I'm still wondering why I wasn't able to ssh in while do-release-upgrade was running, while merging in changes to configuration files. That problem started sometime before I was prompted to merge changes to /etc/bash.bashrc. Guess I'll file a bug against do-release-upgrade.

Where is the -d option to `apt-get dist-upgrade` documented? I did some digging but I couldn't turn it up; the only -d option mentioned was the general one to apt-get that's equivalent to --download-only.

Thanks for the brief history of apt. I was actually under the impression that aptitude required the -r flag in order to install recommends; if they've changed this then I guess I can stop passing that option every time I install something.

I started using backports mostly for the KDE stuff.. that was back when 4.1 was current in Intrepid, so it was really necessary. I thought KDE4 was neat and engaging and seemed quite promising, but even in 4.2 it was too buggy and it seemed like some obvious features were missing. I ended up switching over to using Openbox as my WM, which is actually pretty ideal for me.

In general I find that even with backports, stuff gets pretty out of date, especially on an LTS installation. My decision to uupgrade was mostly based on this. For example I couldn't run some ruby utilities because the package manager (rubygems) was out of date.

I don't mind building local versions of a few apps, but when the dependencies start getting out of date it seems like time to move on. I guess I could dig around for a PPA or check getdeb.net.

That being said, I still have a box running Intrepid, mostly because of hardware semi-compatibilities that could take me days to resolve after upgrading. IMHO this is the biggest problem with recommending linux to people: even if it works, there's no guarantee that it will continue to, unless they hold back from upgrading.

Mackenzie said...

Sorry I said that wrong. -d is for update-manager or do-release-upgrade. With apt-get dist-upgrade, you have to manually change /etc/apt/sources.list

As long as I've used Ubuntu, aptitude has installed recommends by default. It's apt-get that had its apt preferences changed to follow that behaviour now.

There's a difference between the normal ubuntu-backports repo and the Kubuntu Backports PPA, by the way.

Intrepid goes EOL in a few months... Have you filed bugs for those hardware issues (and tested with a Lucid CD)?

Jamesy said...

Wouldn't it be nice if the man pages offered the normal technical information and also a lamens (spelling) explination... I think that would help a lot of people just getting there feet wet...

Pamster said...

Unrelated, but i really like (and am glad you posted) what you did on the HacDC list. many thanks.

hhh said...

Sorry to post this so late, but I just noticed it.

"Where is the -d option to `apt-get dist-upgrade` documented? I did some digging but I couldn't turn it up; the only -d option mentioned was the general one to apt-get that's equivalent to --download-only."

"Sorry I said that wrong. -d is for update-manager or do-release-upgrade. With apt-get dist-upgrade, you have to manually change /etc/apt/sources.list".

I'm not a programmer, but apt-get dist-upgrade -d is for downloading packages without installing them, as stated in man apt-get. I use that option frequently as I currently use sidux (a distro based on Debian sid). Upgrades to sidux are supposed to be done via dist-upgrade and only in init 3 to avoid breaking X. The -d option allows the packages to be downloaded in X. They can then be checked for upgrade problems that get posted in the sidux news page and forums, and then installed in init 3 at a later time.


Динар said...

"is there a recommended way of upgrading to a new release over ssh? "
you think packages can be replaced to hacked versions if they are downloaded not over ssh? no, they have gpg keys.

HenryL said...

I found that while 'apt-get upgrade' installs all package updates, if I follow up with an 'apt-get dist-upgrade', it then proceeds to update the linux kernel only.

Mackenzie said...

That's because at that point it's all that's left to do. You already installed the rest :)

Dustmite said...

Based on your initial description it would imply that you would not normally want to use a dist-upgrade. Yet it is the only way to upgrade the kernel (that I am aware of), and sometimes other applications get locked behind that upgrade method. Is there another way to install kernels or any other applications held back while still using apt-get upgrade but not apt-get dist-upgrade?

Mackenzie said...

Any package can be individually upgraded by using "sudo apt-get install packagename" When the package is already installed, that's interpreted as a request to update that package.