On Sun, Jan 10, 2010 at 7:34 PM, Jeff Horelick <jdhore1@gmail.com> wrote:

2. pacman -Qm has 2 flaws that i see. For one, it'll also list all your AUR packages, it'd be nice to maybe just list packages that were installed from the repos but are no longer there and ignore any manually installed packages. For two, i was thinking something a bit more obvious to the user (i've been using Arch for 2 years now and i didn't know about pacman -Qm) that would display on 'pacman -Syu' that after the packages were installed, something like: "libpng12 and libjpeg7 are no longer being used by any packages, would you like to remove them [Y/n]". I'm thinking this would be good because there could be a lot of clutter on the system if you have to check for "deprecated" packages manually.

Right, you would have AUR packages too with pacman -Qm. Btw you also have pacman -Qtd for the "no longer used dependencies" i.e. orphans.

On Mon, Feb 8, 2010 at 1:49 PM, Xavier Chantry <shiningxc@gmail.com> wrote:

With every big rebuilds we get new breakage stories. It seems like it's the norm nowadays rather than the exception.

I am wondering if it's really only the users that are to blame.. or if Arch is also to blame. Or if Arch was supposed to be an elitist distribution and is victim of its success.

More importantly, I am wondering if the sodepends/soprovides proposal would not actually be a more complex solution than the libfoo2/libfoo3/libfoo4 way.

I haven't seen a single reported problem from any of the recent big rebuilds that wasn't the result of a user doing something they ought not to do (usually piecemeal updates), an out-of-sync mirror (plus users that can't even recognize this when they see it), or AUR packages not rebuilt (again, along with users that don't know they have to do this for themselves). I think "blame the users" is quite adequate as a solution to these kinds of problems. There truly is a minimum level of understanding and competency required to run Arch, and a huge number of users that apparently don't come anywhere close to meeting it.

Keeping old versions of libs would violate Arch's policy of being bleeding edge and also complicate things. -1 from me. On Feb 8, 2010 2:03 PM, "Thomas Bächler" <thomas@archlinux.org> wrote: Am 08.02.2010 19:56, schrieb Ray Kohler:

I haven't seen a single reported problem from any of the recent big rebuilds that wasn't the res... A popular variant of this apparently was cairo-lcd, which everybody forgot to rebuild, thus breaking gtk2.

I should also had that the problems have been getting less due to the community-testing repository and the fact that we could synchronize extra and community rebuilds this way.

On Mon, 2010-02-08 at 13:37 -0700, Brendan Long wrote:

Couldn't the piecemeal update problem be fixed by just putting version numbers in the depends() section in each updated package, so for the libpng rebuild for example, depends(... libpng>=#.#)? It would fix the problem in the most obvious/arch-like way.

Some of us do that, so what happens next: very-old-system# pacman -Sy firefox pacman downloads new repo db, checks for firefox and downloads all the versioned dependencies. The user can run firefox fine now, but everything else on the system is broken because it wasn't updated for the new libpng. Now this can be caught by using depends=('libpng>=1.4.0' 'libpng<1.5.0'), but what if upstream decides to fuck up the ABI in a release and bump soname in the middle of the 1.4 branch? I don't expect libpng to do that, but there's examples of others around. We can never be sure that the next incompatible version will be 1.5.0.

The thing is: 1) Since arch is a rolling release, the package of a library is named such as 'libpng' and not 'libpng-1.1' or 'libpng-1.2', as at any given time only the latest package should exist in the system. 2) The files and symlinks are created during the package building process and the package merely holds them. Because of that, when there is an updated version of the package (may it be a library with an so bump or any other), all the files and content of the previous version are removed, and the content of the new one is installed. 3) In order to keep the old library, which are just the contents of a package, when there's a so bump one could do any of: a) Modify the library .install file to keep the old files -> the files won't be tracked by pacman, which is something NOT WANTED b) Include the old files in the new package, making it trackable by pacman -> this creates an inconsistency as the package for version X contains files of versions before X Therefore, the solution to this is: if the user want to keep the old version, he should either of: - maintain a backup in place (bad bad bad) - create a package with the previous files and install it with pacman, making it trackable and removable when not needed anymore The bottomline: keep an eye on pacman's output, read the mainpage news and decide what to do based on information. -- Guilherme M. Nogueira "Any sufficiently advanced technology is indistinguishable from magic." - Arthur C. Clarke

On 02/08/2010 04:48 PM, Jan de Groot wrote:

On Mon, 2010-02-08 at 13:37 -0700, Brendan Long wrote:

...

Couldn't the piecemeal update problem be fixed by just putting version numbers in the depends() section in each updated package, so for the libpng rebuild for example, depends(... libpng>=#.#)? It would fix the problem in the most obvious/arch-like way.

Some of us do that, so what happens next:

very-old-system# pacman -Sy firefox

pacman downloads new repo db, checks for firefox and downloads all the versioned dependencies. The user can run firefox fine now, but everything else on the system is broken because it wasn't updated for the new libpng.

Now this can be caught by using depends=('libpng>=1.4.0' 'libpng<1.5.0'), but what if upstream decides to fuck up the ABI in a release and bump soname in the middle of the 1.4 branch? I don't expect libpng to do that, but there's examples of others around. We can never be sure that the next incompatible version will be 1.5.0.

But wouldn't the optimal solution be doing the depends correctly on every package, so when your really slow user tries to update Firefox, it correctly informs them that they need to update everything to do that?

On Mon, Feb 08, 2010 at 05:06:07PM -0700, Brendan Long wrote:

But wouldn't the optimal solution be doing the depends correctly on every package, so when your really slow user tries to update Firefox, it correctly informs them that they need to update everything to do that?

That could place the user before a difficult choice. He may want *not* to update a particular package for any good reason (reported regression, adding unwanted dependencies, user base resistance, ...) while still wanting to install a new one that requires a new library version. There is *nothing* wrong with having two (or more) .so versions on the same system. Each appp will use the one it was compiled and linked against. If I have (on my system, not in a repo) - application A depending on libfoo.so.1 and then pacman installs - application B depending on libfoo.so.2 it can ignore the dependency of the installed A since it is already present. And it is the app that depends on the library, not the new library that depends on the app. Installing a new library should just leave A and its dependencies alone. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

On 02/08/2010 05:36 PM, fons@kokkinizita.net wrote:

He may want *not* to update a particular package for any good reason (reported regression, adding unwanted dependencies, user base resistance, ...) while still wanting to install a new one that requires a new library version.

The ability to keep old libraries may be nice, but I'm just saying that if Arch doesn't support keeping old libraries, they should make it explicit in the depends()

On Tue, Feb 09, 2010 at 03:09:00AM +0800, Ray Rashif wrote:

I really like the sodepends/soprovides idea, especially after the last discussion. After this one, I like it even more. The libfoo2/3/4 has always confused me, and is one of the reasons why our repos appear "cleaner" and are, in essence, "easier to navigate".

But really, the only problem we all can recall from the last rebuild was cairo, of which I myself was a victim (it just skipped my mind entirely that I had a foreign cairo). Even Nagy mentioned this as an example of a breakage which sodepends/soprovides could solve, but technically it's still our own short-sightedness at fault while everything was rebuilt and ready.

Here I'd like to chime in, also referring to the other thread (library version conflict) that I started yesterday. It makes perfect sense to me 1. to have a 'rolling release' system in which packages are updated as soon as possible instead of waiting for a bulk update in the form of a new 'version', 2. that all packages in a repo depend on the same library versions. What does *not* make sense is to impose (2) on a user's system. At least not strictly. If I install a new package, and doing that updates a library to one that is incompatible with the previous one (i.e. having a different majer version number) there is *no reason at all* to remove the previous library version. Having a 'rolling release' does not require doing that. If I notice that a new app or an update of an existing one has included a new major version of a library, I will in time probably update or recompile everything to use the new library as well. It would typically be a matter of some days. Just do not expect me to do that *immediately*. I do expect that any apps that I didn't update continue to work until I update them. This means that the previous library version must not be removed. I don't keep a system as a toy and for the pleasure of keeping it up to date. I need it to do my work, and having to do a complete system update at an unexpected time is not helping at all. Even less if that complete system update results in the machine locking up and /me having to spend some hours manually repairing file systems and installed packages. As happened today. Removing the old version means that the user is left with a system that does not work. In my case, installing Latex broke xv, firefox, gnuplot, emacs, and as I discovered only the day after, also my desktop. It left me with an utterly unusable system for no good reason at all. Except maybe some 'religious' one about imposing 'absolute purity' on users. (2) also means that there is no point to include a library version in the package name (Debian style, the cause of lots of misery). Just *do not remove the previous library* when installing one with a different major version. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

On Mon, Feb 08, 2010 at 04:19:04PM -0500, Ray Kohler wrote:

It is *required* to do only complete system updates when using Arch. Partial updates are not supported, *by design*.

If that is true, which I refuse to believe, it means that Arch is a toy system only suitable for kids who take a kick of out if keeping it up to date, in the same way as some others keep their postage stamp collection up to date but never use one of those stamps to actually send a letter. In any *even marginally* professional setting, system maintenance is something that is managed and planned for, and it should not be a side effect of installing some trivial new package. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

Aaron Griffin wrote:

On Mon, Feb 8, 2010 at 3:36 PM, <fons@kokkinizita.net> wrote:

...

On Mon, Feb 08, 2010 at 04:19:04PM -0500, Ray Kohler wrote:

...

It is *required* to do only complete system updates when using Arch. Partial updates are not supported, *by design*.

If that is true, which I refuse to believe, it means that Arch is a toy system only suitable for kids who take a kick of out if keeping it up to date, in the same way as some others keep their postage stamp collection up to date but never use one of those stamps to actually send a letter.

This is not conducive to fixing what you see as a flaw. Please act less like the "kids" you so profane and try to be more constructive. In short: if you're going to call people children, at least act like an adult.

Would you mind explaining HOW, from a pacman perspective, you plan to keep an old library on the system in your ideal system?

Example: installed libfoo 1.0 installed app-bar which links to libfoo 1.0 on server libfoo 2.0 (soname change)

When I attempt to install app-baz, which pulls in libfoo 2.0, how do you expect to resolve all the conflicts that result from keeping libfoo 1.0 on the system at the same time as libfoo 2.0? All sorts of things are in conflict here. There's no way to automatically cover these cases that I can see

Why not make libfoo 2.0 and libfoo 1.0 available all time time until a pacman -Qdt say it is not used/needed any more? Then the user could remove it and not expect anything bad to happen. Could something like that be done? FWIW, I have moved from Arch to Slackware on my main machines, just because of the "I did a sudo pacman -Syy && sudo pacman -Su" and now have @#*%*# it's broke, syndrome . I needed stability on those machines, not update and let's see what I have to fix for the next few hours, or I hope it didn't break anything. I still have arch on a "don't matter if it breaks" machine, just to play with.

On 02/09/2010 12:29 AM, fons@kokkinizita.net wrote:

On Mon, Feb 08, 2010 at 03:47:47PM -0600, Aaron Griffin wrote:

...

When I attempt to install app-baz, which pulls in libfoo 2.0, how do you expect to resolve all the conflicts that result from keeping libfoo 1.0 on the system at the same time as libfoo 2.0? All sorts of things are in conflict here. There's no way to automatically cover these cases that I can see

There are no conflicts resulting form doing this.

Suppose I have a fresh install with: (-> means a symlink)

libfoo.so -> libfoo.so.1.2.3 libfoo.so.1 -> libfoo.so.1.2.3 libfoo.so.1.2.3. // the actual library

and all apps have been linked to libfoo.so.1

When the library is updated as the result of e.g. installing a new package (and meanwhile the library version for the whole repo has changed) there are two possible scenarios:

1. Bugfix, security update or small enhancement: the library is fully compatible with the previous one and has the same major version number. The result is e.g.:

libfoo.so -> libfoo.so.1.3.5 libfoo.so.1 -> libfoo.so.1.3.5 libfoo.so.1.3.5. // the actual library

All apps use the new version.

2. A new major version of the library (very probably incompatible). The result is:

libfoo.so -> libfoo.so.2.1.1 libfoo.so.1 -> libfoo.so.1.2.3 libfoo.so.1.2.3. // the actual library libfoo.so.2 -> libfoo.so.2.1.1 libfoo.so.2.1.1. // the actual library

So pacman has installed libfoo.so.2.1.1, created the symbolic link to it from libfoo.so.2, and modified the symbolic link from libfoo.so to point to the new version.

The result of this is that:

1. The newly installed package will use the new library as it expects 'libfoo.so.2'.

2. The other applications will continue to use the old version, as they expect 'libfoo.so.1'. In other words they will still work. A smart user will probably update them soon, but is not forced to do this immediately.

3. Recompiling anything locally will use the new version as during compilation you use the name 'libfoo.so' without version number, which, using the symbolic link, gets coded into the binary as 'libfoo.so.2'.

*** There is no conflict. *** Pacman can forget about and even delete the package that supplied the old version. It just *should not remove the old library itself*.

you are focusing only on .so which is different but this schema will work only if the package is split in lib, -dev, whatever as now, the headers will conflict since it have the same name on the same location. -- Ionut

http://wiki.archlinux.org/index.php/The_Arch_Way#Code-correctness_over_conve...

On Mon, Feb 08, 2010 at 05:17:15PM -0600, Aaron Griffin wrote:

On Mon, Feb 8, 2010 at 4:54 PM, <fons@kokkinizita.net> wrote:

...

On Tue, Feb 09, 2010 at 12:36:55AM +0200, Ionut Biru wrote:

...

you are focusing only on .so which is different but this schema will work only if the package is split in lib, -dev, whatever as now, the headers will conflict since it have the same name on the same location.

Not true. When a new version is installed, the headers are replaced, and the symlink from 'libfoo.so' is modified to the new version.

pacman doesn't deal in symlinks in this manner. I think your understanding of WHAT a package is is a little off.

So *who* creates the symlink from libfoo.so -> libfoo.so.1.2.3 ? It's either pacman or ldconfig called by pacman. Unless you believe in little gremlins doing it while you sleep. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

On Mon, Feb 08, 2010 at 05:37:19PM -0600, Muhammed Uluyol wrote:

...

So *who* creates the symlink from libfoo.so -> libfoo.so.1.2.3 ? It's either pacman or ldconfig called by pacman. Unless you believe in little gremlins doing it while you sleep.

It is created 'inside' of the package, not during the installation.

$ bsdtar tzf libpng-1.4.0-2-x86_64.pkg.tar.gz .PKGINFO .CHANGELOG usr/ usr/lib/ usr/bin/ usr/share/ usr/include/ usr/include/pngconf.h usr/include/png.h usr/include/libpng14/ usr/include/libpng14/pngconf.h usr/include/libpng14/png.h usr/share/man/ usr/share/licenses/ usr/share/licenses/libpng/ usr/share/licenses/libpng/LICENSE usr/share/man/man3/ usr/share/man/man5/ usr/share/man/man5/png.5.gz usr/share/man/man3/libpngpf.3.gz usr/share/man/man3/libpng.3.gz usr/bin/pnm2png usr/bin/libpng14-config usr/bin/png2pnm usr/bin/libpng-config usr/lib/libpng14.so.14 usr/lib/libpng.so <--- Right there usr/lib/libpng14.a usr/lib/libpng.a usr/lib/libpng14.so.14.0.0 usr/lib/pkgconfig/ usr/lib/libpng14.so usr/lib/pkgconfig/libpng.pc usr/lib/pkgconfig/libpng14.pc

So what ? An update should create/adjust that symlink and it currently does. There's no problem with that. Just don't delete the old *.so, that's all I ask. -- FA O tu, che porte, correndo si ? E guerra e morte !

So what ? An update should create/adjust that symlink and it currently does. There's no problem with that. Just don't delete the old *.so, that's all I ask. Patch pacman, build your version, and add pacman to IgnorePkg

On Mon, Feb 08, 2010 at 05:37:31PM -0600, Aaron Griffin wrote:

"make install" does it

Called by pacman, I suppose. Anyway this is irrelevant. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

On Mon, Feb 8, 2010 at 5:45 PM, <fons@kokkinizita.net> wrote:

On Mon, Feb 08, 2010 at 05:37:31PM -0600, Aaron Griffin wrote:

...

"make install" does it

Called by pacman, I suppose. Anyway this is irrelevant.

No. pacman does not build packages. It installs them

On 09/02/10 08:54, fons@kokkinizita.net wrote:

On Tue, Feb 09, 2010 at 12:36:55AM +0200, Ionut Biru wrote:

...

you are focusing only on .so which is different but this schema will work only if the package is split in lib, -dev, whatever as now, the headers will conflict since it have the same name on the same location.

Not true. When a new version is installed, the headers are replaced, and the symlink from 'libfoo.so' is modified to the new version.

In a link step you refer to the lib as '-lfoo' which gets translated (via that symlink) into 'libfoo.so.N', the newest installed version.

If you have separate -dev and -lib packages (and Arch hasn't AFAICS) it is the package manager's job to always replace both in sync. Nothing magical about that, all distros I used before just did it that way.

I think you miss a point here. After a rebuild NONE of the packages in the repos depend on the old library. So there is no point in us packaging the old library for compatibilty, as none is needed. We only support the latest packages in the repos, so if you have issues with old versions of packages or packages from unsupported sources, then it is up to you to fix them. Allan

On Tue, Feb 09, 2010 at 12:26:59AM +0100, fons@kokkinizita.net wrote:

On Tue, Feb 09, 2010 at 09:18:00AM +1000, Allan McRae wrote:

...

I think you miss a point here. After a rebuild NONE of the packages in the repos depend on the old library. So there is no point in us packaging the old library for compatibilty, as none is needed. We only support the latest packages in the repos, so if you have issues with old versions of packages or packages from unsupported sources, then it is up to you to fix them.

It seems you didn't even read what I wrote.

I do *not* propose to keep old versions in the repo. I did even write explicitly that I consider it normal for all packages in a repo to depend on only one library version.

I *do* propose to not remove the actual old library files when installing a new one with a different major version. That's all.

In other words, do not *force* a user to update all app using a library just because one of them requires a newer version. Doing this leaves the user with a broken system, possibly at the most inconvenient time.

Heh? The idea of a rolling distro also applies to the AUR. The AUR as a part of Arch is as "rooling" as Arch is. Btw, there are workarounds like "libpng12" in the AUR, if you don't want to rebuild all of your own built application. --

On Mon, Feb 8, 2010 at 6:26 PM, <fons@kokkinizita.net> wrote:

In other words, do not *force* a user to update all app using a library just because one of them requires a newer version. Doing this leaves the user with a broken system, possibly at the most inconvenient time.

Arch doesn't force you to update. It's been repeated ad nauseum that partial updates are dangerous. I propose you don't update at all if you don't have time to update everything. Problem solved. You're trying to apply an ugly bandaid a non-existant wound.

On 02/08/2010 04:49 PM, fons@kokkinizita.net wrote:

On Mon, Feb 08, 2010 at 06:42:56PM -0500, dave reisner wrote:

...

On Mon, Feb 8, 2010 at 6:26 PM,<fons@kokkinizita.net> wrote:

...

In other words, do not *force* a user to update all app using a library just because one of them requires a newer version. Doing this leaves the user with a broken system, possibly at the most inconvenient time.

Arch doesn't force you to update. It's been repeated ad nauseum that partial updates are dangerous. I propose you don't update at all if you don't have time to update everything. Problem solved. You're trying to apply an ugly bandaid a non-existant wound.

Installing a new app is not an update. It may require an update of a library, and my point is that such an update should not disable other apps. Which is achieved by just leaving the old *.so in place. That's all.

It's more and more clear that the only resistance to this is of a very nonrational kind.

Ciao,

Why are you even using Arch? You sound like the kind of person who would want a "stable" distro like Ubuntu or Debian. Your changes sound like they would break the best parts of Arch (updates and simple packages).

On Mon, Feb 8, 2010 at 11:20 PM, <fons@kokkinizita.net> wrote:

But then: pacman knows that A is installed and depends on libfoo.so.1. But still it removes that library. Why ? I'd just say it fails to do its job, part of which is being aware of dependencies.

In fact, pacman doesn't know that application A needs libfoo.so.1. It just knows that package (which contains application A0 requires package libfoo (which contains library libfoo.so.1). That's the point. Pacman just tracks dependencies as described in the depends array of the PKGBUILD file. -- A: Because it obfuscates the reading. Q: Why is top posting so bad? ------------------------------------------- Denis A. Altoe Falqueto -------------------------------------------

On 02/08/2010 06:46 PM, fons@kokkinizita.net wrote:

...

It just knows that package (which contains application A0 requires package libfoo (which contains library libfoo.so.1).

In that case, play it safe and don't remove anything that any app could depend on. It's better than making a system instantly unusable.

Ciao,

If you're going to do that, why use a package manager in the first place? pacman isn't smart enough to keep track of what's a library and what's not, and fixing that would make packaging things a lot more complicated (and I get the idea that Arch isn't exactly overflowing with volunteers). And really, why use Arch if you don't want updates? Isn't the whole point that you want a system where everything is always up to date, even if things might be broken more often (not my experience)? It would be interesting to try to patch yaourt to do what you're wanting though. The simplest solution I can think of is some sort of script that finds out which files in a package are libraries (probably something simple like looking for $pkgname.$pkgver.so, combined with what files are different in the new package). When you update a library and a package that's held back depends on it: * Build a new package named $pkgname$pkgver that only contains the library * Change the package data for the held-back package to depend on the new library package I assume the first part could be done with regular expressions. I don't know where the package info for installed files is, but you could grab the package from the cache and update the depend section of the .PKGINFO, then re-package it. Maybe worth a try.

I guess this is not going to change, Fons. So, may I make one suggestion? You could create a script to do the following: 1) rsync your /usr/lib to something like /libsforever 2) run pacman with provided arguments from commandline 3) rsync back whats in /libsforever to /usr/lib skipping older files Its not pretty, I should say, but would work. -- Guilherme M. Nogueira "Any sufficiently advanced technology is indistinguishable from magic." - Arthur C. Clarke

On Tuesday 09 February 2010 08:32:04 Brendan Long wrote:

On 02/08/2010 07:50 PM, fons@kokkinizita.net wrote:

...

One very simple solution would be to never delete anything named /usr/lib/*.so* unless you really have to. That requires one regexp match. A hack, not perfect but it would help.

Ciao,

Except then you'd annoy everyone who wants their package manager to work properly. When I update a package I expect it to clean up after itself.

+1 Lets say I keep old libraries around outside packman(because pacman ignored them as per the proposal), who and when are they going to be cleaned up? My /usr/lib is 1.6G. If it is not cleaned up, it would baloon in less then 3 months enough to make LFS an attractive choice. Its simple. Update complete system once -or-twice a month. Its ok if you upgrade/install individual apps as needed. But a complete upgrade keeps the system clean and I love that about arch. -- Regards Shridhar

Le Tue, 9 Feb 2010 09:12:29 +0100, fons@kokkinizita.net a écrit :

On Mon, Feb 08, 2010 at 08:02:04PM -0700, Brendan Long wrote:

...

Except then you'd annoy everyone who wants their package manager to work properly. When I update a package I expect it to clean up after itself.

Work properly = not remove things that are still needed. Clean up after itself = remove them when no longer needed.

The situation is now very clear. Pacman in its current form just *can't* do this, and this defect is being presented as a virtue, which is a classical marketing scenario.

There are package managers that can do that, I think apt-get is one. Pacman can not by design because it is made for rolling release distributions: the system is considered to be always up to date, all of it. You are not supposed to update just part of it. Now if you agree to that you have too problems: 1) Sometimes installing new packages drag updates of libraries. Simple solution: never use pacman -Sy, always -Syu; if you want to install a package without updating everything use -S without the y. 2) There are problems with mirror sync. This is a *mirror sync* problem, not a Pacman problem. And by the way, the mirror I use doesn't have that problem because it updates the database files before the packages themselves. Because of that, if some packages are out of date, you get transient resolver failures at download time on -Syu and pacman just doesn't update. All you have to do is try again later... Updating all the database files *last* would be even better: there would never be any sync problem (if I understand well how it works). -- catwell

On Tue, Feb 09, 2010 at 09:49:50AM +1000, Allan McRae wrote:

We only support the latest packages in the repos, so if you have issues with old versions of packages or packages from unsupported sources, then it is up to _you_ to fix them.

I don't have issues with them, except that they get deleted when they shouldn't be. What happens now is that if I have a package that depends on a library that has been updated in the repos, you take the liberty to disable that package when by chance I installl something completely unrelated that just happens to use the same library in a new version. I don't ask you to 'support' my old package. Just to leave it alone. And that is *not* in conflict with your desire to use the a new library in the all repo packages. It is completely unrelated to maintaining a repo. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

On Mon, Feb 08, 2010 at 11:54:43PM +0100, fons@kokkinizita.net wrote:

On Tue, Feb 09, 2010 at 12:36:55AM +0200, Ionut Biru wrote:

...

you are focusing only on .so which is different but this schema will work only if the package is split in lib, -dev, whatever as now, the headers will conflict since it have the same name on the same location.

Not true. When a new version is installed, the headers are replaced, and the symlink from 'libfoo.so' is modified to the new version.

In a link step you refer to the lib as '-lfoo' which gets translated (via that symlink) into 'libfoo.so.N', the newest installed version.

If you have separate -dev and -lib packages (and Arch hasn't AFAICS) it is the package manager's job to always replace both in sync. Nothing magical about that, all distros I used before just did it that way.

Afais a pointless discussion. Arch _is_ a rolling distro. I use Arch for almost 3 years now and I never ran into a major upgrade breakage since. Upgrading in cycles of 4-6 days is a sane behaviour on produtive systems; this also covers almost all mirror sync delays. I think this should be mentioned in the wiki. A simple "pacman -Qu" returns the upgrade situation, so it's quite easy to decide if this is a major or "normal" upgrade. Sure this requires some knowledge of the system, but this is what Arch is all about. And real major upgrades (like libpng) are quite seldom anyway. --

On Mon, Feb 8, 2010 at 4:29 PM, <fons@kokkinizita.net> wrote:

On Mon, Feb 08, 2010 at 03:47:47PM -0600, Aaron Griffin wrote:

...

When I attempt to install app-baz, which pulls in libfoo 2.0, how do you expect to resolve all the conflicts that result from keeping libfoo 1.0 on the system at the same time as libfoo 2.0? All sorts of things are in conflict here. There's no way to automatically cover these cases that I can see

There are no conflicts resulting form doing this.

Suppose I have a fresh install with: (-> means a symlink)

libfoo.so -> libfoo.so.1.2.3 libfoo.so.1 -> libfoo.so.1.2.3 libfoo.so.1.2.3.     // the actual library

and all apps have been linked to libfoo.so.1

When the library is updated as the result of e.g. installing a new package (and meanwhile the library version for the whole repo has changed) there are two possible scenarios:

1. Bugfix, security update or small enhancement: the library is fully compatible with the previous one and has the same major version number. The result is e.g.:

libfoo.so -> libfoo.so.1.3.5 libfoo.so.1 -> libfoo.so.1.3.5 libfoo.so.1.3.5.     // the actual library

All apps use the new version.

2. A new major version of the library (very probably incompatible). The result is:

libfoo.so -> libfoo.so.2.1.1 libfoo.so.1 -> libfoo.so.1.2.3 libfoo.so.1.2.3.     // the actual library libfoo.so.2 -> libfoo.so.2.1.1 libfoo.so.2.1.1.     // the actual library

So pacman has installed libfoo.so.2.1.1, created the symbolic link to it from libfoo.so.2, and modified the symbolic link from libfoo.so to point to the new version.

The result of this is that:

1. The newly installed package will use the new library as it expects 'libfoo.so.2'.

2. The other applications will continue to use the old version, as they expect 'libfoo.so.1'. In other words they will still work. A smart user will probably update them soon, but is not forced to do this immediately.

3. Recompiling anything locally will use the new version as during compilation you use the name 'libfoo.so' without version number, which, using the symbolic link, gets coded into the binary as 'libfoo.so.2'.

*** There is no conflict. *** Pacman can forget about and even delete the package that supplied the old version. It just *should not remove the old library itself*.

The system with major/minor version numbers and the symbolic links is actually designed to be used like this, and to permit different versions to co-exist on a system.

A package is not a single .so file, unless that is your proposal - to split all .so files into their own packages. Here is a list of files that would conflict if this was done with libpng: libpng /usr/bin/libpng-config libpng /usr/bin/libpng12-config libpng /usr/bin/png2pnm libpng /usr/bin/pnm2png libpng /usr/include/libpng12/png.h libpng /usr/include/libpng12/pngconf.h libpng /usr/include/png.h libpng /usr/include/pngconf.h libpng /usr/lib/pkgconfig/libpng.pc libpng /usr/lib/pkgconfig/libpng12.pc libpng /usr/share/licenses/libpng/LICENSE libpng /usr/share/man/man3/libpng.3.gz libpng /usr/share/man/man3/libpngpf.3.gz libpng /usr/share/man/man5/png.5.gz

On 02/08/2010 04:05 PM, fons@kokkinizita.net wrote:

On Mon, Feb 08, 2010 at 04:38:46PM -0600, Aaron Griffin wrote:

...

A package is not a single .so file, unless that is your proposal - to split all .so files into their own packages.

Here is a list of files that would conflict if this was done with libpng: libpng /usr/bin/libpng-config libpng /usr/bin/libpng12-config libpng /usr/bin/png2pnm libpng /usr/bin/pnm2png libpng /usr/include/libpng12/png.h libpng /usr/include/libpng12/pngconf.h libpng /usr/include/png.h libpng /usr/include/pngconf.h libpng /usr/lib/pkgconfig/libpng.pc libpng /usr/lib/pkgconfig/libpng12.pc libpng /usr/share/licenses/libpng/LICENSE libpng /usr/share/man/man3/libpng.3.gz libpng /usr/share/man/man3/libpngpf.3.gz libpng /usr/share/man/man5/png.5.gz

If these files are in the same package, they will be replaced together with the library. As far as package management is concerned the old version doesn't exist anymore. The only thing that remains of it is the actual binary and one symlink, to be used by apps that have been linked with it. All the rest can be forgotten.

If these files are not in the same package, there should be a dependency relation so they are still replaced together with the binary library file.

All other distros I've used before just did this, no fuss, no problems.

Besides, what is the point of a 'rolling release' if you are still forced to do bulk updates as a side effect ? With a non-rolling versioned release at least this happens only when you expect it.

Ciao,

I assume the reason Arch doesn't do this is to make it easy to build packages. If you want an example of how "amazing" splitting packages up is, take a look at Ubuntu's package for Pidgin, then look at Arch's PKGBUILD. Good look replacing any Ubuntu package with your own custom version without spending 3 days editing files every time there's an update. Honestly, building a library from the AUR when you need it is much simpler than packaging everything Ubuntu style.

On Mon, Feb 08, 2010 at 11:07:41PM +0000, Pierre Chapuis wrote:

Le Mon, 8 Feb 2010 23:29:48 +0100, fons@kokkinizita.net a écrit :

...

*** There is no conflict. *** Pacman can forget about and even delete the package that supplied the old version. It just *should not remove the old library itself*.

And you end up with .so files not tracked by Pacman in your filesystem? Imagine what will happen to a 2 year-old installation...

The worst that can happen is that you have number of old *.so that no app uses anymore. They don't harm at all, and they don't get in the way of anything new. An update of a library replaces the symlink from the unversioned name, so nothing you do locally will ever refer to the old libs. Any binary updates will use the new ones as well. So you have to clean up every now and then if the disk space is critical. It's much less a nuisance than seeing your system broken just by installing one small trivial app that happened to require a new library version. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

Excerpts from Allan McRae's message of 2010-02-09 00:26:37 +0100:

On 09/02/10 04:49, Xavier Chantry wrote:

...

With every big rebuilds we get new breakage stories. It seems like it's the norm nowadays rather than the exception.

I am wondering if it's really only the users that are to blame.. or if Arch is also to blame. Or if Arch was supposed to be an elitist distribution and is victim of its success.

I think the answer to that is in the question: What did we do different previously that resulted in far less of these issues?

My impression is that nothing has particularly change in terms of how rebuilds are handled. If anything, the whole process has become a lot more streamlined and cases of missing a package rebuild are now almost non-existent.

So the cause must be... A change in user-base? Maybe just an increase in user-base resulting in more people who think Arch should be done their way and not the Arch way?

I don't know whether you (I don't mean you alone) are just being cocky or blind or I don't know what, but I've seen this attitude all over the place and I don't get it. By this attitude I refer to the total ignorance regarding these serious problems, bye developers and regular users on IRC or right here. Lots of people had issues when those new versions hit [extra], ranging from the very common file corruptions due to incompletely synced mirrors to minor and major breakage due to a lacking version check. I alone observed lots of issues, on my system, on IRC and here. There was one bug I reported which was flagged as not a bug (wrong version in dependency array) and who knows how many more. What's annoying here was the general attitude and 'good advices' given. "works for me, you must be doing something wrong" or "you should have read the frontpage, it tells you not to upgrade until the mirrors are synced" which for example isn't what it says, and there's no way to check whether they really are synced, but you get "good advice" anyway: "you should have checked https://www.archlinux.de/?page=MirrorStatus" which doesn't tell you in any way which mirror has fully synced. Anyway, the point is that the update process is obviously flawed, otherwise there wouldn't be so many people having issues. A user with issues gets bad/useless advice served with a cocky attitude. If a user makes actual improvement suggestions, like Fons does, he also gets mostly useless responses with the same attitude. The only one who has attempted to be somewhat constructive so far was Aaron, so kudos for that. I'm sorry for my language in this mail but I'm somewhat angry about how this seems to turn out. Regards, Philipp

On 09/02/10 10:05, hollunder wrote:

...

Excerpts from Allan McRae's message of 2010-02-09 00:26:37 +0100:

...

On 09/02/10 04:49, Xavier Chantry wrote:

...

With every big rebuilds we get new breakage stories. It seems like it's the norm nowadays rather than the exception.

I am wondering if it's really only the users that are to blame.. or if Arch is also to blame. Or if Arch was supposed to be an elitist distribution and is victim of its success.

I think the answer to that is in the question: What did we do different previously that resulted in far less of these issues?

My impression is that nothing has particularly change in terms of how rebuilds are handled. If anything, the whole process has become a lot more streamlined and cases of missing a package rebuild are now almost non-existent.

So the cause must be... A change in user-base? Maybe just an increase in user-base resulting in more people who think Arch should be done their way and not the Arch way?

I don't know whether you (I don't mean you alone) are just being cocky or blind or I don't know what, but I've seen this attitude all over the place and I don't get it. By this attitude I refer to the total ignorance regarding these serious problems, bye developers and regular users on IRC or right here.

It might be being elitist, but saying so does not explain why there were not such big issues earlier in Arch's history. Maybe the target of "competent linux users" does not accurately reflect the user base. So, should the target change or should the user base change? I agree that Arch should stay focused on competent Linux users, but the update process does have flaws. The fact that users can do incomplete updates (because devs don't set dependencies strictly enough) is one

On 02/08/2010 05:50 PM, Allan McRae wrote: that I can think of. On top of that, no one wants to check the mirror status every time they do an update, it's not unreasonable to expect the package manager to keep things working.

On Mon, Feb 8, 2010 at 8:52 PM, hollunder <hollunder@lavabit.com> wrote:

Excerpts from Brendan Long's message of 2010-02-09 02:09:13 +0100:

...

...

On 09/02/10 10:05, hollunder wrote:

...

Excerpts from Allan McRae's message of 2010-02-09 00:26:37 +0100:

...

On 09/02/10 04:49, Xavier Chantry wrote:

...

With every big rebuilds we get new breakage stories. It seems like it's the norm nowadays rather than the exception.

I am wondering if it's really only the users that are to blame.. or if Arch is also to blame. Or if Arch was supposed to be an elitist distribution and is victim of its success.

I think the answer to that is in the question: What did we do different previously that resulted in far less of these issues?

My impression is that nothing has particularly change in terms of how rebuilds are handled.  If anything, the whole process has become a lot more streamlined and cases of missing a package rebuild are now almost non-existent.

So the cause must be... A change in user-base? Maybe just an increase in user-base resulting in more people who think Arch should be done their way and not the Arch way?

I don't know whether you (I don't mean you alone) are just being cocky or blind or I don't know what, but I've seen this attitude all over the place and I don't get it. By this attitude I refer to the total ignorance regarding these serious problems, bye developers and regular users on IRC or right here.

It might be being elitist, but saying so does not explain why there were not such big issues earlier in Arch's history.  Maybe the target of "competent linux users" does not accurately reflect the user base. So, should the target change or should the user base change? I agree that Arch should stay focused on competent Linux users, but the update process does have flaws. The fact that users can do incomplete updates (because devs don't set dependencies strictly enough) is one

On 02/08/2010 05:50 PM, Allan McRae wrote: that I can think of. On top of that, no one wants to check the mirror status every time they do an update, it's not unreasonable to expect the package manager to keep things working.

Mirrorstatus doesn't really help you, I've played with it during the big upgrade because pretty much everyone just said something along the lines of: "look at the startpage, dumbass" and even a dev couldn't come up with a better practical suggestion and didn't know what the mirrorstatus info really meant.. It doesn't know when a mirror is in sync, what it presents to you is probably the last sync starttime or something similar.

The only information you can get from the mirrorstatus page is when the mirror started its last sync and how fast the connection is more or less.

As far as mirror are concernced, we (the devs) are aware of the problem and knew that the move of the rebuild out of testing would create havoc on the mirrors. However, we are working toward fixing that problem: http://mailman.archlinux.org/pipermail/arch-dev-public/2010-February/015335.... So we shouldn't have these mirrors sync problem anymore.

Excerpts from Allan McRae's message of 2010-02-09 01:50:02 +0100:

On 09/02/10 10:05, hollunder wrote:

...

Excerpts from Allan McRae's message of 2010-02-09 00:26:37 +0100:

...

On 09/02/10 04:49, Xavier Chantry wrote:

...

With every big rebuilds we get new breakage stories. It seems like it's the norm nowadays rather than the exception.

I am wondering if it's really only the users that are to blame.. or if Arch is also to blame. Or if Arch was supposed to be an elitist distribution and is victim of its success.

I think the answer to that is in the question: What did we do different previously that resulted in far less of these issues?

My impression is that nothing has particularly change in terms of how rebuilds are handled. If anything, the whole process has become a lot more streamlined and cases of missing a package rebuild are now almost non-existent.

So the cause must be... A change in user-base? Maybe just an increase in user-base resulting in more people who think Arch should be done their way and not the Arch way?

I don't know whether you (I don't mean you alone) are just being cocky or blind or I don't know what, but I've seen this attitude all over the place and I don't get it. By this attitude I refer to the total ignorance regarding these serious problems, bye developers and regular users on IRC or right here.

It might be being elitist, but saying so does not explain why there were not such big issues earlier in Arch's history. Maybe the target of "competent linux users" does not accurately reflect the user base. So, should the target change or should the user base change?

I can't speak of the past, I only use Arch since about a year. It might not have to do with userbase competence at all, for example I had few issues, a handfull of packages that wouldn't work because they were already compiled against the new version while I still had the old one installed, corrupted packages from pretty much all mirrors I tried for a while, nothing serious, and I also don't run any critical system. Fons on the other hand is certainly far more experienced and competent than me, runs critical systems and had apparently bigger problems. So if we look at this admittedly small sample then your theory of the now suddenly dumb userbase seems wrong. What might be true is that with growing popularity the ratio of users who know all of Archs quirks inside out to normal users is likely changing. Maybe the issues are somewhere else? I don't know. One observation I, as a stinking normal user, could make is that there are few devs around where users hang out. Let's see.. I know of one dev active in #archlinux on IRC and about three devs active on this mailinglist, including you, Aaron and the IRC-dev. So maybe it's the devs who detached from the users and there's simply no-one around who could pass on the knowledge? Sure you get pissed off by all those dumb users, but how should they get wiser when everyone else is just as dumb? How should the userbase change? One thing is for sure, elitist and cocky behavior will alienate users, competent ones as well, and alienated users wont help, they wont write patches, they'll switch distro at best. Right now I see, especially on IRC, a culture of slightly less dumb users with elite attitude alienating other dumb users. That's not helping anyone. I know, it doesn't really answer your question, but maybe it raises a few new ones. Regards, Philipp

On Tue, Feb 09, 2010 at 02:21:58AM +0100, hollunder wrote:

Fons ... runs critical systems and had apparently bigger problems.

I should mention that while there have been some problems I am in general very satisfied with Arch, the documentation and the support. The really 'big' problem occured yesterday when installing Latex instantly broke a lot of other packages, including (but that I only discovered this morning) my desktop, and for no good reason at all. But then doing a full system upgrade (as advised on this list) completely locked up the machine - frozen screen, dead capslock led etc. No other choice but a power cycle, broken filesystems, and hours of work repairing them and putting bits and pieces back in place. I still have a about 700 files in lost+found, most of them seem *.pyc and man pages, so probably no big losses. I'm facing a decision to change five 'critical' machines in need of an upgrade to Arch or not. Over the last months I've installed Arch on three less critical ones to get to know the system. Up to yesterday the decision looked quite clear, but today I'm having a hard time trying to ignore the 'cocky attitude' and keeping focused on technical arguments. Which as you wrote have been lacking somewhat, with some nice exceptions. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !

On Tuesday 09 February 2010 08:35:14 Mauro Santos wrote:

Currently for my personal use I don't want anything else other than Arch but for machines that have more users and _need_ to keep working I'm using centos (devs I'm sorry to hinder your world domination plans but I think I'm not competent enough to manage Arch on a server so better use something that in theory will give me less surprises).

There are multiple issues at work here - updateing multiple machines automatically, possibly from one master copy. e.g. I have 3 desktop machines out of which I use 2 regularly and I have to spend sometime upgrading even if the cache is shared. - for server install minimal is better and arch can be tweaked down to bare minimum necessary. - testing for upgrades. I think something like linux containers would be handy here to upgrade things and test. If the multiple sync. method is available, then things can be easily synced. to-and-from. Kernle upgrades are the only exception. -- Regards Shridhar

fons@... wrote:

But then doing a full system upgrade (as advised on this list) completely locked up the machine - frozen screen, dead capslock led etc. No other choice but a power cycle, broken filesystems, and hours of work repairing them and putting bits and pieces back in place. I still have a about 700 files in lost+found, most of them seem *.pyc and man pages, so probably no big losses.

"lost+found" is a filesystem thing: it's the place where files are put that cannot be linked up eg. in the fsck phase. Other than that: if you need old libs or headers, you will have to save them some place, the arch goods will not help you with this. Ironically, if pacman was such a tidy package manager and devs were so keen to keep things that way, why is there ever any output in "pacman -Qdt"? clemens

Nathan Wayde wrote:

On 10/03/10 20:03, clemens fischer wrote:

...

Ironically, if pacman was such a tidy package manager and devs were so keen to keep things that way, why is there ever any output in "pacman -Qdt"?

Could it be because you explicitly requested that? i.e `pacman -R pkgname` which removes that pkg and that pkg only, see `pacman -Rh`.

I always use "pacman -Rs". It's a wrapper script, I normally don't use a bare "pacman", so there's not even neglect at play. clemens

On 11/03/10 08:22, Daenyth Blank wrote:

On Wed, Mar 10, 2010 at 16:54, clemens fischer <ino-news@spotteswoode.dnsalias.org> wrote:

...

I always use "pacman -Rs". It's a wrapper script, I normally don't use a bare "pacman", so there's not even neglect at play.

That's the problem. You want to add -c. -s removes in one direction, -c removes in the other.

Not really... The problem is that pacman does not clean up packages installed as a dependency for a package that are no longer needed due to an update which removed that dep. Also, using "makepkg -s" leaves makedepends as orphans. Allan

On Wed, Mar 10, 2010 at 11:45 PM, Allan McRae <allan@archlinux.org> wrote:

Not really...

The problem is that pacman does not clean up packages installed as a dependency for a package that are no longer needed due to an update which removed that dep.

And by the way, we cannot be 100% sure that the 'no longer needed dependency' is not useful in itself. A package foo might have been installed as a dep for package bar, but it can also have some other purposes, unknown to pacman. I don't know if this is just a theoretical problem that has no practical value. But well.. pacman -Qtd exists for a reason..

On Tue, Feb 9, 2010 at 3:19 AM, Arvid Picciani <aep@exys.org> wrote:

if someone actually posts patches or other constructive stuff, please CC me. We're rewriting pacman anyway and looking for a solution to handle this mess in particular.

Right now the only idea i got is versioned deps which is sort of flawed since that would assert a certain awareness upstream. And if we had that, we didnt have the problem in the first place...

maybe there is a smarter way to force update of packages that would break when their dependencies are updated.

If people actually read my 10 lines mail, and spent more time to think and less time to write, maybe the thread could have gone forward rather than backward ? The last discussions on the sodepends/soprovides proposal are there : http://mailman.archlinux.org/pipermail/pacman-dev/2010-January/010386.html http://mailman.archlinux.org/pipermail/pacman-dev/2010-February/ It's not the first time I regret posting to arch-general, I should have kept this on pacman-dev only, discussions tend to be much more constructive there with much less noise. Anyway, care to explain what you are rewriting pacman for ? There are probably plenty of good reasons to do that, I am just curious to know if you have any :)

On Tue, Feb 9, 2010 at 11:08 AM, Allan McRae <allan@archlinux.org> wrote:

On 09/02/10 19:50, Xavier Chantry wrote:

...

Anyway, care to explain what you are rewriting pacman for ? There are probably plenty of good reasons to do that, I am just curious to know if you have any:)

I guess it is for some ideas here: http://www.hereticlinux.org/wiki/pacideas

# user can then diff /usr/etc/foo.conf /usr/etc/foo.conf and take action I don't expect to see much differences between /usr/etc/foo.conf and /usr/etc/foo.conf, do you ? :P Anyway, nothing interesting there, but also nothing justifying a rewrite. Minor hacks at most.