[pacman-dev] Remove directory symlink support in pacman-4.2 - Was: Use resolved paths for filelist operations

[Andrew Gregory]

On Sun, 17 Feb 2013 11:14:29 +1000
Allan McRae <allan at archlinux.org> wrote:

> On 16/02/13 12:02, Andrew Gregory wrote:
> > Properly handling symlinks requires using resolved paths.  I'm not sure if
> > returning strings instead of file_t structs is the best solution, but it seemed
> > like the least drastic one as we prepare for 4.1 and we weren't using the extra
> > information anyway.
> 
> Thanks for catching this and the patches - I made one small comment.
> 
> 
> This whole situation is beginning to really annoy me...   I know we
> (especially you) have put a lot of work to get symlinks to directories
> working so that when /lib -> usr/lib we now can deal with /lib/foo and
> /usr/lib/foo conflicting etc, but having to stat every directory to
> determine if it is a symlink and the resolve it is just wrong.
> 
> So here is what I am proposing.  Get pacman-4.1 out the door with these
> patches included.   Then the sole focus of pacman-4.2 should be
> stripping support of symlinks to directories out of pacman.  If /lib is
> a symlink on your system, no packages should put files in /lib.  Just
> make that a conflict.  If a package MUST put files there, the package()
> function in the PKGBUILD should look like:
> 
> package() {
>   ln -s usr/lib ${pkgdir}/lib
>   make install
>   rm ${pkgdir}/lib
> }
> 
> Done...
> 
> The only other reason these are used (that I have seen...), is poor
> man's mount points.  For example, your /usr partition is getting full
> and you decide to put it on your /home partition in /home/share (yes - I
> have seen that done...).  The are more correct options that should be
> used here - preferably resize your partitions or create a new partition.
>  On Linux, a bind mount could be used.
> 
> Overall, I see no need to keep this mis-feature.
> 
> 
> So, I am proposing, get this patchset merged and get pacman-4.1 out the
> door.  The just concentrate removing all directory symlink support from
> pacman.
> 
> The only thing that would need to be done here is to check all current
> packages do not have paths in them that require resolving symlinks to
> directories.   E.g. on machines with /lib -> usr/lib, no package should
> have a file in /lib.  I propose that in a 4.1.x point release, we should
> add a tool (or perhaps extend testdb) to detect these packages and print
> a warning.  So a bit of planning is needed, but I think it can be done.
> 
> Comments?
> 
> Allan

I found the following places where we currently use resolved filelists
(or will after my patchset):

Removal:
- check if an upgraded package owns a directory before removing it
- check if any installed packages own a directory before removing it
- check if backup files are in new packages

Conflicts:
- check for inter-target conflicts
- remove currently installed package's files from the conflict check
  for upgrades
- check if all files in a directory are owned by a package for a 
  directory -> symlink transition
- check if a package being removed solves a conflict
- check if file ownership changed from one install target to another
- check if a file conflict is due to the new package using a symlink
- check if a conflicted file is unowned and in the backup array

Having reviewed the related code far more than I never wanted to, I
believe I can get the number of resolved paths down to a manageable
level while keeping full symlink support.  We should be able to take
advantage of the fact that symlinks in paths are not the norm and file
basename collisions are actually quite rare to selectively resolve
individual paths as necessary instead of resolving whole filelists.  I
haven't worked out all of the specifics for doing this efficiently,
though, so it may not be worth it in the end.

If we do remove support, we should probably at least still use resolved
paths to check for inter-target conflicts.  If one target has a symlink
in its paths I don't see any other way to catch a conflict prior to
actually extracting the files.  As long as we only resolve files that
are new in the package and packages don't constantly move files around,
this should have only a small performance hit most of the time.