[pacman-dev] Pacman Disk Space Issues

[Aaron Griffin]

On Fri, Feb 20, 2009 at 8:29 AM, Teran McKinney <sega01 at go-beyond.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Hey,
>
> I've personally ran into this before, but don't think that it is worth
> "fixing" in the current Pacman setup. User error is usually the cause of
> this, since an admin should always have adequate disk space before
> installing new packages. Preventing this is difficult, like Brendan
> said, available disk space may be taken by other causes when Pacman is
> running. Not to mention, some users like to split the filesystem into a
> multitude of partitions.
>
> However, I do have one unlikely suggestion that *could* combat this
> without much difficulty. I was thinking of this more to (partially)
> solve the issue of package updates being cut short by a hard shutdown.
> Basically, packages would be extracted to /var/lib/pacman/extract (or
> similair), and if extraction has not errored, files would be moved to
> their corresponding locations. This would keep the new, possibly
> incompatible files out of the system until they can almost instantly be
> moved over, except in the case of /var, or others being on different
> partitions. This has the drawback that the user must have roughly
> installed size*2 disk space free, since the old package and new package
> would both have to be extracted at once (I personally think this is
> almost a non-issue). The current Pacman implementation may already
> require that much free disk space, but I'm not sure.
>
> An alternative to the above is to create directories in the folders
> that the package extracts to, such as .pacextract in /bin. This would
> insure that all mounted partitions get a fast rename/move, mitigating
> potential data loss and allowing for perfect disk space checking. It
> could be done so that the .pacextract feature only runs on the bottom 2
> or 3 directories (/usr, /usr/local, but not /usr/share/man/man8, which
> would have .pacextract/man8 in /usr/share/man), or on a predetermined
> set of directories in pacman.conf. I think my explanation might be a
> little confusing, so I will write it out in approximate psuedo-shell
> script.
>
> mkdir /usr/bin/.pacextract
> tar xf xz-4.999.8beta-1.pkg.tar.gz -C /usr/bin/.pacextract (with other
> flags so that tar only extracts the archive contents of /usr/bin)
> mv /usr/bin/.pacextract/* /usr/bin && rm -r /usr/bin/.pacextract
>
> The simpler (extract to /var/lib/pacman/extract) method does not offer
> the fast move when multiple partitions are used and perfect disk space
> utilization checking, but would be easier to code. Anyways, this
> idea is quite experimental. I do think it would thoroughly solve the
> disk space without too much difficulty, but I personally wouldn't use it
> soley for that.
>
> Cheers,
> Teran

This was done by the freebsd guys who also make libarchive and they
abandoned it.
I discussed it at length, with the following idea:
* packages are extracted to some dir
    i.e. /var/lib/pacman/extract/$pkgname-$pkgver-$pkgrel/
* for each file to be installed
    if file in filesystem, move existing file out of the way (rename
to *.pactrans.$PID)
    copy temp file to filesystem
    if error, remove ALL file, rename ALL .pactrans files back in place

This would solve things, but the big problem with this approach is
speed. It adds HUGE amounts of disk IO and, according to the people
who made libarchive, slows the process down so noticeably, it isn't
even worth it.

If I were to modify this process, I would use a tmpfs dir for the
/extract/ dir, and move existing files THERE (with the .pactrans
addition). This would save us all the disk IO, but is going to get all
the more complex.

This is, in effect, what Dan is talking about when he says "real
transactions" - the ability for pacman to abort in the middle of an
install and roll everything back successfully.

If we implement "real transactions", the disk space issue becomes a moot point.