[pacman-dev] Disk space checking branch "complete"

[Dan McGee]

On Tue, Nov 16, 2010 at 5:36 PM, Allan McRae <allan at archlinux.org> wrote:
> On 17/11/10 03:25, Nezmer wrote:
>>
>> On Tue, Nov 16, 2010 at 09:03:37PM +1000, Allan McRae wrote:
>>>
>>> On 16/11/10 18:02, Bryce Gibson wrote:
>>>>
>>>> Hi Allan,
>>>> I just wanted to ask, it looks like your patches will make a sync fail
>>>> if it
>>>> finds there's not enough space, is that correct?
>>>> Because I'd suggest a warning may be more appropriate, especially for
>>>> use
>>>> cases like compressed filesystems.
>>>> Cheers
>>>> Bryce
>>>> ps. I noticed that space checking can be completely disabled via
>>>> pacman.conf, which may be seen as a suitable solution for people in
>>>> these
>>>> types of situation.
>>>
>>> Disabling checking in pacman.conf is what I would recommend in these
>>> sort of cases.  These are the sort of things we will only find out
>>> after we get some widespread usage of the feature.  I would be
>>> interested in what the size calculations actually do on compressed
>>> filesystems.
>>>
>>> Allan
>>>
>>>
>>
>> On the topic of FS compression(also FS caching and delayed allocation).
>>
>> The real issue I had was the calculation of install size when building
>> the package. The calculated size was always *very* small. I addressed
>> this issue with 2 patches. The 1st one (adding sync before running du)
>> didn't fix anything. The 2nd patch worked pretty well.
>>
>>
>> http://gitorious.org/pacman-bsd/pacman-bsd/commit/8b367d4441ba85b5548285d987afcfc84c4fcb3e
>>
>> http://gitorious.org/pacman-bsd/pacman-bsd/commit/cf3341fe591293a493bc925e0433a1f696b37d90
>
> I think that initial "sync" is not needed because du does one anyway. The
> second one looks interesting so I have bookmarked it to look at later.

We've putzed around with this a few times, haven't we? This basically
"reverts" this one:

commit 149839c5391e9a93465f86dbb8d095a0150d755d
Author: Xavier Chantry <shiningxc at gmail.com>
Date:   Mon May 26 23:46:01 2008 +0200

    du -b is not available on BSD, use du -k instead.

    This fixes FS#10459.

    There is apparently no portable ways to get the apparent size of a file,
    like du -b does. So the best compromise seems to get the block size in kB,
    and then convert that to byte so that we keep compatibility.

    Signed-off-by: Xavier Chantry <shiningxc at gmail.com>
    Signed-off-by: Dan McGee <dan at archlinux.org>

There are three or so file sizes that matter:
1) du -s : what we do now, sums up actual taken space so would be more
accurate with many < 4K files.
2) du -sb: what we did before and what is proposed, sums up apparent
size, so does not necessarily best represent installed size (either
sparse files or many <4K files would throw the number off). The 4K
assumption also may not always hold...
3) du --tell-me-how-many-blocks-but-not-compressed : what seems like
perhaps the ideal? I'm not sure, but this would basically
    for file in tree:
        total += ceil(filesize to 4K)

1 is the most portable; 2 we need different flags all over the place;
3 we definitely don't seem to be able to use exiting tools but this
would not be an awful one to write.

-Dan