Re: [pacman-dev] [PATCH] [RFC] makepkg: calculate exact total file size

On 26/12/11 03:27, Dave Reisner wrote:

On Sun, Dec 25, 2011 at 06:20:27PM +0100, Florian Pritz wrote:

...

On 25.12.2011 16:06, Dave Reisner wrote:

...

On Sun, Dec 25, 2011 at 08:37:24PM +1000, Allan McRae wrote:

...

The current calculation of the total file size for a package using "du" suffers from issues in portability and correctness. Especially on btrfs, this can result in clearly wrong package information such as:

Download Size : 14684.29 KiB Installed Size : 7628.00 KiB

Use a slower but more accurate method involving "cat" and "wc" to calculate total file size.

Signed-off-by: Allan McRae --- diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in index 8c6984d..c78db86 100644 --- a/scripts/makepkg.sh.in +++ b/scripts/makepkg.sh.in @@ -1132,8 +1132,7 @@ write_pkginfo() { else local packager="Unknown Packager" fi - local size="$(@DUPATH@ -sk)" - size="$(( ${size%%[^0-9]*} * 1024 ))" + local size="$(find . | xargs cat 2>/dev/null | wc -c)"

Unsafe xargs usage.

find . -print0 | xargs -0 2>/dev/null | wc -c

You forgot the cat.

find . -print0 | xargs -0 cat 2>/dev/null | wc -c

meow. Sorry, mittens.

...

...

Why can't we use @SIZECMD@ here? Same issues as du?

SIZECMD returns one file size per line so we'd also have to add them up.

Yup. I do this in paccache:

@SIZECMD@ "${candidates[@]}" | awk '{ sum += $1 } END { print sum }'

find . -print0 | xargs -0 stat -L -c %s | awk '{sum += $1 } END {

I'm happy using @SIZECMD@: allan@mugen ~/tmp/libreoffice print sum }' 196464312 allan@mugen ~/tmp/libreoffice

find . -print0 | xargs -0 cat 2>/dev/null | wc -c 195276472

allan@mugen ~/tmp/libreoffice

du -sb 196444140 .

Of course the numbers between the stat and wc approach are different because stat adds a "block size" amount for each directory of which there is 290 in the libreoffice package: (196464312 - 195276472) / 4096 = 290 So the SIZECMD approach is filesystem dependent, but in a way that is creates minimal difference, unlike the current approach which can wildly vary. It is also about the same speed as the current du based approach.

...

Small test (1003 bytes PKGBUILD on btrfs with default mount options): SIZECMD (stat -L -c %s) 1003 du -skh 512 du -sb 1003 wc -c 1003

If du -sb is portable that might be the easiest way.

It is not... which is why currently -k is used and then we multiply by 1024. There are many messages on pacman-dev about this back when that change was made to makepkg.