[pacman-dev] [PATCH 1/2] Revert "makepkg: calculate exact total file size"

[Dave Reisner]

On Wed, Mar 14, 2012 at 07:48:06PM -0400, Dave Reisner wrote:
> On Thu, Mar 15, 2012 at 07:11:07AM +1000, Allan McRae wrote:
> > On 15/03/12 06:41, Dave Reisner wrote:
> > > btrfs's cow snapshots seem to do very strange things, causing du to not
> > > count data still in filesystem buffers. Repackaging the same build of
> > > handbrake within a chroot on brynhild, I witnessed (via bash xtrace) du
> > > reporting a different $pkgdir size every time the write_pkginfo()
> > > function ran.
> > > 
> > > Unfortunately, replacing du with stat has its own slew of problems,
> > > mostly due to hard links (e.g. git, with 106 hardlinks to the same
> > > file). Working around this is neither fun, nor practical.
> > > 
> > > As it turns out, all we needed here all along was a simple call to sync
> > > to flush writes to disk before calling du.
> > > 
> > > This reverts commit b264fb9e9ddcc31dc8782390309421965e507383.
> > > 
> > > Signed-off-by: Dave Reisner <dreisner at archlinux.org>
> > 
> > Argh...   look at the commit message in 14474a32...
> > 
> > That sync results in the Arch buildserver stalling for minutes.
> > 
> > Allan
> > 
> 
> Ok fine. Back to the trenches I go...
> 
> Some careful changes to makepkg and a simple stack trace catches du in
> the act of being broken. It receives a statbuf with the correct size of
> /usr/bin/ghb:
> 
>   newfstatat(AT_FDCWD, "usr/bin/ghb", {st_mode=S_IFREG|0755,
>   st_size=16037808, ...}, AT_SYMLINK_NOFOLLOW) = 0
> 
> And then it writes the size completely wrong:
> 
>   write(1, "3456\tusr/bin/ghb\n", 17)     = 17
> 
> I have no idea what's going on, but I'll keep digging.
> 
> d

Replying to myself, because I've been having an monologue with myself
all goddamn day over this. It seems that while the stat buffer reports
the size properly, there's a measureable delay of a half second or so
before the st_blocks member is populated.

I can stick a 'sleep 1' into makepkg before the du call, and everything
works. I hacked up du and added some lovely printf calls (since you can't
use gdb for this...)

without the delay...

sb->st_size=16037808, sb->st_blocks=0 <- ghb

^ that's when we see problems, because du is measuring size based on
(st_blkcnt*st_blocksize).

With a 1 second delay...

sb->st_size=16037808, sb->st_blocks=31328 <- ghb

Yay!

d