[pacman-dev] [PATCH 1/2] Revert "makepkg: calculate exact total file size"
btrfs's cow snapshots seem to do very strange things, causing du to not
count data still in filesystem buffers. Repackaging the same build of
handbrake within a chroot on brynhild, I witnessed (via bash xtrace) du
reporting a different $pkgdir size every time the write_pkginfo()
function ran.
Unfortunately, replacing du with stat has its own slew of problems,
mostly due to hard links (e.g. git, with 106 hardlinks to the same
file). Working around this is neither fun, nor practical.
As it turns out, all we needed here all along was a simple call to sync
to flush writes to disk before calling du.
This reverts commit b264fb9e9ddcc31dc8782390309421965e507383.
Signed-off-by: Dave Reisner
Use a simple default value parameter expansion instead of using an
intermediate local variable to write the packager name in write_pkginfo.
Signed-off-by: Dave Reisner
On 15/03/12 06:41, Dave Reisner wrote:
btrfs's cow snapshots seem to do very strange things, causing du to not count data still in filesystem buffers. Repackaging the same build of handbrake within a chroot on brynhild, I witnessed (via bash xtrace) du reporting a different $pkgdir size every time the write_pkginfo() function ran.
Unfortunately, replacing du with stat has its own slew of problems, mostly due to hard links (e.g. git, with 106 hardlinks to the same file). Working around this is neither fun, nor practical.
As it turns out, all we needed here all along was a simple call to sync to flush writes to disk before calling du.
This reverts commit b264fb9e9ddcc31dc8782390309421965e507383.
Signed-off-by: Dave Reisner
Argh... look at the commit message in 14474a32... That sync results in the Arch buildserver stalling for minutes. Allan
On Thu, Mar 15, 2012 at 07:11:07AM +1000, Allan McRae wrote:
On 15/03/12 06:41, Dave Reisner wrote:
btrfs's cow snapshots seem to do very strange things, causing du to not count data still in filesystem buffers. Repackaging the same build of handbrake within a chroot on brynhild, I witnessed (via bash xtrace) du reporting a different $pkgdir size every time the write_pkginfo() function ran.
Unfortunately, replacing du with stat has its own slew of problems, mostly due to hard links (e.g. git, with 106 hardlinks to the same file). Working around this is neither fun, nor practical.
As it turns out, all we needed here all along was a simple call to sync to flush writes to disk before calling du.
This reverts commit b264fb9e9ddcc31dc8782390309421965e507383.
Signed-off-by: Dave Reisner
Argh... look at the commit message in 14474a32...
That sync results in the Arch buildserver stalling for minutes.
Allan
Ok fine. Back to the trenches I go... Some careful changes to makepkg and a simple stack trace catches du in the act of being broken. It receives a statbuf with the correct size of /usr/bin/ghb: newfstatat(AT_FDCWD, "usr/bin/ghb", {st_mode=S_IFREG|0755, st_size=16037808, ...}, AT_SYMLINK_NOFOLLOW) = 0 And then it writes the size completely wrong: write(1, "3456\tusr/bin/ghb\n", 17) = 17 I have no idea what's going on, but I'll keep digging. d
On Wed, Mar 14, 2012 at 07:48:06PM -0400, Dave Reisner wrote:
On Thu, Mar 15, 2012 at 07:11:07AM +1000, Allan McRae wrote:
On 15/03/12 06:41, Dave Reisner wrote:
btrfs's cow snapshots seem to do very strange things, causing du to not count data still in filesystem buffers. Repackaging the same build of handbrake within a chroot on brynhild, I witnessed (via bash xtrace) du reporting a different $pkgdir size every time the write_pkginfo() function ran.
Unfortunately, replacing du with stat has its own slew of problems, mostly due to hard links (e.g. git, with 106 hardlinks to the same file). Working around this is neither fun, nor practical.
As it turns out, all we needed here all along was a simple call to sync to flush writes to disk before calling du.
This reverts commit b264fb9e9ddcc31dc8782390309421965e507383.
Signed-off-by: Dave Reisner
Argh... look at the commit message in 14474a32...
That sync results in the Arch buildserver stalling for minutes.
Allan
Ok fine. Back to the trenches I go...
Some careful changes to makepkg and a simple stack trace catches du in the act of being broken. It receives a statbuf with the correct size of /usr/bin/ghb:
newfstatat(AT_FDCWD, "usr/bin/ghb", {st_mode=S_IFREG|0755, st_size=16037808, ...}, AT_SYMLINK_NOFOLLOW) = 0
And then it writes the size completely wrong:
write(1, "3456\tusr/bin/ghb\n", 17) = 17
I have no idea what's going on, but I'll keep digging.
d
Replying to myself, because I've been having an monologue with myself all goddamn day over this. It seems that while the stat buffer reports the size properly, there's a measureable delay of a half second or so before the st_blocks member is populated. I can stick a 'sleep 1' into makepkg before the du call, and everything works. I hacked up du and added some lovely printf calls (since you can't use gdb for this...) without the delay... sb->st_size=16037808, sb->st_blocks=0 <- ghb ^ that's when we see problems, because du is measuring size based on (st_blkcnt*st_blocksize). With a 1 second delay... sb->st_size=16037808, sb->st_blocks=31328 <- ghb Yay! d
participants (2)
-
Allan McRae
-
Dave Reisner