[pacman-dev] Disk Space

newer
[pacman-dev] Dan's and Xavier's...

older
[pacman-dev] Possible bug in pacman

Matthew Wood

23 Feb 2009 23 Feb '09

4:11 a.m.

What if we checked for disk space on all volumes, prior to the processing of the users command. If the function detects that disk space is lower than a certain amount, say 100 mb, to at least warn the user? This would of course be a temporary solution, for a more permanent possibility we could implement two functions that work in conjunction with each other. The first function would check the downloads to ensure they completed successfully, if they did then it would pass the work to a second function that checks the extracted package size against all mounted volumes, or at least the size of the volumes of the major installation location (/usr, /etc, /var, etc) Does this make any sense or am I barking up the wrong tree? Matt

Show replies by date

Brendan Hide

23 Feb 23 Feb

8:47 a.m.

...

What if we checked for disk space on all volumes, prior to the processing of the users command. If the function detects that disk space is lower than a certain amount, say 100 mb, to at least warn the user? This would of course be a temporary solution, for a more permanent possibility we could implement two functions that work in conjunction with each other. The first function would check the downloads to ensure they completed successfully, if they did then it would pass the work to a second function that checks the extracted package size against all mounted volumes, or at least the size of the volumes of the major installation location (/usr, /etc, /var, etc) Does this make any sense or am I barking up the wrong tree?

Matt The extra checks you mention are partially sensible sanity checks however they shouldn't replace the currently broken part: pacman does not correctly take into account errors given by the deployment part of

Matthew Wood wrote: the installation. On that note however, what would be an appropriate amount of disk space to consider enough? OpenOffice.org is the biggest package I have installed. It is a little over 150MB in size just for the tar.gz and over 330MB for the installation. This number would be wholly inappropriate for most packages though. Also, a full system upgrade could download well over 700MB, which so far I have had occur (mostly) due to the last few KDE 4.2 updates. One negative use-case is that a random memory stick full of music that happens to be plugged in is likely to give a false positive. Most Archers would recognise the warning as such and press "Continue" appropriately. ;) The downloaded files are already being checked against md5sums so files which are not fully downloaded will fail this check. Again, this will be properly fixed as soon as pacman correctly takes into account the errors given when doing the actual deployment. Personally, I like the idea of adding sanity checks - but only if they're done properly. ;) -- __________ Brendan Hide

Aaron Griffin

6:56 p.m.

On Mon, Feb 23, 2009 at 1:47 AM, Brendan Hide wrote:

...

Again, this will be properly fixed as soon as pacman correctly takes into account the errors given when doing the actual deployment. Personally, I like the idea of adding sanity checks - but only if they're done properly. ;)

I wouldn't say that - we've talked about "proper transactions" for a long time, and are still nowhere near a proper solution. So don't expect this to be fixed soon, unless you're writing the patches :) The problem with errors given during actual file operations is... what do we do when extracting a file fails? How do we return to normalcy? We can't. The best we can do is bail out, perhaps before updating the DB. For instance, if I have foobar-1.0 installed, and upgrading to 1.1 fails due to disk space in the middle of the upgrade, what do we do? Do we leave foobar half upgraded and keep the DB entry at 1.0? This would allow us to reinstall foobar at the next upgrade when disk space is fixed, but leaves foobar in a possibly non-functioning state then. What happens if this is a critical package like, say, glibc. You can't clean up your disk space manually if glibc is borked and you can't even use "rm"

Dan McGee

24 Feb 24 Feb

3:44 a.m.

On Mon, Feb 23, 2009 at 11:56 AM, Aaron Griffin wrote:

...

On Mon, Feb 23, 2009 at 1:47 AM, Brendan Hide wrote:

...
Again, this will be properly fixed as soon as pacman correctly takes into account the errors given when doing the actual deployment. Personally, I like the idea of adding sanity checks - but only if they're done properly. ;)

I wouldn't say that - we've talked about "proper transactions" for a long time, and are still nowhere near a proper solution. So don't expect this to be fixed soon, unless you're writing the patches :)

The problem with errors given during actual file operations is... what do we do when extracting a file fails? How do we return to normalcy? We can't. The best we can do is bail out, perhaps before updating the DB.

For instance, if I have foobar-1.0 installed, and upgrading to 1.1 fails due to disk space in the middle of the upgrade, what do we do? Do we leave foobar half upgraded and keep the DB entry at 1.0? This would allow us to reinstall foobar at the next upgrade when disk space is fixed, but leaves foobar in a possibly non-functioning state then. What happens if this is a critical package like, say, glibc. You can't clean up your disk space manually if glibc is borked and you can't even use "rm"

This is not an easy problem by any means as Aaron stated. A few other things: * Say we warn a user if they have less than 100 MB left. You already pointed out that doesn't work for some large packages. What if I have 180 GB left, but my /boot partition is super-low on space? It isn't like Windows where I pretty much need to check C:\ and that is it. * The current order of operations is download, remove 1 package, install 1 package, remove 1, install 1, ... Without making this a more atomic sequence, we can really run into problems. Whatever we do, we can't add in what we had before- that method was horrible. I do have two patches in my working branch dealing with space issues, but I'm not super happy with them: http://code.toofishes.net/cgit/dan/pacman.git/log/?h=working (see the top two patches). -Dan

Sebastian Nowicki

4:57 a.m.

On 24/02/2009, at 2:56 AM, Aaron Griffin wrote:

...

On Mon, Feb 23, 2009 at 1:47 AM, Brendan Hide
...
wrote: Again, this will be properly fixed as soon as pacman correctly takes into account the errors given when doing the actual deployment. Personally, I like the idea of adding sanity checks - but only if they're done properly. ;)

I wouldn't say that - we've talked about "proper transactions" for a long time, and are still nowhere near a proper solution. So don't expect this to be fixed soon, unless you're writing the patches :)

Has there been any (real) discussion about transactions? Perhaps a wiki about design ideas would be good.

Aaron Griffin

7:13 p.m.

On Mon, Feb 23, 2009 at 9:57 PM, Sebastian Nowicki wrote:

...

On 24/02/2009, at 2:56 AM, Aaron Griffin wrote:

...
On Mon, Feb 23, 2009 at 1:47 AM, Brendan Hide wrote:

...
Again, this will be properly fixed as soon as pacman correctly takes into account the errors given when doing the actual deployment. Personally, I like the idea of adding sanity checks - but only if they're done properly. ;)

I wouldn't say that - we've talked about "proper transactions" for a long time, and are still nowhere near a proper solution. So don't expect this to be fixed soon, unless you're writing the patches :)

Has there been any (real) discussion about transactions? Perhaps a wiki about design ideas would be good.

I know lots of people like wiki pages for this sort of thing, but I never have - to use a relevant analogy (due to list discussion): a wiki page is like looking at a working dir, whereas for design decisions, it's important to see the changelog :) I like seeing WHY certain decisions were made, and a wiki page generally won't tell me that. It will tell me that Bob deleted 3 lines and Jim added a new bullet point. But, I digress. I think the only conversation on this topic was between Dan and myself over email, and Dan, Xavier, and myself on IRC. But this was long ago and it may have been brought up since then. The process is simple: if any stage in a given package install or upgrade fails, we should be able to "roll back" to the same state before that install/upgrade began. Ideas are welcome, but I haven't seen a really good idea regarding this that isn't slow as hell.

5547

Age (days ago)

5548

Last active (days ago)

List overview

Download

5 comments

5 participants

participants (5)

Aaron Griffin
Brendan Hide
Dan McGee
Matthew Wood
Sebastian Nowicki