[pacman-dev] filesystem performance

[Nagy Gabor]

> but we haven't got reassuring answers to our "database corruption"
> fear.
> >
> > So I would like to ask you to convince us, why sqlite is safe (well,
> > personally I have very limited sql[ite] knowledge now). Please try
> > to understand that for most of the pacman devels/contributors
> > "stability" is more important than speed [obviously corrupted
> > localdb == unusable system]. That's why I can guess that this idea
> > won't be accepted until we cannot see the proof of the fact that
> > the new db back-end is as safe as the old one (or more safer ;-P).
> 
> 
> Of course. Although I hope you don't mean 'proof' as in 'mathematical
> proof'. It might take me some while to complete that...
> 
> Sqlite implements ACID except for Consistency, and that because it
> ignores foreign key constraints. But otherwise, I believe that part
> would be immediately safer than what we have now.
> 
> As for filesystem corruption (how often does this even happen with
> current sensible filesystems?), one option would be to back up the
> localdb after every commiting pacman operation. Probably with some
> autoexpiration logic. For instance, extra.db is 337KB unpacked (using
> sqlize.py -- might be missing some fields for now), extra.db.bz2 is
> 126K, so we could clone those quite a lot of times before using more
> space than what /var/lib/pacman/ always does (which may be as high as
> 50MB on a fresh install, ext3).
> 
> By the way, how safe do you see the current DB currently? I remember
> having at least a couple of disasters with the filedb when trees were
> truncated. Sure it was my fault for trying out alternative
> filesystems, but still, this is exactly the sort of corruption you're
> afraid of, is it not?
> 

First, I need some more info (deeplinks) about this ACID stuff :-P

Second, about the current db:
You are in the middle of a db write, and your little sister
accidentally presses reset button.
What will happen? At most one corrupted db "entry" [written desc, but
missing depends for example], which can be easily deleted by hand, and
fixed by -Udf package reinstall. The other parts of the db still
readable and valid.
I don't know what happens in the same situation with sqlite:
The worst thing I can imagine, that I get "database corrupted" after
reboot, which would be a nightmare. <- This is where we you must
convince us.
Or since sqlite is a "black box", I got some spam/corrupt info inside
the db, which I cannot clean-up so easily.

So if you can "guarantee", that with sqlite we get an "atomic update"
capable (<- this is missing with current method), "reset-button-safe"
db, it might worth thinking about the change.

Bye