[pacman-dev] Changing how repo dbs are updated

[Allan McRae]

On 03/12/10 02:14, Dan McGee wrote:
> On Tue, Nov 30, 2010 at 11:12 PM, Allan McRae<allan at archlinux.org>  wrote:
>> Hi,
>>
>> This message is more of a sounding board for me to get the issues
>> surrounding this sorted and point out what I am planning to do.  But any
>> comments on this would also be appreciated.  Especially #3 below.
>>
>>
>> Issue I see currently and in the future with signed databases:
>>
>> 1) Currently the repo dbs are updated just like downloading a package file.
>>   If the update is started and canceled part way though, you get a
>> repo.db.part file which pacman attempts to continue downloading. However,
>> unlike package files, this file is not static content and so we should never
>> continue the download.   See https://bugs.archlinux.org/task/15657 . This
>> can be handled by just deleting the repo.db.part file if present, but it
>> might be better just never create .part files in the first place for repo
>> dbs by downloading to a temporary location and moving/deleting based on
>> successful completion.  That would mean having a different download function
>> for repo dbs and packages.  See #2 for additional reasons to split this...
>
> False? At least if the remote server is not broken, commit d2dbb04a9a
> should have definitely fixed this. With that said, the only (external
> of dload.c) user of download_single_file() is the db download code at
> the moment.

Correct.   Looks like that bug can be closed...


>> 2) Database signing.   Currently the code downloads the database, deletes
>> the old now invalid signature, then downloads the new signature.  If the
>> signature is valid, then all is fine.   However, if it fails to download or
>> is invalid, pacman issues an error about failing to update the database.
>>   The database on your system is now not correctly signed (which is bad given
>> its signature is only checked on update...).
>>
>> I think that the old database and signature should only be overwritten if
>> the new database download is successful _and_ its signature is valid.  This
>> requires downloading the database and its signature to a temporary location
>> and then moving the files only once they are confirmed valid.  That would
>> require a different download interface for package and database downloads,
>> but that is a good thing as we can get rid of the force crap from the one
>> used for packages.
>
> But you can't, unless you are required to provide two callbacks for
> downloading files. :/ And don't forget that it would be good to
> support standalone package sigs; e.g. if I do
>
> pacman -U http://example.com/mypkgs/foobar-1.0-arch.pkg.tar.xz
>
> I would expect it to "do the right thing" and also look for a .sig
> there as well.

I had not thought of the remote pacman -U case.   I need to think how 
best to download signatures some more and come up with a new plan.


>> 3) pacman -Syy behavior.  Instead of adding a "force" flag to overwrite the
>> old database, would it be better to just delete the old database first?
>>   Currently, if you use pacman -Syy and a database download fails, you are
>> left with the old sync database you told pacman to get rid of. Is leaving
>> pacman with no database for that repo a better solution?
>
> I don't think so- this seems similar to #1 and leaving the user in a
> worse situation than they originally had. We really interpret -yy as
> "download the remote DB regardless of whether you think it has been
> updated", but if we wanted to change that, I guess I'm not completely
> opposed if we document it as so and change peoples expectations. A
> failed -Syy (by a superuser) keeps any non-superuser from doing -Si,
> etc. so that is no good.

OK - I'm now convinced that this is a bad idea.  :P