[pacman-dev] backup handling
stonecrest noticed during last pacman upgrade that the /etc/pacman.d/current file was removed, although that file was in the backup array of previous packages, and might have been modified by the user. pacman 3.0.5-3 contained only /etc/pacman.d/current file and had it in its backup array. pacman 3.0.5-4 contained only /etc/pacman.d/core file and had it in its backup array. And on 3.0.5-3 -> 3.0.5-4 upgrade, the current file is removed, without any backup. While in this case, it was totally harmless, in the general case, it might not be the desired behavior. As we know, in pacman, an upgrade is a remove + add progress. The remove is a special one though, since we dont want to lose the config files. So pacman adds all files in the backup array to NoUpgrade, so that these files are not removed. But it picks only the files of the new backup array, as seen there : 298 /* Add files in the NEW package's backup array to the noupgrade array 299 * so this removal operation doesn't kill them */ 300 /* TODO if we add here, all backup=() entries for all targets, new and 301 * old, we cover all bases, including backup=() locations changing hands. 302 * But is this viable? */ So during 3.0.5-3 -> 3.0.5-4 , the current file was not added to noupgrade and was then lost. Implementing the above TODO would probably fix the issue, and it should be pretty easy to do. But I'm a bit concerned by the last comment : "But is this viable?" What are really the downsides of doing this? While writing this, I may seen one inconvenient, it's that it might not be obvious to the user that this current file is no longer used and owned by pacman. Maybe it should be renamed to current.pacsave then ?
2007/9/19, Xavier <shiningxc@gmail.com>:
stonecrest noticed during last pacman upgrade that the /etc/pacman.d/current file was removed, although that file was in the backup array of previous packages, and might have been modified by the user.
pacman 3.0.5-3 contained only /etc/pacman.d/current file and had it in its backup array. pacman 3.0.5-4 contained only /etc/pacman.d/core file and had it in its backup array. And on 3.0.5-3 -> 3.0.5-4 upgrade, the current file is removed, without any backup.
While in this case, it was totally harmless, in the general case, it might not be the desired behavior.
As we know, in pacman, an upgrade is a remove + add progress. The remove is a special one though, since we dont want to lose the config files. So pacman adds all files in the backup array to NoUpgrade, so that these files are not removed. But it picks only the files of the new backup array, as seen there :
298 /* Add files in the NEW package's backup array to the noupgrade array 299 * so this removal operation doesn't kill them */ 300 /* TODO if we add here, all backup=() entries for all targets, new and 301 * old, we cover all bases, including backup=() locations changing hands. 302 * But is this viable? */
So during 3.0.5-3 -> 3.0.5-4 , the current file was not added to noupgrade and was then lost.
Implementing the above TODO would probably fix the issue, and it should be pretty easy to do. But I'm a bit concerned by the last comment : "But is this viable?" What are really the downsides of doing this?
While writing this, I may seen one inconvenient, it's that it might not be obvious to the user that this current file is no longer used and owned by pacman. Maybe it should be renamed to current.pacsave then ?
Thanks for bringing this up. I think it would be nice to have such files saved as .pacsave. But how many of those files (that were in NoUpgrade) still exist without backup information now? I think they are <0.1%. -- Roman Kyrylych (Роман Кирилич)
On Wed, Sep 19, 2007 at 11:21:20AM +0300, Roman Kyrylych wrote:
Thanks for bringing this up. I think it would be nice to have such files saved as .pacsave. But how many of those files (that were in NoUpgrade) still exist without backup information now? I think they are <0.1%.
Hmm, I was afraid this would be confusing/misleading :) All my references to NoUpgrade don't refer to the static NoUpgrade definition in /etc/pacman.conf , but the representation of it in pacman at runtime. So pacman temporarily adds files a noupgrade list, so that remove_commit() doesn't kill them. And when it's done, it removes them from the noupgrade list. It never edits the NoUpgrade files in pacman.conf. The case I described is when a config file that was in the backup array leaves the backup array, like what happened with /etc/pacman.d/current.
2007/9/19, Xavier <shiningxc@gmail.com>:
On Wed, Sep 19, 2007 at 11:21:20AM +0300, Roman Kyrylych wrote:
Thanks for bringing this up. I think it would be nice to have such files saved as .pacsave. But how many of those files (that were in NoUpgrade) still exist without backup information now? I think they are <0.1%.
Hmm, I was afraid this would be confusing/misleading :) All my references to NoUpgrade don't refer to the static NoUpgrade definition in /etc/pacman.conf , but the representation of it in pacman at runtime.
So pacman temporarily adds files a noupgrade list, so that remove_commit() doesn't kill them. And when it's done, it removes them from the noupgrade list. It never edits the NoUpgrade files in pacman.conf.
The case I described is when a config file that was in the backup array leaves the backup array, like what happened with /etc/pacman.d/current.
Aha, it seems I confused it with a hack for files that were in NoUpgrade since pacman 2. And I thought that files that left backups array were already saved as .pacsave. If this is not the case then they should definetely be saved. -- Roman Kyrylych (Роман Кирилич)
On 9/18/07, Xavier <shiningxc@gmail.com> wrote:
stonecrest noticed during last pacman upgrade that the /etc/pacman.d/current file was removed, although that file was in the backup array of previous packages, and might have been modified by the user.
pacman 3.0.5-3 contained only /etc/pacman.d/current file and had it in its backup array. pacman 3.0.5-4 contained only /etc/pacman.d/core file and had it in its backup array. And on 3.0.5-3 -> 3.0.5-4 upgrade, the current file is removed, without any backup.
While in this case, it was totally harmless, in the general case, it might not be the desired behavior.
As we know, in pacman, an upgrade is a remove + add progress. The remove is a special one though, since we dont want to lose the config files. So pacman adds all files in the backup array to NoUpgrade, so that these files are not removed. But it picks only the files of the new backup array, as seen there :
298 /* Add files in the NEW package's backup array to the noupgrade array 299 * so this removal operation doesn't kill them */ 300 /* TODO if we add here, all backup=() entries for all targets, new and 301 * old, we cover all bases, including backup=() locations changing hands. 302 * But is this viable? */
So during 3.0.5-3 -> 3.0.5-4 , the current file was not added to noupgrade and was then lost.
Implementing the above TODO would probably fix the issue, and it should be pretty easy to do. But I'm a bit concerned by the last comment : "But is this viable?" What are really the downsides of doing this?
While writing this, I may seen one inconvenient, it's that it might not be obvious to the user that this current file is no longer used and owned by pacman. Maybe it should be renamed to current.pacsave then ?
What would seem smart to me: 1. If file has been removed from backup array and no longer exists in the package, save it as .pacold. 2. If file has been removed from backup array and is still in the package, install new package as pacnew. Does this seem valid? -Dan
On Wed, Sep 19, 2007 at 08:33:01AM -0500, Dan McGee wrote:
What would seem smart to me: 2. If file has been removed from backup array and is still in the package, install new package as pacnew.
What happens on the next upgrade though, won't the custom file be overwritten by the package one? Because, on the next upgrade, the file will be neither in old nor in new backup array.
On 9/19/07, Dan McGee <dpmcgee@gmail.com> wrote:
What would seem smart to me: 1. If file has been removed from backup array and no longer exists in the package, save it as .pacold.
And this is already done anyway - previous backup array vs new backup array is checked for files in the NEW backup array instead of the OLD. We could easily do the converse check for .pacold
On Wed, Sep 19, 2007 at 10:50:35AM -0500, Aaron Griffin wrote:
On 9/19/07, Dan McGee <dpmcgee@gmail.com> wrote:
What would seem smart to me: 1. If file has been removed from backup array and no longer exists in the package, save it as .pacold.
And this is already done anyway - previous backup array vs new backup array is checked for files in the NEW backup array instead of the OLD. We could easily do the converse check for .pacold
Hmm, what? This is exactly the case of the current mirror list during 3.0.5-3 -> 3.0.5-4, and current is lost. Also, do you mean pacold or pacsave? I grepped both 3.0 and 3.1 for pacold, and couldn't find any references of it. Attaching a pactest that should reproduce the problem.
On 9/19/07, Xavier <shiningxc@gmail.com> wrote:
On Wed, Sep 19, 2007 at 10:50:35AM -0500, Aaron Griffin wrote:
On 9/19/07, Dan McGee <dpmcgee@gmail.com> wrote:
What would seem smart to me: 1. If file has been removed from backup array and no longer exists in the package, save it as .pacold.
And this is already done anyway - previous backup array vs new backup array is checked for files in the NEW backup array instead of the OLD. We could easily do the converse check for .pacold
Hmm, what? This is exactly the case of the current mirror list during 3.0.5-3 -> 3.0.5-4, and current is lost. Also, do you mean pacold or pacsave? I grepped both 3.0 and 3.1 for pacold, and couldn't find any references of it.
What I'm saying is that we check the old and new backup arrays for pacsave files... I termed this "proactive backups" back in the day. That is, the first time a file is added to the backup array, it is not overwritten like it was in the past. So this check is _already_ there, it is just a check to see if a file is in the *new* backup array and not the *old* - we just need to reverse this to check for files in the *old* array no longer in the *new* array
On Wed, Sep 19, 2007 at 11:18:55AM -0500, Aaron Griffin wrote:
What I'm saying is that we check the old and new backup arrays for pacsave files... I termed this "proactive backups" back in the day. That is, the first time a file is added to the backup array, it is not overwritten like it was in the past.
So this check is _already_ there, it is just a check to see if a file is in the *new* backup array and not the *old* - we just need to reverse this to check for files in the *old* array no longer in the *new* array
I am confused.. I thought the proactive backup was mostly done in the part I already copy/pasted in my original post : 298 /* Add files in the NEW package's backup array to the noupgrade array 299 * so this removal operation doesn't kill them */ 300 /* TODO if we add here, all backup=() entries for all targets, new and 301 * old, we cover all bases, including backup=() locations changing hands. 302 * But is this viable? */ 303 alpm_list_t *old_noupgrade = alpm_list_strdup(handle->noupgrade); 304 for(b = alpm_pkg_get_backup(newpkg); b; b = b->next) { 305 const char *backup = b->data; 306 _alpm_log(PM_LOG_DEBUG, "adding %s to the NoUpgrade array temporarily\n", 307 backup); 308 handle->noupgrade = alpm_list_add(handle->noupgrade, strdup(backup)); 309 } IE, it just uses the new backup array, disregarding the old (it doesn't matter if the file is in the old backup array or not). To be honest, I am a bit confused by this whole backup handling / proactive backup stuff (but same for 90% of the pacman code), so I'll probably just give up :)
On 9/19/07, Xavier <shiningxc@gmail.com> wrote:
To be honest, I am a bit confused by this whole backup handling / proactive backup stuff (but same for 90% of the pacman code), so I'll probably just give up :)
Looks like Dan got this one (yay!) Removing some gmail labels, and closing this out if there's nothing else actionable here.
participants (4)
-
Aaron Griffin
-
Dan McGee
-
Roman Kyrylych
-
Xavier