[pacman-dev] [PATCH] pacman: don't nuke local repos with -Sc
When using a local repo as a cachedir pacman -Scc or pacman -Sc without KeepCurrent will delete the repo database and packages resulting in a lost database and non functional pacman. So special case file:/// repos in cache to not get nuked and foce KeepCurrent on. --- src/pacman/sync.c | 64 ++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 55 insertions(+), 9 deletions(-) diff --git a/src/pacman/sync.c b/src/pacman/sync.c index b6da1a36..129d3f5c 100644 --- a/src/pacman/sync.c +++ b/src/pacman/sync.c @@ -151,34 +151,71 @@ static int sync_cleandb_all(void) return ret; } -static int sync_cleancache(int level) +static int is_local_repo(char *dir) +{ + alpm_list_t *j, *k; + char *sanatized_dir = strdup(dir); + int len = strlen(sanatized_dir); + int ret = 0; + + if(sanatized_dir[len - 1] == '/') { + sanatized_dir[len - 1] = '\0'; + } + + for(j = alpm_get_syncdbs(config->handle); j; j = alpm_list_next(j)) { + alpm_db_t *db = j->data; + + for(k = alpm_db_get_servers(db); k; k = alpm_list_next(k)) { + char *server = k->data; + + if(strncmp(server, "file://", 7) == 0) { + server += 7; + + if(strcmp(server, sanatized_dir) == 0) { + ret = 1; + goto cleanup; + } + } + } + + } + +cleanup: + free(sanatized_dir); + return ret; +} + +static int sync_cleancache(int config_level) { alpm_list_t *i; alpm_list_t *sync_dbs = alpm_get_syncdbs(config->handle); alpm_db_t *db_local = alpm_get_localdb(config->handle); alpm_list_t *cachedirs = alpm_option_get_cachedirs(config->handle); + int config_cleanmethod = config->cleanmethod; int ret = 0; - if(!config->cleanmethod) { + if(config_cleanmethod) { /* default to KeepInstalled if user did not specify */ - config->cleanmethod = PM_CLEAN_KEEPINST; + config_cleanmethod = PM_CLEAN_KEEPINST; } - if(level == 1) { + if(config_level == 1) { printf(_("Packages to keep:\n")); - if(config->cleanmethod & PM_CLEAN_KEEPINST) { + if(config_cleanmethod & PM_CLEAN_KEEPINST) { printf(_(" All locally installed packages\n")); } - if(config->cleanmethod & PM_CLEAN_KEEPCUR) { + if(config_cleanmethod & PM_CLEAN_KEEPCUR) { printf(_(" All current sync database packages\n")); } } printf("\n"); for(i = cachedirs; i; i = alpm_list_next(i)) { - const char *cachedir = i->data; + char *cachedir = i->data; DIR *dir; struct dirent *ent; + int cleanmethod = config_cleanmethod; + int level = config_level; printf(_("Cache directory: %s\n"), (const char *)i->data); @@ -196,6 +233,15 @@ static int sync_cleancache(int level) printf(_("removing all files from cache...\n")); } + if(is_local_repo(cachedir)) { + pm_printf(ALPM_LOG_DEBUG, + _("cachedir '%s' is a local repo, keeping packages\n"), cachedir); + + level = 1; + cleanmethod |= PM_CLEAN_KEEPCUR; + cleanmethod &= ~PM_CLEAN_KEEPINST; + } + dir = opendir(cachedir); if(dir == NULL) { pm_printf(ALPM_LOG_ERROR, @@ -258,7 +304,7 @@ static int sync_cleancache(int level) local_name = alpm_pkg_get_name(localpkg); local_version = alpm_pkg_get_version(localpkg); - if(config->cleanmethod & PM_CLEAN_KEEPINST) { + if(cleanmethod & PM_CLEAN_KEEPINST) { /* check if this package is in the local DB */ pkg = alpm_db_get_pkg(db_local, local_name); if(pkg != NULL && alpm_pkg_vercmp(local_version, @@ -269,7 +315,7 @@ static int sync_cleancache(int level) delete = 0; } } - if(config->cleanmethod & PM_CLEAN_KEEPCUR) { + if(cleanmethod & PM_CLEAN_KEEPCUR) { alpm_list_t *j; /* check if this package is in a sync DB */ for(j = sync_dbs; j && delete; j = alpm_list_next(j)) { -- 2.32.0
On 15/6/21 5:11 am, morganamilo wrote:
When using a local repo as a cachedir pacman -Scc or pacman -Sc without KeepCurrent will delete the repo database and packages resulting in a lost database and non functional pacman.
So special case file:/// repos in cache to not get nuked and foce KeepCurrent on. ---
This behaviour change is undocumented. I don't think we should do this for -Scc. That is explicitly asking all package files in cache directories to be removed, so that is what you get. I also am leaning towards this behaviour for -Sc being unexpected (even if documented). Pacman's job is really not to manage complex cache set-ups and I'm not convinced this should be supported. Allan
On 24/06/2021 06:38, Allan McRae wrote:
On 15/6/21 5:11 am, morganamilo wrote:
When using a local repo as a cachedir pacman -Scc or pacman -Sc without KeepCurrent will delete the repo database and packages resulting in a lost database and non functional pacman.
So special case file:/// repos in cache to not get nuked and foce KeepCurrent on. ---
This behaviour change is undocumented.
I don't think we should do this for -Scc. That is explicitly asking all package files in cache directories to be removed, so that is what you get.
I also am leaning towards this behaviour for -Sc being unexpected (even if documented). Pacman's job is really not to manage complex cache set-ups and I'm not convinced this should be supported.
Allan
I want this to go ahead, especially with -Scc. Otherwise -Scc can end up losing you your .db file, breaking pacman, the local repo's .db and all the build packages. This use case is not complex in my opinion. It's just adding one line to your pacman.conf to stop every single package being duplicated. If you're still against this do you have any alternative solutions in mind to handle this?
On 24/6/21 6:19 pm, Morgan Adamiec wrote:
On 24/06/2021 06:38, Allan McRae wrote:
On 15/6/21 5:11 am, morganamilo wrote:
When using a local repo as a cachedir pacman -Scc or pacman -Sc without KeepCurrent will delete the repo database and packages resulting in a lost database and non functional pacman.
So special case file:/// repos in cache to not get nuked and foce KeepCurrent on. ---
This behaviour change is undocumented.
I don't think we should do this for -Scc. That is explicitly asking all package files in cache directories to be removed, so that is what you get.
I also am leaning towards this behaviour for -Sc being unexpected (even if documented). Pacman's job is really not to manage complex cache set-ups and I'm not convinced this should be supported.
Allan
I want this to go ahead, especially with -Scc. Otherwise -Scc can end up losing you your .db file, breaking pacman, the local repo's .db and all the build packages.
This use case is not complex in my opinion. It's just adding one line to your pacman.conf to stop every single package being duplicated.
If you're still against this do you have any alternative solutions in mind to handle this?
Not using -Scc. Why would you use that and expect it not to delete packages? A
On 24/06/2021 09:25, Allan McRae wrote:
On 24/6/21 6:19 pm, Morgan Adamiec wrote:
On 24/06/2021 06:38, Allan McRae wrote:
On 15/6/21 5:11 am, morganamilo wrote:
When using a local repo as a cachedir pacman -Scc or pacman -Sc without KeepCurrent will delete the repo database and packages resulting in a lost database and non functional pacman.
So special case file:/// repos in cache to not get nuked and foce KeepCurrent on. ---
This behaviour change is undocumented.
I don't think we should do this for -Scc. That is explicitly asking all package files in cache directories to be removed, so that is what you get.
I also am leaning towards this behaviour for -Sc being unexpected (even if documented). Pacman's job is really not to manage complex cache set-ups and I'm not convinced this should be supported.
Allan
I want this to go ahead, especially with -Scc. Otherwise -Scc can end up losing you your .db file, breaking pacman, the local repo's .db and all the build packages.
This use case is not complex in my opinion. It's just adding one line to your pacman.conf to stop every single package being duplicated.
If you're still against this do you have any alternative solutions in mind to handle this?
Not using -Scc. Why would you use that and expect it not to delete packages?
A
Because I want everything deleted in the main /var/cache/pacman/pkg dir.
On 24/6/21 9:33 pm, Morgan Adamiec wrote:
On 24/06/2021 09:25, Allan McRae wrote:
On 24/6/21 6:19 pm, Morgan Adamiec wrote:
On 24/06/2021 06:38, Allan McRae wrote:
On 15/6/21 5:11 am, morganamilo wrote:
When using a local repo as a cachedir pacman -Scc or pacman -Sc without KeepCurrent will delete the repo database and packages resulting in a lost database and non functional pacman.
So special case file:/// repos in cache to not get nuked and foce KeepCurrent on. ---
This behaviour change is undocumented.
I don't think we should do this for -Scc. That is explicitly asking all package files in cache directories to be removed, so that is what you get.
I also am leaning towards this behaviour for -Sc being unexpected (even if documented). Pacman's job is really not to manage complex cache set-ups and I'm not convinced this should be supported.
Allan
I want this to go ahead, especially with -Scc. Otherwise -Scc can end up losing you your .db file, breaking pacman, the local repo's .db and all the build packages.
This use case is not complex in my opinion. It's just adding one line to your pacman.conf to stop every single package being duplicated.
If you're still against this do you have any alternative solutions in mind to handle this?
Not using -Scc. Why would you use that and expect it not to delete packages?
A
Because I want everything deleted in the main /var/cache/pacman/pkg dir.
Pacman asks what to do for each cache directory and you could say yes to removing files from /var/cache/pacman/pkg/ and no for other cache directories. So that already achieves what you are doing without adding a special case to the codebase. Pacman's job is not to be a specialised cache cleaning utility. Hence why the suggestion of adding the ability to select how many versions of each package is kept was implemented in paccache. Allan
On 24/06/2021 12:51, Allan McRae wrote:
On 24/6/21 9:33 pm, Morgan Adamiec wrote:
On 24/06/2021 09:25, Allan McRae wrote:
On 24/6/21 6:19 pm, Morgan Adamiec wrote:
On 24/06/2021 06:38, Allan McRae wrote:
On 15/6/21 5:11 am, morganamilo wrote:
When using a local repo as a cachedir pacman -Scc or pacman -Sc without KeepCurrent will delete the repo database and packages resulting in a lost database and non functional pacman.
So special case file:/// repos in cache to not get nuked and foce KeepCurrent on. ---
This behaviour change is undocumented.
I don't think we should do this for -Scc. That is explicitly asking all package files in cache directories to be removed, so that is what you get.
I also am leaning towards this behaviour for -Sc being unexpected (even if documented). Pacman's job is really not to manage complex cache set-ups and I'm not convinced this should be supported.
Allan
I want this to go ahead, especially with -Scc. Otherwise -Scc can end up losing you your .db file, breaking pacman, the local repo's .db and all the build packages.
This use case is not complex in my opinion. It's just adding one line to your pacman.conf to stop every single package being duplicated.
If you're still against this do you have any alternative solutions in mind to handle this?
Not using -Scc. Why would you use that and expect it not to delete packages?
A
Because I want everything deleted in the main /var/cache/pacman/pkg dir.
Pacman asks what to do for each cache directory and you could say yes to removing files from /var/cache/pacman/pkg/ and no for other cache directories. So that already achieves what you are doing without adding a special case to the codebase.
Pacman's job is not to be a specialised cache cleaning utility. Hence why the suggestion of adding the ability to select how many versions of each package is kept was implemented in paccache.
Allan
Yes, that does work but then you have to be careful about what to wipe and can't automate it. I would agree if it wasn't for the case accidentally deleting that dir loses you your repo. I believe as a workaround aurutils bind mounts the repo into cache as readonly. Would you be more accepting of it straight up just skipping any cachedirs that happen to be a local repo and not inspect it at all?
On 06/24/21 at 12:58pm, Morgan Adamiec wrote:
On 24/06/2021 12:51, Allan McRae wrote:
On 24/6/21 9:33 pm, Morgan Adamiec wrote:
On 24/06/2021 09:25, Allan McRae wrote:
On 24/6/21 6:19 pm, Morgan Adamiec wrote:
On 24/06/2021 06:38, Allan McRae wrote:
On 15/6/21 5:11 am, morganamilo wrote: > When using a local repo as a cachedir pacman -Scc or pacman -Sc > without KeepCurrent will delete the repo database and packages > resulting in a lost database and non functional pacman. > > So special case file:/// repos in cache to not get nuked and foce > KeepCurrent on. > ---
This behaviour change is undocumented.
I don't think we should do this for -Scc. That is explicitly asking all package files in cache directories to be removed, so that is what you get.
I also am leaning towards this behaviour for -Sc being unexpected (even if documented). Pacman's job is really not to manage complex cache set-ups and I'm not convinced this should be supported.
Allan
I want this to go ahead, especially with -Scc. Otherwise -Scc can end up losing you your .db file, breaking pacman, the local repo's .db and all the build packages.
This use case is not complex in my opinion. It's just adding one line to your pacman.conf to stop every single package being duplicated.
If you're still against this do you have any alternative solutions in mind to handle this?
Not using -Scc. Why would you use that and expect it not to delete packages?
A
Because I want everything deleted in the main /var/cache/pacman/pkg dir.
Pacman asks what to do for each cache directory and you could say yes to removing files from /var/cache/pacman/pkg/ and no for other cache directories. So that already achieves what you are doing without adding a special case to the codebase.
Pacman's job is not to be a specialised cache cleaning utility. Hence why the suggestion of adding the ability to select how many versions of each package is kept was implemented in paccache.
Allan
Yes, that does work but then you have to be careful about what to wipe and can't automate it. I would agree if it wasn't for the case accidentally deleting that dir loses you your repo.
I believe as a workaround aurutils bind mounts the repo into cache as readonly.
Would you be more accepting of it straight up just skipping any cachedirs that happen to be a local repo and not inspect it at all?
-1. -Sc/-Scc are destructive operations. There are a variety of weird cache setups where they could be more destructive than intended. People with weird cache setups really probably just shouldn't use them.
On 24/06/2021 13:31, Andrew Gregory wrote:
On 06/24/21 at 12:58pm, Morgan Adamiec wrote:
On 24/06/2021 12:51, Allan McRae wrote:
On 24/6/21 9:33 pm, Morgan Adamiec wrote:
On 24/06/2021 09:25, Allan McRae wrote:
On 24/6/21 6:19 pm, Morgan Adamiec wrote:
On 24/06/2021 06:38, Allan McRae wrote: > On 15/6/21 5:11 am, morganamilo wrote: >> When using a local repo as a cachedir pacman -Scc or pacman -Sc >> without KeepCurrent will delete the repo database and packages >> resulting in a lost database and non functional pacman. >> >> So special case file:/// repos in cache to not get nuked and foce >> KeepCurrent on. >> --- > > This behaviour change is undocumented. > > I don't think we should do this for -Scc. That is explicitly asking all > package files in cache directories to be removed, so that is what you get. > > I also am leaning towards this behaviour for -Sc being unexpected (even > if documented). Pacman's job is really not to manage complex cache > set-ups and I'm not convinced this should be supported. > > Allan >
I want this to go ahead, especially with -Scc. Otherwise -Scc can end up losing you your .db file, breaking pacman, the local repo's .db and all the build packages.
This use case is not complex in my opinion. It's just adding one line to your pacman.conf to stop every single package being duplicated.
If you're still against this do you have any alternative solutions in mind to handle this?
Not using -Scc. Why would you use that and expect it not to delete packages?
A
Because I want everything deleted in the main /var/cache/pacman/pkg dir.
Pacman asks what to do for each cache directory and you could say yes to removing files from /var/cache/pacman/pkg/ and no for other cache directories. So that already achieves what you are doing without adding a special case to the codebase.
Pacman's job is not to be a specialised cache cleaning utility. Hence why the suggestion of adding the ability to select how many versions of each package is kept was implemented in paccache.
Allan
Yes, that does work but then you have to be careful about what to wipe and can't automate it. I would agree if it wasn't for the case accidentally deleting that dir loses you your repo.
I believe as a workaround aurutils bind mounts the repo into cache as readonly.
Would you be more accepting of it straight up just skipping any cachedirs that happen to be a local repo and not inspect it at all?
-1. -Sc/-Scc are destructive operations. There are a variety of weird cache setups where they could be more destructive than intended. People with weird cache setups really probably just shouldn't use them.
Do you have any alternate solution in mind?
On 24/6/21 10:48 pm, Morgan Adamiec wrote:
On 24/06/2021 13:31, Andrew Gregory wrote:
On 06/24/21 at 12:58pm, Morgan Adamiec wrote:
On 24/06/2021 12:51, Allan McRae wrote:
On 24/6/21 9:33 pm, Morgan Adamiec wrote:
On 24/06/2021 09:25, Allan McRae wrote:
On 24/6/21 6:19 pm, Morgan Adamiec wrote: > > > On 24/06/2021 06:38, Allan McRae wrote: >> On 15/6/21 5:11 am, morganamilo wrote: >>> When using a local repo as a cachedir pacman -Scc or pacman -Sc >>> without KeepCurrent will delete the repo database and packages >>> resulting in a lost database and non functional pacman. >>> >>> So special case file:/// repos in cache to not get nuked and foce >>> KeepCurrent on. >>> --- >> >> This behaviour change is undocumented. >> >> I don't think we should do this for -Scc. That is explicitly asking all >> package files in cache directories to be removed, so that is what you get. >> >> I also am leaning towards this behaviour for -Sc being unexpected (even >> if documented). Pacman's job is really not to manage complex cache >> set-ups and I'm not convinced this should be supported. >> >> Allan >> > > > I want this to go ahead, especially with -Scc. Otherwise -Scc can end up > losing you your .db file, breaking pacman, the local repo's .db and all > the build packages. > > This use case is not complex in my opinion. It's just adding one line to > your pacman.conf to stop every single package being duplicated. > > If you're still against this do you have any alternative solutions in > mind to handle this? >
Not using -Scc. Why would you use that and expect it not to delete packages?
A
Because I want everything deleted in the main /var/cache/pacman/pkg dir.
Pacman asks what to do for each cache directory and you could say yes to removing files from /var/cache/pacman/pkg/ and no for other cache directories. So that already achieves what you are doing without adding a special case to the codebase.
Pacman's job is not to be a specialised cache cleaning utility. Hence why the suggestion of adding the ability to select how many versions of each package is kept was implemented in paccache.
Allan
Yes, that does work but then you have to be careful about what to wipe and can't automate it. I would agree if it wasn't for the case accidentally deleting that dir loses you your repo.
I believe as a workaround aurutils bind mounts the repo into cache as readonly.
Would you be more accepting of it straight up just skipping any cachedirs that happen to be a local repo and not inspect it at all?
-1. -Sc/-Scc are destructive operations. There are a variety of weird cache setups where they could be more destructive than intended. People with weird cache setups really probably just shouldn't use them.
Do you have any alternate solution in mind?
There are a range of possibilities. e.g. a script containing: pacman --cachedir=/var/cache/pacman/pkg/ -Scc
On 24/06/2021 13:57, Allan McRae wrote:
On 24/6/21 10:48 pm, Morgan Adamiec wrote:
On 24/06/2021 13:31, Andrew Gregory wrote:
On 06/24/21 at 12:58pm, Morgan Adamiec wrote:
On 24/06/2021 12:51, Allan McRae wrote:
On 24/6/21 9:33 pm, Morgan Adamiec wrote:
On 24/06/2021 09:25, Allan McRae wrote: > On 24/6/21 6:19 pm, Morgan Adamiec wrote: >> >> >> On 24/06/2021 06:38, Allan McRae wrote: >>> On 15/6/21 5:11 am, morganamilo wrote: >>>> When using a local repo as a cachedir pacman -Scc or pacman -Sc >>>> without KeepCurrent will delete the repo database and packages >>>> resulting in a lost database and non functional pacman. >>>> >>>> So special case file:/// repos in cache to not get nuked and foce >>>> KeepCurrent on. >>>> --- >>> >>> This behaviour change is undocumented. >>> >>> I don't think we should do this for -Scc. That is explicitly asking all >>> package files in cache directories to be removed, so that is what you get. >>> >>> I also am leaning towards this behaviour for -Sc being unexpected (even >>> if documented). Pacman's job is really not to manage complex cache >>> set-ups and I'm not convinced this should be supported. >>> >>> Allan >>> >> >> >> I want this to go ahead, especially with -Scc. Otherwise -Scc can end up >> losing you your .db file, breaking pacman, the local repo's .db and all >> the build packages. >> >> This use case is not complex in my opinion. It's just adding one line to >> your pacman.conf to stop every single package being duplicated. >> >> If you're still against this do you have any alternative solutions in >> mind to handle this? >> > > Not using -Scc. Why would you use that and expect it not to delete > packages? > > A >
Because I want everything deleted in the main /var/cache/pacman/pkg dir.
Pacman asks what to do for each cache directory and you could say yes to removing files from /var/cache/pacman/pkg/ and no for other cache directories. So that already achieves what you are doing without adding a special case to the codebase.
Pacman's job is not to be a specialised cache cleaning utility. Hence why the suggestion of adding the ability to select how many versions of each package is kept was implemented in paccache.
Allan
Yes, that does work but then you have to be careful about what to wipe and can't automate it. I would agree if it wasn't for the case accidentally deleting that dir loses you your repo.
I believe as a workaround aurutils bind mounts the repo into cache as readonly.
Would you be more accepting of it straight up just skipping any cachedirs that happen to be a local repo and not inspect it at all?
-1. -Sc/-Scc are destructive operations. There are a variety of weird cache setups where they could be more destructive than intended. People with weird cache setups really probably just shouldn't use them.
Do you have any alternate solution in mind?
There are a range of possibilities. e.g. a script containing:
pacman --cachedir=/var/cache/pacman/pkg/ -Scc
--cachedir appends not overrides
participants (4)
-
Allan McRae
-
Andrew Gregory
-
Morgan Adamiec
-
morganamilo