[pacman-dev] [PATCH 0/2][WIP][RFC] hooks
This is a lazy conversion of a work-in-progress hook implementation from my pacutils library [1] (hence all the pu_* prefixes). I have left out several previously outlined features that I do not see as essential to an initial implementation and can be easily added later if they are actually needed: * mid-transaction hooks (PreInstall, PostRemove, etc..) * passing triggering packages/files to the hook * system vs user hook directories with overriding (use backup=() instead) * specifying hook order Some included features not previously discussed that I know of: * pre-transaction hooks can cancel the transaction * multiple triggers per-hook Major TODO's: * documentation * error handling * use _alpm_run_chroot or similar to run the hook instead of system() I have chosen to implement hooks entirely in the front-end as the only alternatives I came up with were having alpm do the file parsing itself or having pacman repeatedly reset the list of hooks as packages are added/removed. Because this is a rather minimal implementation, my primary concern at this point is making sure that no essential features are missing and that I don't commit us to anything that would make implementing the remaining features more difficult later. Some example hooks: # basic hook [Trigger] Operation = Install Object = File Target = usr/bin/foo # note no / prefix [Action] When = PostTransaction Exec = /usr/bin/echo "I'm in a hook" # don't uninstall linux* packages [Trigger] Operation = Remove Object = Package Target = linux* [Action] When = PreTransaction Exec = /usr/bin/false Previous discussion: https://mailman.archlinux.org/pipermail/pacman-dev/2013-July/017508.html Wiki: https://wiki.archlinux.org/index.php/User:Allan/Pacman_Hooks [1] https://github.com/andrewgregory/pacutils Andrew Gregory (2): add TRANS_COMMIT_{START,END} events add basic hook support lib/libalpm/alpm.h | 5 +- lib/libalpm/sync.c | 6 ++ lib/libalpm/trans.c | 7 +++ src/pacman/Makefile.am | 1 + src/pacman/callback.c | 42 +++++++++++++ src/pacman/hook.c | 156 +++++++++++++++++++++++++++++++++++++++++++++++++ src/pacman/hook.h | 52 +++++++++++++++++ 7 files changed, 268 insertions(+), 1 deletion(-) create mode 100644 src/pacman/hook.c create mode 100644 src/pacman/hook.h -- 1.9.1
--- lib/libalpm/alpm.h | 5 ++++- lib/libalpm/sync.c | 6 ++++++ lib/libalpm/trans.c | 7 +++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index b0adb95..af33181 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -358,7 +358,10 @@ typedef enum _alpm_event_type_t { ALPM_EVENT_PACSAVE_CREATED, /** A .pacorig file was created; See alpm_event_pacorig_created_t for * arguments */ - ALPM_EVENT_PACORIG_CREATED + ALPM_EVENT_PACORIG_CREATED, + + ALPM_EVENT_TRANS_COMMIT_START, + ALPM_EVENT_TRANS_COMMIT_END, } alpm_event_type_t; /** Events. diff --git a/lib/libalpm/sync.c b/lib/libalpm/sync.c index a025b68..99592df 100644 --- a/lib/libalpm/sync.c +++ b/lib/libalpm/sync.c @@ -1348,6 +1348,9 @@ int _alpm_sync_commit(alpm_handle_t *handle, alpm_list_t **data) EVENT(handle, &event); } + event.type = ALPM_EVENT_TRANS_COMMIT_START; + EVENT(handle, &event); + /* remove conflicting and to-be-replaced packages */ if(trans->remove) { _alpm_log(handle, ALPM_LOG_DEBUG, "removing conflicting and to-be-replaced packages\n"); @@ -1365,6 +1368,9 @@ int _alpm_sync_commit(alpm_handle_t *handle, alpm_list_t **data) return -1; } + event.type = ALPM_EVENT_TRANS_COMMIT_END; + EVENT(handle, &event); + return 0; } diff --git a/lib/libalpm/trans.c b/lib/libalpm/trans.c index e5328c5..a6bbfe7 100644 --- a/lib/libalpm/trans.c +++ b/lib/libalpm/trans.c @@ -178,10 +178,17 @@ int SYMEXPORT alpm_trans_commit(alpm_handle_t *handle, alpm_list_t **data) trans->state = STATE_COMMITING; if(trans->add == NULL) { + alpm_event_t event; + event.type = ALPM_EVENT_TRANS_COMMIT_START; + EVENT(handle, &event); + if(_alpm_remove_packages(handle, 1) == -1) { /* pm_errno is set by _alpm_remove_packages() */ return -1; } + + event.type = ALPM_EVENT_TRANS_COMMIT_END; + EVENT(handle, &event); } else { if(_alpm_sync_commit(handle, data) == -1) { /* pm_errno is set by _alpm_sync_commit() */ -- 1.9.1
--- src/pacman/Makefile.am | 1 + src/pacman/callback.c | 39 +++++++++++++ src/pacman/hook.c | 156 +++++++++++++++++++++++++++++++++++++++++++++++++ src/pacman/hook.h | 52 +++++++++++++++++ 4 files changed, 248 insertions(+) create mode 100644 src/pacman/hook.c create mode 100644 src/pacman/hook.h diff --git a/src/pacman/Makefile.am b/src/pacman/Makefile.am index 5f10308..1834e3e 100644 --- a/src/pacman/Makefile.am +++ b/src/pacman/Makefile.am @@ -31,6 +31,7 @@ pacman_SOURCES = \ conf.h conf.c \ database.c \ deptest.c \ + hook.h hook.c \ ini.h ini.c \ package.h package.c \ pacman.h pacman.c \ diff --git a/src/pacman/callback.c b/src/pacman/callback.c index 340a3a1..35ff077 100644 --- a/src/pacman/callback.c +++ b/src/pacman/callback.c @@ -34,6 +34,7 @@ #include "pacman.h" #include "callback.h" #include "util.h" +#include "hook.h" #include "conf.h" /* download progress bar */ @@ -148,6 +149,35 @@ static void fill_progress(const int bar_percent, const int disp_percent, fflush(stdout); } +static int run_hooks(pu_hook_when_t when) +{ + alpm_list_t *i, *hooks = NULL, *files = pu_hook_find_files(); + int ret = 0; + for(i = files; i; i = alpm_list_next(i)) { + pu_hook_t *hook = pu_hook_load_file(i->data); + if(hook) { + hooks = alpm_list_add(hooks, hook); + } else { + ret = 1; + } + } + + for(i = hooks; i; i = alpm_list_next(i)) { + pu_hook_t *hook = i->data; + if(hook->when == when + && pu_hook_match_transaction(hook, config->handle) + && pu_hook_run(hook) != 0) { + ret = 1; + } + } + + FREELIST(files); + /*alpm_list_free_inner(pu_hook_free);*/ + alpm_list_free(hooks); + + return ret; +} + /* callback to handle messages/notifications from libalpm transactions */ void cb_event(alpm_event_t *event) { @@ -336,6 +366,15 @@ void cb_event(alpm_event_t *event) } } break; + case ALPM_EVENT_TRANS_COMMIT_START: + if(run_hooks(PU_HOOK_WHEN_BEFORE_TRANSACTION) != 0) { + printf("interrupting transaction...\n"); + alpm_trans_interrupt(config->handle); + } + break; + case ALPM_EVENT_TRANS_COMMIT_END: + run_hooks(PU_HOOK_WHEN_AFTER_TRANSACTION); + break; /* all the simple done events, with fallthrough for each */ case ALPM_EVENT_FILECONFLICTS_DONE: case ALPM_EVENT_CHECKDEPS_DONE: diff --git a/src/pacman/hook.c b/src/pacman/hook.c new file mode 100644 index 0000000..34c2f62 --- /dev/null +++ b/src/pacman/hook.c @@ -0,0 +1,156 @@ +#include "hook.h" +#include "ini.h" + +#include <fnmatch.h> +#include <string.h> +#include <dirent.h> + +#define FNMATCH(pattern, string) fnmatch(pattern, string, 0) + +alpm_list_t *pu_hook_find_files(void) +{ + alpm_list_t *files = NULL; + DIR *dd = opendir(PU_HOOK_USER_DIR); + struct dirent entry, *result; + if(!dd) { + return NULL; + } + + while(readdir_r(dd, &entry, &result) == 0 && result) { + char *path = malloc(strlen(PU_HOOK_USER_DIR) + strlen(entry.d_name) + 2); + sprintf(path, "%s/%s", PU_HOOK_USER_DIR, entry.d_name); + files = alpm_list_add(files, path); + } + + return files; +} + +static int _filelist_contains_glob(alpm_filelist_t *files, const char *glob) +{ + unsigned int i; + for(i = 0; i < files->count; ++i) { + if(FNMATCH(glob, files->files[i].name) == 0) { + return 1; + } + } + return 0; +} + +static int _pu_hook_trigger_match_packages(pu_trigger_t *trigger, alpm_list_t *pkgs) +{ + alpm_list_t *i; + for(i = pkgs; i; i = alpm_list_next(i)) { + alpm_pkg_t *p = i->data; + switch(trigger->target) { + case PU_TRIGGER_OBJECT_PACKAGE: + if(FNMATCH(trigger->glob, alpm_pkg_get_name(p)) == 0) { + return 1; + } + break; + case PU_TRIGGER_OBJECT_FILE: + if(_filelist_contains_glob(alpm_pkg_get_files(p), trigger->glob)) { + return 1; + } + break; + } + } + return 0; +} + +static int _pu_hook_trigger_match_transaction(pu_trigger_t *trigger, alpm_handle_t *handle) +{ + switch(trigger->operation) { + case PU_TRIGGER_OPERATION_INSTALL: + return _pu_hook_trigger_match_packages(trigger, alpm_trans_get_add(handle)); + break; + case PU_TRIGGER_OPERATION_REMOVE: + return _pu_hook_trigger_match_packages(trigger, alpm_trans_get_remove(handle)); + break; + } + return 0; +} + +int pu_hook_match_transaction(pu_hook_t *hook, alpm_handle_t *handle) +{ + alpm_list_t *i; + for(i = hook->triggers; i; i = alpm_list_next(i)) { + if(_pu_hook_trigger_match_transaction(i->data, handle)) { + return 1; + } + } + return 0; +} + +int pu_hook_run(pu_hook_t *hook) +{ + return system(hook->cmd); +} + +static int _parse_hook(const char __attribute__((unused)) *file, + int __attribute__((unused)) linenum, const char *section, + char *key, char *val, void *data) +{ + pu_hook_t *hook = data; + + if(section && !key && !val) { + if(strcmp(section, "Trigger") == 0) { + hook->triggers = alpm_list_add(hook->triggers, calloc(sizeof(pu_trigger_t), 1)); + } + } + + if(!section || !key || !val) { + return 0; + } + + if(strcmp(section, "Trigger") == 0) { + pu_trigger_t *trig = alpm_list_last(hook->triggers)->data; + if(strcmp(key, "Object") == 0) { + if(strcmp(val, "File") == 0) { + trig->target = PU_TRIGGER_OBJECT_FILE; + } else if(strcmp(val, "Package") == 0) { + trig->target = PU_TRIGGER_OBJECT_PACKAGE; + } else { + fprintf(stderr, "unknown object '%s'\n", val); + } + } else if(strcmp(key, "Operation") == 0) { + if(strcmp(val, "Install") == 0) { + trig->operation = PU_TRIGGER_OPERATION_INSTALL; + } else if(strcmp(val, "Remove") == 0) { + trig->operation = PU_TRIGGER_OPERATION_REMOVE; + } else { + fprintf(stderr, "unknown action '%s'\n", val); + } + } else if(strcmp(key, "Target") == 0) { + trig->glob = strdup(val); + } else { + fprintf(stderr, "unknown key '%s'\n", key); + } + } else if(strcmp(section, "Action") == 0) { + if(strcmp(key, "When") == 0) { + if(strcmp(val, "PreTransaction") == 0) { + hook->when = PU_HOOK_WHEN_BEFORE_TRANSACTION; + } else if(strcmp(val, "PostTransaction") == 0) { + hook->when = PU_HOOK_WHEN_AFTER_TRANSACTION; + } else { + fprintf(stderr, "unknown When '%s'\n", val); + } + } else if(strcmp(key, "Exec") == 0) { + hook->cmd = strdup(val); + } else { + fprintf(stderr, "unknown key '%s'\n", key); + } + } else { + fprintf(stderr, "unknown section '%s'\n", section); + } + + return 0; +} + +pu_hook_t *pu_hook_load_file(const char *path) +{ + pu_hook_t *hook = calloc(sizeof(pu_hook_t), 1); + parse_ini(path, _parse_hook, hook); + return hook; +} + +/* vim: set ts=2 sw=2 noet: */ diff --git a/src/pacman/hook.h b/src/pacman/hook.h new file mode 100644 index 0000000..812a9a4 --- /dev/null +++ b/src/pacman/hook.h @@ -0,0 +1,52 @@ +#ifndef PU_HOOK_H +#define PU_HOOK_H + +#include <alpm.h> + +#ifndef PU_HOOK_SYS_DIR +#define PU_HOOK_SYS_DIR "/usr/lib/pacutils/hooks/" +#endif + +#ifndef PU_HOOK_USER_DIR +#define PU_HOOK_USER_DIR "/etc/pacman.d/hooks/" +#endif + +typedef enum { + PU_HOOK_WHEN_BEFORE_TRANSACTION, + PU_HOOK_WHEN_AFTER_TRANSACTION, +} pu_hook_when_t; + +typedef enum { + PU_TRIGGER_OBJECT_PACKAGE, + PU_TRIGGER_OBJECT_FILE, +} pu_hook_trigger_object_t; + +typedef enum { + PU_TRIGGER_OPERATION_INSTALL, + PU_TRIGGER_OPERATION_REMOVE, +} pu_hook_trigger_operation_t; + +typedef struct { + pu_hook_trigger_object_t target; + pu_hook_trigger_operation_t operation; + char *glob; +} pu_trigger_t; + +typedef struct { + char *name; + pu_hook_when_t when; + alpm_list_t *triggers; + char *cmd; +} pu_hook_t; + +alpm_list_t *pu_hook_find_files(void); + +pu_hook_t *pu_hook_load_file(const char *path); + +int pu_hook_run(pu_hook_t *hook); + +int pu_hook_match_transaction(pu_hook_t *hooks, alpm_handle_t *handle); + +#endif /* PU_HOOK_H */ + +/* vim: set ts=2 sw=2 noet: */ -- 1.9.1
On 10/04/14 04:59 PM, Andrew Gregory wrote:
Because this is a rather minimal implementation, my primary concern at this point is making sure that no essential features are missing and that I don't commit us to anything that would make implementing the remaining features more difficult later. I have long looked forward to hooks, but the use cases I have in mind might be unreasonable. Could the framework be written to expose certain details about the package (file list, dependencies, etc) to the Exec = script? Hooks for packages that are already installed, can determine this with pacman -Q. But it might be nice to have a hook that changes the indentation of any newly installed *.py say.
A crazier idea is "changing dependencies" during installation instead of using -d. One complaint about Arch is e.g. python2-matplotlib "depending" on python2-pyqt4 when really it can be configured to use gtk instead. If there was something like When = PostExtract, a script could edit the .PKGINFO file and switch qt to gtk. However, it would probably be stupid to make pacman extract all the .PKGINFO files before resolving dependencies.
On Fri, Apr 11, 2014 at 9:36 AM, Connor Behan <connor.behan@gmail.com> wrote:
A crazier idea is "changing dependencies" during installation instead of using -d. One complaint about Arch is e.g. python2-matplotlib "depending" on python2-pyqt4 when really it can be configured to use gtk instead. If there was something like When = PostExtract, a script could edit the .PKGINFO file and switch qt to gtk. However, it would probably be stupid to make pacman extract all the .PKGINFO files before resolving dependencies.
If you can then add the qt dep (and capability) back when installing qt4 to your final scenario, things get even weirder. Because then the package doesn't depend on just one, but both these packages, in case you were going to remove gtk (and needed -dd to do so). This again makes me think about "at least one of" groups of dependencies, which aren't really compatible with the current approach with bash arrays (except for some ugly prefix or suffix magic I'm not willing to call a solution). cheers! mar77i
On 04/11/14 01:59, Andrew Gregory wrote:
This is a lazy conversion of a work-in-progress hook implementation from my pacutils library [1] (hence all the pu_* prefixes). I have left out several previously outlined features that I do not see as essential to an initial implementation and can be easily added later if they are actually needed: * mid-transaction hooks (PreInstall, PostRemove, etc..) * passing triggering packages/files to the hook * system vs user hook directories with overriding (use backup=() instead) * specifying hook order
Some included features not previously discussed that I know of: * pre-transaction hooks can cancel the transaction * multiple triggers per-hook
Major TODO's: * documentation * error handling * use _alpm_run_chroot or similar to run the hook instead of system()
I have chosen to implement hooks entirely in the front-end as the only alternatives I came up with were having alpm do the file parsing itself or having pacman repeatedly reset the list of hooks as packages are added/removed.
Although I welcome the addition of hooks, I don't think it's a good idea to make it a frontend/pacman-only implementation. Just like today one wouldn't consider a package operation (e.g. install) to be complete without dependencies handling or the scriptlets run, tomorrow it won't be considered complete without hooks having been processed. So it should definitely be an ALPM thing. In fact, one of the reason for hooks has been said to replace "common" install files with hooks, so it is thought of as part of the whole operation process, and therefore would mean any frontend not reimplementing hooks exactly like pacman does would be broken! This doesn't sound good/right to me. Frontends could do the parsing themselves, but then only set things up in ALPM, much like they set repos or which packages/groups to ignore. But then, it's ALPM who makes use of it, and maybe ignores a package or asks user about it, not the frontend. Additionally, since there was talk that if a package installs hook files, they should be taken into account within the transaction (for remaining package operations), it seems it might be better to have ALPM actually do the parsing, since it could then also load any new hook files added as part of a package operation. Frontends would then only set which folders are to be used/looked in for hook files. (And there'd be an event INVALID_HOOK or something, for when ALPM fails to parse a hook file, and will therefore ignore it.) What is the problem with ALPM doing the parsing, that you didn't want to do it? Cheers, -j
Because this is a rather minimal implementation, my primary concern at this point is making sure that no essential features are missing and that I don't commit us to anything that would make implementing the remaining features more difficult later.
Some example hooks:
# basic hook [Trigger] Operation = Install Object = File Target = usr/bin/foo # note no / prefix
[Action] When = PostTransaction Exec = /usr/bin/echo "I'm in a hook"
# don't uninstall linux* packages [Trigger] Operation = Remove Object = Package Target = linux*
[Action] When = PreTransaction Exec = /usr/bin/false
Previous discussion: https://mailman.archlinux.org/pipermail/pacman-dev/2013-July/017508.html
Wiki: https://wiki.archlinux.org/index.php/User:Allan/Pacman_Hooks
[1] https://github.com/andrewgregory/pacutils
Andrew Gregory (2): add TRANS_COMMIT_{START,END} events add basic hook support
lib/libalpm/alpm.h | 5 +- lib/libalpm/sync.c | 6 ++ lib/libalpm/trans.c | 7 +++ src/pacman/Makefile.am | 1 + src/pacman/callback.c | 42 +++++++++++++ src/pacman/hook.c | 156 +++++++++++++++++++++++++++++++++++++++++++++++++ src/pacman/hook.h | 52 +++++++++++++++++ 7 files changed, 268 insertions(+), 1 deletion(-) create mode 100644 src/pacman/hook.c create mode 100644 src/pacman/hook.h
On 11/04/14 17:36, Connor Behan wrote:
On 10/04/14 04:59 PM, Andrew Gregory wrote:
Because this is a rather minimal implementation, my primary concern at this point is making sure that no essential features are missing and that I don't commit us to anything that would make implementing the remaining features more difficult later. I have long looked forward to hooks, but the use cases I have in mind might be unreasonable. Could the framework be written to expose certain details about the package (file list, dependencies, etc) to the Exec = script? Hooks for packages that are already installed, can determine this with pacman -Q. But it might be nice to have a hook that changes the indentation of any newly installed *.py say.
If you read the email and the linked wiki design page, that is a future consideration.
A crazier idea is "changing dependencies" during installation instead of using -d. One complaint about Arch is e.g. python2-matplotlib "depending" on python2-pyqt4 when really it can be configured to use gtk instead. If there was something like When = PostExtract, a script could edit the .PKGINFO file and switch qt to gtk. However, it would probably be stupid to make pacman extract all the .PKGINFO files before resolving dependencies.
We already provide a tool to "achieve" that... makepkg! :P
On 04/11/14 at 11:49am, Olivier Brunel wrote:
On 04/11/14 01:59, Andrew Gregory wrote:
This is a lazy conversion of a work-in-progress hook implementation from my pacutils library [1] (hence all the pu_* prefixes). I have left out several previously outlined features that I do not see as essential to an initial implementation and can be easily added later if they are actually needed: * mid-transaction hooks (PreInstall, PostRemove, etc..) * passing triggering packages/files to the hook * system vs user hook directories with overriding (use backup=() instead) * specifying hook order
Some included features not previously discussed that I know of: * pre-transaction hooks can cancel the transaction * multiple triggers per-hook
Major TODO's: * documentation * error handling * use _alpm_run_chroot or similar to run the hook instead of system()
I have chosen to implement hooks entirely in the front-end as the only alternatives I came up with were having alpm do the file parsing itself or having pacman repeatedly reset the list of hooks as packages are added/removed.
Although I welcome the addition of hooks, I don't think it's a good idea to make it a frontend/pacman-only implementation.
Just like today one wouldn't consider a package operation (e.g. install) to be complete without dependencies handling or the scriptlets run, tomorrow it won't be considered complete without hooks having been processed. So it should definitely be an ALPM thing.
In fact, one of the reason for hooks has been said to replace "common" install files with hooks, so it is thought of as part of the whole operation process, and therefore would mean any frontend not reimplementing hooks exactly like pacman does would be broken! This doesn't sound good/right to me.
Frontends could do the parsing themselves, but then only set things up in ALPM, much like they set repos or which packages/groups to ignore. But then, it's ALPM who makes use of it, and maybe ignores a package or asks user about it, not the frontend.
Additionally, since there was talk that if a package installs hook files, they should be taken into account within the transaction (for remaining package operations), it seems it might be better to have ALPM actually do the parsing, since it could then also load any new hook files added as part of a package operation. Frontends would then only set which folders are to be used/looked in for hook files. (And there'd be an event INVALID_HOOK or something, for when ALPM fails to parse a hook file, and will therefore ignore it.)
What is the problem with ALPM doing the parsing, that you didn't want to do it?
Cheers, -j
Right now there is a clear separation between what alpm and pacman are responsible for reading. pacman handles all of the configuration and alpm handles the actual package (which includes any install scripts). Despite their similarity to install scripts, every discussion of hooks so far has anticipated them being user-configurable, making them configuration files more in front-end territory. Having alpm be responsible for reading them would alter the relationship between alpm and the front-end. I don't think that's the correct solution and won't be implementing it without a clear consensus first. The fact that this would introduce additional work for front-ends is unfortunate, but doesn't necessarily mean that the back-end is the correct place for this. Any alpm front-end already likely parses pacman.conf which is much more difficult than parsing the hook files. Even our own scripts in contrib/ don't parse it the same way pacman does. Having the front-end parse and configure the hooks for alpm upfront doesn't really work because we would need to update the hooks in alpm as the hook files were added/removed/updated which would involve the same amount of work as the current implementation. I do hope to move *some* of this into alpm, particularly the actual execution of the individual hooks and possibly the selection of which hooks to run at a given time. The parsing and triggering I currently intend to keep in the front-end. apg
On 04/11/14 16:48, Andrew Gregory wrote:
On 04/11/14 at 11:49am, Olivier Brunel wrote:
On 04/11/14 01:59, Andrew Gregory wrote:
This is a lazy conversion of a work-in-progress hook implementation from my pacutils library [1] (hence all the pu_* prefixes). I have left out several previously outlined features that I do not see as essential to an initial implementation and can be easily added later if they are actually needed: * mid-transaction hooks (PreInstall, PostRemove, etc..) * passing triggering packages/files to the hook * system vs user hook directories with overriding (use backup=() instead) * specifying hook order
Some included features not previously discussed that I know of: * pre-transaction hooks can cancel the transaction * multiple triggers per-hook
Major TODO's: * documentation * error handling * use _alpm_run_chroot or similar to run the hook instead of system()
I have chosen to implement hooks entirely in the front-end as the only alternatives I came up with were having alpm do the file parsing itself or having pacman repeatedly reset the list of hooks as packages are added/removed.
Although I welcome the addition of hooks, I don't think it's a good idea to make it a frontend/pacman-only implementation.
Just like today one wouldn't consider a package operation (e.g. install) to be complete without dependencies handling or the scriptlets run, tomorrow it won't be considered complete without hooks having been processed. So it should definitely be an ALPM thing.
In fact, one of the reason for hooks has been said to replace "common" install files with hooks, so it is thought of as part of the whole operation process, and therefore would mean any frontend not reimplementing hooks exactly like pacman does would be broken! This doesn't sound good/right to me.
Frontends could do the parsing themselves, but then only set things up in ALPM, much like they set repos or which packages/groups to ignore. But then, it's ALPM who makes use of it, and maybe ignores a package or asks user about it, not the frontend.
Additionally, since there was talk that if a package installs hook files, they should be taken into account within the transaction (for remaining package operations), it seems it might be better to have ALPM actually do the parsing, since it could then also load any new hook files added as part of a package operation. Frontends would then only set which folders are to be used/looked in for hook files. (And there'd be an event INVALID_HOOK or something, for when ALPM fails to parse a hook file, and will therefore ignore it.)
What is the problem with ALPM doing the parsing, that you didn't want to do it?
Cheers, -j
Right now there is a clear separation between what alpm and pacman are responsible for reading. pacman handles all of the configuration and alpm handles the actual package (which includes any install scripts). Despite their similarity to install scripts, every discussion of hooks so far has anticipated them being user-configurable, making them configuration files more in front-end territory. Having alpm be responsible for reading them would alter the relationship between alpm and the front-end. I don't think that's the correct solution and won't be implementing it without a clear consensus first.
The fact that this would introduce additional work for front-ends is unfortunate, but doesn't necessarily mean that the back-end is the correct place for this. Any alpm front-end already likely parses pacman.conf which is much more difficult than parsing the hook files. Even our own scripts in contrib/ don't parse it the same way pacman does.
Having the front-end parse and configure the hooks for alpm upfront doesn't really work because we would need to update the hooks in alpm as the hook files were added/removed/updated which would involve the same amount of work as the current implementation.
I do hope to move *some* of this into alpm, particularly the actual execution of the individual hooks and possibly the selection of which hooks to run at a given time. The parsing and triggering I currently intend to keep in the front-end.
Well, I don't think the triggering should be up to the front-end really. They could do the parsing and setting up hooks in ALPM, much like they do repos, but deciding if/when to trigger hooks, and actually triggering them, should be up to ALPM in my mind. Just like a frontend might tell ALPM not to run scriptlets, but it won't decide when to run them, nor ran them itself. IOW, if the frontend is the one deciding when to trigger hooks and actually triggering them, then hooks are nothing else than a user feature, but cannot be thought of as part of the package operation, since it's not part of ALPM. Yes, hook files are user-configurable and therefore more like configuration files, and I can see why that makes them more frontend territory, but they are also intended to replace (some) install files, and they will also be provided by/within packages (in Arch, though I would assume other places it's used as well), so they clearly aren't just user configuration. By which I mean, regardless of what frontend a user uses, which may also parse pacman.conf or could use another configuration file completely, every package operation done will remain consistent, in that it will be processed just as with pacman (assuming ALPM is setup the same, ofc). That's the point of having a library w/ frontends. But if hook files are provided by packages themselves, clearly the syntax of those files isn't pacman-defined, but ALPM-defined, and that even if pacman were to do the parsing. Or, that would mean hooks are a user feature of pacman (the frontend), and Arch decides to "abuse" that feature, making pacman the only frontend that *can* be used, because it (Arch) doesn't just rely only on ALPM to do its package management anymore (of which pacman was merely the official frontend) but (via its package-provided hook files) would actually rely on a pacman-specific feature, for package operations to be performed completely/correctly. That doesn't sound right to me. -j
apg
On 12/04/14 01:21, Olivier Brunel wrote:
On 04/11/14 16:48, Andrew Gregory wrote:
On 04/11/14 at 11:49am, Olivier Brunel wrote:
On 04/11/14 01:59, Andrew Gregory wrote:
This is a lazy conversion of a work-in-progress hook implementation from my pacutils library [1] (hence all the pu_* prefixes). I have left out several previously outlined features that I do not see as essential to an initial implementation and can be easily added later if they are actually needed: * mid-transaction hooks (PreInstall, PostRemove, etc..) * passing triggering packages/files to the hook * system vs user hook directories with overriding (use backup=() instead) * specifying hook order
Some included features not previously discussed that I know of: * pre-transaction hooks can cancel the transaction * multiple triggers per-hook
Major TODO's: * documentation * error handling * use _alpm_run_chroot or similar to run the hook instead of system()
I have chosen to implement hooks entirely in the front-end as the only alternatives I came up with were having alpm do the file parsing itself or having pacman repeatedly reset the list of hooks as packages are added/removed.
Although I welcome the addition of hooks, I don't think it's a good idea to make it a frontend/pacman-only implementation.
Just like today one wouldn't consider a package operation (e.g. install) to be complete without dependencies handling or the scriptlets run, tomorrow it won't be considered complete without hooks having been processed. So it should definitely be an ALPM thing.
In fact, one of the reason for hooks has been said to replace "common" install files with hooks, so it is thought of as part of the whole operation process, and therefore would mean any frontend not reimplementing hooks exactly like pacman does would be broken! This doesn't sound good/right to me.
Frontends could do the parsing themselves, but then only set things up in ALPM, much like they set repos or which packages/groups to ignore. But then, it's ALPM who makes use of it, and maybe ignores a package or asks user about it, not the frontend.
Additionally, since there was talk that if a package installs hook files, they should be taken into account within the transaction (for remaining package operations), it seems it might be better to have ALPM actually do the parsing, since it could then also load any new hook files added as part of a package operation. Frontends would then only set which folders are to be used/looked in for hook files. (And there'd be an event INVALID_HOOK or something, for when ALPM fails to parse a hook file, and will therefore ignore it.)
What is the problem with ALPM doing the parsing, that you didn't want to do it?
Cheers, -j
Right now there is a clear separation between what alpm and pacman are responsible for reading. pacman handles all of the configuration and alpm handles the actual package (which includes any install scripts). Despite their similarity to install scripts, every discussion of hooks so far has anticipated them being user-configurable, making them configuration files more in front-end territory. Having alpm be responsible for reading them would alter the relationship between alpm and the front-end. I don't think that's the correct solution and won't be implementing it without a clear consensus first.
The fact that this would introduce additional work for front-ends is unfortunate, but doesn't necessarily mean that the back-end is the correct place for this. Any alpm front-end already likely parses pacman.conf which is much more difficult than parsing the hook files. Even our own scripts in contrib/ don't parse it the same way pacman does.
Having the front-end parse and configure the hooks for alpm upfront doesn't really work because we would need to update the hooks in alpm as the hook files were added/removed/updated which would involve the same amount of work as the current implementation.
I do hope to move *some* of this into alpm, particularly the actual execution of the individual hooks and possibly the selection of which hooks to run at a given time. The parsing and triggering I currently intend to keep in the front-end.
Well, I don't think the triggering should be up to the front-end really. They could do the parsing and setting up hooks in ALPM, much like they do repos, but deciding if/when to trigger hooks, and actually triggering them, should be up to ALPM in my mind. Just like a frontend might tell ALPM not to run scriptlets, but it won't decide when to run them, nor ran them itself.
I agree fully here.
IOW, if the frontend is the one deciding when to trigger hooks and actually triggering them, then hooks are nothing else than a user feature, but cannot be thought of as part of the package operation, since it's not part of ALPM.
Yes, hook files are user-configurable and therefore more like configuration files, and I can see why that makes them more frontend territory, but they are also intended to replace (some) install files, and they will also be provided by/within packages (in Arch, though I would assume other places it's used as well), so they clearly aren't just user configuration.
By which I mean, regardless of what frontend a user uses, which may also parse pacman.conf or could use another configuration file completely, every package operation done will remain consistent, in that it will be processed just as with pacman (assuming ALPM is setup the same, ofc). That's the point of having a library w/ frontends.
But if hook files are provided by packages themselves, clearly the syntax of those files isn't pacman-defined, but ALPM-defined, and that even if pacman were to do the parsing.
Or, that would mean hooks are a user feature of pacman (the frontend), and Arch decides to "abuse" that feature, making pacman the only frontend that *can* be used, because it (Arch) doesn't just rely only on ALPM to do its package management anymore (of which pacman was merely the official frontend) but (via its package-provided hook files) would actually rely on a pacman-specific feature, for package operations to be performed completely/correctly. That doesn't sound right to me.
I am going to propose moving the configuration parsing code (ini.h,c) to the backend. (I am assuming this is used for hook config parsing without looking at the patches.) All front-ends to libalpm to need to be able to handle the hooks so common code for this is needed. So...: - ini.h,c moved to backend - functions exposed to library - pacman.conf reading adjusted - hook parsing and triggering -> backend Allan
On 11/04/14 09:59, Andrew Gregory wrote:
This is a lazy conversion of a work-in-progress hook implementation from my pacutils library [1] (hence all the pu_* prefixes). I have left out several previously outlined features that I do not see as essential to an initial implementation and can be easily added later if they are actually needed: * mid-transaction hooks (PreInstall, PostRemove, etc..) * passing triggering packages/files to the hook * system vs user hook directories with overriding (use backup=() instead) * specifying hook order
Some included features not previously discussed that I know of: * pre-transaction hooks can cancel the transaction * multiple triggers per-hook
Major TODO's: * documentation * error handling * use _alpm_run_chroot or similar to run the hook instead of system()
I have chosen to implement hooks entirely in the front-end as the only alternatives I came up with were having alpm do the file parsing itself or having pacman repeatedly reset the list of hooks as packages are added/removed.
Because this is a rather minimal implementation, my primary concern at this point is making sure that no essential features are missing and that I don't commit us to anything that would make implementing the remaining features more difficult later.
I have finally had a decent look at your patches. As I said in an earlier email, hooks needs to be handled by the backend. Despite what I said in the previous email, that does not necessarily mean moving the ini parsing code to the backend, although common hook parsing code there would be useful for frontend developers. Allowing pretransaction hooks to stop the transaction seems fine to me. Although in your example, it is clear that you are relying on return value. Given post-package hooks could return non-zero, could we do this another way for consistency. BTW, the proposal on the wiki also allowed multiple targets to trigger a hook.
Some example hooks:
# basic hook [Trigger] Operation = Install Object = File Target = usr/bin/foo # note no / prefix
[Action] When = PostTransaction Exec = /usr/bin/echo "I'm in a hook"
# don't uninstall linux* packages [Trigger] Operation = Remove Object = Package Target = linux*
[Action] When = PreTransaction Exec = /usr/bin/false
This is yet another format! It would make me joyously happy (I'm bad at superlatives) to have the format of the hooks completely defined on the wiki page before we go forward. We (you...) can still implement post transaction hooks first as these are my main concern (updating desktop and font cache), but we need the full spec fleshed out so that we do not end in a mess.
Previous discussion: https://mailman.archlinux.org/pipermail/pacman-dev/2013-July/017508.html
Wiki: https://wiki.archlinux.org/index.php/User:Allan/Pacman_Hooks
participants (5)
-
Allan McRae
-
Andrew Gregory
-
Connor Behan
-
Martti Kühne
-
Olivier Brunel